From patchwork Thu Oct 10 18:23:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830836 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3C2E9CFC5EC for ; Thu, 10 Oct 2024 18:33:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=YxrYeQ29bTb2udoxQtf7r/ekFVNOr4pMlk2GbU44JeY=; b=ow5f3s1RZamOVy egXEpKN37EYTJEhw5yrEaokkkvx971bE38jlAUhbp1qU4aaWp4wvQQn0pFyAoiXiV8zbvj+J4J2Q8 ts4Qk4O8RjGQ/eyyUcTX+8sY4VfBTRQ0D/kePJYu11CNG3JutzWnNRgSX0eHDW++uc0ZZ+qgT7xVv LRHHugEC3QcySCsDZmD4cxvEOYENkQUrgHGXwynkPrL6rjthCXrPIpAKheWRgM7ky4iUpcfI0FyMi Ff1CG24NTTuxTE2dR8T67P4i/bZmFejw7QSua6U0IFoSu0ZRXLTHtTnMn/4kej7g5uuknhwS0vHbb 3pfa9JpMJ7ckSU4sWTlA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syxyF-0000000DuRp-1xT7; Thu, 10 Oct 2024 18:33:07 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqH-0000000Dpc4-15jY for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:24:55 +0000 Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-7ea0dbb7cc1so2052586a12.0 for ; Thu, 10 Oct 2024 11:24:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584691; x=1729189491; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=Om/x5O1cFoCu45xFQlZl7muX4OGvvwFhFfu/OqGssvQ=; b=tCsBk8HaNFJgE7k6da8O+927jDUhAdilLbD6Ct0HgJr6ILzIqoH7HqDbdGX3904isM mgjRlaLXsMfuyAoxkw1WPhJdK3pB5VO/dxbxxKMIal5c9Z8EWR5FbmnyBwNxrl584hj8 llhunBIIUNFcrX/3VjyNPRQ3o0OsEX7z8X0VE4Bgz/2ziMssp34F8oGWo+58KaUGxXnp GgSspRdccjVn7EAlku3QQ/kf2DmnFRd4xHkQRA0RG2GhInoi/RZeZJ1lmT5zInJExN8C nubiDEnlewvl8u2EFKmo4455ddHS15hNF7FMoQlwgmnzw50YNrRY+bHWh95QruI1FQ0a A0vA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584691; x=1729189491; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Om/x5O1cFoCu45xFQlZl7muX4OGvvwFhFfu/OqGssvQ=; b=OxXJCLlmnvAYzOTu7matjitzzYr917el1p1y3JoaZ6GIMGkXDHkb7KljEw4PM2vBRw GOILxwINZ7ecfk47TqNLcbC94VJJZA1+j8ESWo1Aqfdi7EitkgjnifRlCQfnQRFjOky6 zWwh6v5VF3ks7P88aA/SrdDkU0MNucjUPnIKphLWslQ0bIsIPLbdnZPQlpHNoeZK76rS D0V/qeh0tou8rsFsD/C9rhz/B3aKSjLGzTw9FUzQWSs3tac+HM4+hr1QG8/GHv5eJ2hd KGNaKf79VXxs9l1x/X7/gkopgvKGMwZ1vK6m8zfDnwExdX1tRkiY3ZCrFKSdBkythaUP jWMA== X-Forwarded-Encrypted: i=1; AJvYcCU60EdINAEdovJtoaTQzGKSFFwPFoLOy07HUSxs14TDVzHHwFYnjWRsXc4uV+gECwoQSvKv0GpOz8ruaA==@lists.infradead.org X-Gm-Message-State: AOJu0YwM09C364RN3mobcVSALkqN9kuUsTLbLhX/kgHMkPgcyVSy+ra5 ovJ1M1LGOGHHHtkHp5t990iejmAtLkkgEZyYYdwOpAJB5EHELdGIgwtPgjjlMm2mFJQCufyV8sd d8g== X-Google-Smtp-Source: AGHT+IG89v4lIFRhnNqxe9hFvIIxqDvldHGwH//6ib2L4odxYyk8SJA2JGOjXIHuxBeh8ylp7HgA6abaBbM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:eac4:b0:2e2:8f4d:457 with SMTP id 98e67ed59e1d1-2e2c80b5015mr7266a91.2.1728584689990; Thu, 10 Oct 2024 11:24:49 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:03 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-2-seanjc@google.com> Subject: [PATCH v13 01/85] KVM: Drop KVM_ERR_PTR_BAD_PAGE and instead return NULL to indicate an error From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112453_380243_83DC830D X-CRM114-Status: GOOD ( 12.25 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Remove KVM_ERR_PTR_BAD_PAGE and instead return NULL, as "bad page" is just a leftover bit of weirdness from days of old when KVM stuffed a "bad" page into the guest instead of actually handling missing pages. See commit cea7bb21280e ("KVM: MMU: Make gfn_to_page() always safe"). Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_pr.c | 2 +- arch/powerpc/kvm/book3s_xive_native.c | 2 +- arch/s390/kvm/vsie.c | 2 +- arch/x86/kvm/lapic.c | 2 +- include/linux/kvm_host.h | 7 ------- virt/kvm/kvm_main.c | 15 ++++++--------- 6 files changed, 10 insertions(+), 20 deletions(-) diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c index 7b8ae509328f..d7721297b9b6 100644 --- a/arch/powerpc/kvm/book3s_pr.c +++ b/arch/powerpc/kvm/book3s_pr.c @@ -645,7 +645,7 @@ static void kvmppc_patch_dcbz(struct kvm_vcpu *vcpu, struct kvmppc_pte *pte) int i; hpage = gfn_to_page(vcpu->kvm, pte->raddr >> PAGE_SHIFT); - if (is_error_page(hpage)) + if (!hpage) return; hpage_offset = pte->raddr & ~PAGE_MASK; diff --git a/arch/powerpc/kvm/book3s_xive_native.c b/arch/powerpc/kvm/book3s_xive_native.c index 6e2ebbd8aaac..d9bf1bc3ff61 100644 --- a/arch/powerpc/kvm/book3s_xive_native.c +++ b/arch/powerpc/kvm/book3s_xive_native.c @@ -654,7 +654,7 @@ static int kvmppc_xive_native_set_queue_config(struct kvmppc_xive *xive, } page = gfn_to_page(kvm, gfn); - if (is_error_page(page)) { + if (!page) { srcu_read_unlock(&kvm->srcu, srcu_idx); pr_err("Couldn't get queue page %llx!\n", kvm_eq.qaddr); return -EINVAL; diff --git a/arch/s390/kvm/vsie.c b/arch/s390/kvm/vsie.c index 89cafea4c41f..763a070f5955 100644 --- a/arch/s390/kvm/vsie.c +++ b/arch/s390/kvm/vsie.c @@ -661,7 +661,7 @@ static int pin_guest_page(struct kvm *kvm, gpa_t gpa, hpa_t *hpa) struct page *page; page = gfn_to_page(kvm, gpa_to_gfn(gpa)); - if (is_error_page(page)) + if (!page) return -EINVAL; *hpa = (hpa_t)page_to_phys(page) + (gpa & ~PAGE_MASK); return 0; diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 2098dc689088..20526e4d6c62 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2664,7 +2664,7 @@ int kvm_alloc_apic_access_page(struct kvm *kvm) } page = gfn_to_page(kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT); - if (is_error_page(page)) { + if (!page) { ret = -EFAULT; goto out; } diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index db567d26f7b9..ee186a1fbaad 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -153,13 +153,6 @@ static inline bool kvm_is_error_gpa(gpa_t gpa) return gpa == INVALID_GPA; } -#define KVM_ERR_PTR_BAD_PAGE (ERR_PTR(-ENOENT)) - -static inline bool is_error_page(struct page *page) -{ - return IS_ERR(page); -} - #define KVM_REQUEST_MASK GENMASK(7,0) #define KVM_REQUEST_NO_WAKEUP BIT(8) #define KVM_REQUEST_WAIT BIT(9) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 05cbb2548d99..4b659a649dfa 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3078,19 +3078,14 @@ EXPORT_SYMBOL_GPL(gfn_to_page_many_atomic); */ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) { - struct page *page; kvm_pfn_t pfn; pfn = gfn_to_pfn(kvm, gfn); if (is_error_noslot_pfn(pfn)) - return KVM_ERR_PTR_BAD_PAGE; + return NULL; - page = kvm_pfn_to_refcounted_page(pfn); - if (!page) - return KVM_ERR_PTR_BAD_PAGE; - - return page; + return kvm_pfn_to_refcounted_page(pfn); } EXPORT_SYMBOL_GPL(gfn_to_page); @@ -3184,7 +3179,8 @@ static void kvm_set_page_accessed(struct page *page) void kvm_release_page_clean(struct page *page) { - WARN_ON(is_error_page(page)); + if (WARN_ON(!page)) + return; kvm_set_page_accessed(page); put_page(page); @@ -3208,7 +3204,8 @@ EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); void kvm_release_page_dirty(struct page *page) { - WARN_ON(is_error_page(page)); + if (WARN_ON(!page)) + return; kvm_set_page_dirty(page); kvm_release_page_clean(page); From patchwork Thu Oct 10 18:23:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830837 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 86C23CFC5EB for ; Thu, 10 Oct 2024 18:33:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=p9iNq5uFWBSWrXrtqbE+vO8YAM4H9lNNm1jE6e0bCjo=; b=ag6kMJL1btwOjb 6RaalQOGlAKt0LLBF+MFO46F8cKBaTHZt7GyEbbiLQNJSEa2kz0Gs2i37MCpEg/CvjINNxy93Ue3R 1oA0YvdE33PHjOZWELPA0pIlJ9+Gzg3TgA76G7k8lbYgVYYZsNKD3XK9DjdA0sI6gvA75NKT5x+rh cbKvq1FeEF+SHpj5Te0iBc0YhN4D5y2U35ZpNCCb4tMqWC7h11RWpISazkw4yboP0HWNMHulLi/lG ABV+22UOGAOH6AHFl/u1rdKy2kMu5zinzCfujwOTiwusxEo3TCpe3KYHXG2/OTmUPZA3oWIqigmvJ G/Oazqf5iFuLc7TuCI2w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syxyH-0000000DuTW-1PAB; Thu, 10 Oct 2024 18:33:09 +0000 Received: from mail-pl1-x64a.google.com ([2607:f8b0:4864:20::64a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqI-0000000Dpcx-32EH for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:24:56 +0000 Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-20b6144cc2aso12164435ad.0 for ; Thu, 10 Oct 2024 11:24:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584693; x=1729189493; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=yUMe47Ku2d+lU53n+UV6eA4G9zH9pOaBnB7PUsg7n0E=; b=shU7ea7RlpvjkLopneCrUBkZlJ/eYg9PruxHXuYYqoC41aPvYR8MviFIZwfv2Bs/38 V6DM4Rhjol1hLNMqyclecumybgC3FqZubBkU0Ging5vsyF/ycStxP1eNRZ8RqYcwUO6+ QIFFjFEsNwFdIFpKQOMcyhdiOrIwnUVQ0aczRiDG3U5DVsHpbLHR2gdJH5XPGINnkyOW FA4jebfmXxekNvf9FQp5hWO5lvvjLPmzuDXreg+TSJZ7IRbakavxEH2Txn31TG40lqKU +aVDrvuoXBNGaoduvnd40hFU1e+OVk+YXrp2xtTayTEAIRuw2evu92DsRJUkYmp4M0rX Kp8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584693; x=1729189493; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=yUMe47Ku2d+lU53n+UV6eA4G9zH9pOaBnB7PUsg7n0E=; b=bg0avh3eCdFIuHVk3lrCBERPbRQazaBTgXtNMnkUO73NJlV1aY21U3stmBZOUQR/uX hBv1wI/gGxYfZyECff/plVGvAno85HTlEVHqODgQPjYTvtzxi0Tdn75kAWZvr9f8NCs8 vxWBTOVATptg0oq/X5BflDkCgdEeruF2Wl1UJcl07R7bCwJx6n+aZu5sF8AvHvLxVKHN vsY3m92DcHNmvPJN2lOqPWzs/SmEJvHIMT3EwqBJjeQZTNbzU+50xWpZ452LiBn0R2gO tjZuqEalHPnFIi4WUSBgg4J8/nRf356J5SEcftA6t3+ks8r/TukW4iLMKlCJ3mZELrY8 nm3A== X-Forwarded-Encrypted: i=1; AJvYcCVqwI3LXVJ1yKPKOG/O9q5tP8/PSEHkQ2lSLCJOwgJgjdftXcJOdnO3oIspxR7z1zJYzjSd3sVb++k/bw==@lists.infradead.org X-Gm-Message-State: AOJu0Yybz4kRsUC3GaZqwwPlxDDrc+zyrBdiiyT8xmmooFi+K+jrYMRi EF8vOQHc9Xk0xed9K9AblvG7co2ngsiCjUCZvgq48Iqo3ssgT/4iOUFB8sqp9czCWMzFM2oQq0k gsg== X-Google-Smtp-Source: AGHT+IHgBvu4/6UpOEaDnS659CK1Rn1/E5r8mTk00ZsUtMRUIXJafj8Ignnkw5WxXqtPYIrr50vXY4UIGHU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:ff02:b0:205:58ee:1567 with SMTP id d9443c01a7336-20c80362278mr43175ad.0.1728584692824; Thu, 10 Oct 2024 11:24:52 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:04 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-3-seanjc@google.com> Subject: [PATCH v13 02/85] KVM: Allow calling kvm_release_page_{clean,dirty}() on a NULL page pointer From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112454_782389_E7951E08 X-CRM114-Status: GOOD ( 10.09 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Allow passing a NULL @page to kvm_release_page_{clean,dirty}(), there's no tangible benefit to forcing the callers to pre-check @page, and it ends up generating a lot of duplicate boilerplate code. Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 4b659a649dfa..2032292df0b0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3179,7 +3179,7 @@ static void kvm_set_page_accessed(struct page *page) void kvm_release_page_clean(struct page *page) { - if (WARN_ON(!page)) + if (!page) return; kvm_set_page_accessed(page); @@ -3204,7 +3204,7 @@ EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); void kvm_release_page_dirty(struct page *page) { - if (WARN_ON(!page)) + if (!page) return; kvm_set_page_dirty(page); From patchwork Thu Oct 10 18:23:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830842 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E2197CFC5EC for ; Thu, 10 Oct 2024 18:36:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=KH3NacRBIN/eDv72HTUiB+a+DPKyBscJoHj0mB25/ko=; b=JXYOEU1Z5YhSJj TMddypyW+rAsEZDXEMgp3o3p7AKyltLUR6bIvK8OFeSQLemmADxs8aB0x5UH7KJqsBTRPP886ZI7o 6XjFk0RdYpYhmXWEK15PofpMCj3EbhrfyoFbq8yXJIYCO9f1TezBamF2XJb72Bb/Iaxc+B4OiwuWO MLvbiuUpAxHRd18++kmGC0H5rpu5QpGc7nsrmUX8Bi/DpxtY8dSY82Yzp4tLTtxibZtVjtyoLJ9Xb gh9cYZAq0UoL2eDNK99r6Elg76pJamyGVpuLBO6BBbsJwFI8ICQmDyFzM/CSr3nLYJfpQfgG9GgnD iSoUYNhC7xCUEJXVcqqw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syy1H-0000000Dvfd-1L95; Thu, 10 Oct 2024 18:36:15 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqK-0000000DpeD-2C5F for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:24:58 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e3245ed6b8so23817577b3.0 for ; Thu, 10 Oct 2024 11:24:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584695; x=1729189495; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=8PiI9IzFtQVAZLsCzTZ7ta7WGdR9XnSo4A5BSh1uxRg=; b=3tpcWmDhC2p5JkkW8aB29EYcT5Tc9T3eDSrIYpr4d6VdBPy/WHXsR4WShFJm6c9Rgp IwMeB1Akf45dADb/TeQKAfku3V7l/dzk41yiqET2SfY9fONopnJnvP4hE5Hi/kSiofPx 7m6lAJganol+7NKovRFL2zZudjYJRd+Ve4FMWnQ8wtVVq2/6aCMfJM9zEq5h12F9kJoc EXxzXUN61izp/gYQafC5ezdxs0Hmt2qr6ciEnfI2pE51ck2xY2nfnsRpO0j2R6TmnFtR tvKzRl3/fD9ufDJ5YS//pg6k3swCPzJTaPvEKExRNguqGGAS0pSD+9d7EBjVWsvevfaN /u3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584695; x=1729189495; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=8PiI9IzFtQVAZLsCzTZ7ta7WGdR9XnSo4A5BSh1uxRg=; b=wght1Cewub1Pmc3cd6iUoVAnxHKRLr2HlSFshgVEUTSeXvHwtdNiUfGbN9xYcqW96W W2aUtmVEx02lltdDl9mAlWZeLvpFiTjyK0YP86KCsUsAJ2txox4Cz310ABUTZVWVZuam 0HR4tejB3d59HKIoEtARczjg5jVPkmHHFCyytPh0uHtLBy44J/fSiVZnmjsLTAE3jIEw CU+Yt/XJTUO8NwkJ4pi4IuqSFp4cm14HVo5mG1kTfP3r0ZCJfueL+y/Pg81mDtaKAFpZ R8INhtzmBJMBEjATySSS1KmDq8Lq7nmNMv5jLwDiCVUYxrMD/k1keYXyQpmehUoh8asY 91jw== X-Forwarded-Encrypted: i=1; AJvYcCX26lZEZoDqmtQNWGh/krWWGPwDytipdMrt7HBgf0YgrAT/J3k6YF2UumJL66anhfDZVaQZkFlXFjCt5Q==@lists.infradead.org X-Gm-Message-State: AOJu0YyyEHiYzBfA7gmfrhAHHeJJabR4kreKaFbWkGEleBEYPq5f9Qum p38f++hblzZpqMIZ9xum4o35L38BdWjydgQU5wRVIW+L7qWsiiUpOtg5x34xE8AamPxFBl+iPbN Mfg== X-Google-Smtp-Source: AGHT+IGz6mODxZTH5agCMYiGZey+8PwFJu50se4JqpXBNG0SwyKrN9TWGGQJqqmZYCdc2Fnc7eka1IGyLLE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a81:ad16:0:b0:6c1:298e:5a7 with SMTP id 00721157ae682-6e3224662ffmr1632867b3.5.1728584694780; Thu, 10 Oct 2024 11:24:54 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:05 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-4-seanjc@google.com> Subject: [PATCH v13 03/85] KVM: Add kvm_release_page_unused() API to put pages that KVM never consumes From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112456_733753_33309287 X-CRM114-Status: UNSURE ( 8.44 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Add an API to release an unused page, i.e. to put a page without marking it accessed or dirty. The API will be used when KVM faults-in a page but bails before installing the guest mapping (and other similar flows). Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ee186a1fbaad..ab4485b2bddc 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1216,6 +1216,15 @@ unsigned long gfn_to_hva_prot(struct kvm *kvm, gfn_t gfn, bool *writable); unsigned long gfn_to_hva_memslot(struct kvm_memory_slot *slot, gfn_t gfn); unsigned long gfn_to_hva_memslot_prot(struct kvm_memory_slot *slot, gfn_t gfn, bool *writable); + +static inline void kvm_release_page_unused(struct page *page) +{ + if (!page) + return; + + put_page(page); +} + void kvm_release_page_clean(struct page *page); void kvm_release_page_dirty(struct page *page); From patchwork Thu Oct 10 18:23:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830844 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BFB4BCFC5EC for ; Thu, 10 Oct 2024 18:37:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=MTKe1beU7gaOEt5N0uCG5+dvHKG2PDFM6i+SzQjP9lI=; b=mOMLiR9tAhx5lL oyON5Y7i1LLu2XdQKU/ydzglqa4mu+0/xv8dvTxcefI/6ZKTfaoWHA/VnV0uat3qL/biIvvIBPbhU 5eOprBmKNVGq0ZnmWzSRTGaWVVTssQ1bkn8998B7Lxj+PJHD5tw2VqIqeX1PvM2xI40YuYBuepu9r BO9kBy4JAFPlREUWUx/YWfdboxegPHzvCpvPsk4+yHgIxu5qwyeQ38sx7V8iQgZgUFMLjQuE7zygC UZtXdAATbqVWPE9H1v2nLJ+g/Cq4ZRHsCapC7ZJjEsoXQjEb8spjzcej2R/vMxQ267lS0qAyJ/TFG Oa1uTFFDPEW7FNlv9eMA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syy2e-0000000Dw4H-2SSy; Thu, 10 Oct 2024 18:37:40 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqM-0000000DpfE-2L6Q for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:24:59 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e26ba37314so24841917b3.0 for ; Thu, 10 Oct 2024 11:24:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584697; x=1729189497; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=X4CpCR2xmh5xLGpBn2Bl3P0Y4RCeyU3sCMc2ynGYEmk=; b=M7Snr5N22pikPCYYXMJYsOlgIEBf4+x3Uqg6sVxeZVadYTICrG4ulpFRgPtXVG0X0n JZ2/nX9phcaEtKWHj1yPru+AarfrxgYbtzXswb2v92ZPGKNmtdah06jEFW/mllb+oibh CBKgB1cMUdcgVGm+aTjLPjjrmXe2Hfze6XuIV0ZeaSFds53FKVgiaPzajxfWki+VusBG 0ZxzebIV31ZFXuzuGmx7TJqF8oZ/P8kgCd05nBiQQ6We33hwrDG5iyqppNKEN7BXJoyY U1tRYYoYlyeYl0xm2A4lPSen0EvpIggWblDdQu+MQKURLXfflhNdB/DNtQyuAg89xkyJ 7Jtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584697; x=1729189497; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=X4CpCR2xmh5xLGpBn2Bl3P0Y4RCeyU3sCMc2ynGYEmk=; b=HQlOybqBxZtTAd788lqNVwBs+W8K+dUAgRMhE7JBP/KeEe/oizCG6Vh19LoAB2WmwO G7kGNO10oGIPLLbh8kSYHLgKWalotUD0AkH4sffOcjqjZDOBo+SYo/6s4V+iqrt3RFzJ yBMODAJHj84U+QvxV7N290ffyoB9i5y5UY2Ni2HfajCu9xX5dJDyEXFbFj1pL78JGPgs 5OLoqMJ3pk9qcBRp2OP+jKtOl3Cvg4wEQ26KBUFa6h8ZGS5n+VWWC3SnpU0Td18HNy8y yqr3KDrUcpWQWyXtwucV8MAGlD99Zwr5Vw6Npb0QQ+MOLzEHeAZBPmRDL+KNCrKmzGNV gTPA== X-Forwarded-Encrypted: i=1; AJvYcCWcwY32ODg1EzKgd1N0WCEkHElAu6EsAXrJenGJH499ajikO5qZ8Uqiq4tndRMKNZx16eYhzeJC9hKADA==@lists.infradead.org X-Gm-Message-State: AOJu0Yxchsze+JlWmzL/lIcYoPbo7OyFe4IkGEUfBNV4cw28a0GrU5UU wth7ATydSMfMv9cSPEchfE8C3OiPyVGAh3f6NI3vvaOBIb/iGMXaL5t/OxcT2Ne4Yof/IRHPFQO zww== X-Google-Smtp-Source: AGHT+IGrULPD5dT5NWC9dFAfgTsoL9JIWGzHU9t8+6e7flVyO5QHPCRlr/jDUZCu+nhXBHYl+WGzejB3KuQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a0d:e486:0:b0:6e2:371f:4aef with SMTP id 00721157ae682-6e322168931mr204147b3.3.1728584696651; Thu, 10 Oct 2024 11:24:56 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:06 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-5-seanjc@google.com> Subject: [PATCH v13 04/85] KVM: x86/mmu: Skip the "try unsync" path iff the old SPTE was a leaf SPTE From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112458_676789_343792F5 X-CRM114-Status: GOOD ( 12.18 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Apply make_spte()'s optimization to skip trying to unsync shadow pages if and only if the old SPTE was a leaf SPTE, as non-leaf SPTEs in direct MMUs are always writable, i.e. could trigger a false positive and incorrectly lead to KVM creating a SPTE without write-protecting or marking shadow pages unsync. This bug only affects the TDP MMU, as the shadow MMU only overwrites a shadow-present SPTE when synchronizing SPTEs (and only 4KiB SPTEs can be unsync). Specifically, mmu_set_spte() drops any non-leaf SPTEs *before* calling make_spte(), whereas the TDP MMU can do a direct replacement of a page table with the leaf SPTE. Opportunistically update the comment to explain why skipping the unsync stuff is safe, as opposed to simply saying "it's someone else's problem". Cc: stable@vger.kernel.org Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/spte.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 8f7eb3ad88fc..5521608077ec 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -226,12 +226,20 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, spte |= PT_WRITABLE_MASK | shadow_mmu_writable_mask; /* - * Optimization: for pte sync, if spte was writable the hash - * lookup is unnecessary (and expensive). Write protection - * is responsibility of kvm_mmu_get_page / kvm_mmu_sync_roots. - * Same reasoning can be applied to dirty page accounting. + * When overwriting an existing leaf SPTE, and the old SPTE was + * writable, skip trying to unsync shadow pages as any relevant + * shadow pages must already be unsync, i.e. the hash lookup is + * unnecessary (and expensive). + * + * The same reasoning applies to dirty page/folio accounting; + * KVM will mark the folio dirty using the old SPTE, thus + * there's no need to immediately mark the new SPTE as dirty. + * + * Note, both cases rely on KVM not changing PFNs without first + * zapping the old SPTE, which is guaranteed by both the shadow + * MMU and the TDP MMU. */ - if (is_writable_pte(old_spte)) + if (is_last_spte(old_spte, level) && is_writable_pte(old_spte)) goto out; /* From patchwork Thu Oct 10 18:23:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830843 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C5788CFC5EC for ; Thu, 10 Oct 2024 18:37:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=CUS0d2hawXWqOtuR6vYeQ5C6CsYCPK72Bn5AzTSzUyA=; b=iyxMoeuQiSWw3w /j6pW0yyCqsS96nLwmB17Cgk2oeXSP6m1deRXpECMoYRIFd+qvpoavVc/xzvFNRUKm52RYsD+JdrY 30U1ctARoKgeP54C4Yd5NA/UWgubLDxOyris4a+v1OaGJfTAs00GnYxgQ0Xuu0H21/tbqqo3kyeL5 2aLmA3NFnGYLJN8Fg/dCtQ9it4oGsrbnhnyjyP5Dp9QT2yF1b1ba2LM/1M5cST6MuhquZWX//1p6N rBBrrVyvsTNNyBD4eJb6Gk/PWVpAUI2a8ttRxj+g43sCjUJ6O/OQ9fOKRjj4sFzgUXHKyQTFgpy0t +Rj2zuM/N+OOxbGiP/Bg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syy2f-0000000Dw4X-1MCC; Thu, 10 Oct 2024 18:37:41 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqQ-0000000Dphq-3yHw for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:05 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-e290947f6f8so1842485276.2 for ; Thu, 10 Oct 2024 11:25:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584701; x=1729189501; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=pPAj0F/7+N3G0fgp8rSBmKqwV8uR3Qu6XgX2gyD+fUo=; b=uho4rH1Ltyn1x521PuC9kj2h8uorL+rNfA819qT8YPYqGmkARzx2HIJ5RX8b2Scx4o pw/8G2HLJn/wBNNPeQ6BOz9/y0sV2YGDI5/YkZ17dMebLtTQ+0YWwHuVk4SmdTx5MAgN FHZZCWw/6icoy28KbaiYRQKg61SsybK9ggVaoLwVN3TJAUYXjGF9WxagSKJzPcFWeglj f9BIxNwTn3XGX8Vv/64aL2hbxWVunNr959SYvURuL3nwI+eRAI4hyJWuaoB/ScpM0t93 kQuWM2/o9Ivo3R1Z7sbhcoTCwzmOGAQTziNvlKTG8eLizopDKY05CWMPLITK8cJOPl76 waBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584701; x=1729189501; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pPAj0F/7+N3G0fgp8rSBmKqwV8uR3Qu6XgX2gyD+fUo=; b=s6hO4cLY5uPl3A5xrt0Q56jXZqOyeEkxGRNq5gCac8dElf31DWhxJcDbbU3/T5+2M1 iqKMit32GW9XzRwxP4p3mJYbdf+W5Yl9U21dvBnOQkXTWvlHJpgFA6NTP39bpPwxNULT FY0NX7UqP7+QtciFdcNxl4pCNWXcZ6STrJ/iTREx63IYUImLFAo619YW4C+uXsu8QNRm D9pxwAhTBcsDX+6KVbO4kfMHy+TnXj4Evx180z5l3gBMPt4QTOM3X+E9SqyWatJQclk5 Xcrr6G9imL8wF3TOvxFAI2P9MHCDI3N73itDruM+c08Ay+ompljZpfSblKLWrIsK8rDY zc0g== X-Forwarded-Encrypted: i=1; AJvYcCV7BxWh5D+cqIJOvyhKFBfXVIM4MebFMeUlW1F5bSou1qcG4MpIj7i8DcOoS0ZlPpOsfxyF0zVSplwDAQ==@lists.infradead.org X-Gm-Message-State: AOJu0Yzq6oblKDlfUXpqPYbF4D8KoYviby9dUr8AWmbuGBmPV3+kE76e wcFuvnJKoWYiCKOlGlTg8ZlksL4AXHQAXNQWoEKF5dd8B+Oxjc0iReocbxvzbCoIjGbF7JCTEe2 xCg== X-Google-Smtp-Source: AGHT+IHQgMXiDY3ifHkkle2NgOnYNSR5nXtws6LnwLsJ4J6Fqip1dtdD5j5lAqBgp/4MJdCcEXFALInKEMI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:69c3:0:b0:e1a:9ed2:67f4 with SMTP id 3f1490d57ef6-e28fe4313ffmr4949276.2.1728584700890; Thu, 10 Oct 2024 11:25:00 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:07 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-6-seanjc@google.com> Subject: [PATCH v13 05/85] KVM: x86/mmu: Don't overwrite shadow-present MMU SPTEs when prefaulting From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112503_159300_477B7906 X-CRM114-Status: GOOD ( 11.85 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Treat attempts to prefetch/prefault MMU SPTEs as spurious if there's an existing shadow-present SPTE, as overwriting a SPTE that may have been create by a "real" fault is at best confusing, and at worst potentially harmful. E.g. mmu_try_to_unsync_pages() doesn't unsync when prefetching, which creates a scenario where KVM could try to replace a Writable SPTE with a !Writable SPTE, as sp->unsync is checked prior to acquiring mmu_unsync_pages_lock. Note, this applies to three of the four flavors of "prefetch" in KVM: - KVM_PRE_FAULT_MEMORY - Async #PF (host or PV) - Prefetching The fourth flavor, SPTE synchronization, i.e. FNAME(sync_spte), _only_ overwrites shadow-present SPTEs when calling make_spte(). But SPTE synchronization specifically uses mmu_spte_update(), and so naturally avoids the @prefetch check in mmu_set_spte(). Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 3 +++ arch/x86/kvm/mmu/tdp_mmu.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a9a23e058555..a8c64069aa89 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2919,6 +2919,9 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct kvm_memory_slot *slot, } if (is_shadow_present_pte(*sptep)) { + if (prefetch) + return RET_PF_SPURIOUS; + /* * If we overwrite a PTE page pointer with a 2MB PMD, unlink * the parent of the now unreachable PTE. diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 3b996c1fdaab..3c6583468742 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1026,6 +1026,9 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu, if (WARN_ON_ONCE(sp->role.level != fault->goal_level)) return RET_PF_RETRY; + if (fault->prefetch && is_shadow_present_pte(iter->old_spte)) + return RET_PF_SPURIOUS; + if (unlikely(!fault->slot)) new_spte = make_mmio_spte(vcpu, iter->gfn, ACC_ALL); else From patchwork Thu Oct 10 18:23:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830845 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 95BC5CFC5EC for ; Thu, 10 Oct 2024 18:39:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=sZen87NPBVBI+6yLvUfIheMPNtB+PniZzA1i3kCPIA8=; b=nKfBiSCJjMnamn qT9vzNpW/C5KBnsVuD2uan4hQXV3Fct03/BXXGC05qgO/bXpy/giz1+9eHNAYkiHvpB8pH+fYD9pY PJ0DdrQ5FtbwsHY+eZYMXS6fqX/B18quN6awUTxxtLbQA49TlLzyFpdpkqLsFFEE/yJ0zCTJPTiV8 N9LorpvaxdAH81RxqwctkXdFtBVe7/lVoJXOfvKCuOMY5P2E0+lWRedlevaF7Ir8o6hrA/DCtAacY wvmhFuWrjPVBUc0tH7L5BnS0KGUZNmdTjDvQLXZIx1MV9oFYlJlvDdaYusi+XjOH2he9C9MqajHN0 x0n3+fwJVg4t28DWAQXw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syy42-0000000DwW0-31iR; Thu, 10 Oct 2024 18:39:06 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqT-0000000Dpjz-0Rn0 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:08 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e290d41291bso1588430276.1 for ; Thu, 10 Oct 2024 11:25:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584704; x=1729189504; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=zmCOkT+wuJeT9H1MjXHCrAkGhLQoiMWK376kHF4kwCg=; b=Kq1WOn3ILTCesOLo5TwG98ce4X4S2wBk+Zed3w07pth0mmR27m6+zcIyUDuJ90K+tm H+FQy9IEhYfF0RVk1sN+uPV9VcfkmlC388brN9oIUwZNBfslQrpSLQpjPEekeKwAECAW 2yGteXV6Zs9Ai7X0iKtDFBT6EI5+bcL0iQSZm+JwuYIL/GZ6kqokrCisnWfZI2cp/w52 DNf/U/JvuC2w3tIklSm+kpLK8DDwtz7HtLUb62f7F49qPLWneNZV6W3Chjdfb5rxikNY YWlXE+zrdqJb3OTa3cM6srJ+EqsaVuKwpF+4ltKq8VJKibN0Sh4+lO62nT1rXvo2yqGY OoRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584704; x=1729189504; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=zmCOkT+wuJeT9H1MjXHCrAkGhLQoiMWK376kHF4kwCg=; b=vGDoIriAV4iFJou7wCfdHtNlx3zUd1S9bSW2XCZClJ4+IoZDQuhM2R/lh/zGAGzE5x PR9nkq74kBCD0OIsI7ckW+daYozYietpvn5KFSqaw45rAcbKvyefHQBao9GZkQZ++jZK rdQWPKAOHV1L9iyIUu8H7n6qJAtp3dKPiZjoh6rPWawMLMl/Xud5ja5+nzLEj4Cua67E O5wt4FPDqlPnNwJgnRSJSxO/YVJRokBOkUoDh6PzpmR4pXtXRqOFMLnUjE5aRsS4CCXb 1V6RyjwMte/ByDOrOaMmsyfo+4ebNNLEptH5auG+2gr6b34QCUEj7+h1g1LavnVqL9+0 NLHw== X-Forwarded-Encrypted: i=1; AJvYcCWuxFt4rJqc+eMflQO4k/bxDaa+N1xZxN0DAR0K63YbY3qe0vwEXqSjioJjwNNkhcKT+KaLcjmNSdN75A==@lists.infradead.org X-Gm-Message-State: AOJu0Yxw61JDUMFnMZfbOopsuDQdLKRvrcN8U7z59lBPcGoecECpTN0c sBHu8Wnhmzrvm6/MU74ZbZotGeTsIh1aNvPBDY/upPp3i+nPmcboFEnIqg6bOc5Gqs6QEvDVWyw Ffw== X-Google-Smtp-Source: AGHT+IHLS32ZnUy4bY0gdVqRjJSlR5N1zmb3UxYk5drKGwVtxhq/Y8NIZIwMp1duovDzYwXPQsOfO5WBu4o= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6902:1812:b0:e24:9f58:dd17 with SMTP id 3f1490d57ef6-e28fe32f042mr66754276.1.1728584702932; Thu, 10 Oct 2024 11:25:02 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:08 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-7-seanjc@google.com> Subject: [PATCH v13 06/85] KVM: x86/mmu: Invert @can_unsync and renamed to @synchronizing From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112505_245091_7124CC70 X-CRM114-Status: GOOD ( 15.47 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Invert the polarity of "can_unsync" and rename the parameter to "synchronizing" to allow a future change to set the Accessed bit if KVM is synchronizing an existing SPTE. Querying "can_unsync" in that case is nonsensical, as the fact that KVM can't unsync SPTEs doesn't provide any justification for setting the Accessed bit. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 12 ++++++------ arch/x86/kvm/mmu/mmu_internal.h | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/mmu/spte.c | 4 ++-- arch/x86/kvm/mmu/spte.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 4 ++-- 6 files changed, 13 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a8c64069aa89..0f21d6f76cab 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2795,7 +2795,7 @@ static void kvm_unsync_page(struct kvm *kvm, struct kvm_mmu_page *sp) * be write-protected. */ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot, - gfn_t gfn, bool can_unsync, bool prefetch) + gfn_t gfn, bool synchronizing, bool prefetch) { struct kvm_mmu_page *sp; bool locked = false; @@ -2810,12 +2810,12 @@ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot, /* * The page is not write-tracked, mark existing shadow pages unsync - * unless KVM is synchronizing an unsync SP (can_unsync = false). In - * that case, KVM must complete emulation of the guest TLB flush before - * allowing shadow pages to become unsync (writable by the guest). + * unless KVM is synchronizing an unsync SP. In that case, KVM must + * complete emulation of the guest TLB flush before allowing shadow + * pages to become unsync (writable by the guest). */ for_each_gfn_valid_sp_with_gptes(kvm, sp, gfn) { - if (!can_unsync) + if (synchronizing) return -EPERM; if (sp->unsync) @@ -2941,7 +2941,7 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct kvm_memory_slot *slot, } wrprot = make_spte(vcpu, sp, slot, pte_access, gfn, pfn, *sptep, prefetch, - true, host_writable, &spte); + false, host_writable, &spte); if (*sptep == spte) { ret = RET_PF_SPURIOUS; diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index c98827840e07..4da83544c4e1 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -164,7 +164,7 @@ static inline gfn_t gfn_round_for_level(gfn_t gfn, int level) } int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot, - gfn_t gfn, bool can_unsync, bool prefetch); + gfn_t gfn, bool synchronizing, bool prefetch); void kvm_mmu_gfn_disallow_lpage(const struct kvm_memory_slot *slot, gfn_t gfn); void kvm_mmu_gfn_allow_lpage(const struct kvm_memory_slot *slot, gfn_t gfn); diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index ae7d39ff2d07..6e7bd8921c6f 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -963,7 +963,7 @@ static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, int host_writable = spte & shadow_host_writable_mask; slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn); make_spte(vcpu, sp, slot, pte_access, gfn, - spte_to_pfn(spte), spte, true, false, + spte_to_pfn(spte), spte, true, true, host_writable, &spte); return mmu_spte_update(sptep, spte); diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 5521608077ec..0e47fea1a2d9 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -157,7 +157,7 @@ bool spte_has_volatile_bits(u64 spte) bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, const struct kvm_memory_slot *slot, unsigned int pte_access, gfn_t gfn, kvm_pfn_t pfn, - u64 old_spte, bool prefetch, bool can_unsync, + u64 old_spte, bool prefetch, bool synchronizing, bool host_writable, u64 *new_spte) { int level = sp->role.level; @@ -248,7 +248,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, * e.g. it's write-tracked (upper-level SPs) or has one or more * shadow pages and unsync'ing pages is not allowed. */ - if (mmu_try_to_unsync_pages(vcpu->kvm, slot, gfn, can_unsync, prefetch)) { + if (mmu_try_to_unsync_pages(vcpu->kvm, slot, gfn, synchronizing, prefetch)) { wrprot = true; pte_access &= ~ACC_WRITE_MASK; spte &= ~(PT_WRITABLE_MASK | shadow_mmu_writable_mask); diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index 2cb816ea2430..c81cac9358e0 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -499,7 +499,7 @@ bool spte_has_volatile_bits(u64 spte); bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, const struct kvm_memory_slot *slot, unsigned int pte_access, gfn_t gfn, kvm_pfn_t pfn, - u64 old_spte, bool prefetch, bool can_unsync, + u64 old_spte, bool prefetch, bool synchronizing, bool host_writable, u64 *new_spte); u64 make_huge_page_split_spte(struct kvm *kvm, u64 huge_spte, union kvm_mmu_page_role role, int index); diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 3c6583468742..76bca7a726c1 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1033,8 +1033,8 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu, new_spte = make_mmio_spte(vcpu, iter->gfn, ACC_ALL); else wrprot = make_spte(vcpu, sp, fault->slot, ACC_ALL, iter->gfn, - fault->pfn, iter->old_spte, fault->prefetch, true, - fault->map_writable, &new_spte); + fault->pfn, iter->old_spte, fault->prefetch, + false, fault->map_writable, &new_spte); if (new_spte == iter->old_spte) ret = RET_PF_SPURIOUS; From patchwork Thu Oct 10 18:23:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831008 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F3B21D24442 for ; Thu, 10 Oct 2024 19:46:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LU5s+KB2Nmh6nl6V6rE3YW8uNys1tnFcjDpY+vGsEAE=; b=ykYOgQN7Hf8/WA tqa54YLLbVIwvZTnMJWxUOGa8kuoReT4ShjE0Xa9fL46u4aCfRLywWN8MHf0h7XEeuEe4tfY6lpU4 mmRXU5TZqmrezUtJ3o+93XXjMhMbDM7MvuTY6NUN8xAbXz8hWMFhujhHwmKm6cZbV8cgk2jfi0Ls8 k4Wbxg5Zyl7XRzTN41tCFREdezs524yzw20mI7SttLyq+oHb/+2Ls8VCGcbbjd6+bDC/8DVaOgLkv 7qoMTz3ekRvAaihc2svNXup928NOflezuLeMIty/6FS+fhMwvIeeOyHoGHyHItxgyvdKmR2vudAMp m5BJ0mCY/FklFUvJeolQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syz76-0000000EAk6-2Na2; Thu, 10 Oct 2024 19:46:20 +0000 Received: from mail-pj1-x1049.google.com ([2607:f8b0:4864:20::1049]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqW-0000000DpmI-1u0w for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:11 +0000 Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2e295559c37so1096520a91.0 for ; Thu, 10 Oct 2024 11:25:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584707; x=1729189507; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=GZoCfd92gqYHwqsSYOai9KQfQ7K6dxh+JWKccmiJTPU=; b=h8kgmq5KMhiBhtsbuTJw6kU2LJ3f2iQP13uN81By+PdbuW2nwg9GE7cWKJK4SIKxe9 aqQXNd0DGyY4m+d5ZAQ6AwjC5GPE0e8hdo5ZQn8POJbht1P2W0EOFPEf/SqHVj9VIEPG mTauj/V2pKKa8GfckVKm8AicpL2PyF2yZCj6/iuFPJChYGGr7qKDGMKW02z+t4+bIiJY 2yCoRqfFPRISvLKxNhzo+zUQG2ARex/YqnovbrM3qvS88K07YwkBncJVeTp3tRusFkom BIOx4xdIwVY9lgLVDyE6HEqvbD5bvi0UkH09epNU5s/AYlbnV/veBkjfvxbSlpss22U0 qsAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584707; x=1729189507; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=GZoCfd92gqYHwqsSYOai9KQfQ7K6dxh+JWKccmiJTPU=; b=ZQAcfOlAh5UwGnVbqusItCDIWV1ggN4OBvp14N1CGheKwT2Z2pwSOh0aD00hO2a7YX fxeBghpraFNRnX1TeKO0Vdckfy/30I6FBNp8kgCr6a5GLCdrE6grpE/06z0bzoLDNU5c b00rp/cASfSesIYPxqymySi5qWeU8p5lnNggSjc91QhHMDhmOOTzmiftnz7xsXXFOSeN N7KzJrILCSLCFGzNvaZpOWcqJzWT4OMsG16TqKBO320TkiLkC8KqZ/Nb9X9FPOXiDqHe fibNSVvzoLiK4BxWxITgU7Udg3NNxwlH2xhRmT3PzsZVYn6WtM8Y3OZoR38En2jeRwSE 7USg== X-Forwarded-Encrypted: i=1; AJvYcCUcYS/4ALlx6GuZW8QWA3Vmj/FPvtm6Xo9JopZD65okTY8xG5MxE/9hWP4oP5ZLKIKMvNQtnk5+jaftzQ==@lists.infradead.org X-Gm-Message-State: AOJu0YzkU0+GSjxPD7K0aALBeO2q/JRm7+f9qi/sk/kSHFkVTmQAmrxj SnSSa91p+NgGC86gvUqZw3TpQWHOP80j1yf4zJyzlwdfWpYOQdWJH5AvYCO32MGbpm0bwaJ7KOJ Now== X-Google-Smtp-Source: AGHT+IH8UDk7ruln14ZDsSWqbj1uDcx7028dIle1wpc1As3ec3XNWN4cizl8I1jAsp9rsgJdiTjtCNc1iNw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:fa84:b0:2e2:da81:40c6 with SMTP id 98e67ed59e1d1-2e2f0a524d4mr44a91.2.1728584705592; Thu, 10 Oct 2024 11:25:05 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:09 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-8-seanjc@google.com> Subject: [PATCH v13 07/85] KVM: x86/mmu: Mark new SPTE as Accessed when synchronizing existing SPTE From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112508_586377_41A6A042 X-CRM114-Status: GOOD ( 12.94 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Set the Accessed bit when making a "new" SPTE during SPTE synchronization, as _clearing_ the Accessed bit is counter-productive, and even if the Accessed bit wasn't set in the old SPTE, odds are very good the guest will access the page in the near future, as the most common case where KVM synchronizes a shadow-present SPTE is when the guest is making the gPTE read-only for Copy-on-Write (CoW). Preserving the Accessed bit will allow dropping the logic that propagates the Accessed bit to the underlying struct page when overwriting an existing SPTE, without undue risk of regressing page aging. Note, KVM's current behavior is very deliberate, as SPTE synchronization was the only "speculative" access type as of commit 947da5383069 ("KVM: MMU: Set the accessed bit on non-speculative shadow ptes"). But, much has changed since 2008, and more changes are on the horizon. Spurious clearing of the Accessed (and Dirty) was mitigated by commit e6722d9211b2 ("KVM: x86/mmu: Reduce the update to the spte in FNAME(sync_spte)"), which changed FNAME(sync_spte) to only overwrite SPTEs if the protections are actually changing. I.e. KVM is already preserving Accessed information for SPTEs that aren't dropping protections. And with the aforementioned future change to NOT mark the page/folio as accessed, KVM's SPTEs will become the "source of truth" so to speak, in which case clearing the Accessed bit outside of page aging becomes very undesirable. Suggested-by: Yan Zhao Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/spte.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 0e47fea1a2d9..618059b30b8b 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -178,7 +178,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, spte |= SPTE_TDP_AD_WRPROT_ONLY; spte |= shadow_present_mask; - if (!prefetch) + if (!prefetch || synchronizing) spte |= spte_shadow_accessed_mask(spte); /* @@ -259,7 +259,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, spte |= spte_shadow_dirty_mask(spte); out: - if (prefetch) + if (prefetch && !synchronizing) spte = mark_spte_for_access_track(spte); WARN_ONCE(is_rsvd_spte(&vcpu->arch.mmu->shadow_zero_check, spte, level), From patchwork Thu Oct 10 18:23:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C0DD2D24442 for ; Thu, 10 Oct 2024 19:46:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=rMfZSnL+t8/GGyI81TcilGmPd4V6JO370Q+jFYqbKv0=; b=tneiWJA1BMmNav tnjAEeBMT0xS+8EFDGmSKIxPWovNTmYbG6JpxxdZGS+B5cNVCHSYsTMjtqwY9uHx2Sm4D3ut6X3vq +W/0wNE4S5EXB5bF9YKNbgCoWE7/HloYsBT4F8Ce2Ki6Wa/a8Cgq9dF3UDPqfme2Gl5MXMYlbDHiB yjQ/ztlLMh/XFKiwU20BbFpiOcd3YEqhDsFFSmFdtuEeqM2G+8f187KHGJS4NK+EHvb8Zs7rx5yVJ rZKDTP/cHlmmWBNaEWps5PgDYl27HHME2hgA0jRY8W5w9i7GV09F0J14VTyarngRPlXUXCe7ZrAuU YO0w259wrGOwusEx8SyQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syz78-0000000EAlE-2JX5; Thu, 10 Oct 2024 19:46:22 +0000 Received: from mail-pj1-x1049.google.com ([2607:f8b0:4864:20::1049]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqY-0000000Dpob-24p8 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:13 +0000 Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2e2d3d5fb4eso896023a91.2 for ; Thu, 10 Oct 2024 11:25:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584709; x=1729189509; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=e3d7DBw/sg5cuh+IKhcM0BkpGN0EJ0d/2DHG844pD6k=; b=hYflAaWqfkt/9sjlBCfgQhFqXMVl+3HoIOZuTqZBbZVlTAMz/0CpZvy1EZ+Xt5U4Vk l3zYswVa/pShqe5WMq/UNsp7ryiz3zV1sn09NLFFufXCrWgkQS652k3DpQJX4VjkxCmw f/tHxYugYCdPNcfMSFwFC6865MJNcPOMzPRfnym2tadGXORr9qCJV/KqXgLE5MpBA/aA 7soIPZ+JxQvzUIgQhY/STcnxBzIl/cT2FagOECDlV+N7zK+0n1/lzQF+/Hv80tKiESJ6 fucd/zt5i39oJGbDFBQH6RdIvBQtZD0pVk7EQrmh8wU9eSrwNsW/5H/j6r3BZvIfa2hZ DjCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584709; x=1729189509; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=e3d7DBw/sg5cuh+IKhcM0BkpGN0EJ0d/2DHG844pD6k=; b=rZVS9Xb/2aT1kv5w7ji17vhXwvsg+zVmwuYiXUQiKCrSDXWlipied/ShWG/kOegvn7 CHJpBv8Rr9WtmPA7ATefv2mLGkdO62ztc5DeQxUKqfPApdEWRvibYLVl7JnLKsFVQNV/ QirWYmFZNUyQ9yzwx0plNIKIfuixacc5vhkWX4J1pJhdVpsRXPRicI1SBFiijE2Yyxhd BQTRYLmmO4BxpGTt5dIgduaL8QuphKPW716a508AJnVK1rKONUpUbLUb9aPXyFHLdDHY Y/pv93i6yZMN19EpVms8eudhuR4HQDZRBTPKn6nE4+Ci9s/UAJImOiNiQLueOCki6C1j YxkQ== X-Forwarded-Encrypted: i=1; AJvYcCWHxYDu85LYHEqZd9Xxs6hjtq0sOW3U0H3dmfqdM0uXhvEG6cLch+z1wkvy/d/H+ZtzjwMM4It1etSEqw==@lists.infradead.org X-Gm-Message-State: AOJu0Yzcn2v2A2DuT/ZY1tVhu7vUCTQcXQZRlWmErXdPbGibZFGw+1Fm lw5VFdEKNo1UEx76z72Rlf9rX2VIA9NFLECUGUnTQOVMdJVlI3WkrARu+MzNvflCchJrHEhhuKg QYw== X-Google-Smtp-Source: AGHT+IFvOj1YmixVIm2giqrxikolLV70YR7/zWjxlJNmhfeSXdlwSACqKd64r4Is1alzH3q275ZQuugRXZo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90b:153:b0:2e2:af66:c33e with SMTP id 98e67ed59e1d1-2e2f0ae73f3mr37a91.1.1728584708343; Thu, 10 Oct 2024 11:25:08 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:10 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-9-seanjc@google.com> Subject: [PATCH v13 08/85] KVM: x86/mmu: Mark folio dirty when creating SPTE, not when zapping/modifying From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112510_759360_60180F11 X-CRM114-Status: GOOD ( 28.48 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Mark pages/folios dirty when creating SPTEs to map PFNs into the guest, not when zapping or modifying SPTEs, as marking folios dirty when zapping or modifying SPTEs can be extremely inefficient. E.g. when KVM is zapping collapsible SPTEs to reconstitute a hugepage after disbling dirty logging, KVM will mark every 4KiB pfn as dirty, even though _at least_ 512 pfns are guaranteed to be in a single folio (the SPTE couldn't potentially be huge if that weren't the case). The problem only becomes worse for 1GiB HugeTLB pages, as KVM can mark a single folio dirty 512*512 times. Marking a folio dirty when mapping is functionally safe as KVM drops all relevant SPTEs in response to an mmu_notifier invalidation, i.e. ensures that the guest can't dirty a folio after access has been removed. And because KVM already marks folios dirty when zapping/modifying SPTEs for KVM reasons, i.e. not in response to an mmu_notifier invalidation, there is no danger of "prematurely" marking a folio dirty. E.g. if a filesystems cleans a folio without first removing write access, then there already exists races where KVM could mark a folio dirty before remote TLBs are flushed, i.e. before guest writes are guaranteed to stop. Furthermore, x86 is literally the only architecture that marks folios dirty on the backend; every other KVM architecture marks folios dirty at map time. x86's unique behavior likely stems from the fact that x86's MMU predates mmu_notifiers. Long, long ago, before mmu_notifiers were added, marking pages dirty when zapping SPTEs was logical, and perhaps even necessary, as KVM held references to pages, i.e. kept a page's refcount elevated while the page was mapped into the guest. At the time, KVM's rmap_remove() simply did: if (is_writeble_pte(*spte)) kvm_release_pfn_dirty(pfn); else kvm_release_pfn_clean(pfn); i.e. dropped the refcount and marked the page dirty at the same time. After mmu_notifiers were introduced, commit acb66dd051d0 ("KVM: MMU: don't hold pagecount reference for mapped sptes pages") removed the refcount logic, but kept the dirty logic, i.e. converted the above to: if (is_writeble_pte(*spte)) kvm_release_pfn_dirty(pfn); And for KVM x86, that's essentially how things have stayed over the last ~15 years, without anyone revisiting *why* KVM marks pages/folios dirty at zap/modification time, e.g. the behavior was blindly carried forward to the TDP MMU. Practically speaking, the only downside to marking a folio dirty during mapping is that KVM could trigger writeback of memory that was never actually written. Except that can't actually happen if KVM marks folios dirty if and only if a writable SPTE is created (as done here), because KVM always marks writable SPTEs as dirty during make_spte(). See commit 9b51a63024bd ("KVM: MMU: Explicitly set D-bit for writable spte."), circa 2015. Note, KVM's access tracking logic for prefetched SPTEs is a bit odd. If a guest PTE is dirty and writable, KVM will create a writable SPTE, but then mark the SPTE for access tracking. Which isn't wrong, just a bit odd, as it results in _more_ precise dirty tracking for MMUs _without_ A/D bits. To keep things simple, mark the folio dirty before access tracking comes into play, as an access-tracked SPTE can be restored in the fast page fault path, i.e. without holding mmu_lock. While writing SPTEs and accessing memslots outside of mmu_lock is safe, marking a folio dirty is not. E.g. if the fast path gets interrupted _just_ after setting a SPTE, the primary MMU could theoretically invalidate and free a folio before KVM marks it dirty. Unlike the shadow MMU, which waits for CPUs to respond to an IPI, the TDP MMU only guarantees the page tables themselves won't be freed (via RCU). Opportunistically update a few stale comments. Cc: David Matlack Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 30 ++++-------------------------- arch/x86/kvm/mmu/paging_tmpl.h | 6 +++--- arch/x86/kvm/mmu/spte.c | 20 ++++++++++++++++++-- arch/x86/kvm/mmu/tdp_mmu.c | 12 ------------ 4 files changed, 25 insertions(+), 43 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 0f21d6f76cab..1ae823ebd12b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -547,10 +547,8 @@ static bool mmu_spte_update(u64 *sptep, u64 new_spte) kvm_set_pfn_accessed(spte_to_pfn(old_spte)); } - if (is_dirty_spte(old_spte) && !is_dirty_spte(new_spte)) { + if (is_dirty_spte(old_spte) && !is_dirty_spte(new_spte)) flush = true; - kvm_set_pfn_dirty(spte_to_pfn(old_spte)); - } return flush; } @@ -593,9 +591,6 @@ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep) if (is_accessed_spte(old_spte)) kvm_set_pfn_accessed(pfn); - if (is_dirty_spte(old_spte)) - kvm_set_pfn_dirty(pfn); - return old_spte; } @@ -1250,16 +1245,6 @@ static bool spte_clear_dirty(u64 *sptep) return mmu_spte_update(sptep, spte); } -static bool spte_wrprot_for_clear_dirty(u64 *sptep) -{ - bool was_writable = test_and_clear_bit(PT_WRITABLE_SHIFT, - (unsigned long *)sptep); - if (was_writable && !spte_ad_enabled(*sptep)) - kvm_set_pfn_dirty(spte_to_pfn(*sptep)); - - return was_writable; -} - /* * Gets the GFN ready for another round of dirty logging by clearing the * - D bit on ad-enabled SPTEs, and @@ -1275,7 +1260,8 @@ static bool __rmap_clear_dirty(struct kvm *kvm, struct kvm_rmap_head *rmap_head, for_each_rmap_spte(rmap_head, &iter, sptep) if (spte_ad_need_write_protect(*sptep)) - flush |= spte_wrprot_for_clear_dirty(sptep); + flush |= test_and_clear_bit(PT_WRITABLE_SHIFT, + (unsigned long *)sptep); else flush |= spte_clear_dirty(sptep); @@ -1628,14 +1614,6 @@ static bool kvm_rmap_age_gfn_range(struct kvm *kvm, clear_bit((ffs(shadow_accessed_mask) - 1), (unsigned long *)sptep); } else { - /* - * Capture the dirty status of the page, so that - * it doesn't get lost when the SPTE is marked - * for access tracking. - */ - if (is_writable_pte(spte)) - kvm_set_pfn_dirty(spte_to_pfn(spte)); - spte = mark_spte_for_access_track(spte); mmu_spte_update_no_track(sptep, spte); } @@ -3415,7 +3393,7 @@ static bool fast_pf_fix_direct_spte(struct kvm_vcpu *vcpu, * harm. This also avoids the TLB flush needed after setting dirty bit * so non-PML cases won't be impacted. * - * Compare with set_spte where instead shadow_dirty_mask is set. + * Compare with make_spte() where instead shadow_dirty_mask is set. */ if (!try_cmpxchg64(sptep, &old_spte, new_spte)) return false; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 6e7bd8921c6f..fbaae040218b 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -892,9 +892,9 @@ static gpa_t FNAME(gva_to_gpa)(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, /* * Using the information in sp->shadowed_translation (kvm_mmu_page_get_gfn()) is - * safe because: - * - The spte has a reference to the struct page, so the pfn for a given gfn - * can't change unless all sptes pointing to it are nuked first. + * safe because SPTEs are protected by mmu_notifiers and memslot generations, so + * the pfn for a given gfn can't change unless all SPTEs pointing to the gfn are + * nuked first. * * Returns * < 0: failed to sync spte diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 618059b30b8b..8e8d6ee79c8b 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -232,8 +232,8 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, * unnecessary (and expensive). * * The same reasoning applies to dirty page/folio accounting; - * KVM will mark the folio dirty using the old SPTE, thus - * there's no need to immediately mark the new SPTE as dirty. + * KVM marked the folio dirty when the old SPTE was created, + * thus there's no need to mark the folio dirty again. * * Note, both cases rely on KVM not changing PFNs without first * zapping the old SPTE, which is guaranteed by both the shadow @@ -266,12 +266,28 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, "spte = 0x%llx, level = %d, rsvd bits = 0x%llx", spte, level, get_rsvd_bits(&vcpu->arch.mmu->shadow_zero_check, spte, level)); + /* + * Mark the memslot dirty *after* modifying it for access tracking. + * Unlike folios, memslots can be safely marked dirty out of mmu_lock, + * i.e. in the fast page fault handler. + */ if ((spte & PT_WRITABLE_MASK) && kvm_slot_dirty_track_enabled(slot)) { /* Enforced by kvm_mmu_hugepage_adjust. */ WARN_ON_ONCE(level > PG_LEVEL_4K); mark_page_dirty_in_slot(vcpu->kvm, slot, gfn); } + /* + * If the page that KVM got from the primary MMU is writable, i.e. if + * it's host-writable, mark the page/folio dirty. As alluded to above, + * folios can't be safely marked dirty in the fast page fault handler, + * and so KVM must (somewhat) speculatively mark the folio dirty even + * though it isn't guaranteed to be written as KVM won't mark the folio + * dirty if/when the SPTE is made writable. + */ + if (host_writable) + kvm_set_pfn_dirty(pfn); + *new_spte = spte; return wrprot; } diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 76bca7a726c1..517b384473c1 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -511,10 +511,6 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, if (is_leaf != was_leaf) kvm_update_page_stats(kvm, level, is_leaf ? 1 : -1); - if (was_leaf && is_dirty_spte(old_spte) && - (!is_present || !is_dirty_spte(new_spte) || pfn_changed)) - kvm_set_pfn_dirty(spte_to_pfn(old_spte)); - /* * Recursively handle child PTs if the change removed a subtree from * the paging structure. Note the WARN on the PFN changing without the @@ -1249,13 +1245,6 @@ static bool age_gfn_range(struct kvm *kvm, struct tdp_iter *iter, iter->level); new_spte = iter->old_spte & ~shadow_accessed_mask; } else { - /* - * Capture the dirty status of the page, so that it doesn't get - * lost when the SPTE is marked for access tracking. - */ - if (is_writable_pte(iter->old_spte)) - kvm_set_pfn_dirty(spte_to_pfn(iter->old_spte)); - new_spte = mark_spte_for_access_track(iter->old_spte); iter->old_spte = kvm_tdp_mmu_write_spte(iter->sptep, iter->old_spte, new_spte, @@ -1596,7 +1585,6 @@ static void clear_dirty_pt_masked(struct kvm *kvm, struct kvm_mmu_page *root, trace_kvm_tdp_mmu_spte_changed(iter.as_id, iter.gfn, iter.level, iter.old_spte, iter.old_spte & ~dbit); - kvm_set_pfn_dirty(spte_to_pfn(iter.old_spte)); } rcu_read_unlock(); From patchwork Thu Oct 10 18:23:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3C2D8CFC5EB for ; Thu, 10 Oct 2024 18:43:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=EZjcCAr8GXqjYtOv6SCM8qN/l70/JByu3sz+z7Oo5PY=; b=m+tlwgsq+zFt/q 2aGIAipXT+tTUGaKzFX9yxFIT6tun8QTbNAV58u0uE8rUtOOc+XWJLHFRbmqfyd4hDWpzFaedFuZu PjcsRQi57egLifLwl5pxDbsNEmUicGoqoil+SIve1aytiOmnmlgwNOL76rWXnJ6RWZVEOG2OeHnqD iiEL5yQ/CZbJir5aRSNADI61fIh1/weOTuaeXnUZfHPi46LiQzqMZ37SCsRD6dTwASKyRizcXtPTJ W+cWQsmi/Xi4DtiW5YdC2gM2gXSqvg4LxV3Ye+A/VPEXPVPERnb5/VppOHaWGboxG0LnQZoBMrtC2 aoQsoyLhAqZ+sac/SyRQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syy82-0000000Dxba-1qfm; Thu, 10 Oct 2024 18:43:14 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqa-0000000Dpq0-29s4 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:16 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e2baf2ff64so22478427b3.0 for ; Thu, 10 Oct 2024 11:25:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584711; x=1729189511; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=0ix/Xs4/Qn/2DvOT3Bh000Q598gzWwOGIiO61AJ4Bto=; b=wg6l/e68EEItXXcfocqTbRUpJ/oGTThYQm/2n+G9Mw+E0+rgd/O8ALeeTI0zso3low A4IuPTpimz0NQuRMW8Uj3CXmwznPneCME+/ePXkyHtXW57BceBMbIX5zxGj2TGtYC1iA LVFdYAaz40Ujvh2KG/1U9Je1xNngtGgYPesb2oS8XSfpalOoYSptDzyZkgRIHj1XxtS7 jfP03JdayC4nQ8fEyV8m4KHGF+R0vtK1thWL3EAbusKhT/mQEQYcHogc/b3dLaeVXhMW KAfRABQIpN8xmuoHrALjhVLDeEQTIydOsBSvlnSmtEpSpnJBoskxtDWeYkSylb8yh/GR bXCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584711; x=1729189511; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=0ix/Xs4/Qn/2DvOT3Bh000Q598gzWwOGIiO61AJ4Bto=; b=I6yXFjD8LjIVn+lGjI/59D1jdkeD87ADoGszVL6gQyeU4Q9M6R3dOppBnOINGp0gBy eIj+yWUDs0am6apoXBbv/x6neGkf9VzDnAi55SXEVlSqCfVjgM4Z7gnZ0IPRdBuG73BL Zz/yfEYN0M5JxCwo9C0hYN91StsRny8KSBQMSupHxMje01bnpOTMkH8fDZC4Vr68u/dO p9mYPVacpJxA4RsnK1ip7YBinfeU2NIQm2xGOk6hgTsN4cSzHaz/7Ic9VkjznYPtTowa e3XU0be1cfSjGkGORoYvEn2kgtdDPXu82GDD7cfyemADAfhPcB+ksclec9En+mNx8I6N Go/w== X-Forwarded-Encrypted: i=1; AJvYcCUC+ECKR4SXVwT7OBq8edSwdXWYT1q3ZeEPXunONOewNoEEnTCQmjQOvK3dkVlAX9fNIy6dw2CcI42lZw==@lists.infradead.org X-Gm-Message-State: AOJu0Yx7VIIlai76zcs1GDPlkXIVLuHGb+5vAuFlhPWWRt5JleY2JeU3 U2Jv1AMJoX5W6+7Nlns3z2oeG3M+RO1xIlhlO/jOavhqUI35oYZloJcU/ZRNRl/yORqQYTYey+b KBQ== X-Google-Smtp-Source: AGHT+IGD2TfP2dloLIlDYfRlI1lvmWPD7YUtu8EYW3dxRXY7JxJbZQIAJnuRF8jcBfCYGhk25T1qJs7AAuY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:5b41:b0:6db:c6ac:62a0 with SMTP id 00721157ae682-6e322305467mr243067b3.5.1728584711093; Thu, 10 Oct 2024 11:25:11 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:11 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-10-seanjc@google.com> Subject: [PATCH v13 09/85] KVM: x86/mmu: Mark page/folio accessed only when zapping leaf SPTEs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112512_755809_D99D09BD X-CRM114-Status: GOOD ( 17.34 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Now that KVM doesn't clobber Accessed bits of shadow-present SPTEs, e.g. when prefetching, mark folios as accessed only when zapping leaf SPTEs, which is a rough heuristic for "only in response to an mmu_notifier invalidation". Page aging and LRUs are tolerant of false negatives, i.e. KVM doesn't need to be precise for correctness, and re-marking folios as accessed when zapping entire roots or when zapping collapsible SPTEs is expensive and adds very little value. E.g. when a VM is dying, all of its memory is being freed; marking folios accessed at that time provides no known value. Similarly, because KVM marks folios as accessed when creating SPTEs, marking all folios as accessed when userspace happens to delete a memslot doesn't add value. The folio was marked access when the old SPTE was created, and will be marked accessed yet again if a vCPU accesses the pfn again after reloading a new root. Zapping collapsible SPTEs is a similar story; marking folios accessed just because userspace disable dirty logging is a side effect of KVM behavior, not a deliberate goal. As an intermediate step, a.k.a. bisection point, towards *never* marking folios accessed when dropping SPTEs, mark folios accessed when the primary MMU might be invalidating mappings, as such zappings are not KVM initiated, i.e. might actually be related to page aging and LRU activity. Note, x86 is the only KVM architecture that "double dips"; every other arch marks pfns as accessed only when mapping into the guest, not when mapping into the guest _and_ when removing from the guest. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- Documentation/virt/kvm/locking.rst | 76 +++++++++++++++--------------- arch/x86/kvm/mmu/mmu.c | 4 +- arch/x86/kvm/mmu/tdp_mmu.c | 7 ++- 3 files changed, 43 insertions(+), 44 deletions(-) diff --git a/Documentation/virt/kvm/locking.rst b/Documentation/virt/kvm/locking.rst index 20a9a37d1cdd..3d8bf40ca448 100644 --- a/Documentation/virt/kvm/locking.rst +++ b/Documentation/virt/kvm/locking.rst @@ -147,49 +147,51 @@ Then, we can ensure the dirty bitmaps is correctly set for a gfn. 2) Dirty bit tracking -In the origin code, the spte can be fast updated (non-atomically) if the +In the original code, the spte can be fast updated (non-atomically) if the spte is read-only and the Accessed bit has already been set since the Accessed bit and Dirty bit can not be lost. But it is not true after fast page fault since the spte can be marked writable between reading spte and updating spte. Like below case: -+------------------------------------------------------------------------+ -| At the beginning:: | -| | -| spte.W = 0 | -| spte.Accessed = 1 | -+------------------------------------+-----------------------------------+ -| CPU 0: | CPU 1: | -+------------------------------------+-----------------------------------+ -| In mmu_spte_clear_track_bits():: | | -| | | -| old_spte = *spte; | | -| | | -| | | -| /* 'if' condition is satisfied. */| | -| if (old_spte.Accessed == 1 && | | -| old_spte.W == 0) | | -| spte = 0ull; | | -+------------------------------------+-----------------------------------+ -| | on fast page fault path:: | -| | | -| | spte.W = 1 | -| | | -| | memory write on the spte:: | -| | | -| | spte.Dirty = 1 | -+------------------------------------+-----------------------------------+ -| :: | | -| | | -| else | | -| old_spte = xchg(spte, 0ull) | | -| if (old_spte.Accessed == 1) | | -| kvm_set_pfn_accessed(spte.pfn);| | -| if (old_spte.Dirty == 1) | | -| kvm_set_pfn_dirty(spte.pfn); | | -| OOPS!!! | | -+------------------------------------+-----------------------------------+ ++-------------------------------------------------------------------------+ +| At the beginning:: | +| | +| spte.W = 0 | +| spte.Accessed = 1 | ++-------------------------------------+-----------------------------------+ +| CPU 0: | CPU 1: | ++-------------------------------------+-----------------------------------+ +| In mmu_spte_update():: | | +| | | +| old_spte = *spte; | | +| | | +| | | +| /* 'if' condition is satisfied. */ | | +| if (old_spte.Accessed == 1 && | | +| old_spte.W == 0) | | +| spte = new_spte; | | ++-------------------------------------+-----------------------------------+ +| | on fast page fault path:: | +| | | +| | spte.W = 1 | +| | | +| | memory write on the spte:: | +| | | +| | spte.Dirty = 1 | ++-------------------------------------+-----------------------------------+ +| :: | | +| | | +| else | | +| old_spte = xchg(spte, new_spte);| | +| if (old_spte.Accessed && | | +| !new_spte.Accessed) | | +| flush = true; | | +| if (old_spte.Dirty && | | +| !new_spte.Dirty) | | +| flush = true; | | +| OOPS!!! | | ++-------------------------------------+-----------------------------------+ The Dirty bit is lost in this case. diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 1ae823ebd12b..04228a7da69a 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -542,10 +542,8 @@ static bool mmu_spte_update(u64 *sptep, u64 new_spte) * to guarantee consistency between TLB and page tables. */ - if (is_accessed_spte(old_spte) && !is_accessed_spte(new_spte)) { + if (is_accessed_spte(old_spte) && !is_accessed_spte(new_spte)) flush = true; - kvm_set_pfn_accessed(spte_to_pfn(old_spte)); - } if (is_dirty_spte(old_spte) && !is_dirty_spte(new_spte)) flush = true; diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 517b384473c1..8aa0d7a7602b 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -520,10 +520,6 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, if (was_present && !was_leaf && (is_leaf || !is_present || WARN_ON_ONCE(pfn_changed))) handle_removed_pt(kvm, spte_to_child_pt(old_spte, level), shared); - - if (was_leaf && is_accessed_spte(old_spte) && - (!is_present || !is_accessed_spte(new_spte) || pfn_changed)) - kvm_set_pfn_accessed(spte_to_pfn(old_spte)); } static inline int __must_check __tdp_mmu_set_spte_atomic(struct tdp_iter *iter, @@ -865,6 +861,9 @@ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct kvm_mmu_page *root, tdp_mmu_iter_set_spte(kvm, &iter, SHADOW_NONPRESENT_VALUE); + if (is_accessed_spte(iter.old_spte)) + kvm_set_pfn_accessed(spte_to_pfn(iter.old_spte)); + /* * Zappings SPTEs in invalid roots doesn't require a TLB flush, * see kvm_tdp_mmu_zap_invalidated_roots() for details. From patchwork Thu Oct 10 18:23:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830850 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F14D0CFC5EC for ; Thu, 10 Oct 2024 18:43:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=d4yKJHNdpIuH+avuAxe96G0IpM/S0JvEXt03Qo0zun0=; b=cPhU+5JV7O5prb MBr+gCCpOk0L01ebelHNedsHaJR1SJK9yo3k5mDy0tiuIQcI+HHmizUff3wt9IUQN0BNIABGurKg0 0IlOk4We7oz1flIH+c3jAC9i/WtXxXZVuShDZoVpjMWr4JMfP9/VWnGgbc5L20IbOVmaPpKv2ZJ/f 84eGGcKLMB2mZAyUrIPP3ZEXXxmlTVBJniyIvJ3x8TdZZaqYtOvEB9mSIIEc/NaUaYw96VVBE1c2N vSi1EGlNEfNucisoyO8ZGGuKdnk4Fel27ZyfVMwOj627jFPhAmkkiLcV6tD/hWbiX87qeyLd8jE4r 6v70ktNPSzYaDytRXnZg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syy8D-0000000Dxls-0NBN; Thu, 10 Oct 2024 18:43:25 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqc-0000000Dpru-2XyL for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:18 +0000 Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-70ac9630e3aso1194440a12.1 for ; Thu, 10 Oct 2024 11:25:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584713; x=1729189513; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=k3/xcOX0JmpZSdxUAeDWhRU0v4mqVA9cM67JfDYoNsA=; b=25I4FmM9q8/zakq/tajBcDwOrvaX/2yJF+noHaaqm7+nhtCEEA/vHiU69MABElw42H B0+z7J4m0eAxHcEysEV5vjKCvS07QkxRcPK+VkmZpmuUDp6NfwO3Hvjtsgw809Xkf7cK xEWsOB6J1lk8cjv54hI8ebnqvdiE0V8pyzSHiEy5o33n4d83JNM2vXUxpVVLfIIBy2Gk vnelAIZWHr1D4pc4Ntp/6Ckp76P4LiMNNaMPcqx8leW2ZC1NVk0dvQPNTg2yv+KCnTwe kjkpA8Kzuy4XL/IX5EnjFjgQ3ax4KyRzS75vIqG1SBagCjy18+WtNgYR9ZX90VOYzF7L 2vtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584713; x=1729189513; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=k3/xcOX0JmpZSdxUAeDWhRU0v4mqVA9cM67JfDYoNsA=; b=NHy5+s4Xx+HQIM9Vf7/KN7Gd8R0aHV/xy5IsARd+IFAecRu+7TDAN24G974mKRwL8S 2CjnVYV1LCGV3ErMBhQnQa5H4hOO8GaZxTFxk9opb9B39jdlYrkwv/02I8g1BE7opZQf /PzYcprEuWvNGInMD7QjYZeF/+ftwsHTDz8GXOog92vmAT/3awlG/PdNYBxDKZnV9ztg kqfv7s8PMG/D4LtS4g/sdWnSvpkn6HBO150ZVJEbu63CJPV0XIu+KRoFYfcFPMN/6kac nt0CNaObcMiBWgTwPq2+FlBtytqVx6VqIf1SLkQ6qeHUKDoV5Jnoqlis4XT3co60dNts 7KmQ== X-Forwarded-Encrypted: i=1; AJvYcCWk9I4AZae99RDbH2/4XoX4BxJ0v7TQuRRLk6BMNAvs2LrLsKFnXyFwWdXToSa/4jIS8FfmnbSKrDNugw==@lists.infradead.org X-Gm-Message-State: AOJu0Yxi8yq8bK4sycfSiLHoET+37ZDQgTEzFZMOFfM+cnj4c7xWs0n9 J6Dtu7GVVwX/3QLen8w58UuWMItntVy+f+TYQ8v0iTEDOkJea7Du7yNva1h2fphcM1P3E3foU9r 6hg== X-Google-Smtp-Source: AGHT+IEzZdX7U43FtBqfYwOSCvhQrFRs7MoNj08p1am6l1FbD4QaVhgMc/3KfB78OlyTOzXtpjGPG3KIosU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:ea0e:b0:2e2:d843:4880 with SMTP id 98e67ed59e1d1-2e2f0c82a2fmr30a91.5.1728584713017; Thu, 10 Oct 2024 11:25:13 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:12 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-11-seanjc@google.com> Subject: [PATCH v13 10/85] KVM: x86/mmu: Use gfn_to_page_many_atomic() when prefetching indirect PTEs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112514_832953_973BADBA X-CRM114-Status: UNSURE ( 8.86 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Use gfn_to_page_many_atomic() instead of gfn_to_pfn_memslot_atomic() when prefetching indirect PTEs (direct_pte_prefetch_many() already uses the "to page" APIS). Functionally, the two are subtly equivalent, as the "to pfn" API short-circuits hva_to_pfn() if hva_to_pfn_fast() fails, i.e. is just a wrapper for get_user_page_fast_only()/get_user_pages_fast_only(). Switching to the "to page" API will allow dropping the @atomic parameter from the entire hva_to_pfn() callchain. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/paging_tmpl.h | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index fbaae040218b..36b2607280f0 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -535,8 +535,8 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, { struct kvm_memory_slot *slot; unsigned pte_access; + struct page *page; gfn_t gfn; - kvm_pfn_t pfn; if (FNAME(prefetch_invalid_gpte)(vcpu, sp, spte, gpte)) return false; @@ -549,12 +549,11 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, if (!slot) return false; - pfn = gfn_to_pfn_memslot_atomic(slot, gfn); - if (is_error_pfn(pfn)) + if (gfn_to_page_many_atomic(slot, gfn, &page, 1) != 1) return false; - mmu_set_spte(vcpu, slot, spte, pte_access, gfn, pfn, NULL); - kvm_release_pfn_clean(pfn); + mmu_set_spte(vcpu, slot, spte, pte_access, gfn, page_to_pfn(page), NULL); + kvm_release_page_clean(page); return true; } From patchwork Thu Oct 10 18:23:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830851 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0769FCFC5EB for ; Thu, 10 Oct 2024 18:43:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=BVRFdXZb7oxga7XSL+bjNemAsFT4t2hzS03Y0hFyrHc=; b=q3soJRAeQ7A7jR BqLh/BE3q1Uf4jWcwpfKGh3sx7DJfgiX0EWYHjBhTyV2qgNENRsy4K8IYEwNZ/syP2r+M+OZRxTP9 ccUkmU/Xw6fiGJWy4oU3V/s1sx/abZcGo1FyMP+oaepGWIM+/um6FfLWvU/MUT+4M0G0iiuMFyNwD MLMwpErRdbyNiQ7s4FfQH9CTvgCChOiTWECj0c/GwBbDwqdWViL7aLMr/k/g+iGilcM5qmFPtsOR+ X4qtIZTiIHUdEY8PeRaBy8yXiFZsb8dXVNT0XcDrwEM0iVWbWeOCN6D21uDZUfFONu5QAmq5Zo8Cv WSZa1WCPNNkT/S4OwpqQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syy8F-0000000Dxnu-194O; Thu, 10 Oct 2024 18:43:27 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqj-0000000Dpy4-42GE for linux-riscv@bombadil.infradead.org; Thu, 10 Oct 2024 18:25:22 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:Content-Type :Cc:To:From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Reply-To:Sender:Content-ID:Content-Description; bh=WDoARMTIexYqUKDMQLAC2OQWFTeknat7PpvqQx990/M=; b=WC5mBfewmhr0oTCmlmNwSCj/Pp pyZFqDuD9jdf1OyDYyY6Z+780w+rFPt1b3N6St2vNJy6Xo40msLkaeO0EXEzw2hy7pUSmSAtbFbcn hjyMvRQdYxXlrbYo4u93w7sohWmmtcarVfPKc9k6P8/iq1ezmx6/6EsW18ui6lMOw+0wsex7g+rY7 ORB9rl6uaqhLFjFxYHGp+CeCpakIH5y9O2nNXDlUnukB927Pt5FvlmrGt/kv5ygzT3VgqD+aRUKNZ c7Qzsc6v9S3hoMLhyRe4zPTS/zgMsLfZO0IiDygb8bqQuljmk+J7sfJgdBUAHj/9RZAcZykdL7jkn 6LbSo/Pg==; Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqg-00000005UVO-3iO5 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:20 +0000 Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-20b6144cc2aso12169005ad.0 for ; Thu, 10 Oct 2024 11:25:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584715; x=1729189515; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=WDoARMTIexYqUKDMQLAC2OQWFTeknat7PpvqQx990/M=; b=l0w9lnO34VRhpw9k4OCEu+5qWOSo/Odt1wTWLT30hnPFDBw9mjJcx13yjIxqVQ+ozn RZRohhTr//PIMU5+d6u0p2Ybm0pluL9Trct+kP4ZdpaBMtyn+vjVDJ4xtjrW3W5rr4J1 xQqPMgmVruN1GprlW/XdHvx+FDJGuSIERXlP2nBikfRbtTWoVBUOgV9j0JftuApvZJ19 qVe6ULQk5PLsMp/4wZJBa8kk3LNbuqSLiixvIPjkPnYXNjwzO/ralU4fWg9M65kFhe3i Fgza66jUadX1Vy06tXYPpkdLyYki95qkQQQ/etSfZX5Hb2a4gT4fVGxo1ccleuS6iOy4 dVNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584715; x=1729189515; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=WDoARMTIexYqUKDMQLAC2OQWFTeknat7PpvqQx990/M=; b=Gn3HuUPEw9FgZRXcX2Rh0PWQXBaUVl7f3/hlexMHA2h7J11yeYGMr084hMZyeE+ov4 CGNJz9ka8Pu5ykbvFQI0gM8lVYiMQLYTlz6wga7o0StS+WusM3ByJHj8ihPnA7oWyBOD WHf7FZRIbB9UoW1npggmfSzyQRyEGv1Mw1WOelHq9deZMv2s9OTfySI85sfMa+caE01o +qwFXmX8RAK6AMsPZsb6/Or1f0Hwr9GzQ3tfSUM0Z77iLZrBhsvdAP6PvBMoiubAPTrF 1d4uBPVdrGqtHnVDl14RGQhYniX8F2mMP8KNppuHZpRsHOvLsv5KP0abKGOFHMUjQVCk 38CQ== X-Forwarded-Encrypted: i=1; AJvYcCUEZyYX3Ltk8PSFn41HI2IAI7t3EaXvzCDmIM91ZGGJ5gFokrqf/d08fALGHYXdBjAGgL2qd1CPcRo6Eg==@lists.infradead.org X-Gm-Message-State: AOJu0YwB/wGmSnb5HxrlOI9D/SfLnQMY8faaO+b7ET15yLYc9OnkR+UW g0L6Lx+uu73L6HzxAh5ohc8P9iTgErZApJK3DOZkAHaiPBzBSrW3vmnXGEE2YxNK/ITN6L5IYXA m6g== X-Google-Smtp-Source: AGHT+IFdLsiMiQR9ssEjYgvfJ+TOiOlvv8h202qp7bAbI4MCMFrC9IRsz+IufRcGyfoerq3Ms7MkoMKw8Uo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:f143:b0:20c:5beb:9c6a with SMTP id d9443c01a7336-20c80510505mr43225ad.4.1728584715291; Thu, 10 Oct 2024 11:25:15 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:13 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-12-seanjc@google.com> Subject: [PATCH v13 11/85] KVM: Rename gfn_to_page_many_atomic() to kvm_prefetch_pages() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_192518_991859_8E708C61 X-CRM114-Status: GOOD ( 11.39 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Rename gfn_to_page_many_atomic() to kvm_prefetch_pages() to try and communicate its true purpose, as the "atomic" aspect is essentially a side effect of the fact that x86 uses the API while holding mmu_lock. E.g. even if mmu_lock weren't held, KVM wouldn't want to fault-in pages, as the goal is to opportunistically grab surrounding pages that have already been accessed and/or dirtied by the host, and to do so quickly. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- include/linux/kvm_host.h | 4 ++-- virt/kvm/kvm_main.c | 6 +++--- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 04228a7da69a..5fe45ab0e818 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2958,7 +2958,7 @@ static int direct_pte_prefetch_many(struct kvm_vcpu *vcpu, if (!slot) return -1; - ret = gfn_to_page_many_atomic(slot, gfn, pages, end - start); + ret = kvm_prefetch_pages(slot, gfn, pages, end - start); if (ret <= 0) return -1; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 36b2607280f0..143b7e9f26dc 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -549,7 +549,7 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, if (!slot) return false; - if (gfn_to_page_many_atomic(slot, gfn, &page, 1) != 1) + if (kvm_prefetch_pages(slot, gfn, &page, 1) != 1) return false; mmu_set_spte(vcpu, slot, spte, pte_access, gfn, page_to_pfn(page), NULL); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ab4485b2bddc..56e7cde8c8b8 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1207,8 +1207,8 @@ void kvm_arch_flush_shadow_all(struct kvm *kvm); void kvm_arch_flush_shadow_memslot(struct kvm *kvm, struct kvm_memory_slot *slot); -int gfn_to_page_many_atomic(struct kvm_memory_slot *slot, gfn_t gfn, - struct page **pages, int nr_pages); +int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, + struct page **pages, int nr_pages); struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn); unsigned long gfn_to_hva(struct kvm *kvm, gfn_t gfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 2032292df0b0..957b4a6c9254 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3053,8 +3053,8 @@ kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn) } EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn); -int gfn_to_page_many_atomic(struct kvm_memory_slot *slot, gfn_t gfn, - struct page **pages, int nr_pages) +int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, + struct page **pages, int nr_pages) { unsigned long addr; gfn_t entry = 0; @@ -3068,7 +3068,7 @@ int gfn_to_page_many_atomic(struct kvm_memory_slot *slot, gfn_t gfn, return get_user_pages_fast_only(addr, nr_pages, FOLL_WRITE, pages); } -EXPORT_SYMBOL_GPL(gfn_to_page_many_atomic); +EXPORT_SYMBOL_GPL(kvm_prefetch_pages); /* * Do not use this helper unless you are absolutely certain the gfn _must_ be From patchwork Thu Oct 10 18:23:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831010 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BE1F6D24448 for ; Thu, 10 Oct 2024 19:46:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=tS4W1e3QMH6bsLfKjBD2u9hhmclW2bPr40M4sgxY66Y=; b=zf330057Eg0Mjc Q8vARKQTYZDd2hQa4PpvPtsEMeUpFGxoNZcZ6dAgbE5Y2X2GWh7ag+DkGbqfRj4nLuqn1i62s+Q+L fQJsaTIIuuZR15oMbFlgaDzPdOQ35jWnljw7xi2djUpqFZlICbVAmetzxtJOqUhrpwm+IOINbVj81 WMX2x5NWbBjXt3zHTq+S+blSnU/axOS7lFuNUqneDjklj3Z8QSPhQrWs23umfgiQcXBEBUubRo++t swmSlP8OdisMrmKZ9RzM60XOClZ928Ty2Oq/QQmBSmmUrg9Jjnjn62IEHmwejFDA5RoT4+HseEzPc yToQ4H97HpbNLtiqj0ag==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syz7A-0000000EAnO-3Gnl; Thu, 10 Oct 2024 19:46:24 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqh-0000000DpuQ-3kjL for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:23 +0000 Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-71df4f10044so1398122b3a.1 for ; Thu, 10 Oct 2024 11:25:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584717; x=1729189517; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=IMk6Ksq+DkJF8Cmr0W3poBl1lc0GSthNwEC5P5onZMQ=; b=sYztD1QUHkHCxhJUevPDoJF4UDpBqIWxrLp4Fsc5G3u67OwUqDTyKbqhAKCZPzJ0d8 qRK1Fa9bB06nko7fzVcGEhCI+y1w5VdhwVp/LLs1a6pVEGIvhu5ntvEhQJ1kNfZZGWFv iGi2Nwxi8R1PVsHxD+9i09eW1iQvDjX1p7b4InpT4z5RV0k9HvhqonQpuuYXH69vwNdu ZY6CSpu+KVE8A5eaTeR4l5pqAT5Fuhdgy0cYd8NxvfQdZ3nRB4FbkRSE5MYs4z/KR6yb I+ApWd+1p8f6iY4xfplNq8eqYOJv97lfMldu9Wvpdnu8LB+vryaM+CIWcGjlL47ZHZuP auGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584717; x=1729189517; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=IMk6Ksq+DkJF8Cmr0W3poBl1lc0GSthNwEC5P5onZMQ=; b=PSoxlmWUsb/dR2panFDgVXrrrWDbSWSxKTSdCIzxkgqHaf54upoT6pR8BlwjOVWqTq 1QPC8dJcVG2KU4UV4dAMrYB0f9/vySbpxlA1WKXloLcEk8w0wyDGnrHBaIuadx1jIV8X 4KP11gwVYMKUURkBP5fpzkeiXmOUuZHSIdCEVcJIW+vzjim/Wi7MMRSYKihI/lJdAGpq /LIAcxAcZHo0oYXaW43Vzy+9c/CMQA6kt+2D1gr9DYEWsgGMsd/dYkhcB2nkNFhs73SP M36uuJmC9GE73V6BpmGhg1o/on8abjPpb8P7MLvbD2gROXNheJfmzJ8hXL3IIWh6GLYe ucvQ== X-Forwarded-Encrypted: i=1; AJvYcCUPrYpLtD4xQJjgak7BGpdagUSRYUE7xEdIeCyPqNpexNvHLBwGQhVDtXpLp07AY19G76i6E50UBBF1wA==@lists.infradead.org X-Gm-Message-State: AOJu0YyVxpQEg8e8hF7Y5OhMvTRqG4lGqlw2D2ebq04wIgCa0hvyyhdH TPj1apG95XMMyJ6nF8ZzyMqWHVBslMX9AtF+E8GguvnhchMJP56Ue/9th3Cyoc/xTS/acGzd0Kl nHg== X-Google-Smtp-Source: AGHT+IEgBAPk5NggidzeRIz4/KASJ69N4B17cGxFjG2BQTaLRrpMtimTBEcirbwLQ+H7ew0HtHwcTgFgXoI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2d8a:b0:71d:fc7c:4a34 with SMTP id d2e1a72fcca58-71e1dbff219mr24371b3a.6.1728584717336; Thu, 10 Oct 2024 11:25:17 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:14 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-13-seanjc@google.com> Subject: [PATCH v13 12/85] KVM: Drop @atomic param from gfn=>pfn and hva=>pfn APIs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112520_068928_9A8B1907 X-CRM114-Status: GOOD ( 16.94 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Drop @atomic from the myriad "to_pfn" APIs now that all callers pass "false", and remove a comment blurb about KVM running only the "GUP fast" part in atomic context. No functional change intended. Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- Documentation/virt/kvm/locking.rst | 4 +-- arch/arm64/kvm/mmu.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_hv.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_radix.c | 2 +- arch/x86/kvm/mmu/mmu.c | 12 ++++---- include/linux/kvm_host.h | 4 +-- virt/kvm/kvm_main.c | 39 ++++++-------------------- virt/kvm/kvm_mm.h | 4 +-- virt/kvm/pfncache.c | 2 +- 9 files changed, 23 insertions(+), 48 deletions(-) diff --git a/Documentation/virt/kvm/locking.rst b/Documentation/virt/kvm/locking.rst index 3d8bf40ca448..f463ac42ac7a 100644 --- a/Documentation/virt/kvm/locking.rst +++ b/Documentation/virt/kvm/locking.rst @@ -135,8 +135,8 @@ We dirty-log for gfn1, that means gfn2 is lost in dirty-bitmap. For direct sp, we can easily avoid it since the spte of direct sp is fixed to gfn. For indirect sp, we disabled fast page fault for simplicity. -A solution for indirect sp could be to pin the gfn, for example via -kvm_vcpu_gfn_to_pfn_atomic, before the cmpxchg. After the pinning: +A solution for indirect sp could be to pin the gfn before the cmpxchg. After +the pinning: - We have held the refcount of pfn; that means the pfn can not be freed and be reused for another gfn. diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index a509b63bd4dd..a6e62cc9015c 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1569,7 +1569,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, mmu_seq = vcpu->kvm->mmu_invalidate_seq; mmap_read_unlock(current->mm); - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, false, NULL, + pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, write_fault, &writable, NULL); if (pfn == KVM_PFN_ERR_HWPOISON) { kvm_send_hwpoison_signal(hva, vma_shift); diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c index 1b51b1c4713b..8cd02ca4b1b8 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c @@ -613,7 +613,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_vcpu *vcpu, write_ok = true; } else { /* Call KVM generic code to do the slow-path check */ - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, false, NULL, + pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, writing, &write_ok, NULL); if (is_error_noslot_pfn(pfn)) return -EFAULT; diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index 408d98f8a514..26a969e935e3 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -852,7 +852,7 @@ int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, unsigned long pfn; /* Call KVM generic code to do the slow-path check */ - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, false, NULL, + pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, writing, upgrade_p, NULL); if (is_error_noslot_pfn(pfn)) return -EFAULT; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 5fe45ab0e818..0e235f276ee5 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4380,9 +4380,9 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault return kvm_faultin_pfn_private(vcpu, fault); async = false; - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, false, - &async, fault->write, - &fault->map_writable, &fault->hva); + fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, &async, + fault->write, &fault->map_writable, + &fault->hva); if (!async) return RET_PF_CONTINUE; /* *pfn has correct page already */ @@ -4402,9 +4402,9 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault * to wait for IO. Note, gup always bails if it is unable to quickly * get a page and a fatal signal, i.e. SIGKILL, is pending. */ - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, - NULL, fault->write, - &fault->map_writable, &fault->hva); + fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, NULL, + fault->write, &fault->map_writable, + &fault->hva); return RET_PF_CONTINUE; } diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 56e7cde8c8b8..2faafc7a56ae 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1232,9 +1232,8 @@ kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn); -kvm_pfn_t gfn_to_pfn_memslot_atomic(const struct kvm_memory_slot *slot, gfn_t gfn); kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool atomic, bool interruptible, bool *async, + bool interruptible, bool *async, bool write_fault, bool *writable, hva_t *hva); void kvm_release_pfn_clean(kvm_pfn_t pfn); @@ -1315,7 +1314,6 @@ void mark_page_dirty(struct kvm *kvm, gfn_t gfn); struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu); struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn_t gfn); -kvm_pfn_t kvm_vcpu_gfn_to_pfn_atomic(struct kvm_vcpu *vcpu, gfn_t gfn); kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); int kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *map); void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 957b4a6c9254..0bc077213d3e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2756,8 +2756,7 @@ static inline int check_user_page_hwpoison(unsigned long addr) /* * The fast path to get the writable pfn which will be stored in @pfn, - * true indicates success, otherwise false is returned. It's also the - * only part that runs if we can in atomic context. + * true indicates success, otherwise false is returned. */ static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, bool *writable, kvm_pfn_t *pfn) @@ -2922,7 +2921,6 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, /* * Pin guest page in memory and return its pfn. * @addr: host virtual address which maps memory to the guest - * @atomic: whether this function is forbidden from sleeping * @interruptible: whether the process can be interrupted by non-fatal signals * @async: whether this function need to wait IO complete if the * host page is not in the memory @@ -2934,22 +2932,16 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, * 2): @write_fault = false && @writable, @writable will tell the caller * whether the mapping is writable. */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool atomic, bool interruptible, - bool *async, bool write_fault, bool *writable) +kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, + bool write_fault, bool *writable) { struct vm_area_struct *vma; kvm_pfn_t pfn; int npages, r; - /* we can do it either atomically or asynchronously, not both */ - BUG_ON(atomic && async); - if (hva_to_pfn_fast(addr, write_fault, writable, &pfn)) return pfn; - if (atomic) - return KVM_PFN_ERR_FAULT; - npages = hva_to_pfn_slow(addr, async, write_fault, interruptible, writable, &pfn); if (npages == 1) @@ -2986,7 +2978,7 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool atomic, bool interruptible, } kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool atomic, bool interruptible, bool *async, + bool interruptible, bool *async, bool write_fault, bool *writable, hva_t *hva) { unsigned long addr = __gfn_to_hva_many(slot, gfn, NULL, write_fault); @@ -3008,39 +3000,24 @@ kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, writable = NULL; } - return hva_to_pfn(addr, atomic, interruptible, async, write_fault, - writable); + return hva_to_pfn(addr, interruptible, async, write_fault, writable); } EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable) { - return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, false, - NULL, write_fault, writable, NULL); + return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, NULL, + write_fault, writable, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) { - return __gfn_to_pfn_memslot(slot, gfn, false, false, NULL, true, - NULL, NULL); + return __gfn_to_pfn_memslot(slot, gfn, false, NULL, true, NULL, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); -kvm_pfn_t gfn_to_pfn_memslot_atomic(const struct kvm_memory_slot *slot, gfn_t gfn) -{ - return __gfn_to_pfn_memslot(slot, gfn, true, false, NULL, true, - NULL, NULL); -} -EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot_atomic); - -kvm_pfn_t kvm_vcpu_gfn_to_pfn_atomic(struct kvm_vcpu *vcpu, gfn_t gfn) -{ - return gfn_to_pfn_memslot_atomic(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn); -} -EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn_atomic); - kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn) { return gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn); diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index 715f19669d01..a3fa86f60d6c 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -20,8 +20,8 @@ #define KVM_MMU_UNLOCK(kvm) spin_unlock(&(kvm)->mmu_lock) #endif /* KVM_HAVE_MMU_RWLOCK */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool atomic, bool interruptible, - bool *async, bool write_fault, bool *writable); +kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, + bool write_fault, bool *writable); #ifdef CONFIG_HAVE_KVM_PFNCACHE void gfn_to_pfn_cache_invalidate_start(struct kvm *kvm, diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index f0039efb9e1e..58c706a610e5 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -198,7 +198,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) } /* We always request a writeable mapping */ - new_pfn = hva_to_pfn(gpc->uhva, false, false, NULL, true, NULL); + new_pfn = hva_to_pfn(gpc->uhva, false, NULL, true, NULL); if (is_error_noslot_pfn(new_pfn)) goto out_error; From patchwork Thu Oct 10 18:23:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830852 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 84A6DCFC5EB for ; Thu, 10 Oct 2024 18:43:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=G1lsjzJQmUtosG/yjy8UXT/aP+o35xa3pQjU60F5a5E=; b=MwmViu5T2OCiDe ZAz5/AgVthOcGOIMzRE+3WcNn/Xhnn+jzEhaprSQZELILx1/Hztr8mf464HQHotOwnwJUTfZbVkPD /bGIjvFlT3NEJBe7I5cv4toDkzZyGYzw3heUWZbPrLg6Ak4a+08NwOipEXhwFD5vUszOUXhNCI2gG L0AXVY6HCA5p35DXueQ2jHPjJyXMJt3B5M0BLXSkLwyTCu/b7dcVffL5cWVMeUyDfp6l7YAQSK52j 6LtnBYI7HRKBNAL/Xi3p0OUPUezby0XmAEcNncPU8mIUEpu+U9epj6I7yyNUhC5Chhh7bVloPBubP /zSqPo4ICte7eLRgFJFg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syy8H-0000000Dxpj-0YxU; Thu, 10 Oct 2024 18:43:29 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqi-0000000DpwP-2jYI for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:22 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e31e5d1739so25016937b3.1 for ; Thu, 10 Oct 2024 11:25:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584719; x=1729189519; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=F0+KYw/6HzWsmGATvSxbQuKdqCcRS3waqcwxHrAen8Y=; b=xcx2ikl1tbRLIGweKlp3E/XnLAXCMQRCQXm9TtYfNf9DGLbpCrsIFndqhSqsrjmSat qbIwnWEGdAd/nLXZAcHBPN1kvhcatnmCn/IV2PUU4hvQAUC0GtQOG/uesI9giBA68Vdl P8krWUymc0mzim/EeROqSoFIeSbbx3oMeiXZ0tVASZm3sdADhAR1QaWws2RKxwvjQLlU QuZptrSMsSy1XH5GGusvv0oBwN4fcKmhVFZQJ9iPkBVYw0QATlMg251iOM1Fe+gpWxFP XM+6I7qL7tGGMZT5M5+DeqRp9HNjhyKl/Q1MkapmvaxgyFylhv2ThxWRjBfsgRywaBUZ W1aA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584719; x=1729189519; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=F0+KYw/6HzWsmGATvSxbQuKdqCcRS3waqcwxHrAen8Y=; b=q9nuw43miMt/tR3Lwq8UabQ5quAV7hxcvN8+93TO7RIV8TuL34HYDJfJuSiJFa+7w+ IFJ1kwEmmzAP49d84gBhLnHl0pxE4SY432Q/vxYE4Vv0uqx2/fVD0P2wDT6vMsoBx+iz LyCJEDaXpM/O8bQ8YS65I4Hml0YLjEyU6f9mW7OMuIYIMyj9xxdcxOpHnfc7UolZqXCm VL/iyatZwqw3KJpm5C5+ygy/7d4iJxoT1BWUqIZ39XQ8ANyXTGMOM1Zqih3gdbMqc9BM bzY4NqGOnN5Ytr8eNa55SLCjgaoL03I1Pef5o2V4mLzUbUvt1J7J5tA135YnWi3Gkmwc oICw== X-Forwarded-Encrypted: i=1; AJvYcCXH56dPsK4b80A0gG7xbELp7/819Mh/PwWuvpln+w9DyBayjcUbHlJQZ+P728LB51k9GhDzq4OcuD2MvQ==@lists.infradead.org X-Gm-Message-State: AOJu0Yytut0pulnNDVsVeewiRAbIHRWeXgl2rjvLUUMbzlD+AHFvYnoe gz34DubXtewXHE64WTHFwNr1STLwQK13krDPsKdgkpDWC5nV5JRc1WUjB2K9E7FlbiYMHMD6ltx 3hg== X-Google-Smtp-Source: AGHT+IEeBu92qjGJ6/8zutZUBi8WOanw4UupdtztXpUfLE+vAeLHZHNkVuFOksmpHMU427MoGYN+Q9YZlwU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:3204:b0:62c:f976:a763 with SMTP id 00721157ae682-6e322185b22mr897567b3.1.1728584719334; Thu, 10 Oct 2024 11:25:19 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:15 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-14-seanjc@google.com> Subject: [PATCH v13 13/85] KVM: Annotate that all paths in hva_to_pfn() might sleep From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112520_731187_CA7BF306 X-CRM114-Status: UNSURE ( 9.81 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Now that hva_to_pfn() no longer supports being called in atomic context, move the might_sleep() annotation from hva_to_pfn_slow() to hva_to_pfn(). Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 0bc077213d3e..17acc75990a5 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2804,8 +2804,6 @@ static int hva_to_pfn_slow(unsigned long addr, bool *async, bool write_fault, struct page *page; int npages; - might_sleep(); - if (writable) *writable = write_fault; @@ -2939,6 +2937,8 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, kvm_pfn_t pfn; int npages, r; + might_sleep(); + if (hva_to_pfn_fast(addr, write_fault, writable, &pfn)) return pfn; From patchwork Thu Oct 10 18:23:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BE515D24447 for ; Thu, 10 Oct 2024 19:46:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=HsEipXV+8pjn//yweeQfg+LyIyE5S7cyzt78IBnzrUw=; b=2sODuUrRB72Anu BIdSfQCA6yTF+Ly4Yow00qvmEFabh10KZBtU8t9NtVMbAb0UAy/cm49XmNuZdZ3QyqhEgmE9ECS9T d5gjMy4KMNbg7l7dFmfaE6mxO6H5CLPcZ/gej7TGu+aViR9Z3JwBdfbeJpLCi51d0kiPc9mnuqIwm iH9EM7ujpZlCAxoPCOZnVz9155/535hg2hzWv0z6DK2A8BethHKVijyKo17Vi+XhIlRd/uQy7sJsm KdDn7GAgVjH5ryjjk1bS34XuqlUFA+H+94LmyT1Q7e19Z91ThGxIFKvwMBwDW8xwNJB12MIMbcBTK bINXun86M68sfimUQDYw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syz7D-0000000EApr-208V; Thu, 10 Oct 2024 19:46:27 +0000 Received: from mail-pj1-x1049.google.com ([2607:f8b0:4864:20::1049]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxql-0000000Dpyc-2tQ4 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:25 +0000 Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2e2ca4fb175so1247623a91.3 for ; Thu, 10 Oct 2024 11:25:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584722; x=1729189522; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=RdgAxVyxp31c7WibHLiyIF43paER00mxZOAIHhsdwhk=; b=b7p6IY4ClkV+QsXomlwvoGqMh+SB8UDjT7mX9uoM5ZhyMGTHNoOTrGpBZfJp2lwLVX 0x23jdq8biX8z0ZIidDU79yQDeDe6dBrdMgsah/xF71aJn4U5mr4VKqCbENkqredNlpT 0tfmwqY+XQFrBOrUvncPhvTCE4/rbEy5HZJ/ggCCaLBScKOKMlUvJ3pQP7qNuXb3HndF Zsxc8oRV34Z1rzEDTCj2Q4Ps4jWYqYy79NUHweHrj8V03+aXAOEnxXwmcoZQhpeNOYhO B+38kYQySh0koQBeoNmkx1In/nwDJQ3xu5Ei0inEsydspQsXYJq9uOlExrXPKZG/AZXL dJmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584722; x=1729189522; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RdgAxVyxp31c7WibHLiyIF43paER00mxZOAIHhsdwhk=; b=lAwVHfPHGJmwWI/hmrstIH9s9bMfi2a1Kcrx2ki1Blfn7QLSQYlQUTEc0t/Ih5jUsz u+qIDXAO9tFJOyedXEemh+EC5Zv1aFex8qJSikuR/gMoyytFPv6roJG8cJNCsUs7TTEq pUHEZxpJFjEaq7yaoShNNULdR0aAmtPb/KmU5mrfQDlAKJAYCK6+H9J9FWQXpIGn3h4x V5m0t4xT3F6wx9SuK4n8dikthZj3cqkDb6LJWogD/TUx/o+4ipiDBH6X3zz+3AVJJ+sR 0ADe9qczGxZeac8QP3xeMZox5eETxIz3C7s3DeLKKEjwtiruSe6OBjXWV3FPceH3MX0M FNEA== X-Forwarded-Encrypted: i=1; AJvYcCWCtKhypxV4jctZcjYlxRnGGJ5ZwA5NQ+0V0LPCmy5hHQcX/Rjd9Du/ERvoWOWD2tomRSZPTQgFNy01yA==@lists.infradead.org X-Gm-Message-State: AOJu0Yymjyk5pLu7KwUJx8y+LklfWRBjVAMSRlBpQwG1c8zuQX2PoVEv jp3lt8XW+scgKD4qNFFcLvtpdT1lBxrdaNuAHAl6tE/A/rgpz69wKq+h9wmz4zClKenmYoiZEF6 img== X-Google-Smtp-Source: AGHT+IH031wjBK6zBugJiyQwbgM1S3WkZumhuZG0/47H7WWzE24O4tbIqX0eqBdD0TzKWzHgCchY3qP4ics= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:9a94:b0:2e2:da81:40c1 with SMTP id 98e67ed59e1d1-2e2f09f2280mr107a91.1.1728584720854; Thu, 10 Oct 2024 11:25:20 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:16 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-15-seanjc@google.com> Subject: [PATCH v13 14/85] KVM: Return ERR_SIGPENDING from hva_to_pfn() if GUP returns -EGAIN From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112523_867271_DD0F460A X-CRM114-Status: UNSURE ( 9.92 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Treat an -EAGAIN return from GUP the same as -EINTR and immediately report to the caller that a signal is pending. GUP only returns -EAGAIN if the _initial_ mmap_read_lock_killable() fails, which in turn onnly fails if a signal is pending Note, rwsem_down_read_slowpath() actually returns -EINTR, so GUP is really just making life harder than it needs to be. And the call to mmap_read_lock_killable() in the retry path returns its -errno verbatim, i.e. GUP (and thus KVM) is already handling locking failure this way, but only some of the time. Suggested-by: Paolo Bonzini Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 17acc75990a5..ebba5d22db2d 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2946,7 +2946,7 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, writable, &pfn); if (npages == 1) return pfn; - if (npages == -EINTR) + if (npages == -EINTR || npages == -EAGAIN) return KVM_PFN_ERR_SIGPENDING; mmap_read_lock(current->mm); From patchwork Thu Oct 10 18:23:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831012 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9310BD24442 for ; Thu, 10 Oct 2024 19:46:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=SMz3rohP/rD1541BBuHPF24mEDru8OxMRLzuanxS4NY=; b=La6EvMWlI4dXKB 4j8ZSgSJQuNlyoW9QEhzUp/r/fMyEtnI5Yi3f+i5XoZThyzC9taVdrriUF3niraS9/oqczXNEOBVn 0Q1UD7aaaG2hdm95fVnVB+lM1yHVs+39oiem2vthDCAUZrtZdmwhSiAqIQAVi+9pAlIpovzoKoll/ sf1jP2buKJsrKOh7W6vd8v48uHm6j51aGz72FX2O5Vya4SgJ7UeeoGwB6kcnm6tSzYkrG3uMgYEOf ZOlcS7QMQVvFhFXioeVG+OqQibrGSgMIb8EJqb0Mq35tBAuONMw5S3a/X/GCk1eMYl1XI+ffrauFf iK1RByfumMMV2WIrjNeg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syz7I-0000000EAu0-2t72; Thu, 10 Oct 2024 19:46:32 +0000 Received: from mail-yb1-f201.google.com ([209.85.219.201]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqo-0000000Dq0T-0g0y for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:27 +0000 Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e28fc60660dso1617033276.0 for ; Thu, 10 Oct 2024 11:25:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584724; x=1729189524; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=86tNdlyX8HXR/WHFYCj/131p34DPDBwWgQrhOuSjETM=; b=N4gnYeeGDWzC78e3rSacwyrZQ1zS2o0yOL0hz9MvZ+3DIzi+69xBA6zHHWOtGMJ2o4 +J6iT1n1ARNhvB5RJoFHzuv+22xh20DwcLqiRfU5Zh/1hVlsG256CSfzaJoTeA5X+f+O UPmmk1Ezb4cnGxfvJW/RfYmICQgZLA1KjSgMaAfaIwOXS+Pv6DWInlDP9pObpC3C/KlX qpCk6StQ5PCtNMk2Jmk2O/6M9v2d2dl4d70qhqpBoXa+gnO5zGGVilZrXdQ2IvvB5vYN MmC7jNDHog6vNvFcJPnquCCHzR84iug+bqTRoJkTuXfv6KQUKKv6mk+mlcXTwCkqfzBN kuvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584724; x=1729189524; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=86tNdlyX8HXR/WHFYCj/131p34DPDBwWgQrhOuSjETM=; b=K+CePWNZ3jtZx3wq0evWheehaCLcFh/4dVMJGNLujbFrXSys7Lpq5TeIyl77WK+0Zg MmB13yhv+ydA7YZLW6gNdFspVf044QF+YfTWKfprCgvFX7DjR/Qkd/2gOXMtqCSUD8DW xmnQGyKu7+biUQK/H0uMHMiv0iZZqgr8B0XLcybDwlTarG8jVZZsJmAzeh5+tGlJmlpu rwdp1JAumqFhO5VFreSv1JmtLu+4MxBvWzRPAZ/eIMUo4Y/jACdYppQc6CVlkq3Lcv+N TjodAUfoDN469cm4zxQ16y1oTg7eQSAve72pjkaVASq+XleYqwedEGKjQtw7+fc3lBRf KgCA== X-Forwarded-Encrypted: i=1; AJvYcCWZlPx3do3GzI8aLQGEgMY1yJAK4jz6Hy0sh5S5yPctQnwrjv45o1G2XgkLKGfTDUiFL+Idarv2wK6nhg==@lists.infradead.org X-Gm-Message-State: AOJu0YxMmQVz4WJtCz2w/Qa5r5oEtVjybsFMJAh4dd5ZkgcInnqwjZcP 2OYcltwLoq8zFncewnwOS/5oDNiuBnx7Zitp0Ythmgn5qbOshtmwATSJSd0MQC3H0bfw4xazBCX 5XA== X-Google-Smtp-Source: AGHT+IGf82b/hxj2IWFe54/TAnEDS4XyhakmYGUWxryxrCLTzb83+cUU5J1DZhSXbLBNZQvipdjBbhmVkR8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:8e09:0:b0:e28:ef25:5f13 with SMTP id 3f1490d57ef6-e28fe0e7352mr200743276.0.1728584723790; Thu, 10 Oct 2024 11:25:23 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:17 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-16-seanjc@google.com> Subject: [PATCH v13 15/85] KVM: Drop extra GUP (via check_user_page_hwpoison()) to detect poisoned page From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112526_316810_53522E39 X-CRM114-Status: GOOD ( 10.98 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Remove check_user_page_hwpoison() as it's effectively dead code. Prior to commit 234b239bea39 ("kvm: Faults which trigger IO release the mmap_sem"), hva_to_pfn_slow() wasn't actually a slow path in all cases, i.e. would do get_user_pages_fast() without ever doing slow GUP with FOLL_HWPOISON. Now that hva_to_pfn_slow() is a straight shot to get_user_pages_unlocked(), and unconditionally passes FOLL_HWPOISON, it is impossible for hva_to_pfn() to get an -errno that needs to be morphed to -EHWPOISON. There are essentially four cases in KVM: - npages == 0, then FOLL_NOWAIT, a.k.a. @async, must be true, and thus check_user_page_hwpoison() will not be called - npages == 1 || npages == -EHWPOISON, all good - npages == -EINTR || npages == -EAGAIN, bail early, all good - everything else, including -EFAULT, can go down the vma_lookup() path, as npages < 0 means KVM went through hva_to_pfn_slow() which passes FOLL_HWPOISON Suggested-by: Paolo Bonzini Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 17 ++--------------- 1 file changed, 2 insertions(+), 15 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index ebba5d22db2d..87f81e74cbc0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2746,14 +2746,6 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *w return gfn_to_hva_memslot_prot(slot, gfn, writable); } -static inline int check_user_page_hwpoison(unsigned long addr) -{ - int rc, flags = FOLL_HWPOISON | FOLL_WRITE; - - rc = get_user_pages(addr, 1, flags, NULL); - return rc == -EHWPOISON; -} - /* * The fast path to get the writable pfn which will be stored in @pfn, * true indicates success, otherwise false is returned. @@ -2948,14 +2940,10 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, return pfn; if (npages == -EINTR || npages == -EAGAIN) return KVM_PFN_ERR_SIGPENDING; + if (npages == -EHWPOISON) + return KVM_PFN_ERR_HWPOISON; mmap_read_lock(current->mm); - if (npages == -EHWPOISON || - (!async && check_user_page_hwpoison(addr))) { - pfn = KVM_PFN_ERR_HWPOISON; - goto exit; - } - retry: vma = vma_lookup(current->mm, addr); @@ -2972,7 +2960,6 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, *async = true; pfn = KVM_PFN_ERR_FAULT; } -exit: mmap_read_unlock(current->mm); return pfn; } From patchwork Thu Oct 10 18:23:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831013 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 94A8DD24442 for ; Thu, 10 Oct 2024 19:46:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Zsfs9266/uXM28/lnxLep26s2ejtcKj6HRKBjCF7jKY=; b=un95AbGvSTUClV lkY7CeZB2qClay866A/6y4W9o+pTJwN1Gm083OO/4Vl0GJ7G48kmpG6jonTrI/AUYYEIeSYdsTPfh PtJ5Lq8gkSlkhRgzPbjdzw36H3LnC/yPaTCDMguRbX5d2P7/UY7c5DlqQtucTSNNWagmfO0kWO5/b KMQtLOp+LS0vn5NMWcTdxAGhX6sRVqZiiKmIfqOQcAEBQSeoq97HfRPaa+PV28Y2kuHwzeHaAuFMU ZdQjm2GDBuLzu7rbVc6FpA7gRe3GHZELDOrVOz+4fMAxqYuaO9YZ5lspjv7f8LQBYEHDodUCCkSb2 xBDdw8xspxlCdp1x//Og==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syz7N-0000000EAxl-0OOU; Thu, 10 Oct 2024 19:46:37 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqp-0000000Dq27-1nas for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:30 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-7190c5e73cdso1782837b3a.0 for ; Thu, 10 Oct 2024 11:25:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584726; x=1729189526; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=H+RZpPtsk3Pe/RqqhWN1oE7ZGzOesPmt7slwmkU09yw=; b=Qjec5b1CwJ9qFJlW4vKnn33w59PK4W9zPFcydX7ZQSuN5EmpIcv79lHfTUVVhozZOs xhYIm58mnWXsUk2OpErOSahqb/ggw6yIcx6eWAgTsaOV+112SciknFzibBcZVB3lSkD6 UfnSN8hm21A7i4UvrvshfZ4Rjp4g1LQuKUmwfXiqDVoacgirXDTYJ4/5bXcw7T47czT0 5vNPZCH3B6swmxQhV1DEi8N3Qn8EaFYT10ZKoiyEkhOaTeWlFf2ocOBVtL+NORRkUTzz sXkQaXKAtNNNW/wNu8MGtO4ATj8rSFarU89qqAWxDNcP+QjhDL92o7YFs4Kg/sl02I7o oJWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584726; x=1729189526; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=H+RZpPtsk3Pe/RqqhWN1oE7ZGzOesPmt7slwmkU09yw=; b=v+QbVz/m+UoITsyFdtkfZoIeVy2pVgtUXCUTO6+9Q0+iox5Nu2HNrYzgFSIM6MAb0w Lk3tOWNWwLC8oevRuf7kLx0QlbI4iHmKqZKoDR7DIrugS1SpcOYf+FOPy0UiaYzGoCpn Fz8JfCS/8FlZTwzVULWfp3exl9V5elQI0LxKAwR+VijYFInNNb+o8Chlt8f2dmA7F2Q8 lQfn/LD96WtPu/evHJ9pyWGrUAb+qPabS52NsNt9KW5ggWi4wSw7qVPf2M8riJ8DykLQ UbqL4+rs9oizd0GRpuFNLDV8rSGW6lV9IuwSnAtlhgXYFCa9Nfpl4qExzHgP1vu1FkHP VWBA== X-Forwarded-Encrypted: i=1; AJvYcCVHcE2zBzhHoN0IUvPgu7S7gF5LTGVMxwXF2znr5kqs95Sgm7kwzNoobPJmTH3pMIHOJxfd3tTnqS3XcA==@lists.infradead.org X-Gm-Message-State: AOJu0Yzj8OpZ4GMReBokO7qNVoAuH31pkMb5MxFaMLXZEuTepv8oJ/PT BKsF5MawIY34lq+HiYrvYJ5AGlWpbrH+5lQt3jeDcwMt6m1xgX4reUrRyzSDcunyEFXkZ4jPTFF rcg== X-Google-Smtp-Source: AGHT+IEgY3dn5eo0Xr6pp3rAg0uVd1WuwbDag/Dk23UVvT7MMuxyKVAtWc2T42TlMp1H8D9GJ1Oltmo921o= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:8c17:b0:71d:f452:ee99 with SMTP id d2e1a72fcca58-71e1dbcac44mr6866b3a.3.1728584725637; Thu, 10 Oct 2024 11:25:25 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:18 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-17-seanjc@google.com> Subject: [PATCH v13 16/85] KVM: Replace "async" pointer in gfn=>pfn with "no_wait" and error code From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112527_542457_FAF187E1 X-CRM114-Status: GOOD ( 20.72 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: David Stevens Add a pfn error code to communicate that hva_to_pfn() failed because I/O was needed and disallowed, and convert @async to a constant @no_wait boolean. This will allow eliminating the @no_wait param by having callers pass in FOLL_NOWAIT along with other FOLL_* flags. Tested-by: Alex Bennée Signed-off-by: David Stevens Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 18 +++++++++++------- include/linux/kvm_host.h | 3 ++- virt/kvm/kvm_main.c | 27 ++++++++++++++------------- virt/kvm/kvm_mm.h | 2 +- virt/kvm/pfncache.c | 4 ++-- 5 files changed, 30 insertions(+), 24 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 0e235f276ee5..fa8f3fb7c14b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4374,17 +4374,21 @@ static int kvm_faultin_pfn_private(struct kvm_vcpu *vcpu, static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { - bool async; - if (fault->is_private) return kvm_faultin_pfn_private(vcpu, fault); - async = false; - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, &async, + fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, fault->write, &fault->map_writable, &fault->hva); - if (!async) - return RET_PF_CONTINUE; /* *pfn has correct page already */ + + /* + * If resolving the page failed because I/O is needed to fault-in the + * page, then either set up an asynchronous #PF to do the I/O, or if + * doing an async #PF isn't possible, retry with I/O allowed. All + * other failures are terminal, i.e. retrying won't help. + */ + if (fault->pfn != KVM_PFN_ERR_NEEDS_IO) + return RET_PF_CONTINUE; if (!fault->prefetch && kvm_can_do_async_pf(vcpu)) { trace_kvm_try_async_get_page(fault->addr, fault->gfn); @@ -4402,7 +4406,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault * to wait for IO. Note, gup always bails if it is unable to quickly * get a page and a fatal signal, i.e. SIGKILL, is pending. */ - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, NULL, + fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, true, fault->write, &fault->map_writable, &fault->hva); return RET_PF_CONTINUE; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 2faafc7a56ae..071a0a1f1c60 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -97,6 +97,7 @@ #define KVM_PFN_ERR_HWPOISON (KVM_PFN_ERR_MASK + 1) #define KVM_PFN_ERR_RO_FAULT (KVM_PFN_ERR_MASK + 2) #define KVM_PFN_ERR_SIGPENDING (KVM_PFN_ERR_MASK + 3) +#define KVM_PFN_ERR_NEEDS_IO (KVM_PFN_ERR_MASK + 4) /* * error pfns indicate that the gfn is in slot but faild to @@ -1233,7 +1234,7 @@ kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn); kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool interruptible, bool *async, + bool interruptible, bool no_wait, bool write_fault, bool *writable, hva_t *hva); void kvm_release_pfn_clean(kvm_pfn_t pfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 87f81e74cbc0..dd5839abef6c 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2778,7 +2778,7 @@ static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, * The slow path to get the pfn of the specified host virtual address, * 1 indicates success, -errno is returned if error is detected. */ -static int hva_to_pfn_slow(unsigned long addr, bool *async, bool write_fault, +static int hva_to_pfn_slow(unsigned long addr, bool no_wait, bool write_fault, bool interruptible, bool *writable, kvm_pfn_t *pfn) { /* @@ -2801,7 +2801,7 @@ static int hva_to_pfn_slow(unsigned long addr, bool *async, bool write_fault, if (write_fault) flags |= FOLL_WRITE; - if (async) + if (no_wait) flags |= FOLL_NOWAIT; if (interruptible) flags |= FOLL_INTERRUPTIBLE; @@ -2912,8 +2912,8 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, * Pin guest page in memory and return its pfn. * @addr: host virtual address which maps memory to the guest * @interruptible: whether the process can be interrupted by non-fatal signals - * @async: whether this function need to wait IO complete if the - * host page is not in the memory + * @no_wait: whether or not this function need to wait IO complete if the + * host page is not in the memory * @write_fault: whether we should get a writable host page * @writable: whether it allows to map a writable host page for !@write_fault * @@ -2922,7 +2922,7 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, * 2): @write_fault = false && @writable, @writable will tell the caller * whether the mapping is writable. */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, +kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, bool write_fault, bool *writable) { struct vm_area_struct *vma; @@ -2934,7 +2934,7 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, if (hva_to_pfn_fast(addr, write_fault, writable, &pfn)) return pfn; - npages = hva_to_pfn_slow(addr, async, write_fault, interruptible, + npages = hva_to_pfn_slow(addr, no_wait, write_fault, interruptible, writable, &pfn); if (npages == 1) return pfn; @@ -2956,16 +2956,17 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, if (r < 0) pfn = KVM_PFN_ERR_FAULT; } else { - if (async && vma_is_valid(vma, write_fault)) - *async = true; - pfn = KVM_PFN_ERR_FAULT; + if (no_wait && vma_is_valid(vma, write_fault)) + pfn = KVM_PFN_ERR_NEEDS_IO; + else + pfn = KVM_PFN_ERR_FAULT; } mmap_read_unlock(current->mm); return pfn; } kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool interruptible, bool *async, + bool interruptible, bool no_wait, bool write_fault, bool *writable, hva_t *hva) { unsigned long addr = __gfn_to_hva_many(slot, gfn, NULL, write_fault); @@ -2987,21 +2988,21 @@ kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, writable = NULL; } - return hva_to_pfn(addr, interruptible, async, write_fault, writable); + return hva_to_pfn(addr, interruptible, no_wait, write_fault, writable); } EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable) { - return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, NULL, + return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, false, write_fault, writable, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) { - return __gfn_to_pfn_memslot(slot, gfn, false, NULL, true, NULL, NULL); + return __gfn_to_pfn_memslot(slot, gfn, false, false, true, NULL, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index a3fa86f60d6c..51f3fee4ca3f 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -20,7 +20,7 @@ #define KVM_MMU_UNLOCK(kvm) spin_unlock(&(kvm)->mmu_lock) #endif /* KVM_HAVE_MMU_RWLOCK */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, +kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, bool write_fault, bool *writable); #ifdef CONFIG_HAVE_KVM_PFNCACHE diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index 58c706a610e5..32dc61f48c81 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -197,8 +197,8 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) cond_resched(); } - /* We always request a writeable mapping */ - new_pfn = hva_to_pfn(gpc->uhva, false, NULL, true, NULL); + /* We always request a writable mapping */ + new_pfn = hva_to_pfn(gpc->uhva, false, false, true, NULL); if (is_error_noslot_pfn(new_pfn)) goto out_error; From patchwork Thu Oct 10 18:23:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830853 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 91945CFC5F2 for ; Thu, 10 Oct 2024 18:44:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5houM4/91BqBg/I9WhJjEW5YAABULlgdtpi5KpWhjuU=; b=VM3AVo4Z3kTPRh PC+hewxmRqrEwirf77WC9LZ6P4R+yISZtr++P7vj8qDX2e98AhDKmZmU1nO18mfA7jWYypSC4pHt4 2Sv/QTz8WhmxEyOg9nosAl7vTWYs5a1wbzBxXOgNMsoSAlIUrFx9azL8ivAayJCkepqAUQcMTTJak fHpRe/3dn7yIoNMFkVCLwHbPVU/rEOiEq9C9WhzciwMZ2/jq44C0H3U7lJFt+WBhnWI/aBpxTQccm SQewORgjTZUbB4VPdIj+B6wOoKB8I4pIRyrfBPpbxraAA1J5RhP3VpOf8nBAXFN4Iw53HVdoxVhGR tnU5F33PLGyiIoscWAuQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syy9S-0000000Dyc0-1wDW; Thu, 10 Oct 2024 18:44:42 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqs-0000000Dq3k-0957 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:32 +0000 Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-20bb491189aso16352695ad.3 for ; Thu, 10 Oct 2024 11:25:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584728; x=1729189528; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=5N9i4qcLR1M4w0icoJwy+0qPwifBeFycVmWqg2S4YD0=; b=uTLUXAD9ftwOAWUh4FjktczO89FGu52aC2rgPp245/lEj1s72YqNR/aIDgmovpnjTw HC3jnAk/7L5Fa0FkPdQPqU2qvNP/FCS9jm5l9QtDKCjIYexrNMlJ6rHK9Qh90mNdQ0gz pQWmLtmO5UuB7V2fnKZL8l7u3utIoX9wgdz/yAcJ39cWN1i86vKz12XVoNvrTXzGldva +W7MUjeaG2WCz28YEVABAqFpH18qx4ouR3U+t7I7QQtA6vEwM/KTV5fdtXtHIk2RZ3nO 0lFQqywKjsqrELmkNAOSZ4dPDOMCodBq3cSpkUx60Gsb4fkH4CB5l/Bi/Eiawj8f3uVP IPLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584728; x=1729189528; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=5N9i4qcLR1M4w0icoJwy+0qPwifBeFycVmWqg2S4YD0=; b=qOIOZRJWYsW4DPTLRgoiUN9GopYdbY9C78gQq6KDs350utzXFg/fLHZbP0Fw4rixTF Ss4auiG5IjBJ0BHgX9PHbRFyQQ25FjjgoiMtcAUCUNahE51GkyxvT+79E4apwFw83j8P 95Ry9DVRosiqOtkmoHnw8Ta7+wsLViGASZyzZdDEibGQYMb+O5ZQAgSX4uuBSbie0I2Q kS93xoYNMxPu60MMUkHOaqlFwXZU6ZcZyKZscvaBA2N7Vj6wSYvQkpsAgk3X27iSd5JY E0vPZ8RlA51niHtE+q6QdL8hPE67ardZNo3/J7nuLO4dti5p5sCzEB16aWU7K8sVie5w oZ4w== X-Forwarded-Encrypted: i=1; AJvYcCV01E42VtifZK6H1ttRp/lBXQEidqVqNUmW7+cEpZGzhozJbAMdE0pOAphfnsYwiRuyseIKR0QTZzGmpg==@lists.infradead.org X-Gm-Message-State: AOJu0YxSXUeetlVfBcwUHZ3AMD5dk19ZZVdYlq9Snp3gj5AiUuK0wZZ2 imQGSAcmjB4/GIQRvuDdWQbeMt45Ed8L7j0RyYCASghyMamLq8KFqsn5O6xvceG0y3nXXgwqVCG uoA== X-Google-Smtp-Source: AGHT+IHx3wOCIkhIacFyS0KkUCfogOtyOnmQIp9NOdA3rn83blMwYV/7exbhoH/uHIfky6alS3vCfOjmfpM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:d502:b0:20b:7ece:3225 with SMTP id d9443c01a7336-20c6358fdd3mr1169885ad.0.1728584727745; Thu, 10 Oct 2024 11:25:27 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:19 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-18-seanjc@google.com> Subject: [PATCH v13 17/85] KVM: x86/mmu: Drop kvm_page_fault.hva, i.e. don't track intermediate hva From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112530_201107_4C97ADFC X-CRM114-Status: GOOD ( 11.32 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Remove kvm_page_fault.hva as it is never read, only written. This will allow removing the @hva param from __gfn_to_pfn_memslot(). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 5 ++--- arch/x86/kvm/mmu/mmu_internal.h | 2 -- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index fa8f3fb7c14b..c67228b46bd5 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3294,7 +3294,6 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *vcpu, fault->slot = NULL; fault->pfn = KVM_PFN_NOSLOT; fault->map_writable = false; - fault->hva = KVM_HVA_ERR_BAD; /* * If MMIO caching is disabled, emulate immediately without @@ -4379,7 +4378,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, fault->write, &fault->map_writable, - &fault->hva); + NULL); /* * If resolving the page failed because I/O is needed to fault-in the @@ -4408,7 +4407,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault */ fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, true, fault->write, &fault->map_writable, - &fault->hva); + NULL); return RET_PF_CONTINUE; } diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index 4da83544c4e1..633aedec3c2e 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -238,7 +238,6 @@ struct kvm_page_fault { /* Outputs of kvm_faultin_pfn. */ unsigned long mmu_seq; kvm_pfn_t pfn; - hva_t hva; bool map_writable; /* @@ -313,7 +312,6 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, .is_private = err & PFERR_PRIVATE_ACCESS, .pfn = KVM_PFN_ERR_FAULT, - .hva = KVM_HVA_ERR_BAD, }; int r; From patchwork Thu Oct 10 18:23:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830860 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 80F42CFC5F2 for ; Thu, 10 Oct 2024 18:46:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZZISHIOh7PKZSM010FEmz6WQlRhjPbD0QuQpOZQKtpw=; b=lChnRJ6BWFvMq0 SURsxtzZxEDJkGwbH8AYKM14tgNR20gaSIYLIi/hHX7z65JsyjdGrcTqP9OswWL7VXIWqXf12P+az xfjm6q/dLqsxmUZQ537iTwZYRYI2Z/vUzt89tI1cYKlgXQe7x2qtTWZ9/rChBaMCWB7+QxAzxy8qF tgp/VzBposOhCMVE8Igj5n+gtRufoHfg1+JPsnwgOqyzSRPlWqLSer1EPy19jYyBbRELqh6dFhi9F YYBB0RqNcOow+nhmrt+DttoFPIS+xnLrxx53Im0tDCcpfNoPXG1v4LnukLo85xvu1ne7Uht1qeDfh MGfOzavaqYqjPk2KtN2A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyB6-0000000Dyxs-3kTW; Thu, 10 Oct 2024 18:46:24 +0000 Received: from mail-pj1-x104a.google.com ([2607:f8b0:4864:20::104a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqu-0000000Dq6T-2SNy for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:34 +0000 Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2e2d3d5fb4eso896236a91.2 for ; Thu, 10 Oct 2024 11:25:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584731; x=1729189531; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=KxFe7qyo4eHIzhTxczhq6OF5aBNVD8CbU/vq6HHGwtQ=; b=awdZis4GuboE2s97liIxH2HBWuUUAZFAC1GR5LT8PbnEmXHC9BSez0m6ZP3mLwxKXM lQ1MevXxzALb1FlxY1y77fDO1z/yeM3kTyp/oML38Xu6q7A/SmpXpmWj9OE6Ic2oUvpP G6PcEUHqZ9s82F1e4DgelmyGmWvUWCcrjUMNuBi8hYHZqkdUNNzyDmqRS5imgnmSEsOe Hwlj5C1c37+lD1ZhDMZabp8YATzJcOtF5IR98HGQw71sfkFR7KqlyyliLqBRHmcynxCc mZz4ZlNs/pnaJQjzxt1mqttaEDYAWQ4/MDsqxIKf+BKKwPrQIijYa3xjhuw1Sz7YKV2J 6Ajg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584731; x=1729189531; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=KxFe7qyo4eHIzhTxczhq6OF5aBNVD8CbU/vq6HHGwtQ=; b=ZUr0sc5uah7Urg889IofNDiPZ8dHUaESDnpg6j6F8re7BBRgz79rozFwvKTujOETA6 buiRnqZrvKWM8Tv3kplqDCLajKbeEx088f5nlJCMSSS++rL5k8LUcDZRqUGditu+QVFL 601Rbj8oLiFOM88NIgjxl3dtoCCksDWFVdkBB1xpfmDauJZnFLlWMGckiIP3IGX6P2+x 9+Ki0wYQUB4C5s1QW0Ijdxl5iKunwgTZH2bajS/WAer7jxPA1xwLvwUzVQBoeTu0BLwU 4VU58uhQu8rSqeq+HiByrxAWtd/aNi1HmP17NCUfzKHiJYoXDl2YUBIoozPDD57QUChk YJ9w== X-Forwarded-Encrypted: i=1; AJvYcCWRvfz/1Gw08LMqH4yZllVqYJKn4YiRARPjUZVttKkmjm3wF+Ib2ktKf3YnteyF3xFd5wAtQgbp8TOisA==@lists.infradead.org X-Gm-Message-State: AOJu0YyesBbOpOAAc3IWgJv3y9GV11pdG3qtxLgaMd/WH0fDOdPWgZOJ 381wptrjErWEDfIV7E5NBCXgRndQOcm+loYdxFNz1onSomHbZQBFKBOafrmUWiqVIoO1h0AMx71 9qQ== X-Google-Smtp-Source: AGHT+IGFsZ8G+8oqHMXYnd6l/b9oSo3/s957bEBlaeuh+YsQIWv/Tzia2RQhXQw4+rIMPn/3epGKWmgCn2Q= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:5107:b0:2e2:ca3e:10fe with SMTP id 98e67ed59e1d1-2e2f0f88a54mr20a91.8.1728584729847; Thu, 10 Oct 2024 11:25:29 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:20 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-19-seanjc@google.com> Subject: [PATCH v13 18/85] KVM: Drop unused "hva" pointer from __gfn_to_pfn_memslot() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112532_804653_6AA7290C X-CRM114-Status: GOOD ( 11.64 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Drop @hva from __gfn_to_pfn_memslot() now that all callers pass NULL. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/arm64/kvm/mmu.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_hv.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_radix.c | 2 +- arch/x86/kvm/mmu/mmu.c | 6 ++---- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 9 +++------ 6 files changed, 9 insertions(+), 14 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index a6e62cc9015c..dd221587fcca 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1570,7 +1570,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, mmap_read_unlock(current->mm); pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - write_fault, &writable, NULL); + write_fault, &writable); if (pfn == KVM_PFN_ERR_HWPOISON) { kvm_send_hwpoison_signal(hva, vma_shift); return 0; diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c index 8cd02ca4b1b8..2f1d58984b41 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c @@ -614,7 +614,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_vcpu *vcpu, } else { /* Call KVM generic code to do the slow-path check */ pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - writing, &write_ok, NULL); + writing, &write_ok); if (is_error_noslot_pfn(pfn)) return -EFAULT; page = NULL; diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index 26a969e935e3..8304b6f8fe45 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -853,7 +853,7 @@ int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, /* Call KVM generic code to do the slow-path check */ pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - writing, upgrade_p, NULL); + writing, upgrade_p); if (is_error_noslot_pfn(pfn)) return -EFAULT; page = NULL; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index c67228b46bd5..28f2b842d6ca 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4377,8 +4377,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault return kvm_faultin_pfn_private(vcpu, fault); fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, - fault->write, &fault->map_writable, - NULL); + fault->write, &fault->map_writable); /* * If resolving the page failed because I/O is needed to fault-in the @@ -4406,8 +4405,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault * get a page and a fatal signal, i.e. SIGKILL, is pending. */ fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, true, - fault->write, &fault->map_writable, - NULL); + fault->write, &fault->map_writable); return RET_PF_CONTINUE; } diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 071a0a1f1c60..cbc7b9c04c14 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1235,7 +1235,7 @@ kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn); kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, bool interruptible, bool no_wait, - bool write_fault, bool *writable, hva_t *hva); + bool write_fault, bool *writable); void kvm_release_pfn_clean(kvm_pfn_t pfn); void kvm_release_pfn_dirty(kvm_pfn_t pfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index dd5839abef6c..10071f31b2ca 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2967,13 +2967,10 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, bool interruptible, bool no_wait, - bool write_fault, bool *writable, hva_t *hva) + bool write_fault, bool *writable) { unsigned long addr = __gfn_to_hva_many(slot, gfn, NULL, write_fault); - if (hva) - *hva = addr; - if (kvm_is_error_hva(addr)) { if (writable) *writable = false; @@ -2996,13 +2993,13 @@ kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable) { return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, false, - write_fault, writable, NULL); + write_fault, writable); } EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) { - return __gfn_to_pfn_memslot(slot, gfn, false, false, true, NULL, NULL); + return __gfn_to_pfn_memslot(slot, gfn, false, false, true, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); From patchwork Thu Oct 10 18:23:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831139 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 488BFD24452 for ; Thu, 10 Oct 2024 21:01:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=lbmXafF2uVHKNbsalganIo7NPvqrdSgpb/bcsG3spRc=; b=qkljlBf+UFubOB gncYMRubLdMNCPGKwU5g7Db3LG1yNHZxs+LoNJNXy0uKdIgdS0JZ8Osycwq9hN3bv4XGVjzOcigB9 1MjGGE7G0XaobdCjYZH/mRLWPYeUb6wll+1Ic8Ne/Z96UFwJOEIz6epWVzaRiaXv4BR6VlQN4jYRR XzvBeoC5E08lAD1LFSrdsgGb+Sf0CNv5ZF6nqquLCJ8INnxPY25JXV/TfKHrbIm/7nmKMvW8RFxNp PMA3LzKcirNkCYCgcospoOdUvnWbUhAUTzH/RgZ3+xugqR3dQKMYFQA71q6QBI6w3hHfw1YCRL9hi qQbBjcfJd7WwnUDBp4yw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0Hk-0000000ELiF-2FYo; Thu, 10 Oct 2024 21:01:24 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqw-0000000Dq7z-2Vod for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:37 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e31d9c8efcso25656207b3.0 for ; Thu, 10 Oct 2024 11:25:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584733; x=1729189533; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=GUnoK8hes+aZNWzzso7nvh51KinEVhA6U4mspQ9USyg=; b=QSA5Sg21GOvgjD1Ji6QRzFZ+R3PP2MPakGWPOI/tPT+1CrRvbx7U8z/Sd1b+fnqA1Q A0AGN19YkH/fcahh2w+MAOQ5qRjYuS47eqno2btw34H/K2nX8nOBoKOz/cGEAqBE9smh CwybwU7CMSzF+dGRGdbQrsgrn0xWROTjrGsCh1QntjZ5OJc9TYkDEDJOpd3AJrIM5x3K 9ulB5CWefMEweMtJTo88ludJG4EwG25DjsKKZ7LdZhrPq8zrlUiAxCon5NbZZrBUmVrS eWGebW8A5hj0X6W7yGnBGdOB59w2+X6x/xS7t1zsL8gPrO4kWpWjcDCH59ZBXYsVwpEt vxyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584733; x=1729189533; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=GUnoK8hes+aZNWzzso7nvh51KinEVhA6U4mspQ9USyg=; b=mHnURumJq1dKSDAHkV21HV26aXMqD4LVRQmsINiRgWhemzupVZe0Nv48xfsbdkB3G8 ZDmdBAzzeITcsfVHFCQI1gr9q84Xyb/e4zyuMnGB+wfmPp4CIXylc+68sIT7o6QnmXdq ibxyf9yyf4EFdKOu3WmF9C3gj58kim8M5QPvWHQCnEaZjPIWqt9eCO/wHmg86rWVSQPB 6Ha9NfQTBZBZgFjJWIA9x4Oej4TeUFAblfdM+0ludwzJ+aE6D5D3OK47V1xO6g/wUJ1s 00/coVHa989Z01TCkCPLZ8OjpKMBIWyVMNc+GWpTOLTKXQijJKPZatPn/oPll4vyUTnw C7FA== X-Forwarded-Encrypted: i=1; AJvYcCUojJFi2DpI44DnW2/qR/6evOdD0zUc6eOuHOQ4N71jymqQQT+Hvdr23MuEe2IqwP9nSWGHCMlHZQ+ETw==@lists.infradead.org X-Gm-Message-State: AOJu0YwCL8xwXvayzSPW+9IpyO+1Z6NfiHNaIPFzltlxPP0YVxqThEo7 wTOLDMoz4/xnKk0Xa/vX9bx9BO/ctAEEW/p9lBlQl5EgMPBsiJT9YmLBuiVYzGqNUgbn70+j5NI i9Q== X-Google-Smtp-Source: AGHT+IHJGx3vYA/ciscOnZoyMGBDqN0A5fNCXYXCqQ4jImN8v4VZ+ud/0qiiFEbLMXTR4U3Rg3+GOt3HIs4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2c01:b0:6db:d257:b98 with SMTP id 00721157ae682-6e32215253bmr563917b3.3.1728584733164; Thu, 10 Oct 2024 11:25:33 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:21 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-20-seanjc@google.com> Subject: [PATCH v13 19/85] KVM: Introduce kvm_follow_pfn() to eventually replace "gfn_to_pfn" APIs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112534_857739_59C3DE7F X-CRM114-Status: GOOD ( 24.98 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: David Stevens Introduce kvm_follow_pfn() to eventually supplant the various "gfn_to_pfn" APIs, albeit by adding more wrappers. The primary motivation of the new helper is to pass a structure instead of an ever changing set of parameters, e.g. so that tweaking the behavior, inputs, and/or outputs of the "to pfn" helpers doesn't require churning half of KVM. In the more distant future, the APIs exposed to arch code could also follow suit, e.g. by adding something akin to x86's "struct kvm_page_fault" when faulting in guest memory. But for now, the goal is purely to clean up KVM's "internal" MMU code. As part of the conversion, replace the write_fault, interruptible, and no-wait boolean flags with FOLL_WRITE, FOLL_INTERRUPTIBLE, and FOLL_NOWAIT respectively. Collecting the various FOLL_* flags into a single field will again ease the pain of passing new flags. Tested-by: Alex Bennée Signed-off-by: David Stevens Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 162 +++++++++++++++++++++++--------------------- virt/kvm/kvm_mm.h | 20 +++++- virt/kvm/pfncache.c | 9 ++- 3 files changed, 109 insertions(+), 82 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 10071f31b2ca..52629ac26119 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2750,8 +2750,7 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *w * The fast path to get the writable pfn which will be stored in @pfn, * true indicates success, otherwise false is returned. */ -static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, - bool *writable, kvm_pfn_t *pfn) +static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) { struct page *page[1]; @@ -2760,14 +2759,13 @@ static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, * or the caller allows to map a writable pfn for a read fault * request. */ - if (!(write_fault || writable)) + if (!((kfp->flags & FOLL_WRITE) || kfp->map_writable)) return false; - if (get_user_page_fast_only(addr, FOLL_WRITE, page)) { + if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, page)) { *pfn = page_to_pfn(page[0]); - - if (writable) - *writable = true; + if (kfp->map_writable) + *kfp->map_writable = true; return true; } @@ -2778,8 +2776,7 @@ static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, * The slow path to get the pfn of the specified host virtual address, * 1 indicates success, -errno is returned if error is detected. */ -static int hva_to_pfn_slow(unsigned long addr, bool no_wait, bool write_fault, - bool interruptible, bool *writable, kvm_pfn_t *pfn) +static int hva_to_pfn_slow(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) { /* * When a VCPU accesses a page that is not mapped into the secondary @@ -2792,34 +2789,30 @@ static int hva_to_pfn_slow(unsigned long addr, bool no_wait, bool write_fault, * Note that get_user_page_fast_only() and FOLL_WRITE for now * implicitly honor NUMA hinting faults and don't need this flag. */ - unsigned int flags = FOLL_HWPOISON | FOLL_HONOR_NUMA_FAULT; - struct page *page; + unsigned int flags = FOLL_HWPOISON | FOLL_HONOR_NUMA_FAULT | kfp->flags; + struct page *page, *wpage; int npages; - if (writable) - *writable = write_fault; - - if (write_fault) - flags |= FOLL_WRITE; - if (no_wait) - flags |= FOLL_NOWAIT; - if (interruptible) - flags |= FOLL_INTERRUPTIBLE; - - npages = get_user_pages_unlocked(addr, 1, &page, flags); + npages = get_user_pages_unlocked(kfp->hva, 1, &page, flags); if (npages != 1) return npages; + if (!kfp->map_writable) + goto out; + + if (kfp->flags & FOLL_WRITE) { + *kfp->map_writable = true; + goto out; + } + /* map read fault as writable if possible */ - if (unlikely(!write_fault) && writable) { - struct page *wpage; - - if (get_user_page_fast_only(addr, FOLL_WRITE, &wpage)) { - *writable = true; - put_page(page); - page = wpage; - } + if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &wpage)) { + *kfp->map_writable = true; + put_page(page); + page = wpage; } + +out: *pfn = page_to_pfn(page); return npages; } @@ -2846,10 +2839,10 @@ static int kvm_try_get_pfn(kvm_pfn_t pfn) } static int hva_to_pfn_remapped(struct vm_area_struct *vma, - unsigned long addr, bool write_fault, - bool *writable, kvm_pfn_t *p_pfn) + struct kvm_follow_pfn *kfp, kvm_pfn_t *p_pfn) { - struct follow_pfnmap_args args = { .vma = vma, .address = addr }; + struct follow_pfnmap_args args = { .vma = vma, .address = kfp->hva }; + bool write_fault = kfp->flags & FOLL_WRITE; kvm_pfn_t pfn; int r; @@ -2860,7 +2853,7 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, * not call the fault handler, so do it here. */ bool unlocked = false; - r = fixup_user_fault(current->mm, addr, + r = fixup_user_fault(current->mm, kfp->hva, (write_fault ? FAULT_FLAG_WRITE : 0), &unlocked); if (unlocked) @@ -2878,8 +2871,8 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, goto out; } - if (writable) - *writable = args.writable; + if (kfp->map_writable) + *kfp->map_writable = args.writable; pfn = args.pfn; /* @@ -2908,22 +2901,7 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, return r; } -/* - * Pin guest page in memory and return its pfn. - * @addr: host virtual address which maps memory to the guest - * @interruptible: whether the process can be interrupted by non-fatal signals - * @no_wait: whether or not this function need to wait IO complete if the - * host page is not in the memory - * @write_fault: whether we should get a writable host page - * @writable: whether it allows to map a writable host page for !@write_fault - * - * The function will map a writable host page for these two cases: - * 1): @write_fault = true - * 2): @write_fault = false && @writable, @writable will tell the caller - * whether the mapping is writable. - */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, - bool write_fault, bool *writable) +kvm_pfn_t hva_to_pfn(struct kvm_follow_pfn *kfp) { struct vm_area_struct *vma; kvm_pfn_t pfn; @@ -2931,11 +2909,10 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, might_sleep(); - if (hva_to_pfn_fast(addr, write_fault, writable, &pfn)) + if (hva_to_pfn_fast(kfp, &pfn)) return pfn; - npages = hva_to_pfn_slow(addr, no_wait, write_fault, interruptible, - writable, &pfn); + npages = hva_to_pfn_slow(kfp, &pfn); if (npages == 1) return pfn; if (npages == -EINTR || npages == -EAGAIN) @@ -2945,18 +2922,19 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, mmap_read_lock(current->mm); retry: - vma = vma_lookup(current->mm, addr); + vma = vma_lookup(current->mm, kfp->hva); if (vma == NULL) pfn = KVM_PFN_ERR_FAULT; else if (vma->vm_flags & (VM_IO | VM_PFNMAP)) { - r = hva_to_pfn_remapped(vma, addr, write_fault, writable, &pfn); + r = hva_to_pfn_remapped(vma, kfp, &pfn); if (r == -EAGAIN) goto retry; if (r < 0) pfn = KVM_PFN_ERR_FAULT; } else { - if (no_wait && vma_is_valid(vma, write_fault)) + if ((kfp->flags & FOLL_NOWAIT) && + vma_is_valid(vma, kfp->flags & FOLL_WRITE)) pfn = KVM_PFN_ERR_NEEDS_IO; else pfn = KVM_PFN_ERR_FAULT; @@ -2965,41 +2943,69 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, return pfn; } +static kvm_pfn_t kvm_follow_pfn(struct kvm_follow_pfn *kfp) +{ + kfp->hva = __gfn_to_hva_many(kfp->slot, kfp->gfn, NULL, + kfp->flags & FOLL_WRITE); + + if (kfp->hva == KVM_HVA_ERR_RO_BAD) + return KVM_PFN_ERR_RO_FAULT; + + if (kvm_is_error_hva(kfp->hva)) + return KVM_PFN_NOSLOT; + + if (memslot_is_readonly(kfp->slot) && kfp->map_writable) { + *kfp->map_writable = false; + kfp->map_writable = NULL; + } + + return hva_to_pfn(kfp); +} + kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, bool interruptible, bool no_wait, bool write_fault, bool *writable) { - unsigned long addr = __gfn_to_hva_many(slot, gfn, NULL, write_fault); - - if (kvm_is_error_hva(addr)) { - if (writable) - *writable = false; - - return addr == KVM_HVA_ERR_RO_BAD ? KVM_PFN_ERR_RO_FAULT : - KVM_PFN_NOSLOT; - } - - /* Do not map writable pfn in the readonly memslot. */ - if (writable && memslot_is_readonly(slot)) { - *writable = false; - writable = NULL; - } - - return hva_to_pfn(addr, interruptible, no_wait, write_fault, writable); + struct kvm_follow_pfn kfp = { + .slot = slot, + .gfn = gfn, + .map_writable = writable, + }; + + if (write_fault) + kfp.flags |= FOLL_WRITE; + if (no_wait) + kfp.flags |= FOLL_NOWAIT; + if (interruptible) + kfp.flags |= FOLL_INTERRUPTIBLE; + + return kvm_follow_pfn(&kfp); } EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable) { - return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, false, - write_fault, writable); + struct kvm_follow_pfn kfp = { + .slot = gfn_to_memslot(kvm, gfn), + .gfn = gfn, + .flags = write_fault ? FOLL_WRITE : 0, + .map_writable = writable, + }; + + return kvm_follow_pfn(&kfp); } EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) { - return __gfn_to_pfn_memslot(slot, gfn, false, false, true, NULL); + struct kvm_follow_pfn kfp = { + .slot = slot, + .gfn = gfn, + .flags = FOLL_WRITE, + }; + + return kvm_follow_pfn(&kfp); } EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index 51f3fee4ca3f..d5a215958f06 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -20,8 +20,24 @@ #define KVM_MMU_UNLOCK(kvm) spin_unlock(&(kvm)->mmu_lock) #endif /* KVM_HAVE_MMU_RWLOCK */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, - bool write_fault, bool *writable); + +struct kvm_follow_pfn { + const struct kvm_memory_slot *slot; + const gfn_t gfn; + + unsigned long hva; + + /* FOLL_* flags modifying lookup behavior, e.g. FOLL_WRITE. */ + unsigned int flags; + + /* + * If non-NULL, try to get a writable mapping even for a read fault. + * Set to true if a writable mapping was obtained. + */ + bool *map_writable; +}; + +kvm_pfn_t hva_to_pfn(struct kvm_follow_pfn *kfp); #ifdef CONFIG_HAVE_KVM_PFNCACHE void gfn_to_pfn_cache_invalidate_start(struct kvm *kvm, diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index 32dc61f48c81..067daf9ad6ef 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -159,6 +159,12 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) kvm_pfn_t new_pfn = KVM_PFN_ERR_FAULT; void *new_khva = NULL; unsigned long mmu_seq; + struct kvm_follow_pfn kfp = { + .slot = gpc->memslot, + .gfn = gpa_to_gfn(gpc->gpa), + .flags = FOLL_WRITE, + .hva = gpc->uhva, + }; lockdep_assert_held(&gpc->refresh_lock); @@ -197,8 +203,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) cond_resched(); } - /* We always request a writable mapping */ - new_pfn = hva_to_pfn(gpc->uhva, false, false, true, NULL); + new_pfn = hva_to_pfn(&kfp); if (is_error_noslot_pfn(new_pfn)) goto out_error; From patchwork Thu Oct 10 18:23:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 85540CFC5F4 for ; Thu, 10 Oct 2024 18:49:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=urB7D0YjB3PrS0IhcdOWFrPf08LFU3ilyDP+CEASzzI=; b=1aUJwUEa5iN5oP NFjmPaNvu18f8Aiple2cPcxqPXe/gdPTsoX/RnsNqtdq+dmEr1WsTxEZ4xCtRbwq5eVAvSeV4c+Zf ub+TSMG9E0gIlVq5ZPOIoSuP28bIJC2rcnzaCFs3exJRcwkuJWBdy7vhZ6t+HWCJQi93PrPNabERo UC/FhTDKr/ajCo1inteJ2hfuilTBNK6BtqtJwX2SptselRhJFGlSIQZ6KQwP31x9gxcwWF4qIlhWU 0onFLp2ugsxeLkb/a74++R5meKs3grp/hmZv9qhHAIQhvtgLavzsMrnwcGf/HdkJLI1DrqqiHqqPM aFTK1/uYUcW1HA+lYWBQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyDm-0000000DzU2-0Q0o; Thu, 10 Oct 2024 18:49:10 +0000 Received: from mail-pj1-x104a.google.com ([2607:f8b0:4864:20::104a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqy-0000000Dq9l-2UwZ for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:39 +0000 Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2e2dcea0f03so997754a91.1 for ; Thu, 10 Oct 2024 11:25:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584735; x=1729189535; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=LNPEWcjaz+dEwpcBQF1ZcKP3HAxUc/LzxURpjzN7wyY=; b=cQVMlHY01P5GUGimI1EwUwV1gMXDWO1YhRNQgRRZRp00tcXkR3VxYVS8SFNZdGUHSM +EnHGdT46sR1K/IGTWajJMmoGBn0sNYOYBl9jKHyXHiRccxDMQz2SiMM8mPBIPgc36qO Po18p81ZhZKkcywvYLdtTn92U0KOy4krcXQi8HumWxBuK6fva1EYc44wJf8gFuc/2fd6 KE22Z93ZbqXtSHgYYN/h+5IQDFU3/qaj+6FgQnJ80jSJFdyR2wG6J4Jnw+f04poLAqTP dsiF1M/Sh525nq3aLtvs5dKB47INji3eZlodpB9Gp1tMy1RepfdgGfZF+37O8j4+KXr8 aD+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584735; x=1729189535; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=LNPEWcjaz+dEwpcBQF1ZcKP3HAxUc/LzxURpjzN7wyY=; b=UouhhKm9lBDf2VSyBNvCkzzFd/QdMjeIrjAZx/vnuNvPhwEddeXMJ9Ypxdz7yDAOJy U+DR+gQwUeFmiEXiPsBhGxu5sm+Ld0f+vZc9A/7VyeyKhJggDr8kfKsgHspAjNGjK9ly KosgzuMsEUkZyT/InVCXl2U7+DCJYlSlcBdwaP6bzBFk7WI40JOGLL1FVy9Mp660Mgu7 8mhKQwceOqCfxZ6w76ng8nzAR/oYY+7vpNIPthha/HdTogFsAFn604hUWzsKWFA2mS+1 wn51sS65ubSWsncvkOvEEZ/pxw+aNvaTgV67xzhZEEenAsqwjCM36f296gageThaEnWW wNfw== X-Forwarded-Encrypted: i=1; AJvYcCUe8ZESBNGNDq3hYprAwtttyFLeNhhpyKiFclselHNY8o8jefd7fLgyKPxMj6i33XDaXvslLUGTnILDPw==@lists.infradead.org X-Gm-Message-State: AOJu0YyyMHSyPHePJnWYE5GlhjgNBPcFtGqkVKwRxM2wUhe+ztWZ5OGz ATEHuUZAXsKegPLsa7j/iYBLQG5D2vxSD/qCAAS9zXxDJf1/4M0qi5C9tj+b9jH8NuV03J6GO7Q Okg== X-Google-Smtp-Source: AGHT+IEnZEGZphhdHlQgb0iOcJfNRdGKMtxwSRuFd5MkDGpZIOhDcg8ohYIeF4ySSwO5bqKUu8yK4lYjZ+4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:8595:b0:2e2:d600:3862 with SMTP id 98e67ed59e1d1-2e2f0d9dbfemr89a91.8.1728584735013; Thu, 10 Oct 2024 11:25:35 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:22 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-21-seanjc@google.com> Subject: [PATCH v13 20/85] KVM: Remove pointless sanity check on @map param to kvm_vcpu_(un)map() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112536_714593_C7809FAA X-CRM114-Status: GOOD ( 10.00 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Drop kvm_vcpu_{,un}map()'s useless checks on @map being non-NULL. The map is 100% kernel controlled, any caller that passes a NULL pointer is broken and needs to be fixed, i.e. a crash due to a NULL pointer dereference is desirable (though obviously not as desirable as not having a bug in the first place). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 52629ac26119..c7691bc40389 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3071,9 +3071,6 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) void *hva = NULL; struct page *page = KVM_UNMAPPED_PAGE; - if (!map) - return -EINVAL; - pfn = gfn_to_pfn(vcpu->kvm, gfn); if (is_error_noslot_pfn(pfn)) return -EINVAL; @@ -3101,9 +3098,6 @@ EXPORT_SYMBOL_GPL(kvm_vcpu_map); void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) { - if (!map) - return; - if (!map->hva) return; From patchwork Thu Oct 10 18:23:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 152A3CFC5F5 for ; Thu, 10 Oct 2024 18:49:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=4FLuYqcYfYKO0YFOyI00fJurPtgxOjQDWgl3qcjFFzs=; b=dpZoV2PWjIHtX8 4yy+VTKqvdjGy1ol4ydrOXUPm510tK2HXD9QX/1haPcpeBCSeHRJD/keLd84Q+LDeKq1UnnwLA0jM qnHG7izy2vsHHEVtBMoLkTuLwhoG+gxhfn6r6bMr1eEnsqQ9wt4ub2JJEt02hB9K8AoKYe7Yy5dbH N0w/P+xNEUkpB3bUiEm8Utaa5z+59n/RR20nGVqgAKOULKeNk6Zree5EC04DtKGm16Zv40vLj3b9y s+jVvV2/37EiflhO+zEP1GvgOcjDePIO3AHx84KUYxeHSPOCxsMjjl8hdCFE88zoYGG3WWLrq3q+s EHgiiDi9tExVdu9dSnQQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyDm-0000000DzUL-3exi; Thu, 10 Oct 2024 18:49:10 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxr0-0000000DqBR-2Jd7 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:40 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e2605ce4276so2179754276.3 for ; Thu, 10 Oct 2024 11:25:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584737; x=1729189537; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=IMgO/JDFhWAWu7Wkr7+zBWYfApHhI977J8rKw7ML51Y=; b=hgOyYvQjo5Iks6jmfsYRhhZ653J3E4BLCYk0mKm4K/P7+ujw/kMs5SrTQfCdhu/xHN Db+HLZdiVwPcx0XNx4loznLbHChfVsgKxjMvnZidzMISqKM77GaWqQSFWintuPV3lTQX 5jNj8pMXtm3F6/AV4ag8agFO4lZ7LmFFe8bcXThkRc/3+NJnsaMCjPEnLChuYTVFTwIB CFbpbYCm8WEOCdmDdlAYfY1rbx6mzZGqqXQreWnDdlkTfbmZbXswSTw+o5ATOZGDgKiM JIQWUJLrwQQWH02X0iNjAYXuGRrfA3GmkUw6L3I4X7spPpuDh3EtI/himbevDZ/o1cjm qkzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584737; x=1729189537; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=IMgO/JDFhWAWu7Wkr7+zBWYfApHhI977J8rKw7ML51Y=; b=fAhL4oRDHKiTzGX8i7G0I099W2UcpncYcNBXXkt1dz6+aTbWnjGuA900Lie+DXmwoC HsQMMTJAIYDg1cBFHgq3I+3igrXG0VqtWQwcNvDvm0RvzqbUI3wQbfgEgfWGF0jqKtIc IDes59v7CfE3IElToNZ8sftNlnqWc0BcyQg54YgF12lsqplpOyrE1OfUG59uUN9HTgGI UBKZIp+YYssQV6RP6ICRMj3cegXkpgJ/tB7n7aro0s/8Ptm5toB0OTW6wqlAEYTqBBeY l6/isNaHc74i5CWOmQaE4navHPuNihfE2ZJajsbCsOTi8kLPoAokKnpOqHLLyqIc82R+ oQkQ== X-Forwarded-Encrypted: i=1; AJvYcCXdhQpmCTwWTJSPNwWMP3ms96DHq6WFBMheFgeoQ9Oaw5Obgjae9rsEyxPepcDAKqCspdjLARl4bGHlhw==@lists.infradead.org X-Gm-Message-State: AOJu0YzxGflMWEUAv/B9gtJVDFTFVqIiOSLO7E3Iy+rsbnujYByN5jSS sLnXq9n4zeI+VB5roBYSSVcXmNpWmEoVMDbY4N4VNsP5cO/leqDjg6kUjD5VAJdmhQdu4fQuW6Z 4Mw== X-Google-Smtp-Source: AGHT+IGlSN4X6DbrP5DPPe6+MPgoNlxMoOT3GZzpJV2jr+xnXsAJlzTjTlsnBYkwN+CsyuU9UJbLGW89hxk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:c70a:0:b0:e28:eae7:f84a with SMTP id 3f1490d57ef6-e28fe0feaecmr5507276.0.1728584736880; Thu, 10 Oct 2024 11:25:36 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:23 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-22-seanjc@google.com> Subject: [PATCH v13 21/85] KVM: Explicitly initialize all fields at the start of kvm_vcpu_map() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112538_693348_2E5AB0CE X-CRM114-Status: GOOD ( 10.01 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Explicitly initialize the entire kvm_host_map structure when mapping a pfn, as some callers declare their struct on the stack, i.e. don't zero-initialize the struct, which makes the map->hva in kvm_vcpu_unmap() *very* suspect. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 40 ++++++++++++++++------------------------ 1 file changed, 16 insertions(+), 24 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index c7691bc40389..f1c9a781315c 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3067,32 +3067,24 @@ void kvm_release_pfn(kvm_pfn_t pfn, bool dirty) int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) { - kvm_pfn_t pfn; - void *hva = NULL; - struct page *page = KVM_UNMAPPED_PAGE; - - pfn = gfn_to_pfn(vcpu->kvm, gfn); - if (is_error_noslot_pfn(pfn)) - return -EINVAL; - - if (pfn_valid(pfn)) { - page = pfn_to_page(pfn); - hva = kmap(page); -#ifdef CONFIG_HAS_IOMEM - } else { - hva = memremap(pfn_to_hpa(pfn), PAGE_SIZE, MEMREMAP_WB); -#endif - } - - if (!hva) - return -EFAULT; - - map->page = page; - map->hva = hva; - map->pfn = pfn; + map->page = KVM_UNMAPPED_PAGE; + map->hva = NULL; map->gfn = gfn; - return 0; + map->pfn = gfn_to_pfn(vcpu->kvm, gfn); + if (is_error_noslot_pfn(map->pfn)) + return -EINVAL; + + if (pfn_valid(map->pfn)) { + map->page = pfn_to_page(map->pfn); + map->hva = kmap(map->page); +#ifdef CONFIG_HAS_IOMEM + } else { + map->hva = memremap(pfn_to_hpa(map->pfn), PAGE_SIZE, MEMREMAP_WB); +#endif + } + + return map->hva ? 0 : -EFAULT; } EXPORT_SYMBOL_GPL(kvm_vcpu_map); From patchwork Thu Oct 10 18:23:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830862 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 258CCCFC5F3 for ; Thu, 10 Oct 2024 18:49:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=nm3CqfBG7D6bxycJqx29G3FK7jIU/aEIfF68/cEimp8=; b=S7PuJNYK3NlOxk DXDkSDI4PnnjmrX+PL1WJKPdVLly5gbETBmb4HjQ7v7ZjWyrW97TGAy+JP5KPEwkuuiJQ4k/SDFJG d7Wta1QaiYvutTqWMKvz1NxUe5v45kOdb1j8Sxg2yS6IMmLppAqivSn48Ye1+e5j1wCI96rv4mVA/ wrNYGLbu2+8mOPIBtL65GTcz0XJypkBDEPmPhJKzsHqn4xah7MZGVwcTeeVV1c/q9fKChLuGvh2gU zq4/XYsL5gZkLLzTMOsclHDf40FAy5DXopJBGlkT5Ql2lB2l1gpe8hLDSMjXzNlDoRLJ+jelnTGm3 YT+nVwEPQo/3CailHLnw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyDn-0000000DzUq-2VpL; Thu, 10 Oct 2024 18:49:11 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxr2-0000000DqCu-1AV1 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:42 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e2261adfdeso21188417b3.2 for ; Thu, 10 Oct 2024 11:25:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584739; x=1729189539; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=wU+jX/zRq5GV793BchOJ2bBXpO8AjDudJh47DKJg70M=; b=FIApe3/ZIx/g/Zfj5ITJmKSVaNpQH8BwtI16v5WF0kzSLo63vWhM9zgGz/f6KZrMod eaM47p1v8g3XI8cRrHCgnlw1lkERt2Es2DkFBS95BMEa8VOtSISuR0JQV/5yE6ZrE9g8 jtFlVPKcjYY26HLTybBZ5JgpE+WFrKho0MxAG6zyMHT/NBJhzkj0OsbHSOdDYe5atDIk dwHJUpgA5PUAp3iX+mX/XwVfXhs7goHJGlBb88Z1FzHwOHL0qZrZ/S3A9GWkwAYPF2Rs cJn+WIYR1eimvF36V83F42MITIu1YXfPQNciFvRCZHqu34NfpiAeRIxNgBjC85XWMPfE z3Uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584739; x=1729189539; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=wU+jX/zRq5GV793BchOJ2bBXpO8AjDudJh47DKJg70M=; b=VRNm5oAyA+cAqWaTB4AuOXBRrNBLeSINBBW2kOi4DxNSh992uQsDltOu4FszxA3Z8a L90r0fmCMCrzs/k9DzjwxPdqYELIX70W0r8R8sUqR87RznBlFKqOFW56VWUXFTxizXcX qMCndUrzh08HjqlE9XhigFGFL8ttSPJnCHb0HI0qIfVdJfd7Oy2YGYGrZGf3lxzynbDj kanN+XkOIjRPuybnDyFJZSINqqVjvLgY1G3IwyPtj87YVECThlMLS6dZj1cyoFF47lM4 off5aN5YzgP7gafjraSAprVLFK6XBhj+97nzDOV9Y2mgHyXGAaflLm3sTe85AOSbYFEW pD8Q== X-Forwarded-Encrypted: i=1; AJvYcCWxN/1KXgo0ZLHpx5c5cYDHndq2akPHo+BwOpdothtS4cefUqX8+igeKcjQlAZROXgBhqJZT+2a4Ks6kg==@lists.infradead.org X-Gm-Message-State: AOJu0YzOZbbmxkhbhcNpJ1ORjdwCz7dR2LMaK0evo4folx7kbaNszeoG tss9GPnJxIGeIZMDgqkpS3be8vIWrThYb/gXJyHiOfuJeOliva02tj0wpw7MeunMkMMcLsrirHM pTQ== X-Google-Smtp-Source: AGHT+IGZ+3mddEgfwOZrB7Mg/yEB+clUKORcl3gwcx/GeFypDRI64NMuGnBtjzJ0gTzObJ64BsofGpKHxtA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:ab6d:0:b0:e28:f8e6:f4c6 with SMTP id 3f1490d57ef6-e28fe465652mr58758276.2.1728584738891; Thu, 10 Oct 2024 11:25:38 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:24 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-23-seanjc@google.com> Subject: [PATCH v13 22/85] KVM: Use NULL for struct page pointer to indicate mremapped memory From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112540_385079_63EE7EB4 X-CRM114-Status: GOOD ( 14.28 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Drop yet another unnecessary magic page value from KVM, as there's zero reason to use a poisoned pointer to indicate "no page". If KVM uses a NULL page pointer, the kernel will explode just as quickly as if KVM uses a poisoned pointer. Never mind the fact that such usage would be a blatant and egregious KVM bug. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 4 ---- virt/kvm/kvm_main.c | 4 ++-- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index cbc7b9c04c14..e3c01cbbc41a 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -273,16 +273,12 @@ enum { READING_SHADOW_PAGE_TABLES, }; -#define KVM_UNMAPPED_PAGE ((void *) 0x500 + POISON_POINTER_DELTA) - struct kvm_host_map { /* * Only valid if the 'pfn' is managed by the host kernel (i.e. There is * a 'struct page' for it. When using mem= kernel parameter some memory * can be used as guest memory but they are not managed by host * kernel). - * If 'pfn' is not managed by the host kernel, this field is - * initialized to KVM_UNMAPPED_PAGE. */ struct page *page; void *hva; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index f1c9a781315c..7acb1a8af2e4 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3067,7 +3067,7 @@ void kvm_release_pfn(kvm_pfn_t pfn, bool dirty) int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) { - map->page = KVM_UNMAPPED_PAGE; + map->page = NULL; map->hva = NULL; map->gfn = gfn; @@ -3093,7 +3093,7 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) if (!map->hva) return; - if (map->page != KVM_UNMAPPED_PAGE) + if (map->page) kunmap(map->page); #ifdef CONFIG_HAS_IOMEM else From patchwork Thu Oct 10 18:23:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831140 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CB00FD2444F for ; Thu, 10 Oct 2024 21:01:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=y8lbNB/D/08aZFhDXGcnGhmU21hzz7mkS1sZcBdn5JE=; b=QL942kMa1Vk+ZI euWPsjNDwwPQ61KOAmUKSucoVCPvxtKzdKa6GxwdGO7UOk6IwoAmopabhOnTvsU5z+KCR9qfkV4Tt /DudzzW9uy6u1ulejhIeTLL1rzEsO0LEke5G0VRnMjV8hnaBDSIxQg9eqQn3d2rzWTU5TQxdy4y0B ZCZ0qY2ybfBrxXYKEAR7Zh1Q3G0dhDSA9kYd/+tn+BiEyNPO1VOScyR9dSkP6BaDitltvRd2wEm/O 1EZRQl/Ewi7Uf4hVM2qwKb+O7K4uS+UTDrazCuDvFneIKwnLtmpTw5aPg/CUXyrv2ZYPmBiKyMLlw O/Ms6U4Jo8F/JIJGibjg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0Hm-0000000ELju-35yC; Thu, 10 Oct 2024 21:01:26 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxr4-0000000DqEd-0vzh for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:45 +0000 Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-20b8bf5d09aso15596605ad.3 for ; Thu, 10 Oct 2024 11:25:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584741; x=1729189541; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=UxQVf2h1B1ZgTBX9bfQB+O4u0WEdBy2E8BNaXtNQtEc=; b=cHCxW9OvZGr1yBq6iVLuom+7EfZOc2EiXI7ygoBxdL22tnMY4Rj8QktPQoH9Zvi7rY qyOcJSNL56bk8CzNNK18Qv8diJFnIWO5chCmwCAkSlGhhfKd2k9jLijI97GeNIR3JSb5 KDlg64BqicYf/HRueE0w1SDL7nCmZA4NdT6OUJajraX/1Si4XFYP/vdmBM232nIzJ/Vg 6B+M47Y5OHDhvCmF/DpY/dTExiOIuOVDanc++9fpYZHB1avwuohYC45mdc78wc8C+odL PzeJTkRQxew5bIRj0u4rP3K84+JAXK/nVdKxY+zRIlu3rMcO5RHJD0zj3yKwqUl7by/Z hxkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584741; x=1729189541; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=UxQVf2h1B1ZgTBX9bfQB+O4u0WEdBy2E8BNaXtNQtEc=; b=GoAMQejNn/zySbrOCD7bLEfZCKAcqt2bDfxYBT+GwmzozW+C0iG2sitMlWovr844MG Z1GUoInRg42y/e9sXvs6Kef1Y2kC/3/05tctNcPPxfWdRLg40kOOe+P64xvfhYV3e4cY GZIcjLyymC2uGyHUe3SlIXzsUTgi6f31za/d/ePTothlJ3zlu6NHmZloCLoJm2wXohFe x8YkH34JhDvGymwUqpIeeW5T1DkzfQ4uicudI3zDSOWyQAaRnTFMgCJAWAqcAs1Tou8r 4+yE6jASuvk56o/pCJnxHBosN3ni9ltO/2yTYQ3cwH8Og2ZEmHYutsFy9SvRlQfGEij1 BDjQ== X-Forwarded-Encrypted: i=1; AJvYcCXUNgjNWmAEcfiitz+csOvdOg50wjZw2+ndb9rSUiMWVVj4FEi3vsFa3oxNcpxzluy7dlzHjGxVSV6QRw==@lists.infradead.org X-Gm-Message-State: AOJu0Yy/wxa2Q59bAqDaizGicQjtc3pl52ofAkwNoQuS4gFc0Qa1/lQ3 HOdiU3ZxfRelze9PUnen+VsMTom5mVZLHt51298bKf1Z55O7pWqEb2C+v0byYb5GG2a5FSPdh4o pzQ== X-Google-Smtp-Source: AGHT+IFSsaj7YGs0xXvL7KvHpo4OHZS23TfW901WnGn1dNBw4UEq8HUBbyPz7tDSJuBwPv672jv+MkN6+4M= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:2349:b0:20b:9365:e6e4 with SMTP id d9443c01a7336-20c63782bdcmr1094885ad.9.1728584740809; Thu, 10 Oct 2024 11:25:40 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:25 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-24-seanjc@google.com> Subject: [PATCH v13 23/85] KVM: nVMX: Rely on kvm_vcpu_unmap() to track validity of eVMCS mapping From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112542_369474_37E85BEE X-CRM114-Status: UNSURE ( 9.67 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Remove the explicit evmptr12 validity check when deciding whether or not to unmap the eVMCS pointer, and instead rely on kvm_vcpu_unmap() to play nice with a NULL map->hva, i.e. to do nothing if the map is invalid. Note, vmx->nested.hv_evmcs_map is zero-allocated along with the rest of vcpu_vmx, i.e. the map starts out invalid/NULL. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index a8e7bc04d9bf..e94a25373a59 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -231,11 +231,8 @@ static inline void nested_release_evmcs(struct kvm_vcpu *vcpu) struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(vcpu); struct vcpu_vmx *vmx = to_vmx(vcpu); - if (nested_vmx_is_evmptr12_valid(vmx)) { - kvm_vcpu_unmap(vcpu, &vmx->nested.hv_evmcs_map, true); - vmx->nested.hv_evmcs = NULL; - } - + kvm_vcpu_unmap(vcpu, &vmx->nested.hv_evmcs_map, true); + vmx->nested.hv_evmcs = NULL; vmx->nested.hv_evmcs_vmptr = EVMPTR_INVALID; if (hv_vcpu) { From patchwork Thu Oct 10 18:23:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831241 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3B3D5D2444F for ; Thu, 10 Oct 2024 22:16:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+EhyhYRLCbkN1nieOCUFgm5lAe7eRxcavCexP4oXlMw=; b=GCilZTeo6p+P2i hEjCKa8hI1+Z6hkU0s/kx+kkEmCWluC/+YPNeoqduY+ubnb/V196ke5F05aiU5eneL3nSPF3AYmPM sbH9YTEXm4sizA2nbH5e+OO7xDT49GqNReeO1dYXokhZxT9L5mWTQQBNRNwx3IOQ7JVs7rK+plaea gyXqbLdTDndwh0PJGuebqFqsQeGRju8Nl8H9Gv2axdlYgjvwEmdr1+XgymVfyr49YoBkZ8v9MFrB/ 1Y6flP3E0G18a43IT70d52lK6GydoHM781pK9mmwY20wUEh/vys8krHf4O9jMLxDgycO0/CS1Jw50 FRqzJun+Fk0Ls2B8gJKA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz1SM-0000000EVsi-2jOj; Thu, 10 Oct 2024 22:16:26 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxr6-0000000DqGB-1oXL for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:46 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e25b39871fso27043637b3.0 for ; Thu, 10 Oct 2024 11:25:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584743; x=1729189543; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=eHAZpPaCzYnE+TRrbh6ni4wh+Yys2pChavpSnHk2YEU=; b=Zb0EU3prV44v8rePPhkxu4JaEh7GECx4rDUMV31lO5GUbi2/yQu8jNNiHhLjJrB/2S OXtDhvADkrnimtQeMA+9W0Nz7h1XXpa8MTpyPZWvWureI2+J1i5mAMQ7vU3A0M5bepIA +RpwQGwEWZxr13xXmQeYkgCJkenuOlcASDNr/to7qY1cfgY2h+c4OgaSmVEAAzx8J5Af ZpQx654UMkRAeTlRjIvCxepr15qG8z2MYLnNUHLzyek8s0RMI66Nr33O89odLYjjSxX9 AhGjwOH5ukYEndvRdqJSGnAjfdUuKkw0gWxtU/m+QI8HXgIijYEEPmmVaBGYqA594u03 Cgag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584743; x=1729189543; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=eHAZpPaCzYnE+TRrbh6ni4wh+Yys2pChavpSnHk2YEU=; b=wknW1MvWymN82ogmXyCjWsY6yJfh23A4Aq2lcV3Pd9p3tMGG7+zzz4oEh50QNmNkHr MhglK67EqxWf4uuvI9vuiU3l2TwIx5ZcYh00DsH7Av+WryoHfVn0vGLWvBJKorV5S/Hf WJlyqFSY9onc5s81mvUzltW+qUMTnBJALi2xjPp+5NmpIj2bpQ++j6bT2rbWdbOK9xoM Ayptj890zQ+ERtqzNvy63SWCEwk8FENH3ocP3UXUwGCOKGv7CpdTdx/5isl9Vczk9peW 9M+W+lUw5JDwY2IK6SJjb5kzgIB4QDXTfyQi9v+V/odmSVM4RmemduCzNByQCkgmRXD5 181g== X-Forwarded-Encrypted: i=1; AJvYcCWxAeD0ztfstfmYtYJt0NafaTbgFIebG3LbsrShktj7+WqCt9Qd7oOzR8MIY3i/dVD4LJ0aG5NTfg7Nwg==@lists.infradead.org X-Gm-Message-State: AOJu0Yxnm9CnODw1whN1YYVB+II79NMKrB+CUCxI5an9Z/q1jwmwPEjG baxjrL283ArLu3ua+6gPBIoGk88Fdy7TIMWCYwkPhEHjqKgJE4zYuqlAXpeF6DSrHAXcp6omo+0 xmA== X-Google-Smtp-Source: AGHT+IGUAGxjHNELfO1gnQBoxyShlnhLiinRslmifWPsD1igdN7s2BU3B9tdatRPL4jwPkA1Z0qI9X+JjYQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:9346:0:b0:e28:fc1b:66bb with SMTP id 3f1490d57ef6-e28fe4f0fddmr4836276.6.1728584742775; Thu, 10 Oct 2024 11:25:42 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:26 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-25-seanjc@google.com> Subject: [PATCH v13 24/85] KVM: nVMX: Drop pointless msr_bitmap_map field from struct nested_vmx From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112544_704253_C98C169C X-CRM114-Status: GOOD ( 11.56 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Remove vcpu_vmx.msr_bitmap_map and instead use an on-stack structure in the one function that uses the map, nested_vmx_prepare_msr_bitmap(). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 8 ++++---- arch/x86/kvm/vmx/vmx.h | 2 -- 2 files changed, 4 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index e94a25373a59..fb37658b62c9 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -621,7 +621,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, int msr; unsigned long *msr_bitmap_l1; unsigned long *msr_bitmap_l0 = vmx->nested.vmcs02.msr_bitmap; - struct kvm_host_map *map = &vmx->nested.msr_bitmap_map; + struct kvm_host_map msr_bitmap_map; /* Nothing to do if the MSR bitmap is not in use. */ if (!cpu_has_vmx_msr_bitmap() || @@ -644,10 +644,10 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, return true; } - if (kvm_vcpu_map(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), map)) + if (kvm_vcpu_map(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), &msr_bitmap_map)) return false; - msr_bitmap_l1 = (unsigned long *)map->hva; + msr_bitmap_l1 = (unsigned long *)msr_bitmap_map.hva; /* * To keep the control flow simple, pay eight 8-byte writes (sixteen @@ -711,7 +711,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, MSR_IA32_FLUSH_CMD, MSR_TYPE_W); - kvm_vcpu_unmap(vcpu, &vmx->nested.msr_bitmap_map, false); + kvm_vcpu_unmap(vcpu, &msr_bitmap_map, false); vmx->nested.force_msr_bitmap_recalc = false; diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index 2325f773a20b..40303b43da6c 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -200,8 +200,6 @@ struct nested_vmx { struct kvm_host_map virtual_apic_map; struct kvm_host_map pi_desc_map; - struct kvm_host_map msr_bitmap_map; - struct pi_desc *pi_desc; bool pi_pending; u16 posted_intr_nv; From patchwork Thu Oct 10 18:23:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831245 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D6239D24459 for ; Thu, 10 Oct 2024 22:16:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZjFbaMsOcVaFK16wNz6D28UM72PL9IxBKW8uk1UoomU=; b=qcT+zQ1Or7s78Q r6Jw+U0z+fFsEGXWEsPcIfC4g3PozQn5dJ0OU+gOB+TvE9QyLdor7L31ghNWMjmHjCXiWzLc4S2Na guGHYY6pgXlWVdOYly7NUK3xr7/a5jbyC+zWPwvem7pnvkZeUctk374Dwptz9/IQY3HbV2Oc/hzCz glYEsTd9roHWcX4uk8hM1DZxhEN6QTVBQL4Xx356Va+b0m7Si2k5MaBuZKXZP1fpTkrDzMXEwuG7q XBYvdkb1OKNA0AFHIaiHgOcLOireoxMSgEsPixWeCt1MGJFGkFuES9FKAbX2nL630TJNa3xdBFFuV y9MB68UMjRMFYpNbJDoA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz1SN-0000000EVtH-3sc7; Thu, 10 Oct 2024 22:16:27 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxr8-0000000DqHo-0VaY for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:48 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e25b39871fso27044077b3.0 for ; Thu, 10 Oct 2024 11:25:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584744; x=1729189544; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=xnti8P2TQudE7lvlkZdCl8Wq+TvGW4tpGYbCykNWsXw=; b=iLMIpzBZnXcGg0be6+lfC0lRkhEn5KMOD8i176j/1z6K3vzSEKUgEY2w7Slsux3/sR 3Law3bIdQBGjRc99BGD1QjTrmVGIIdRklEe+BS8USHYkaFZ87LaMNp0wTitj1ABRQEGb +GB2Ly492NrlOCkVLBGlkdVcdkTXT7bd/nxeCgdcefR38PXWMKZfxp+zYbQ2eTwdTLLi Cr0gabDJOvk1eneM9z0RaBxYhv4itP1lyjhdFwUNWGM/9rjT6blS0JrqB6TqwHJeOUuj OXlSTptN9xWHtLzCgYAxhDDTihYLI7dD3QW4kF/L5ZcA16r+jrF0UOiapmRfSZAtHtOM hG6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584744; x=1729189544; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=xnti8P2TQudE7lvlkZdCl8Wq+TvGW4tpGYbCykNWsXw=; b=S/Jiat/yI0nuKQ7p6TQvQkzgGLywPv/3KshCdq/g+5b9eKOm3anjlZ2+mKIsdw0MnK L9MspFOEGhl9A4h5GEm4KwbF7HC5YpgxRiIRVdpwFUQr1cVbArF7laXRVRCjMqesklNi MJzru+CHe2zHvBbjH0Z/DyroypUJwEFKMFLuTW7z744ZaH+eTqQn6Ip/DrB6/64A5Ftg gl5XJK408SvwoYsmRcXWiQigtCFQ0EHFNzURxD/N224yNfjDmxVpH4Yh9SuJHVmjDInT PaqSufnUOMtcxJEksI2CaOld/6s8+AOS+0dTecb5VXRh5TVdc/WCXZTllPGW0XgSlcKt V2Xw== X-Forwarded-Encrypted: i=1; AJvYcCUtwFdGpCGOcq6nJmFRrSpDRpAcmBE3awF4NADEUMwsrTBTTePpRNxKsDHH3We+cXcPMKvSBrDUYR1UCQ==@lists.infradead.org X-Gm-Message-State: AOJu0YzWrwWQ9XIlLNjwALFK4vDFaWD48WPK8M2Cu4ZEd2V1ISwALaZd zn9oNkm4j/PR6w9A69dQDREjUveilj7jKQyD9xfTFibHtPqZ+cg77ZzkPq8XoyV4V9qh63Gc7CZ 3Vw== X-Google-Smtp-Source: AGHT+IEdX//xUJq0LlzHSFsWnzcNANrEE9KtnndGvFzP5BAu23zXYFVXA6eeBi51F5M5i08FgAsxiLe5HbY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:300b:b0:6e3:f32:5fc8 with SMTP id 00721157ae682-6e32217bbe0mr502817b3.1.1728584744665; Thu, 10 Oct 2024 11:25:44 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:27 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-26-seanjc@google.com> Subject: [PATCH v13 25/85] KVM: nVMX: Add helper to put (unmap) vmcs12 pages From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112546_277858_F264935A X-CRM114-Status: GOOD ( 11.41 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Add a helper to dedup unmapping the vmcs12 pages. This will reduce the amount of churn when a future patch refactors the kvm_vcpu_unmap() API. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 32 ++++++++++++++++++-------------- 1 file changed, 18 insertions(+), 14 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index fb37658b62c9..81865db18e12 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -314,6 +314,21 @@ static void vmx_switch_vmcs(struct kvm_vcpu *vcpu, struct loaded_vmcs *vmcs) vcpu->arch.regs_dirty = 0; } +static void nested_put_vmcs12_pages(struct kvm_vcpu *vcpu) +{ + struct vcpu_vmx *vmx = to_vmx(vcpu); + + /* + * Unpin physical memory we referred to in the vmcs02. The APIC access + * page's backing page (yeah, confusing) shouldn't actually be accessed, + * and if it is written, the contents are irrelevant. + */ + kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, false); + kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); + kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); + vmx->nested.pi_desc = NULL; +} + /* * Free whatever needs to be freed from vmx->nested when L1 goes down, or * just stops using VMX. @@ -346,15 +361,8 @@ static void free_nested(struct kvm_vcpu *vcpu) vmx->nested.cached_vmcs12 = NULL; kfree(vmx->nested.cached_shadow_vmcs12); vmx->nested.cached_shadow_vmcs12 = NULL; - /* - * Unpin physical memory we referred to in the vmcs02. The APIC access - * page's backing page (yeah, confusing) shouldn't actually be accessed, - * and if it is written, the contents are irrelevant. - */ - kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, false); - kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); - kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); - vmx->nested.pi_desc = NULL; + + nested_put_vmcs12_pages(vcpu); kvm_mmu_free_roots(vcpu->kvm, &vcpu->arch.guest_mmu, KVM_MMU_ROOTS_ALL); @@ -5010,11 +5018,7 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason, vmx_update_cpu_dirty_logging(vcpu); } - /* Unpin physical memory we referred to in vmcs02 */ - kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, false); - kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); - kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); - vmx->nested.pi_desc = NULL; + nested_put_vmcs12_pages(vcpu); if (vmx->nested.reload_vmcs01_apic_access_page) { vmx->nested.reload_vmcs01_apic_access_page = false; From patchwork Thu Oct 10 18:23:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831141 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 392C9D24451 for ; Thu, 10 Oct 2024 21:01:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=RKCBrglvO/zPjXTOB57X7Dsl3NA2TagzlSyp3M756Cw=; b=Jr8bnWHy5g4DX4 Pm+8pJdtCJvw0KIMhf8DWabTSxQVovtqvb+lPBcpiYl+9a3GEkzTfLcUjuhO4qdbkEJrmMsr1wF0n w4pTDKq8SEr0SdBAoCREjT4ARa5cdfRyBynoC6AzvmGyP81CzkPIpTVPoqmc5gP0X/QNL4wBVrnAN Si7xfOKHvJvCJ+vCMazWRFODeGuGE7MCw5TwnHX3wey+T6dlhkk7YS3kD5uG5Yb4HUGXMhwWjTugV yslATgSt9sAzHSPb9JAQAOR97GzXM8y59SyYB0GayNC73U10fdCgE0pUOs9RkVcIdBlpntN7RHx3o JJ6mhptUxOa5d/8MVTBA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0Hn-0000000ELkx-3ef8; Thu, 10 Oct 2024 21:01:27 +0000 Received: from mail-pj1-x104a.google.com ([2607:f8b0:4864:20::104a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrA-0000000DqK4-2tTw for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:50 +0000 Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2e148d4550bso1322254a91.1 for ; Thu, 10 Oct 2024 11:25:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584747; x=1729189547; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=/W6JK9yAcorM7mqEFQq2hRBqvFCoz2C4pz3yOEmiGek=; b=3ljqXwYT9KU7vL/bvKPf9S0Yn/qBntGvThCT2wzCd8Z12f/66N5sIoVWA14cY3wEXx g0HCLPAfoQ28OuiX6nzyAfawBv92qW+HPQQu37BsUa/V7uUuXJiLYMSM601iEwSqsUOf /NM7uP351CB79XT86j42gqhFKlgX/vbHgHz5DTqinJBvK+PcdaEbIaL+QrMPgT0FIbtp HGVX7X1ioRXdTGYtEGA8WTxtd6LOh1q2++W09WQfgTr1feSdFE+UnwE/XERuI/cMc2G4 mKN8Qf+/P7jUbv/vuDxwiZEcUlFeFFt/L98/apl6pTYPYAUmkiq5SNEFXz3+LRw5g5bM ZyYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584747; x=1729189547; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=/W6JK9yAcorM7mqEFQq2hRBqvFCoz2C4pz3yOEmiGek=; b=NNCVgfLRQRn9jnl1WequyMdRWb5nDoG+fWAoLAB3Xk0WIDsmzF1qqXsgOnAB4/QYJp j8dr81zkIfslkKNldAtEO6ZMdinx2QuVNlP59pDySFbyYkIpnCyiJQmBXNFL/ckOcdLT JIbExVYpse62Lg9RS4XgJgzEslnqqRhcTM/8owzcckVXbG0DlAtbVZYm4hERGA+Uapn2 mqx21fUtHDsYYsUZ/Ih8bBlAvfasg7b9bGD+S0fvo0+thQHARhq2MPiIqE9uFHGR1xoM swC6HhH78AYeT0kn60H9tMIaxBMC8FU2cUwwu8cNfSkP8pZXlU4kAI0Ywe10O3jeSgZI xoqQ== X-Forwarded-Encrypted: i=1; AJvYcCUlj4ZOdRALb9ir323yQ1UsvBoJm3X1OO3eBtGMu830TeyU7c0IWxX6dkGmNQgfB7SpT+M4zNRexqnasw==@lists.infradead.org X-Gm-Message-State: AOJu0YxmKe2NOLODxJ5sW9UvLi9gR7eA+FfzmkXmlPk0eIH19ufovdQT lCAounqo1+JA58K9XVUAzxXYFhlyaWiDmIROGSKGLDWC+7E9N4O+gHKvs71ML20vtBgBTBsPdll +Pw== X-Google-Smtp-Source: AGHT+IFGWHaFUEsQRxn4rcRWabj8vzxqpLHdNOXqOmiiQlX4dD6MZqF5BpShV2hVfQIOX/PTyNMVtEMSya8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:9a94:b0:2e2:da81:40c1 with SMTP id 98e67ed59e1d1-2e2f09f2280mr109a91.1.1728584746508; Thu, 10 Oct 2024 11:25:46 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:28 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-27-seanjc@google.com> Subject: [PATCH v13 26/85] KVM: Use plain "struct page" pointer instead of single-entry array From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112548_865888_6F3901E2 X-CRM114-Status: GOOD ( 10.74 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Use a single pointer instead of a single-entry array for the struct page pointer in hva_to_pfn_fast(). Using an array makes the code unnecessarily annoying to read and update. No functional change intended. Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 7acb1a8af2e4..d3e48fcc4fb0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2752,7 +2752,7 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *w */ static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) { - struct page *page[1]; + struct page *page; /* * Fast pin a writable pfn only if it is a write fault request @@ -2762,8 +2762,8 @@ static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) if (!((kfp->flags & FOLL_WRITE) || kfp->map_writable)) return false; - if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, page)) { - *pfn = page_to_pfn(page[0]); + if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &page)) { + *pfn = page_to_pfn(page); if (kfp->map_writable) *kfp->map_writable = true; return true; From patchwork Thu Oct 10 18:23:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831242 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 258DDD24456 for ; Thu, 10 Oct 2024 22:16:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GsgjRju6y33J6xi13KgbkeyQw0CD37A6KEF+3fDIaSM=; b=XkT2UeyvHMydmE WNapBYbSmYbgJ/BWaLhrezRJvxltevOT9efpHh6FdHQTBCZGFV5tboFb012737qOtb+FE1q8jnYDz Q0y1hudkVcA71MY3MT+MnRnco4OOys8hWSgYziPNmWhpBKOfApzJI5dJoxiniUK3BBYmgNGxBCZzl MzjI56/AVGg1PgestHrZyieWeY1rAEDPzFmPIXZjaJvv0kwiHfrhuah4lz9G+p6Vc6Jys2nZ6gPhO UPmZhdST8u0EavlXBb6YsMDQre6u5NhAJc7Sv3XTLqrXGfSg8htF3H8Srxi3B+JtslhMNwC/azxZ1 dOLP+PuWHi8edKey/cig==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz1SP-0000000EVuD-094Z; Thu, 10 Oct 2024 22:16:29 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrC-0000000DqLt-1ut7 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:52 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-71e00a395b1so1476156b3a.0 for ; Thu, 10 Oct 2024 11:25:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584749; x=1729189549; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=txhtY6exXDaGFQAWYGvmt09bjeoYz319F03IR2k/C3M=; b=QOiWlENQw1uIbr6nUNrqjMhma3ERcOVij9+O8o2BjPvO4mrYl4XsAmuLiusexGR8og wjRSA7WvE5P8YAxpwHCCocnQY3fc2JIYkO+hhKk6wpWzFsfI8BAMyortVof5wkh28VS0 Qt6uIQVR43FVlVhGrf1uAMsyy/6cvsQrPWf167zPqypNnCLXzoYxB2jdJWRtBimBpcWt uoYysw3iFUs8BsYcyGBmeWFo8/VrIQKNDreM/iOY/RkbKvOAdmKLBqSp17mNc5/WUjiD njASJ7YlaTL5HOieVnKQztyEjeiufrz2c0OKSD/CKjo1u2aS16zJOMO10Q+JaDo62pQq 63gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584749; x=1729189549; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=txhtY6exXDaGFQAWYGvmt09bjeoYz319F03IR2k/C3M=; b=j6Nd4G5j/bGKjtT8n6x6+uqDn19xW8e+2uvChL7Ncl+l7QZWFSkCCMczoWxJ5U6ZND qnzFUto29p8xenXAF+MaubivBfK/nicD2tKbnJ6IHciBpbJYwAYnpPhYEMPyqJMciDQI S13O7uqpCzIbHoTncS7E2YwwFJMONfniHF48Ng/Ot3kXjl9YLxBDGn47KQ9aK6YylSHn rHdWHTFdzYS2crQokiMBT6aSTrIJgQZ79q5RIphWhEx97a9JQriX7LEdixpBX/VflfI+ 4F6UPbGK3MIDMZKSPLqctO8AzAPy92NeXCXaBMfpRfymjRml4qbslxaM2WlR451qdJpY QdxA== X-Forwarded-Encrypted: i=1; AJvYcCUOMLd6TgzOWlkiV8qiGMNSgZkFP9aDGKf64hw0MJQ4m24GKPV80MxzuSdgfdQXOwcNeNk1/XrLQBH2Og==@lists.infradead.org X-Gm-Message-State: AOJu0YyX+DZnJmmZ9lZlcP1QjVpB6rgqkSHF/RWOwJ/h+S1WI7FG58zK FUMFq8bBE9zUZIE1aUlvkzPuVZW0vMrMxopOWyCod9Ep7IILhCOu1GK0iHvAfkpMvk8sNCeBivT uJA== X-Google-Smtp-Source: AGHT+IE/uLa8aldzd4vJMnq+De69DiThAuVt+3q8zF6rqxlBj2/ZKo7+FHCoJyy45GpCD4mZjJfGV3U05Tk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2d8a:b0:71d:f1f9:b982 with SMTP id d2e1a72fcca58-71e1dc00b9cmr38199b3a.6.1728584749206; Thu, 10 Oct 2024 11:25:49 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:29 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-28-seanjc@google.com> Subject: [PATCH v13 27/85] KVM: Provide refcounted page as output field in struct kvm_follow_pfn From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112550_735001_45D015E9 X-CRM114-Status: GOOD ( 22.69 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Add kvm_follow_pfn.refcounted_page as an output for the "to pfn" APIs to "return" the struct page that is associated with the returned pfn (if KVM acquired a reference to the page). This will eventually allow removing KVM's hacky kvm_pfn_to_refcounted_page() code, which is error prone and can't detect pfns that are valid, but aren't (currently) refcounted. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 99 +++++++++++++++++++++------------------------ virt/kvm/kvm_mm.h | 9 +++++ 2 files changed, 56 insertions(+), 52 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index d3e48fcc4fb0..e29f78ed6f48 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2746,6 +2746,46 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *w return gfn_to_hva_memslot_prot(slot, gfn, writable); } +static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, + struct follow_pfnmap_args *map, bool writable) +{ + kvm_pfn_t pfn; + + WARN_ON_ONCE(!!page == !!map); + + if (kfp->map_writable) + *kfp->map_writable = writable; + + /* + * FIXME: Remove this once KVM no longer blindly calls put_page() on + * every pfn that points at a struct page. + * + * Get a reference for follow_pte() pfns if they happen to point at a + * struct page, as KVM will ultimately call kvm_release_pfn_clean() on + * the returned pfn, i.e. KVM expects to have a reference. + * + * Certain IO or PFNMAP mappings can be backed with valid struct pages, + * but be allocated without refcounting, e.g. tail pages of + * non-compound higher order allocations. Grabbing and putting a + * reference to such pages would cause KVM to prematurely free a page + * it doesn't own (KVM gets and puts the one and only reference). + * Don't allow those pages until the FIXME is resolved. + */ + if (map) { + pfn = map->pfn; + page = kvm_pfn_to_refcounted_page(pfn); + if (page && !get_page_unless_zero(page)) + return KVM_PFN_ERR_FAULT; + } else { + pfn = page_to_pfn(page); + } + + if (kfp->refcounted_page) + *kfp->refcounted_page = page; + + return pfn; +} + /* * The fast path to get the writable pfn which will be stored in @pfn, * true indicates success, otherwise false is returned. @@ -2763,9 +2803,7 @@ static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) return false; if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &page)) { - *pfn = page_to_pfn(page); - if (kfp->map_writable) - *kfp->map_writable = true; + *pfn = kvm_resolve_pfn(kfp, page, NULL, true); return true; } @@ -2797,23 +2835,15 @@ static int hva_to_pfn_slow(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) if (npages != 1) return npages; - if (!kfp->map_writable) - goto out; - - if (kfp->flags & FOLL_WRITE) { - *kfp->map_writable = true; - goto out; - } - /* map read fault as writable if possible */ - if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &wpage)) { - *kfp->map_writable = true; + if (!(flags & FOLL_WRITE) && kfp->map_writable && + get_user_page_fast_only(kfp->hva, FOLL_WRITE, &wpage)) { put_page(page); page = wpage; + flags |= FOLL_WRITE; } -out: - *pfn = page_to_pfn(page); + *pfn = kvm_resolve_pfn(kfp, page, NULL, flags & FOLL_WRITE); return npages; } @@ -2828,22 +2858,11 @@ static bool vma_is_valid(struct vm_area_struct *vma, bool write_fault) return true; } -static int kvm_try_get_pfn(kvm_pfn_t pfn) -{ - struct page *page = kvm_pfn_to_refcounted_page(pfn); - - if (!page) - return 1; - - return get_page_unless_zero(page); -} - static int hva_to_pfn_remapped(struct vm_area_struct *vma, struct kvm_follow_pfn *kfp, kvm_pfn_t *p_pfn) { struct follow_pfnmap_args args = { .vma = vma, .address = kfp->hva }; bool write_fault = kfp->flags & FOLL_WRITE; - kvm_pfn_t pfn; int r; r = follow_pfnmap_start(&args); @@ -2867,37 +2886,13 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, } if (write_fault && !args.writable) { - pfn = KVM_PFN_ERR_RO_FAULT; + *p_pfn = KVM_PFN_ERR_RO_FAULT; goto out; } - if (kfp->map_writable) - *kfp->map_writable = args.writable; - pfn = args.pfn; - - /* - * Get a reference here because callers of *hva_to_pfn* and - * *gfn_to_pfn* ultimately call kvm_release_pfn_clean on the - * returned pfn. This is only needed if the VMA has VM_MIXEDMAP - * set, but the kvm_try_get_pfn/kvm_release_pfn_clean pair will - * simply do nothing for reserved pfns. - * - * Whoever called remap_pfn_range is also going to call e.g. - * unmap_mapping_range before the underlying pages are freed, - * causing a call to our MMU notifier. - * - * Certain IO or PFNMAP mappings can be backed with valid - * struct pages, but be allocated without refcounting e.g., - * tail pages of non-compound higher order allocations, which - * would then underflow the refcount when the caller does the - * required put_page. Don't allow those pages here. - */ - if (!kvm_try_get_pfn(pfn)) - r = -EFAULT; + *p_pfn = kvm_resolve_pfn(kfp, NULL, &args, args.writable); out: follow_pfnmap_end(&args); - *p_pfn = pfn; - return r; } diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index d5a215958f06..d3ac1ba8ba66 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -35,6 +35,15 @@ struct kvm_follow_pfn { * Set to true if a writable mapping was obtained. */ bool *map_writable; + + /* + * Optional output. Set to a valid "struct page" if the returned pfn + * is for a refcounted or pinned struct page, NULL if the returned pfn + * has no struct page or if the struct page is not being refcounted + * (e.g. tail pages of non-compound higher order allocations from + * IO/PFNMAP mappings). + */ + struct page **refcounted_page; }; kvm_pfn_t hva_to_pfn(struct kvm_follow_pfn *kfp); From patchwork Thu Oct 10 18:23:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830884 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 47A8ECFC5F4 for ; Thu, 10 Oct 2024 18:56:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=oLl5gvCeE6DiguNlZycGz1P26kPvsUu3bEXe1ovJdmk=; b=Q0Nwmu4zFl7M49 vobq1+dFhsli+IHvFv+dTOqAJe12Y9OUpfbH+PdAgCKV2tdw0n79uDdeIVNk3AERBik7Hvc5xjh/E jiEFXTFkgBdcSONWBjQ5jRqQw3zvxfEwMM6VNV1/vxikgYJ2MCHjwgcjkJ++nETRqg/SqnTMqQIuJ ieB3B/c1F/aSg5Gu9/cmkGG2ZVChnEiriQFoYccindRBxaYnL35Krw0eE/sh0URLvmne2XxvrqFVD uS6T35v1dfP9b1tQDt82fEQFcYwip8fJV49fLMBUDiPWNLZVs0CFHFX0Vdc/Y9NzHpvxM16kDHMr0 hCqyuM6BfVnA8lzeqNxg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyKn-0000000E0Wc-2gO9; Thu, 10 Oct 2024 18:56:25 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrE-0000000DqNT-2Sz0 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:54 +0000 Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-778702b9f8fso1039964a12.1 for ; Thu, 10 Oct 2024 11:25:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584751; x=1729189551; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=MotdKZcknVHCSnrsAh1XZemF3Qz2FYV2KdHfEEknv98=; b=jtLSLBcVgcASnTqlQ1eKxrFJR2feDfmay3NURsIC0axb8C2ARvmHmzCliBGAueotr+ B5XpHud2uLNFerpPsDj3hQwC9F0YTpa1737w8LQD0y5y7LzFTJWgDzF8hgHAoeB98c9h Ori7lZA+YjrLO+Qvs/uyhUsL+FZdPkU+W0JGTGPrJpQDbrwxwoqqJKLrmYcuj3o8IGb9 kg0swTwVKHW8C2cNgOAq8B0ThHBwdCGytcaZqR7NBsDqfTCj3rwXpBwNRbOpdEi/OhiP Gt1beXFBomlvF6zLw3cKb7dJ4CZuwDOsEzDBVH9U+wi6w0FOeGSWx7Vqbfa8yS3y3aJZ XbHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584751; x=1729189551; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=MotdKZcknVHCSnrsAh1XZemF3Qz2FYV2KdHfEEknv98=; b=ZCF/+5ntGRA7HOp3mkfTKj1EgFkedoj3/H6plIzMEo4QtNJ2jOCkn7SCcopd0eZ0JA AZzqcb3gvpNt7SGzeAK2hRW86SQmVwUhyFPw42m9Hdy5M1aSHwj2ns9WtJiYt+5sPuT4 I4bY6LJkEIPlfshGl9ZIUBTZAN0svC68MHciV7Ux0i9YrCt4mnyqGMzifiGSHYAFmSMK AFiaHYO7XvtL4/gg6sFf5lfasYKYUooSSfuy0u57z517taN/eg/yMjiU84K+ededDdY6 UlAWKuTIfGDA585b23LCVSX8CE8ruS4sotDiGsSz6us/68EksvpREFzDZqU/ZgkeZl8Q JfMA== X-Forwarded-Encrypted: i=1; AJvYcCVyqSoyvf9fPHRiF4f/Wd4z+mkM3OrmZnXBxLEG2OTVRn1ApZhCj8p8uQvxzC2r0b0tVCqt0Wm4KFcfsg==@lists.infradead.org X-Gm-Message-State: AOJu0Yy3ldmn/AjajYD0xbmk4BuPyWs+TBBIysH+7/x/LljcqC4lqsGO lwZ8eJMgAxYb0s+4St9qXs8zfV0TUXVQlkijraE0sruGTuNN/JM3t662cDwXO8KmmzLYPuz23Sy Fsw== X-Google-Smtp-Source: AGHT+IHahb+vMbBUybe9JXpaiwYyoag4sz2flsLCX6vY5d7t+wo3V9TaPVQYV32tHeuijXZHqATezWyezMk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:d913:b0:2e2:a810:c3e4 with SMTP id 98e67ed59e1d1-2e2c81d4d19mr7321a91.4.1728584751203; Thu, 10 Oct 2024 11:25:51 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:30 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-29-seanjc@google.com> Subject: [PATCH v13 28/85] KVM: Move kvm_{set,release}_page_{clean,dirty}() helpers up in kvm_main.c From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112552_672180_0C96E617 X-CRM114-Status: UNSURE ( 9.48 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Hoist the kvm_{set,release}_page_{clean,dirty}() APIs further up in kvm_main.c so that they can be used by the kvm_follow_pfn family of APIs. No functional change intended. Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 82 ++++++++++++++++++++++----------------------- 1 file changed, 41 insertions(+), 41 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index e29f78ed6f48..6cdbd0516d58 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2746,6 +2746,47 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *w return gfn_to_hva_memslot_prot(slot, gfn, writable); } +static bool kvm_is_ad_tracked_page(struct page *page) +{ + /* + * Per page-flags.h, pages tagged PG_reserved "should in general not be + * touched (e.g. set dirty) except by its owner". + */ + return !PageReserved(page); +} + +static void kvm_set_page_dirty(struct page *page) +{ + if (kvm_is_ad_tracked_page(page)) + SetPageDirty(page); +} + +static void kvm_set_page_accessed(struct page *page) +{ + if (kvm_is_ad_tracked_page(page)) + mark_page_accessed(page); +} + +void kvm_release_page_clean(struct page *page) +{ + if (!page) + return; + + kvm_set_page_accessed(page); + put_page(page); +} +EXPORT_SYMBOL_GPL(kvm_release_page_clean); + +void kvm_release_page_dirty(struct page *page) +{ + if (!page) + return; + + kvm_set_page_dirty(page); + kvm_release_page_clean(page); +} +EXPORT_SYMBOL_GPL(kvm_release_page_dirty); + static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, struct follow_pfnmap_args *map, bool writable) { @@ -3105,37 +3146,6 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) } EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); -static bool kvm_is_ad_tracked_page(struct page *page) -{ - /* - * Per page-flags.h, pages tagged PG_reserved "should in general not be - * touched (e.g. set dirty) except by its owner". - */ - return !PageReserved(page); -} - -static void kvm_set_page_dirty(struct page *page) -{ - if (kvm_is_ad_tracked_page(page)) - SetPageDirty(page); -} - -static void kvm_set_page_accessed(struct page *page) -{ - if (kvm_is_ad_tracked_page(page)) - mark_page_accessed(page); -} - -void kvm_release_page_clean(struct page *page) -{ - if (!page) - return; - - kvm_set_page_accessed(page); - put_page(page); -} -EXPORT_SYMBOL_GPL(kvm_release_page_clean); - void kvm_release_pfn_clean(kvm_pfn_t pfn) { struct page *page; @@ -3151,16 +3161,6 @@ void kvm_release_pfn_clean(kvm_pfn_t pfn) } EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); -void kvm_release_page_dirty(struct page *page) -{ - if (!page) - return; - - kvm_set_page_dirty(page); - kvm_release_page_clean(page); -} -EXPORT_SYMBOL_GPL(kvm_release_page_dirty); - void kvm_release_pfn_dirty(kvm_pfn_t pfn) { struct page *page; From patchwork Thu Oct 10 18:23:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B8D3BCFC5F6 for ; Thu, 10 Oct 2024 18:57:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=XLdDohDDFqGFXAGyTu3u+ZvGCh81J1T2J9zwRSkoLRw=; b=XXD6Zo0cScOCsY yT60xleSam62u41QigaSlPbr+foorhHgHq6I4Vcbttfwx1eHST/aFGJXB3y3SYbRU79/qrSYMEE8n MaOjToXHwrepNFfolGZEjFYUHgoWlXnpR3Tw3hAUXxQPG8V9FrDs5OXt/b/Ri28Ca2A9sSb6nnMNF NeldXwJUSup63vuW3kg+YzZ0Xvz81vvcQWB0UQWEA+cQKbRVnn3HtEK/bYuwSmGPvb6v1Wd9Aye/F tN5krKdVmYwZVryoLzme8TustIrulmJwR6aLPwxmfETtFVTSHm8WTeFHLhshXaEe3mmpiRxNlSaCa aVJH56VeQ9qkfGqKtHrg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyM7-0000000E0qJ-2f2P; Thu, 10 Oct 2024 18:57:47 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrH-0000000DqP1-0KZ8 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:56 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e02fff66a83so1965993276.0 for ; Thu, 10 Oct 2024 11:25:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584753; x=1729189553; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=nXVCtJlFVF227JmXan6PsTRYgw7//koA3Av9lMmMw70=; b=ttCXfoQrTUVfgH8jwR8KWzplwegscq+j8FDqssv2oxhDJRt4EEVBLhTF1+GaGNqIQl XvdFls0Il6zSW+CKvmB9tAofDO1Ill0yzaCXijICpQhEHTeRAWZnMb43UbmWmuM0vN2I 9sZWepWrJ+nULqgWGU5XrQ46pWviuTndQVuXa3ndAOlS+s/JEa0YjwCfRXMzAf4tzQHd XqRowaXrWrVUe04WhSnbBr0Qc8BnZMsF8YrEnrHvYviJJkJkNbvK6sB89qGjQdFD7cL9 5GOd+zNYLbrYew8HML/Cnf9qrivbBDIL1+HW1EqU4gsgl4lwPm3tZ7yh7tQzleuhQo16 DdYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584753; x=1729189553; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=nXVCtJlFVF227JmXan6PsTRYgw7//koA3Av9lMmMw70=; b=EY8D3yTM7zxeZvW2pfvmJlLn4RaIQ9EyUe1E8Z3TMho7TiUltgA9xDNXgEfjGm9WY9 XEWvsu8WlZMadm2AaJrZ3Kt7twnrNcPtYlPELM8pjVAA5uE01XJGYF1iWoNYlfyS9JJx 4RIwAXkyd3OWTllVxebEvIY8Zpa6TPhhdqq+PBZqg17IZoHNtd5Bu01QiJ8PJDroOSIu HyIglHtjc3ZtqlSg7W9JjE4w/PDd64kaaGTQGrYkAGf+gaIxpWHaXLBObCBktVcwl3WE vufR4snUbTUI5yeBUwecjJwJLjs98iZcOuyC2rKdbbw7QBuWpkJJjvQwk7O4kXqh9oRD Fghw== X-Forwarded-Encrypted: i=1; AJvYcCUXjMERsMV+WZLCYbczSHfp6/fcP4o2KUg5d2aT/XO9psnI4IEbmglvVKen8WNkLgZQ719RVlCRIdljDA==@lists.infradead.org X-Gm-Message-State: AOJu0Yz/kqm3pedx/TC9RkDiiCR1VoU7ysXmoEAtxIVxj71qgHjNdn/z 0tz/L1UWiWDj9uJkt9jD2A4blm2q1yatWqt/fBDWE11pBE241mum8NihgldY82POJZ6dyg62g5k yfg== X-Google-Smtp-Source: AGHT+IFqzRS1xOg2oJDncuDOdOtR7gITdwbfvBFuJ4xWEWgDQEplutdiZUKmbH0mIVak18Oaak6LM032atM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:b205:0:b0:e25:17cb:352e with SMTP id 3f1490d57ef6-e28fe43f3f5mr4095276.9.1728584753336; Thu, 10 Oct 2024 11:25:53 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:31 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-30-seanjc@google.com> Subject: [PATCH v13 29/85] KVM: pfncache: Precisely track refcounted pages From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112555_434415_A8B3C65C X-CRM114-Status: GOOD ( 10.67 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Track refcounted struct page memory using kvm_follow_pfn.refcounted_page instead of relying on kvm_release_pfn_clean() to correctly detect that the pfn is associated with a struct page. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/pfncache.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index 067daf9ad6ef..728d2c1b488a 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -159,11 +159,14 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) kvm_pfn_t new_pfn = KVM_PFN_ERR_FAULT; void *new_khva = NULL; unsigned long mmu_seq; + struct page *page; + struct kvm_follow_pfn kfp = { .slot = gpc->memslot, .gfn = gpa_to_gfn(gpc->gpa), .flags = FOLL_WRITE, .hva = gpc->uhva, + .refcounted_page = &page, }; lockdep_assert_held(&gpc->refresh_lock); @@ -198,7 +201,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) if (new_khva != old_khva) gpc_unmap(new_pfn, new_khva); - kvm_release_pfn_clean(new_pfn); + kvm_release_page_unused(page); cond_resched(); } @@ -218,7 +221,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) new_khva = gpc_map(new_pfn); if (!new_khva) { - kvm_release_pfn_clean(new_pfn); + kvm_release_page_unused(page); goto out_error; } @@ -236,11 +239,11 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) gpc->khva = new_khva + offset_in_page(gpc->uhva); /* - * Put the reference to the _new_ pfn. The pfn is now tracked by the + * Put the reference to the _new_ page. The page is now tracked by the * cache and can be safely migrated, swapped, etc... as the cache will * invalidate any mappings in response to relevant mmu_notifier events. */ - kvm_release_pfn_clean(new_pfn); + kvm_release_page_clean(page); return 0; From patchwork Thu Oct 10 18:23:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830886 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EBF44CFC5F7 for ; Thu, 10 Oct 2024 18:59:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=RqsyyUJa7R9QcnJVFa+nP1sCVW71bP1kxm301L3FJag=; b=HJObgx2mnk+QSF 1FCEdn8hImpLt/mp0OEWQqKepKQurnRPf/23aKXtBGTu8HA2EWNceHyhZlA97DnKezkxYbEtiDpOy KXOQ40CLxoIcHqiQzgVxNroIoq5lC0BCy8pAmTp2K4SsSSo3+6UTDdbL6rq4Z8eolazgcaegYrEj7 e4RgKYXTCYR3PiNJ5lP/wRivXp/Yq2EVJOEWRA1XvgBIaQavvItzF85B/2jrCpb6vtjMOm6TztO7c VBd4WGzjaPIdKnTYlevsWS1kITxXnPbsi8oVd0LDkILUsSTVNQEgSQQIUYIqdzCOOulPacLQmtWtR sEocYgDjMPOpb+ErEExA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyNS-0000000E1Ot-2zzE; Thu, 10 Oct 2024 18:59:10 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrI-0000000DqR6-439o for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:25:58 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e3231725c9so23554677b3.1 for ; Thu, 10 Oct 2024 11:25:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584755; x=1729189555; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=/prKt8Bu+DRISNBcov5NrkfgoAIYHMahITKzCWpSrCI=; b=spm+Iun/rKPn0B8n5jp4vtXW4KoIEaOrEeQhdA6BP3YALJ/n0GskBlDL6e3o6m8fcJ B2FLXrGM9uCwxFoF53aH0EHoH0e8ffaFOJQDwK/LpbkXexqlsNTjFgC4wgFjjWRitsa2 UwACMIiv9Ob9acFXqYVSy594DaZ7OfCpmYZ3aqpwJOcSfJBcl+GOnZoVxguzaWh8MnAj Kpu9mPXvfrTaAVtdQVhqotBwfO2qkl2/kuOdYGCFEGrb3/wC8p5ti+UyIpUO2AHERCmz CJK7plDH++huahh/U/pxEGKc9USpkfmmKt+0HHbSHtgky+x/Xx5DiQah54yFJCS1hted Ighg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584755; x=1729189555; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=/prKt8Bu+DRISNBcov5NrkfgoAIYHMahITKzCWpSrCI=; b=i0Xjn6mX2tmjsay+frGq2cfWDMpn3yNtd6DTiB1jaht2dY37GLOWfcendla1nEWqGZ 6XTjWW72WzWWbSr1N8ROmOA9kOpIx15UxEVUBo4BzIv5Yz7af5ZlO3+IhAKsNbibpqbG FqW0hM1iy6P8itqI/HxdpeFfQJ79Ptk6uyYqgJD4u3qet8NTMY8B1UZK/LTYWyEutcXT 3MQoYsRrEvBndYzbswmHrJkTlkoNxqJOLy20L2BQQeb3BNmBVhlhCPWB1YPiyaFvu10J e3mGU1cTiYGpTw3DzNCVMmLXwOGQshK2M9rdSJUGCTyhX2FOI+DohtZJfsKWktcgrbxY E+PA== X-Forwarded-Encrypted: i=1; AJvYcCUNADgfeJP0dPIdPOyk7QwbX81U6NcoZ5iWzfQGJ0TUOTB+35Bo8kIA1a2GqvqF1Pcdk5jE2FpQ5YQYEw==@lists.infradead.org X-Gm-Message-State: AOJu0YzbRuROYoDAxSZjZzJfLXPfN6OuhgV2chpVvKATrOdfoTnMZ8V8 XiPkTWQF7gFiD5yOZQEujw5UmVAUJKCODNSXIpBVvF5nv0Pp86lmhDFf9Ic3zG2LUDxZWGcruYH ufg== X-Google-Smtp-Source: AGHT+IHNZW39HKgOpo4XLx10sqcp5MldEDD35H2r/7z7SkgDatDPvbEuIFyvlgb9BtXWEl9gaR4EDWCFxGs= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2093:b0:6d4:30f0:3a74 with SMTP id 00721157ae682-6e322171640mr1011347b3.6.1728584755409; Thu, 10 Oct 2024 11:25:55 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:32 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-31-seanjc@google.com> Subject: [PATCH v13 30/85] KVM: Migrate kvm_vcpu_map() to kvm_follow_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112557_064858_019A9329 X-CRM114-Status: GOOD ( 14.65 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: David Stevens Migrate kvm_vcpu_map() to kvm_follow_pfn(), and have it track whether or not the map holds a refcounted struct page. Precisely tracking struct page references will eventually allow removing kvm_pfn_to_refcounted_page() and its various wrappers. Signed-off-by: David Stevens [sean: use a pointer instead of a boolean] Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 26 ++++++++++++++++---------- 2 files changed, 17 insertions(+), 11 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index e3c01cbbc41a..02ab3a657aa6 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -280,6 +280,7 @@ struct kvm_host_map { * can be used as guest memory but they are not managed by host * kernel). */ + struct page *refcounted_page; struct page *page; void *hva; kvm_pfn_t pfn; @@ -1238,7 +1239,6 @@ void kvm_release_pfn_dirty(kvm_pfn_t pfn); void kvm_set_pfn_dirty(kvm_pfn_t pfn); void kvm_set_pfn_accessed(kvm_pfn_t pfn); -void kvm_release_pfn(kvm_pfn_t pfn, bool dirty); int kvm_read_guest_page(struct kvm *kvm, gfn_t gfn, void *data, int offset, int len); int kvm_read_guest(struct kvm *kvm, gpa_t gpa, void *data, unsigned long len); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 6cdbd0516d58..b1c1b7e4f33a 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3093,21 +3093,21 @@ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) } EXPORT_SYMBOL_GPL(gfn_to_page); -void kvm_release_pfn(kvm_pfn_t pfn, bool dirty) -{ - if (dirty) - kvm_release_pfn_dirty(pfn); - else - kvm_release_pfn_clean(pfn); -} - int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) { + struct kvm_follow_pfn kfp = { + .slot = gfn_to_memslot(vcpu->kvm, gfn), + .gfn = gfn, + .flags = FOLL_WRITE, + .refcounted_page = &map->refcounted_page, + }; + + map->refcounted_page = NULL; map->page = NULL; map->hva = NULL; map->gfn = gfn; - map->pfn = gfn_to_pfn(vcpu->kvm, gfn); + map->pfn = kvm_follow_pfn(&kfp); if (is_error_noslot_pfn(map->pfn)) return -EINVAL; @@ -3139,10 +3139,16 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) if (dirty) kvm_vcpu_mark_page_dirty(vcpu, map->gfn); - kvm_release_pfn(map->pfn, dirty); + if (map->refcounted_page) { + if (dirty) + kvm_release_page_dirty(map->refcounted_page); + else + kvm_release_page_clean(map->refcounted_page); + } map->hva = NULL; map->page = NULL; + map->refcounted_page = NULL; } EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); From patchwork Thu Oct 10 18:23:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831165 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6FC21D24454 for ; Thu, 10 Oct 2024 21:16:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=y+EBDty2r0KkiZce4Fyu9yKKAHNGaa25WFTqfloVn8k=; b=mKnzKv0zeXFJwr 391z+55xIAqGNXK/Um56/xn+2CGsGpjBycHbHpbzU39dE+Qhu2YVZYsdp+lAbxpflm8EizXy61sdo paH31P5dCb2igXVC73D7yl1cmeX+klyIzuat/3qRepdFs9fTrdM/Kl+gW3t8o2VbYrA56LlDeWCj5 Ox7XC2YFy+8b2x7eK+WoJvL/7Aczbvlhd+h7IcVKOpf8jWQ97FbeHX89BlmXu/rQ6GtlcxD8NtoT4 mvLWgS/jz6AQhoP1qWDWuHEHDYgPNY9YBWmliPCHAtfPhnlaeDO1YuwpkyII63r2AVdO2Cj2JcmI/ RpoV6IHUgJYrCVwP3QWA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0WK-0000000ENXX-4BCS; Thu, 10 Oct 2024 21:16:29 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrK-0000000DqT0-2D6X for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:02 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e290b8b69f8so2137714276.2 for ; Thu, 10 Oct 2024 11:25:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584757; x=1729189557; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=dQaiAwtn89tr8uPQzQfay17fvGV3F3WPslUlHA+BgDY=; b=OSACH8GjnuyuiXpMfRDQCjz+rzwPNx3aGaepvL099zA8znTQC1egQ2seldu9OeijMU LazKJ2ZGVLT3cJM5U1v/DaYRFVPu0uxfn8zMN/VSc5LNS7kFSdzT1N8jORlYegWBdM/M 6+R11SLiQDzZrRPEGc1qEG0QzXF6NWx2dCA+IXrYAeG9Ro77XlesdpijfZpvz09fesrT q+fr7nWvVHD1GEkg/Q0M4eSGZNQMP2USY0zm3KIrxgzWmSOvWe+t8nR4i6Zd1AWzpjTY FkihywB7cmxcQ55tLY2qp4bZYNfslEZRg29M9xzmTj8+zhGiB5ThGCdBROkdMErWT8rb +SJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584757; x=1729189557; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=dQaiAwtn89tr8uPQzQfay17fvGV3F3WPslUlHA+BgDY=; b=mPFWWQb5oS0RGoq8dguiaT2nqKcp3eTQQdA0sn2hEFvnRvdF7bnzz5VEVxkhDgk5m0 A0ELfKXQzQrEVrMsNYBm/WjVpxRcDN+FH3aG6M3hVm4o94nAYXIE4BhxyqKyv/PywThZ hCI+NZ7MavmgMhKPFjocKaMnqPtdUZYGWEN4YeJ5ZIGaky5aH9dUr7OJYShYtP1osozU HbzidlXbU9XQs8PC9SWy9a10CW/zE2c/6WRRRrwg/dtciRMijuXdOD2tx0BqO29mLfDm kNx40AC/hIgva3xT4rtPNzHiswOevDKfboVFO/ZoE9+lpODMCJZBYVpMhs8m6686b327 1UhQ== X-Forwarded-Encrypted: i=1; AJvYcCUk7F+0jkdM5BQz5NwikGb/cUFxzoyn969mBkLLrd446W8uR/O2dioRV5HyY0pSjoMzXKMSZbgRR7uMsQ==@lists.infradead.org X-Gm-Message-State: AOJu0YxsWmERfmqapbEfan5GNT5vFIul7WjnK+LKXb4EOTYdS8aNPTpq ja8Xbpm1vp0WFe1frts7o/chTRHkrvbm+9mhCxOq5pf9Nc+eHs4ZGOKANclF5B5L93+iiPzzzG3 +ig== X-Google-Smtp-Source: AGHT+IHY0yNiFmgeAXsfVKl7EbEBY2R3q1IXvjRBHtSPoxXhe2HbHq4twyB0Huv0R+/OHvP6V1ZW3nZzpzQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:80cb:0:b0:e20:2502:be14 with SMTP id 3f1490d57ef6-e28fe410672mr4860276.7.1728584757233; Thu, 10 Oct 2024 11:25:57 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:33 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-32-seanjc@google.com> Subject: [PATCH v13 31/85] KVM: Pin (as in FOLL_PIN) pages during kvm_vcpu_map() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112558_743167_7EC3FFEC X-CRM114-Status: GOOD ( 22.60 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Pin, as in FOLL_PIN, pages when mapping them for direct access by KVM. As per Documentation/core-api/pin_user_pages.rst, writing to a page that was gotten via FOLL_GET is explicitly disallowed. Correct (uses FOLL_PIN calls): pin_user_pages() write to the data within the pages unpin_user_pages() INCORRECT (uses FOLL_GET calls): get_user_pages() write to the data within the pages put_page() Unfortunately, FOLL_PIN is a "private" flag, and so kvm_follow_pfn must use a one-off bool instead of being able to piggyback the "flags" field. Link: https://lwn.net/Articles/930667 Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 54 +++++++++++++++++++++++++++++----------- virt/kvm/kvm_mm.h | 7 ++++++ 3 files changed, 47 insertions(+), 16 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 02ab3a657aa6..8739b905d85b 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -280,7 +280,7 @@ struct kvm_host_map { * can be used as guest memory but they are not managed by host * kernel). */ - struct page *refcounted_page; + struct page *pinned_page; struct page *page; void *hva; kvm_pfn_t pfn; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index b1c1b7e4f33a..40a59526d466 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2814,9 +2814,12 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, */ if (map) { pfn = map->pfn; - page = kvm_pfn_to_refcounted_page(pfn); - if (page && !get_page_unless_zero(page)) - return KVM_PFN_ERR_FAULT; + + if (!kfp->pin) { + page = kvm_pfn_to_refcounted_page(pfn); + if (page && !get_page_unless_zero(page)) + return KVM_PFN_ERR_FAULT; + } } else { pfn = page_to_pfn(page); } @@ -2834,16 +2837,24 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) { struct page *page; + bool r; /* - * Fast pin a writable pfn only if it is a write fault request - * or the caller allows to map a writable pfn for a read fault - * request. + * Try the fast-only path when the caller wants to pin/get the page for + * writing. If the caller only wants to read the page, KVM must go + * down the full, slow path in order to avoid racing an operation that + * breaks Copy-on-Write (CoW), e.g. so that KVM doesn't end up pointing + * at the old, read-only page while mm/ points at a new, writable page. */ if (!((kfp->flags & FOLL_WRITE) || kfp->map_writable)) return false; - if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &page)) { + if (kfp->pin) + r = pin_user_pages_fast(kfp->hva, 1, FOLL_WRITE, &page) == 1; + else + r = get_user_page_fast_only(kfp->hva, FOLL_WRITE, &page); + + if (r) { *pfn = kvm_resolve_pfn(kfp, page, NULL, true); return true; } @@ -2872,10 +2883,21 @@ static int hva_to_pfn_slow(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) struct page *page, *wpage; int npages; - npages = get_user_pages_unlocked(kfp->hva, 1, &page, flags); + if (kfp->pin) + npages = pin_user_pages_unlocked(kfp->hva, 1, &page, flags); + else + npages = get_user_pages_unlocked(kfp->hva, 1, &page, flags); if (npages != 1) return npages; + /* + * Pinning is mutually exclusive with opportunistically mapping a read + * fault as writable, as KVM should never pin pages when mapping memory + * into the guest (pinning is only for direct accesses from KVM). + */ + if (WARN_ON_ONCE(kfp->map_writable && kfp->pin)) + goto out; + /* map read fault as writable if possible */ if (!(flags & FOLL_WRITE) && kfp->map_writable && get_user_page_fast_only(kfp->hva, FOLL_WRITE, &wpage)) { @@ -2884,6 +2906,7 @@ static int hva_to_pfn_slow(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) flags |= FOLL_WRITE; } +out: *pfn = kvm_resolve_pfn(kfp, page, NULL, flags & FOLL_WRITE); return npages; } @@ -3099,10 +3122,11 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) .slot = gfn_to_memslot(vcpu->kvm, gfn), .gfn = gfn, .flags = FOLL_WRITE, - .refcounted_page = &map->refcounted_page, + .refcounted_page = &map->pinned_page, + .pin = true, }; - map->refcounted_page = NULL; + map->pinned_page = NULL; map->page = NULL; map->hva = NULL; map->gfn = gfn; @@ -3139,16 +3163,16 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) if (dirty) kvm_vcpu_mark_page_dirty(vcpu, map->gfn); - if (map->refcounted_page) { + if (map->pinned_page) { if (dirty) - kvm_release_page_dirty(map->refcounted_page); - else - kvm_release_page_clean(map->refcounted_page); + kvm_set_page_dirty(map->pinned_page); + kvm_set_page_accessed(map->pinned_page); + unpin_user_page(map->pinned_page); } map->hva = NULL; map->page = NULL; - map->refcounted_page = NULL; + map->pinned_page = NULL; } EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index d3ac1ba8ba66..acef3f5c582a 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -30,6 +30,13 @@ struct kvm_follow_pfn { /* FOLL_* flags modifying lookup behavior, e.g. FOLL_WRITE. */ unsigned int flags; + /* + * Pin the page (effectively FOLL_PIN, which is an mm/ internal flag). + * The page *must* be pinned if KVM will write to the page via a kernel + * mapping, e.g. via kmap(), mremap(), etc. + */ + bool pin; + /* * If non-NULL, try to get a writable mapping even for a read fault. * Set to true if a writable mapping was obtained. From patchwork Thu Oct 10 18:23:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831164 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E1CBED2444F for ; Thu, 10 Oct 2024 21:16:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LQ8DHeflOylF45oiu9ZHJAwAS/KUwYy+XkcIDC57qnA=; b=pt5Ud8GZY2pPhy JyPIy+M6F8Vq3f4YXBfn+1kE2T9JEOHD5DjOPuZ0lUGNgHsxtshOa8s+WGe+KuaAQ83E8DzefvV0J lWjc6eKVkeXZ+sYS4kzSpWo/urC4ivltxndryYZjFAH8xSaA3SEliWfXcr/ADPBov51BO+fQjTo17 l83fFAorVxsjmPaVu3jsrXjUcPio4F3XJ0cDp77TqQbEdKo6S4fex4+eblN7IN1vMi6GeAU853SSV OiOdX/5BdZMsIeaW6yfE8Yyr1lCxBH0JeD11RSzyg+Am19usCorTy/6bMaAx+05/FOpAtT1J5i5jo I4IaKR3xSqioFdtpCjGg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0WM-0000000ENYx-2MVw; Thu, 10 Oct 2024 21:16:30 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrM-0000000DqUA-2Hya for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:02 +0000 Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-7163489149fso1214841a12.3 for ; Thu, 10 Oct 2024 11:25:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584759; x=1729189559; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=VITkHk6wYCk30qmAXnWfpQxFdItqbxje0xl/TrLgm28=; b=DDGV/BA9cByk02/tA4Cb/+AgaPSPEwimwcElwsHI1jiRIZELRkjDSYk3e7ZNGCv0X6 Xi76UfOwi0uHKt2IJK3aT1JjYxjdtbyaBwyAnFT3U3gn/rBV0L9eCzkzxmRa04PKCjLe ArtEmn/ENFhUYin/BrtkhAGwGmHDMtR6KKmWPoh91MCyGM2EwU8EoTsRgoFMiTVWqisT 3pgL2ELhDC4PmAeoYGgmfJl21pGh+O/G6/I+7+8SOGtDtuMlcFEKqLEi0Hi7fzSGVvFB ySrMi7ysavXcWJMDfUQY4qfyFR54JIxt96ot/kZsE2L9OD6Z1BG2yOcBh05vKntTesz9 ibPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584759; x=1729189559; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=VITkHk6wYCk30qmAXnWfpQxFdItqbxje0xl/TrLgm28=; b=QSKjwpOQAXNDEmfWtG/nshIqA5NRUHbEH1IareW+0oLsoIcmupUz0WyFEwa8byhrCn orG98rmHjQQkF1YR020+FMixFJj1a5JPOCZormO3cBEjVXEh5HPhy90/ZZ+3F1JKwyGz SRVeYFqnVsPt9HcuUFb7LDgfRlL1NHAQYSAwSWh7Z3yz569NkAC3rgCO0tZinLpj5rUi 8P2EJzs7DqDS94KNd23qONVzEXrTQ5pJdn9gyGrSnohoLDjIRGUH8Iwa6SQ0iBFYSExh g/QchLuDbzPJHJe+IrZX8ionovfNL2fIM0vZMVdZtCEHXnNM58d5C39CcJbd6mBFi2S8 1Zeg== X-Forwarded-Encrypted: i=1; AJvYcCU2vL64rECyobXzCBN4YFXK62qBYTAmw0jBPKaPIiLy8MgwKJDF0liEA4zv4MLSs2UzFO+7KaZla8l3YQ==@lists.infradead.org X-Gm-Message-State: AOJu0Yxl6iKFst4xEG5GRmTHMA4QhntEhsXzUllgtyi2SxoCkI2c2trw vhOTQZMLAWcY8bVCaz07vCLVT2wTQZenLwXmQolF5jyK6IzJYt5O42gdWuPuIDumEVzlzPW6YlR RKg== X-Google-Smtp-Source: AGHT+IEekecUM0XVWoBmNjTczqp1a2/ixTMw1fOAmBrSm/Dp9XgHLEkyd54mJG+SlDPSB0szYk/zKOq5dxU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a65:678f:0:b0:7e9:f98c:e9f7 with SMTP id 41be03b00d2f7-7ea5359ed14mr32a12.10.1728584759044; Thu, 10 Oct 2024 11:25:59 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:34 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-33-seanjc@google.com> Subject: [PATCH v13 32/85] KVM: nVMX: Mark vmcs12's APIC access page dirty when unmapping From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112600_692670_87095319 X-CRM114-Status: GOOD ( 11.27 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Mark the APIC access page as dirty when unmapping it from KVM. The fact that the page _shouldn't_ be written doesn't guarantee the page _won't_ be written. And while the contents are likely irrelevant, the values _are_ visible to the guest, i.e. dropping writes would be visible to the guest (though obviously highly unlikely to be problematic in practice). Marking the map dirty will allow specifying the write vs. read-only when *mapping* the memory, which in turn will allow creating read-only maps. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 81865db18e12..ff83b56fe2fa 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -318,12 +318,7 @@ static void nested_put_vmcs12_pages(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); - /* - * Unpin physical memory we referred to in the vmcs02. The APIC access - * page's backing page (yeah, confusing) shouldn't actually be accessed, - * and if it is written, the contents are irrelevant. - */ - kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, false); + kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, true); kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); vmx->nested.pi_desc = NULL; From patchwork Thu Oct 10 18:23:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831166 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8AEE9D24456 for ; Thu, 10 Oct 2024 21:16:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=jj2wNaAxCcma+QiviSUbMtgXcol+ckaAW+fieRepz8s=; b=MLD/Q0z+bPSZ0y 3VNS0Nd3aH1jOFlpjCuUsCKGyH+6LY8YkIIg7Sf94aNvTZEbJla6Opwpr10O6GnjqWn+lVdfuLA42 8wfkZZQOODX6ia/u8m34i0xKDWmnqyugbUJAUwqpnDHxfmxIJXjNzs9XwD0E/CmofO36Vxyln7W2w zCXwrymQh09FGWRUPjwPc+wIXAUBiTdsoUwWIlp75rssbtkcu2uKzI59F3EqA1uahehQaQEdIdsRX adSlv4lnxsaSP5vk2tNC5gTvVdwli8pdwzrxHvDcH3yRSX2jYvt23v8KK5rUJNamSG8NjTv4H0pv+ RnaUoCKOtUkdQelXrj1g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0WO-0000000ENa8-13lY; Thu, 10 Oct 2024 21:16:32 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrO-0000000DqVK-1rip for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:04 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e3204db795so21454187b3.2 for ; Thu, 10 Oct 2024 11:26:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584761; x=1729189561; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=+DlXoUFYFzJpPJut8ySNZ/avzuJs4Hk+oSmN6VTWgSk=; b=4EMLSx3Ftaw9gpvv4YrUXwm4Ks7qKUIi5Av2oi+legj4y7IQEeq9x4wlwZS+YWWx8t 0xit1JXqyQbkuGjTIvQTmIh9VVxkJnzZXwCY7NnCxL7Tzx0AQ0WqR/rS8TtjzL7zlSar vqNLhoD9LADMuTHQFqSPrwSjvccSQtNnO1Ox0zkU50RacgMm0RN7WZ6nwppBMtVycT3Q 5VtuL7yKy6hZPQuGehnhFA4hxkc7QeNABSIZ4rdhaX3h1YQOLR+4r+p6RuoQu94V8inI q5M+hfNWjQ/vrjfsKEFYLuelSGw5zlEhMvv7lcNZQL9Y3xwO0lH2zxpMwHOTTMe7gdJJ bUiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584761; x=1729189561; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=+DlXoUFYFzJpPJut8ySNZ/avzuJs4Hk+oSmN6VTWgSk=; b=lxc2dSgcPtsC+k4AuAR1c6Fsv5u1E4J/Z+NSYbrGK0IKyrL9xGfqmujmtLu1VY5Een GTUyMfRk7v5WMOOENO5wUIeFCIO+CbuT90253oWO5BIyC/euWggFgCyXIrObrGy2Mh64 tigwhvErWfPqBUXiQtN/JD2LxocZrx9rnPzCS/S5U5DuhApNanKxDYGiEBkkZ8mk2rhM Ecwyxs5AA60rJLuzP0IR/OOn2GJA6bz8Pd6TwY5hVeEgxicirNGYNAIMIE3kkBIvoiHn ArSvOJPoG/eqiipHCnEJ+gZSq9MPZW4VeGklOQ67U+HyDcZwamTqPDHfiKK345RCSL6D /BEg== X-Forwarded-Encrypted: i=1; AJvYcCXLreD6SRGvKxqwRDe6w4MGCjNNvIyqJvw0U3YqtgrW9W8hSaIZeXgLjmmO9LUPN4wcOv1xhL/wiPSY/w==@lists.infradead.org X-Gm-Message-State: AOJu0YyUtQT34EJl8ncbm9RO2+bFJ7zT6xNY+PZSCVqv9V1JBWLXeF6c KNOulTxJ+/bA84uT7rS8FHg/BCunpxJdzOscT+/wShJ1CAe3R0oTtULN3BuHhMc5wKt9vyYxadS AbQ== X-Google-Smtp-Source: AGHT+IHYBe/gylmYYB76Y4JbCGQqBxyU7NvKFk8CzbtQRqqOSH0kk5njqmBLFW7wxt+Okx9MKc9CHg/F7Og= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:4349:b0:6e2:1713:bdb5 with SMTP id 00721157ae682-6e32217cfd3mr118607b3.5.1728584760829; Thu, 10 Oct 2024 11:26:00 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:35 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-34-seanjc@google.com> Subject: [PATCH v13 33/85] KVM: Pass in write/dirty to kvm_vcpu_map(), not kvm_vcpu_unmap() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112602_571719_09F4B34F X-CRM114-Status: GOOD ( 16.44 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Now that all kvm_vcpu_{,un}map() users pass "true" for @dirty, have them pass "true" as a @writable param to kvm_vcpu_map(), and thus create a read-only mapping when possible. Note, creating read-only mappings can be theoretically slower, as they don't play nice with fast GUP due to the need to break CoW before mapping the underlying PFN. But practically speaking, creating a mapping isn't a super hot path, and getting a writable mapping for reading is weird and confusing. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/nested.c | 4 ++-- arch/x86/kvm/svm/sev.c | 2 +- arch/x86/kvm/svm/svm.c | 8 ++++---- arch/x86/kvm/vmx/nested.c | 16 ++++++++-------- include/linux/kvm_host.h | 20 ++++++++++++++++++-- virt/kvm/kvm_main.c | 12 +++++++----- 6 files changed, 40 insertions(+), 22 deletions(-) diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index d5314cb7dff4..9f9478bdecfc 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -922,7 +922,7 @@ int nested_svm_vmrun(struct kvm_vcpu *vcpu) nested_svm_vmexit(svm); out: - kvm_vcpu_unmap(vcpu, &map, true); + kvm_vcpu_unmap(vcpu, &map); return ret; } @@ -1126,7 +1126,7 @@ int nested_svm_vmexit(struct vcpu_svm *svm) vmcb12->control.exit_int_info_err, KVM_ISA_SVM); - kvm_vcpu_unmap(vcpu, &map, true); + kvm_vcpu_unmap(vcpu, &map); nested_svm_transition_tlb_flush(vcpu); diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 0b851ef937f2..4557ff3804ae 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -3468,7 +3468,7 @@ void sev_es_unmap_ghcb(struct vcpu_svm *svm) sev_es_sync_to_ghcb(svm); - kvm_vcpu_unmap(&svm->vcpu, &svm->sev_es.ghcb_map, true); + kvm_vcpu_unmap(&svm->vcpu, &svm->sev_es.ghcb_map); svm->sev_es.ghcb = NULL; } diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 9df3e1e5ae81..c1e29307826b 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -2299,7 +2299,7 @@ static int vmload_vmsave_interception(struct kvm_vcpu *vcpu, bool vmload) svm_copy_vmloadsave_state(vmcb12, svm->vmcb); } - kvm_vcpu_unmap(vcpu, &map, true); + kvm_vcpu_unmap(vcpu, &map); return ret; } @@ -4714,7 +4714,7 @@ static int svm_enter_smm(struct kvm_vcpu *vcpu, union kvm_smram *smram) svm_copy_vmrun_state(map_save.hva + 0x400, &svm->vmcb01.ptr->save); - kvm_vcpu_unmap(vcpu, &map_save, true); + kvm_vcpu_unmap(vcpu, &map_save); return 0; } @@ -4774,9 +4774,9 @@ static int svm_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram) svm->nested.nested_run_pending = 1; unmap_save: - kvm_vcpu_unmap(vcpu, &map_save, true); + kvm_vcpu_unmap(vcpu, &map_save); unmap_map: - kvm_vcpu_unmap(vcpu, &map, true); + kvm_vcpu_unmap(vcpu, &map); return ret; } diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index ff83b56fe2fa..259fe445e695 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -231,7 +231,7 @@ static inline void nested_release_evmcs(struct kvm_vcpu *vcpu) struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(vcpu); struct vcpu_vmx *vmx = to_vmx(vcpu); - kvm_vcpu_unmap(vcpu, &vmx->nested.hv_evmcs_map, true); + kvm_vcpu_unmap(vcpu, &vmx->nested.hv_evmcs_map); vmx->nested.hv_evmcs = NULL; vmx->nested.hv_evmcs_vmptr = EVMPTR_INVALID; @@ -318,9 +318,9 @@ static void nested_put_vmcs12_pages(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); - kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, true); - kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); - kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); + kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map); + kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map); + kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map); vmx->nested.pi_desc = NULL; } @@ -624,7 +624,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, int msr; unsigned long *msr_bitmap_l1; unsigned long *msr_bitmap_l0 = vmx->nested.vmcs02.msr_bitmap; - struct kvm_host_map msr_bitmap_map; + struct kvm_host_map map; /* Nothing to do if the MSR bitmap is not in use. */ if (!cpu_has_vmx_msr_bitmap() || @@ -647,10 +647,10 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, return true; } - if (kvm_vcpu_map(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), &msr_bitmap_map)) + if (kvm_vcpu_map_readonly(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), &map)) return false; - msr_bitmap_l1 = (unsigned long *)msr_bitmap_map.hva; + msr_bitmap_l1 = (unsigned long *)map.hva; /* * To keep the control flow simple, pay eight 8-byte writes (sixteen @@ -714,7 +714,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, MSR_IA32_FLUSH_CMD, MSR_TYPE_W); - kvm_vcpu_unmap(vcpu, &msr_bitmap_map, false); + kvm_vcpu_unmap(vcpu, &map); vmx->nested.force_msr_bitmap_recalc = false; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 8739b905d85b..9263375d0362 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -285,6 +285,7 @@ struct kvm_host_map { void *hva; kvm_pfn_t pfn; kvm_pfn_t gfn; + bool writable; }; /* @@ -1312,8 +1313,23 @@ void mark_page_dirty(struct kvm *kvm, gfn_t gfn); struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu); struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn_t gfn); kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); -int kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *map); -void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty); + +int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *map, + bool writable); +void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map); + +static inline int kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, + struct kvm_host_map *map) +{ + return __kvm_vcpu_map(vcpu, gpa, map, true); +} + +static inline int kvm_vcpu_map_readonly(struct kvm_vcpu *vcpu, gpa_t gpa, + struct kvm_host_map *map) +{ + return __kvm_vcpu_map(vcpu, gpa, map, false); +} + unsigned long kvm_vcpu_gfn_to_hva(struct kvm_vcpu *vcpu, gfn_t gfn); unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *writable); int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data, int offset, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 40a59526d466..080740f65061 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3116,7 +3116,8 @@ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) } EXPORT_SYMBOL_GPL(gfn_to_page); -int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) +int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map, + bool writable) { struct kvm_follow_pfn kfp = { .slot = gfn_to_memslot(vcpu->kvm, gfn), @@ -3130,6 +3131,7 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) map->page = NULL; map->hva = NULL; map->gfn = gfn; + map->writable = writable; map->pfn = kvm_follow_pfn(&kfp); if (is_error_noslot_pfn(map->pfn)) @@ -3146,9 +3148,9 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) return map->hva ? 0 : -EFAULT; } -EXPORT_SYMBOL_GPL(kvm_vcpu_map); +EXPORT_SYMBOL_GPL(__kvm_vcpu_map); -void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) +void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map) { if (!map->hva) return; @@ -3160,11 +3162,11 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) memunmap(map->hva); #endif - if (dirty) + if (map->writable) kvm_vcpu_mark_page_dirty(vcpu, map->gfn); if (map->pinned_page) { - if (dirty) + if (map->writable) kvm_set_page_dirty(map->pinned_page); kvm_set_page_accessed(map->pinned_page); unpin_user_page(map->pinned_page); From patchwork Thu Oct 10 18:23:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831167 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2AF79D24454 for ; Thu, 10 Oct 2024 21:16:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=m9RFsLbHcB/VBBFEi9BhyNrmyDTNNyLNmiHX+RkU6GM=; b=RwoJLdfSPB5AhN ybHnc0w1sTRWpY0N5urcbFA37JdCrWf32vULixH3qCGmN7D5cu79SlzQU0pHa0yv4TERxltE7Ks4x cvQ8yBsZOhcfrUpjbOB+xEBcCd0mJSbCDFVg+Vwq24UMJ/CN+ajnqNB4tLXMLqDxS+Kmv/dYUCXZe YzJEQJaKogLzU9SMlwXhpqsoJRaOL5F6p6X67eGdO0kWjATEPaVPBlRRiYC4SiJxQeybfgWeERtPC MrjejDHXoCX6e/ztkHza+5R4XWvjD5dVr0rYEvUaUsyh7sIbMHK9vm42VJshAp5jlPjQQMc6CB/tp TPp0khIsuEDM4rb60LAg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0WP-0000000ENbY-3qJE; Thu, 10 Oct 2024 21:16:33 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrQ-0000000DqX3-0PeK for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:06 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-71e00c8adf9so1318775b3a.1 for ; Thu, 10 Oct 2024 11:26:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584763; x=1729189563; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=w3u38MFKj4Hgu0NiviVwRjtFpk/ioMNBGBDQlt4DAgY=; b=wwsVmKdK+SipFg1qIF50TrdP6Oo7jO28ReLYz/VDjQtbPUYfeyWnk0GVNrM0uvKMns yzO1p2PHSyt6/6J6LjJccz4eC9X33n2N2DFqSywzlwGAsAWPs0+b7iefqw5tZpUARPQu 9OG64S1K6L+hc8UcCB61w0V2JL630JzshC0mLEAVFvK8USjgHGCuqzpy7gkmEW0tty66 NRob1YtQ6GGUHdVw6r2QY0qsQ4+DlBD5nwBCrRtcLzLwX9krXorOWvD+nF58Mr7YwjaH LDcNdo4BOKyAvdYIXqY2QFZaKTbi1CdYfH/QTc06gv+b8kc00QbcJRcZEUai+QUcejZU 3v4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584763; x=1729189563; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=w3u38MFKj4Hgu0NiviVwRjtFpk/ioMNBGBDQlt4DAgY=; b=vaZiwV0bifM9vHEGG+Bw+1yCSgszo0WMkKAZv+Lg0TEGnOlGmZseuq19u9w/h9iC3P NhR1Q1yY4JHXIw7xPK+YtkXKx6QrlGg7ZaNQVl7cvM9aBXBc4HreXfE9g3NbzX7ryKuU q2ZCGUwcpOyYAXsFgESrVd6cUMVvDXMEGtfwMOj7Fp1kiz2oF0EmQfSGTMYUBMDf1K/x 50PFr8jXL9MvV/m7/G9pQs84IEnkw/cGxLSBDczleo46SbGLpN9t++rUg4th0v6LOETc CM6Mp8qEMFEH6FkC1bgLIWpvui5SUEKmm4QkbAwFBiBghQrQqo/TA0R5ama+4RU2q2eP KFoQ== X-Forwarded-Encrypted: i=1; AJvYcCX//CUvbU6Jnax3aW0FuZIAoW979xvBea/8dn9uoNZphaNUaHxcmOmIALW8vP020kxgzzgFeYjfWy2AuQ==@lists.infradead.org X-Gm-Message-State: AOJu0YwJfeb/8ShxH50+M/vVALKBPBYNv0UXNyfx+mYOIwMA6/B6y8B+ bia072HuJBrGv8S1rMd5oAZDyH+FldFAFDQKIUT9c9scEA/onxM/rsZ4o0AmWY7fOlesa6FmNix +JQ== X-Google-Smtp-Source: AGHT+IHthU1bxxWE76UmRU/cmSB/2b+tJYVulTyd1SGmZGLYBHQFEbukUNmDJAlwNs2CFth+SfvGFADhDCI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2f51:b0:71e:dd:8f9b with SMTP id d2e1a72fcca58-71e1dbe8750mr6976b3a.5.1728584762866; Thu, 10 Oct 2024 11:26:02 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:36 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-35-seanjc@google.com> Subject: [PATCH v13 34/85] KVM: Get writable mapping for __kvm_vcpu_map() only when necessary From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112604_231226_81B9562F X-CRM114-Status: GOOD ( 10.35 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org When creating a memory map for read, don't request a writable pfn from the primary MMU. While creating read-only mappings can be theoretically slower, as they don't play nice with fast GUP due to the need to break CoW before mapping the underlying PFN, practically speaking, creating a mapping isn't a super hot path, and getting a writable mapping for reading is weird and confusing. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 080740f65061..b845e9252633 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3122,7 +3122,7 @@ int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map, struct kvm_follow_pfn kfp = { .slot = gfn_to_memslot(vcpu->kvm, gfn), .gfn = gfn, - .flags = FOLL_WRITE, + .flags = writable ? FOLL_WRITE : 0, .refcounted_page = &map->pinned_page, .pin = true, }; From patchwork Thu Oct 10 18:23:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831312 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 114DDD24459 for ; Thu, 10 Oct 2024 23:31:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Lc2xoNXRtpGmTofFpPNwLwTx32tqwU/oquAiyzzOPDw=; b=oYkxyIVVU7Xa1k rybxygPhos+TRUpltUiLSRwlWMaM6PeQDUPqavZeysayjVBJAOZ0Or3gw0Vmd1LU4eO0/ZXq6lV2y 1spWMtgjZ3umbWe9swe7x6J3gLF7gc6NDmcSnDeolJVvHxlQ2KC54wXLUoRtYdO3EYJFH6ZFxndep 4iyb508sjbgZm7PQ7snr3+1g0zXKT7AZJjx5sp6d5zsGBXtmCT02UUnCNay2KZXVAwii1cAHlRNDM X7uJqkOr9+jcFq+wFOj4S5PtihoaYnmbUlRn4cFcLT/d7WwJbJeEYZf2Hu2z1OWQg8dkhgsgWafvN BAbz4+PL08HNzqekRKvg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz2cy-0000000Ef4V-2rM2; Thu, 10 Oct 2024 23:31:28 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrS-0000000DqYk-0ra1 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:08 +0000 Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-7e9fb5352dfso1520878a12.3 for ; Thu, 10 Oct 2024 11:26:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584765; x=1729189565; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=ByBZxBThzycQ9+5cWv7LHsEIs8PvxVG+sWS1etQw10Y=; b=VR8H35J0pZub8PS/JM9vp2UKN2Zg/mExpEpDuTzNIH+FW762bLByCaHMK7vSiwjC4T x8MhTpwSZld0moJyvFJhg/1ohUP78VLZJ2X6yOnBjBPRs6ofdVEEpZ+NJRm6GxM2xpIF vvKfeZIRuv9EOalwyRkzpan94NKDtcJUocISdZUZPJJRPvvy2UgbHhkgYUXorzexJZLT NrdOa2be9cJP1lh0sOESqVIGY224l57JOIbn+rYAnjihqpPZGA1yK9opXvvM7mbxD+LE 3SPXzF81rFiZFm76cK9RPr7dhR2qr/NnlTVK8HwKmkY+DzSCLdf0+4721hzjhid+NXrL 3Crg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584765; x=1729189565; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=ByBZxBThzycQ9+5cWv7LHsEIs8PvxVG+sWS1etQw10Y=; b=hOlAz/UN6TmVijU5amNha9CfMrLdQSiogvv1oWvuoB5yQBfO4JF0SRoxD7wCnVwnP2 KRvTVVBjaEFPm/FBpQnum7+uLFwuBdp3Jdifye/4tErnQCreZKw8DuKeOTcoArgTZ49L dob0ZtCQVTJiSMjUW0VC35suM9PZqgeoCYTxVZmjhjdT3AKXpPCrFeYhysVzoo/f3xYy 6htW+BNiUZnNsMkmRb6kG5wFO+o7tuNoC8Baus65uejY+R8y9D/aovEOspew95U7iSeG M+QifhwF+3rnjgy/3aR/Rnc3836a/2DUHhc2UrbGU3Xp5efOo5c6TKB47Sw1T8+SbZYI J4bg== X-Forwarded-Encrypted: i=1; AJvYcCUooy4O51osrXWB6QCkBhH5uYOWsKsSqQ6vhTSqJaASIXXVg0KLbtBb/J6z55AuXrUzmSy/5KfC9iFr2w==@lists.infradead.org X-Gm-Message-State: AOJu0Yw7If4gGnm5N2xSAFgmkjqqemFH5tB4YSdoowkqC2BKSIIp093v FBx4NeGG/yGui2bYyw1xzRN88hIXAvoiHslpnuYcCwYCY5jCGmcPA9SDDPKfOcMEcXqn30u5GSs CYw== X-Google-Smtp-Source: AGHT+IE23QkhjcWr/3QASJ/S9+dapZUNy7Z4CCepJKqapzuUx8eBu6OhWGfM5laLrlsw47tRhKuFSYugMsQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:ff61:0:b0:7cb:c8c3:3811 with SMTP id 41be03b00d2f7-7ea5356a658mr52a12.5.1728584764791; Thu, 10 Oct 2024 11:26:04 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:37 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-36-seanjc@google.com> Subject: [PATCH v13 35/85] KVM: Disallow direct access (w/o mmu_notifier) to unpinned pfn by default From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112606_373308_FF01BC20 X-CRM114-Status: GOOD ( 14.98 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Add an off-by-default module param to control whether or not KVM is allowed to map memory that isn't pinned, i.e. that KVM can't guarantee won't be freed while it is mapped into KVM and/or the guest. Don't remove the functionality entirely, as there are use cases where mapping unpinned memory is safe (as defined by the platform owner), e.g. when memory is hidden from the kernel and managed by userspace, in which case userspace is already fully trusted to not muck with guest memory mappings. But for more typical setups, mapping unpinned memory is wildly unsafe, and unnecessary. The APIs are used exclusively by x86's nested virtualization support, and there is no known (or sane) use case for mapping PFN-mapped memory a KVM guest _and_ letting the guest use it for virtualization structures. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index b845e9252633..6dcb4f0eed3e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -94,6 +94,13 @@ unsigned int halt_poll_ns_shrink = 2; module_param(halt_poll_ns_shrink, uint, 0644); EXPORT_SYMBOL_GPL(halt_poll_ns_shrink); +/* + * Allow direct access (from KVM or the CPU) without MMU notifier protection + * to unpinned pages. + */ +static bool allow_unsafe_mappings; +module_param(allow_unsafe_mappings, bool, 0444); + /* * Ordering of locks: * @@ -2811,6 +2818,9 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, * reference to such pages would cause KVM to prematurely free a page * it doesn't own (KVM gets and puts the one and only reference). * Don't allow those pages until the FIXME is resolved. + * + * Don't grab a reference for pins, callers that pin pages are required + * to check refcounted_page, i.e. must not blindly release the pfn. */ if (map) { pfn = map->pfn; @@ -2929,6 +2939,14 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, bool write_fault = kfp->flags & FOLL_WRITE; int r; + /* + * Remapped memory cannot be pinned in any meaningful sense. Bail if + * the caller wants to pin the page, i.e. access the page outside of + * MMU notifier protection, and unsafe umappings are disallowed. + */ + if (kfp->pin && !allow_unsafe_mappings) + return -EINVAL; + r = follow_pfnmap_start(&args); if (r) { /* From patchwork Thu Oct 10 18:23:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830901 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BCA4DCFC5F8 for ; Thu, 10 Oct 2024 19:08:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=IbrL0P7R5W29256kjSMKEuzaBOVSGtkLbr06InJRoMU=; b=C0qfOsGO4WP/Mn JYyEE2YyuEpys1fgYazLcOENUlftEyUPEi7F+vAJyn2tjMnmnYK/K58uvvwVhbo/hIZypaQmUTYt4 9fRNpLKNhjPhKJ2bAek+sexjcdFdQXjyFigKsbbMQMJHR8TKSWhNODFMA9/slXSJJNY/UbzfeDUwe +C8ThSQvcnkVG9bsSVfpoP/l82fy4us3HDlW6yZ8fsExkXb4ShKW8DuUdy12MAcB4N6Vuzeg6eNuN MAHTmSUATkPkZzzrMRDzunXWHuz1dzMymh0B5P0oMSOk7YeKUJQ3UsNPMjD1BSFOKX1eX2hLawUWF CXOy/OkkWSf6El0thSZQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyWm-0000000E386-1eJF; Thu, 10 Oct 2024 19:08:48 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrU-0000000DqaP-1STM for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:09 +0000 Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-71df07ac9faso1221367b3a.1 for ; Thu, 10 Oct 2024 11:26:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584767; x=1729189567; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=n3l8C52m2RjpLTXcY6dq+UIBo01A1fTC4GBMzDz1kHI=; b=hbUUm6huEsIpvBlGvvHRI3wy9VneLaobE+5HtYZY5ER96gyN7mdBt227GEUK5zzVMn 76vbGKJ41cFK+Ysmvcp7yzuv7ili+YPUjYOxE/ctNAbuu4Nnb7yjxtQS1drQiUhAjqUc Ip97sqxSr37TeirbDgYFFRiSaHoci9HOuvGFeZA6n046j+szLpL2tKQh7Ei2hE6WbqkS ntJU0pnF07qDfzHcM1/OBCL3m4W69iEVbt8ImxGUxYS+T3YNq0sWzfDSXc6BIWrgACxA cLY+/1MsX2oQAbg3Kc7OdW631minPkxe5BLmqpeshd1NcocBBwJNRd9/VTPro4cW+odU d97A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584767; x=1729189567; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=n3l8C52m2RjpLTXcY6dq+UIBo01A1fTC4GBMzDz1kHI=; b=A7aXeTj5qKQxg03pVoLa7mzeynB6lyI/YMkXeA7le2Jg3cMO5E88hvrGDjqMlTpdp5 L5ROJtX/sz2mraR2urrKLET0ybTuC2zj/B5ygbxEW4JZ+XoxgrOpTP0bIViyA7Tc33ZC y88hbOercD8wcBWXyczivQu+G6ALYB1DIbhTSjtlSmr6PuLpLVnLrwE+wJhiVJU+kBCS dVfDwO2wJ+9RaJnEkl+FRAu2Bzd6miXcTUyYJ+MYaUSoNPau5hs/U4FB1yGn4OwQeCZV +Yl+HUXz1pbNlf9X4id8eTZfI5y/oj7zbbeczMBoG87T0jJgcaWs8pP7LYMR51NTwrYM dRbw== X-Forwarded-Encrypted: i=1; AJvYcCWe/2IFSQLzrVKjECHWXI9/Pl/A7O886hM70f/zERe8Da+8wXSuY4YKCg7OqH75J/bT+fgOHvyy270dFA==@lists.infradead.org X-Gm-Message-State: AOJu0Yy77ehOyWnGKQ3GlOHkNYWCQJRzDI9u+YFoCP05LpOP6jxRhB5g fJ3Lx+eI3kRNJNRMGZVowcvH6dkIECGGw6Hn3Yi5BXRQ9M8W7lWfDE1l5nrHcwQP6e4PCIO2P4m WRg== X-Google-Smtp-Source: AGHT+IH5y4h19aQ8D3UFZNHEjFs0l+IkJ/F+GwDOdY5CLNZ6WKY+eP+zfpO4DiXt2PfVXCIgSRJOtCb+AeE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:6f44:b0:71e:268b:845e with SMTP id d2e1a72fcca58-71e26e53c16mr11733b3a.1.1728584766802; Thu, 10 Oct 2024 11:26:06 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:38 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-37-seanjc@google.com> Subject: [PATCH v13 36/85] KVM: x86: Don't fault-in APIC access page during initial allocation From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112608_471574_7335FB84 X-CRM114-Status: GOOD ( 10.14 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Drop the gfn_to_page() lookup when installing KVM's internal memslot for the APIC access page, as KVM doesn't need to immediately fault-in the page now that the page isn't pinned. In the extremely unlikely event the kernel can't allocate a 4KiB page, KVM can just as easily return -EFAULT on the future page fault. Suggested-by: Paolo Bonzini Signed-off-by: Sean Christopherson --- arch/x86/kvm/lapic.c | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 20526e4d6c62..65412640cfc7 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2647,7 +2647,6 @@ void kvm_apic_update_apicv(struct kvm_vcpu *vcpu) int kvm_alloc_apic_access_page(struct kvm *kvm) { - struct page *page; void __user *hva; int ret = 0; @@ -2663,17 +2662,6 @@ int kvm_alloc_apic_access_page(struct kvm *kvm) goto out; } - page = gfn_to_page(kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT); - if (!page) { - ret = -EFAULT; - goto out; - } - - /* - * Do not pin the page in memory, so that memory hot-unplug - * is able to migrate it. - */ - put_page(page); kvm->arch.apic_access_memslot_enabled = true; out: mutex_unlock(&kvm->slots_lock); From patchwork Thu Oct 10 18:23:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830902 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 252FACFC5F9 for ; Thu, 10 Oct 2024 19:10:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=dnMjDDphgmIAm7paQZjZKoXBpwBvkG2jcwklV7zG3Bs=; b=UO+UAgvT226aWv b06U/1QZrM2Z4dfEq/iJksGNrhIfoVfJxAq8Ak6mPWJLS60kz8nnPfscCmjLhK3XsYTbXaQnuEmbR o9iBvwoAiJeEMHDS2/ogaPwQQSI2mR7BewBlCXmaRbeJujHtfuY+4wzFP6CrkdMOgJTyBOF4VLvDA uKJeDNmG/bo38ErDjF8mm28gMxJML3oV8wZgnY/SQYShypcbc1ulq/X9pf2r/4y2z5xWqCn+jj995 2OGVqF1T8FAoAn5vOsO/pYPl6sct7Zh6yTEl7Q4i/JV+UdQOulN4xy3QLZT9Egp4MX+8Jy4yaiRVM iXQZCrt2IJ/SmmBpCaZQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyYF-0000000E3cE-2meK; Thu, 10 Oct 2024 19:10:19 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrW-0000000Dqbw-2DW5 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:11 +0000 Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-20c8b0b0736so7708555ad.3 for ; Thu, 10 Oct 2024 11:26:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584769; x=1729189569; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=551DSOChXG5A+yDanUyc1gC1jxg9ryEf27npUXDW1CE=; b=hHRUq768hWMB0snk3BTDf56BfbwlI/wpDGgFhaIx6MEk7ikWWWPixPGn5ajCAfoaG7 0pbQvb2docQGGHhQNGOEELyTo7Vn6Kg+2FK1Ado67d3dpb9nyxslxTWPdbIhCjSBDz55 ZZHh5rAO98OLYQq30LZnkOFZ8o9x0nvVNieVl9SGp8g1LHKYjOx0z8KSEtcXQHgXxI5a ssz873LT3liXqRaru9m8P5P+ghEymZ2dHwIcTCN5DQoQB/SedtOHs1Ul/MYoHZ+tTuqg rAtROSmEeH/8XJgjKuysN/PJsY+C2BYPd5iklkCaHOn7gwmpxw+U9/Pxtu4Mfcnf5wVt B9Hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584769; x=1729189569; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=551DSOChXG5A+yDanUyc1gC1jxg9ryEf27npUXDW1CE=; b=K37YjCqNDS9NfUQa+AErUGeMpjKXBBM3fFO6MtzVNF5TvLqoz2HarJQR3FjkVpf4mh olsA7njp3a2uAnvG6kVpNdZj2ub862uc4TfbQZImkP95upx6okM0+g06ySBX4cteV9Hj saJMmnFXGD7eYRaTEjAeOmWzg6kRYfFEVxQR812UBjPXbNJ1IBkb5jRbWZxz71eCGLTG +k3qrCL9hrteiN/6kRQbJ9KYgCH2ksBUUZjdpg1Q/CbLtJUzRvBru1G5HdQux/hlLSbs eRIqTdHMOMBJsk2GW82QaEpYZ3Po1v14TOm3hdYzKnvEP6QLVJzgRBAS3PCGMkSDqyol df+g== X-Forwarded-Encrypted: i=1; AJvYcCUA+38Ec/rN+7qThh3DMklDyEFk5oc/9mV7iZcWWFG7LDFqV4fjE6kdpBx9SlzJpyIBiqL+0JprGox1RQ==@lists.infradead.org X-Gm-Message-State: AOJu0YzF0AnQaqkWaFdM9TGjpKrjSUwbGdjUMYwrOHMroz6vlPvBOvVn 81+55wQ2WGrXl0a1V88lDaGQKRqGi92ZIDH/AuRVT996F1vI6pCNh2blEh2b5gUDWhqWrSWKmKz EsA== X-Google-Smtp-Source: AGHT+IGKm7yq47izgr1FmkV+xWCQVhrzT+Qt86SXT8qqzZ7digIfsw6vfVxyVo72E4861BiCHpsXRIV5BzI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:e80f:b0:1fa:2ae7:cc6a with SMTP id d9443c01a7336-20c63722d7fmr648865ad.4.1728584768900; Thu, 10 Oct 2024 11:26:08 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:39 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-38-seanjc@google.com> Subject: [PATCH v13 37/85] KVM: x86/mmu: Add "mmu" prefix fault-in helpers to free up generic names From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112610_618815_9F97831D X-CRM114-Status: GOOD ( 12.05 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Prefix x86's faultin_pfn helpers with "mmu" so that the mmu-less names can be used by common KVM for similar APIs. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 19 ++++++++++--------- arch/x86/kvm/mmu/mmu_internal.h | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- 3 files changed, 12 insertions(+), 11 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 28f2b842d6ca..e451e1b9a55a 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4347,8 +4347,8 @@ static u8 kvm_max_private_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, return max_level; } -static int kvm_faultin_pfn_private(struct kvm_vcpu *vcpu, - struct kvm_page_fault *fault) +static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault) { int max_order, r; @@ -4371,10 +4371,11 @@ static int kvm_faultin_pfn_private(struct kvm_vcpu *vcpu, return RET_PF_CONTINUE; } -static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) +static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault) { if (fault->is_private) - return kvm_faultin_pfn_private(vcpu, fault); + return kvm_mmu_faultin_pfn_private(vcpu, fault); fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, fault->write, &fault->map_writable); @@ -4409,8 +4410,8 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault return RET_PF_CONTINUE; } -static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, - unsigned int access) +static int kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault, unsigned int access) { struct kvm_memory_slot *slot = fault->slot; int ret; @@ -4493,7 +4494,7 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, if (mmu_invalidate_retry_gfn_unsafe(vcpu->kvm, fault->mmu_seq, fault->gfn)) return RET_PF_RETRY; - ret = __kvm_faultin_pfn(vcpu, fault); + ret = __kvm_mmu_faultin_pfn(vcpu, fault); if (ret != RET_PF_CONTINUE) return ret; @@ -4570,7 +4571,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault if (r) return r; - r = kvm_faultin_pfn(vcpu, fault, ACC_ALL); + r = kvm_mmu_faultin_pfn(vcpu, fault, ACC_ALL); if (r != RET_PF_CONTINUE) return r; @@ -4661,7 +4662,7 @@ static int kvm_tdp_mmu_page_fault(struct kvm_vcpu *vcpu, if (r) return r; - r = kvm_faultin_pfn(vcpu, fault, ACC_ALL); + r = kvm_mmu_faultin_pfn(vcpu, fault, ACC_ALL); if (r != RET_PF_CONTINUE) return r; diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index 633aedec3c2e..59e600f6ff9d 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -235,7 +235,7 @@ struct kvm_page_fault { /* The memslot containing gfn. May be NULL. */ struct kvm_memory_slot *slot; - /* Outputs of kvm_faultin_pfn. */ + /* Outputs of kvm_mmu_faultin_pfn(). */ unsigned long mmu_seq; kvm_pfn_t pfn; bool map_writable; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 143b7e9f26dc..9bd3d6f5db91 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -812,7 +812,7 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault if (r) return r; - r = kvm_faultin_pfn(vcpu, fault, walker.pte_access); + r = kvm_mmu_faultin_pfn(vcpu, fault, walker.pte_access); if (r != RET_PF_CONTINUE) return r; From patchwork Thu Oct 10 18:23:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830908 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 17956CFC5F9 for ; Thu, 10 Oct 2024 19:11:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=D+NXRZEOQlRWOfJNuDdLxh/Kw3af5CgdS1a0xsSEdOk=; b=qclBH+vu98NGmt +3dUklJyj0/mBNlvGlq51uicv1CsrrTxG+JIqp8XR1TI+w3xqs49ip66twhMzfRJZpTHRg88E+uIF WLvJJArU1WXX+NoXk72Nf6eA2R8NodTzwPuPVcTGU5dFoqcN57YBDiPV0xTamhKWll4Ym/61VW1E/ g5E5ChRaWaqM+MZJsyJ/SZ8T2CXkzjWTL/0rG/2WkHnTKvqxijjpwczj/j/Z75g8CowUBScDZa1Fb bETqp1uAkPwfPkGeMFSzBPs5bfSDKjq92jc3JTqesJV+kP804dU1hdg1lCuKyiwrBk9d97c+iGfT/ hDylHR9Rd7+zMHd6j2jQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyZe-0000000E3zv-3eI8; Thu, 10 Oct 2024 19:11:46 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrZ-0000000DqeS-1ABV for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:14 +0000 Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-7ea38f581c9so1274182a12.0 for ; Thu, 10 Oct 2024 11:26:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584772; x=1729189572; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=UQkfEcPm4BCGA2NCPxYbIcRTjXNaFQKnzNzKtIM0PSc=; b=a9tvknbu3mVAwdSk/bWW5Mn6EIDWzmh5YcjYhMVkkIs1GvQLH6PIVzkh/eOSJc+rpn 3ueOHIXlNOai8fANmCvUpo+xVDPEYFJTvnkaiYtt1Lx+b1NC0b6SUoONHAXEP/bKZSui rgPHbPmU/oMAwui5lHJhk4Vj2+lb3sZyIjFzW/OIQ4ntLUfMJmYF7D1ILFmOtChZCw6+ CTNBGGncSEfZ4h5yf5X2BrgnM3j86XxhiCzCWwarWP1oaMehmhQAqRF8CTp4GI62hfIu HitGGHrCddX+1k1ox1ejB/nmPZCOPRAWnV8+5pJzeDfC5Z7i3iI7WxVGBWshaWSaoTMj mfaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584772; x=1729189572; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=UQkfEcPm4BCGA2NCPxYbIcRTjXNaFQKnzNzKtIM0PSc=; b=Qzhk/kYxYHp5yWKpdFPGeEfY780+qFTg4dTDV0QYEC8P4swgcovAOoa/sfYGZKyyBt q3wY1McHQXZWiIMS2PV1FzxTVEk2W0+q26/DsvuyH1d94XBsz1d/Eck7n+xyYrH0B0YM iu3i5SyccIofK2UPbYER3X2A98k3kIzAZj4/fm26V0dTCD7usUr9Laxmsg7dvoTyAssn 2OyklI8OOActthON5qmO0iKEsP1O35qGr+cqd3JCAEW/Mputtqy8eB1tGIDVj9XqHhjm Dp+sbo1dIg0vYiiBVS6pFNlme/LB5y1FFZairBNoj2AWVXprZWQkGZQDmDSQeq2BRCRa U4kQ== X-Forwarded-Encrypted: i=1; AJvYcCULn+4i7pQ3jf870tQcLXAM7aQ0h1aXQwBCHBASbKqw67gqieRzmb5QjX9SPAEp7u3nZjkG6S3CXaL0bA==@lists.infradead.org X-Gm-Message-State: AOJu0YyQ755BgenGDt+vUDfh3RHCu3EAtUzifFttx9iSzIY8G1UlXvaY YVZxdBMUugJW1RmVgcr0WEd85DtARRkBoD/FfKVqOUya8oHI/n02jIfYcoSSkugh8eQh1gCC4vv zpQ== X-Google-Smtp-Source: AGHT+IHq9hr54ORGaKAudrJMY0Hy9bRYjmUvRYz8zAm0RAj3x+bD9YD3f2uWGLgDblXeQYOnnP141duUJXA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:1d03:0:b0:7db:539:893c with SMTP id 41be03b00d2f7-7ea535a65c6mr17a12.9.1728584770873; Thu, 10 Oct 2024 11:26:10 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:40 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-39-seanjc@google.com> Subject: [PATCH v13 38/85] KVM: x86/mmu: Put direct prefetched pages via kvm_release_page_clean() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112613_383986_F2A66D8E X-CRM114-Status: GOOD ( 10.94 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Use kvm_release_page_clean() to put prefeteched pages instead of calling put_page() directly. This will allow de-duplicating the prefetch code between indirect and direct MMUs. Note, there's a small functional change as kvm_release_page_clean() marks the page/folio as accessed. While it's not strictly guaranteed that the guest will access the page, KVM won't intercept guest accesses, i.e. won't mark the page accessed if it _is_ accessed by the guest (unless A/D bits are disabled, but running without A/D bits is effectively limited to pre-HSW Intel CPUs). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index e451e1b9a55a..62924f95a398 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2965,7 +2965,7 @@ static int direct_pte_prefetch_many(struct kvm_vcpu *vcpu, for (i = 0; i < ret; i++, gfn++, start++) { mmu_set_spte(vcpu, slot, start, access, gfn, page_to_pfn(pages[i]), NULL); - put_page(pages[i]); + kvm_release_page_clean(pages[i]); } return 0; From patchwork Thu Oct 10 18:23:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830909 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 86FC0CFC5FB for ; Thu, 10 Oct 2024 19:13:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=PK0F5NNpRY1szFbOVcwu+xmmY3jSitclElANjvMK+jI=; b=VRYDZYylolgfSF FC7p5uG5cQJ7ti2QxjKzHfx4T0Ny3GBh+aiu95s9hLtlkkbN707WlvtzYG/nJG/BAPY57nwI+T9UK Z97zJV55qiY9c/wmH6rpzes1noAOU9whCM7+Ldb4MTgQOcLl5P3MQ7EHSV2un9oDUus18yxwAsDNm fQMD97q8ZRRZCkotqeon5n0P0aSQGF31GVbtnEkUzTRp3TeHn2I0R+K38sCpSqLr951xlDh+22WZg VnVT32hEhhHrFSbxFKztwJCoDQfWo56nvLJ+dz2uqF2BEsnkM0sf3NIoPlw2nj6rcFPtKBgSGObaK EiohSCGwsxMyFzSLozSw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyb5-0000000E4Rp-28GW; Thu, 10 Oct 2024 19:13:15 +0000 Received: from mail-pl1-x64a.google.com ([2607:f8b0:4864:20::64a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxra-0000000Dqfd-3xAl for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:16 +0000 Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-20c3d9a0eb2so16543075ad.0 for ; Thu, 10 Oct 2024 11:26:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584774; x=1729189574; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=bLMllFEXz5NbCB79T5EDeXxXCOdT0E/lPN9LlBgR2FM=; b=iqLFJk08lU4sXCqW4q3PDDrFFs5ZbTmdrSdsuuvf3BYW0ezc3qn0B+PT1xdp95wJKP v8vCmQJzR9uGRWWX3xM5EFUHYKi2EyUNEbk9tyfb71n60zeMppgYPZ/Vj4gdSD5N6RvJ FkgKdr3Z6z8/NYvrrP30gumqXo23lNuT93fQVxGcHiqCRnlW4E/mnK7E1UGmNlyGu+b8 Y5W97QXCMKKOXRoadOfj7pRQ5An6esrBRxEqAAv4FVktbDLfZXuA1uVEw7KCqLLpPIOx JbdwK5ygGHHjecSj53zztRzMuX8uHhsU/M+w90oMCOX27vE9+USr78CsT5jhtEz3h4c7 2qkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584774; x=1729189574; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=bLMllFEXz5NbCB79T5EDeXxXCOdT0E/lPN9LlBgR2FM=; b=AE4JgjxZbTJrvgkKWNQUa/OEBy06ys0UdbOmHga/Fm2YH0rQt87EFtzVu9A82ruZzO ///uT9b4SCuGFaoOvqgTnJZpxCCZSO8/s7KnILuDfO/nDHw5wOw5Lb1wj+DprZF9l+87 2BxRMHPEd5lHvqpo2o/F+J3waXLLbREndWuOFUFgHZyfXMFE/ogli/kU31lFG3f+FgRN coA8o+57smGoUugNWBtqIuBTkxRMhFqm2nsy96ZjlN5c4LwGHeKyfAsDf5ddFeZeKXhw goalLgrut7qxq0bAr6SWFuJzynFkLXcqpY1c7Iw8RIewHzduWsnHAYrvxxTgmncIgyrg FOKw== X-Forwarded-Encrypted: i=1; AJvYcCUCP4JK6kc2oGnhRRfQ6q02AQzvBHWFUlYc6dIXfAeu8hZWFS9lkxJFXhb0N6+mCtQBEQiR3/M3a4X+Hg==@lists.infradead.org X-Gm-Message-State: AOJu0Yz8hknjqd8qW51GHgLD9V5oltAwAIYCybEmF0Vp+J8GmU6T5BHs 9yUaGudV8WG/e6kcEVBBY4jreP1hPQ5+BrFxJ8NfhVGRrfM3hPz4A+BDGrSUz1t6vhGeQEppjgY czw== X-Google-Smtp-Source: AGHT+IHTeV3dFyfdACUyzit3PmgjtoD4THwZ77SaAB8Cv7KtK6sEvfc4IjBdPT+nDHGdhMcMXcKC7R1qaBw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:1c8:b0:20c:716c:5af with SMTP id d9443c01a7336-20c716c078dmr946605ad.3.1728584773496; Thu, 10 Oct 2024 11:26:13 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:41 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-40-seanjc@google.com> Subject: [PATCH v13 39/85] KVM: x86/mmu: Add common helper to handle prefetching SPTEs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112615_045138_94955CB4 X-CRM114-Status: GOOD ( 13.48 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Deduplicate the prefetching code for indirect and direct MMUs. The core logic is the same, the only difference is that indirect MMUs need to prefetch SPTEs one-at-a-time, as contiguous guest virtual addresses aren't guaranteed to yield contiguous guest physical addresses. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 40 +++++++++++++++++++++------------- arch/x86/kvm/mmu/paging_tmpl.h | 13 +---------- 2 files changed, 26 insertions(+), 27 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 62924f95a398..65d3a602eb2c 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2943,32 +2943,41 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct kvm_memory_slot *slot, return ret; } -static int direct_pte_prefetch_many(struct kvm_vcpu *vcpu, - struct kvm_mmu_page *sp, - u64 *start, u64 *end) +static bool kvm_mmu_prefetch_sptes(struct kvm_vcpu *vcpu, gfn_t gfn, u64 *sptep, + int nr_pages, unsigned int access) { struct page *pages[PTE_PREFETCH_NUM]; struct kvm_memory_slot *slot; - unsigned int access = sp->role.access; - int i, ret; - gfn_t gfn; + int i; + + if (WARN_ON_ONCE(nr_pages > PTE_PREFETCH_NUM)) + return false; - gfn = kvm_mmu_page_get_gfn(sp, spte_index(start)); slot = gfn_to_memslot_dirty_bitmap(vcpu, gfn, access & ACC_WRITE_MASK); if (!slot) - return -1; + return false; - ret = kvm_prefetch_pages(slot, gfn, pages, end - start); - if (ret <= 0) - return -1; + nr_pages = kvm_prefetch_pages(slot, gfn, pages, nr_pages); + if (nr_pages <= 0) + return false; - for (i = 0; i < ret; i++, gfn++, start++) { - mmu_set_spte(vcpu, slot, start, access, gfn, + for (i = 0; i < nr_pages; i++, gfn++, sptep++) { + mmu_set_spte(vcpu, slot, sptep, access, gfn, page_to_pfn(pages[i]), NULL); kvm_release_page_clean(pages[i]); } - return 0; + return true; +} + +static bool direct_pte_prefetch_many(struct kvm_vcpu *vcpu, + struct kvm_mmu_page *sp, + u64 *start, u64 *end) +{ + gfn_t gfn = kvm_mmu_page_get_gfn(sp, spte_index(start)); + unsigned int access = sp->role.access; + + return kvm_mmu_prefetch_sptes(vcpu, gfn, start, end - start, access); } static void __direct_pte_prefetch(struct kvm_vcpu *vcpu, @@ -2986,8 +2995,9 @@ static void __direct_pte_prefetch(struct kvm_vcpu *vcpu, if (is_shadow_present_pte(*spte) || spte == sptep) { if (!start) continue; - if (direct_pte_prefetch_many(vcpu, sp, start, spte) < 0) + if (!direct_pte_prefetch_many(vcpu, sp, start, spte)) return; + start = NULL; } else if (!start) start = spte; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 9bd3d6f5db91..a476a5428017 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -533,9 +533,7 @@ static bool FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, u64 *spte, pt_element_t gpte) { - struct kvm_memory_slot *slot; unsigned pte_access; - struct page *page; gfn_t gfn; if (FNAME(prefetch_invalid_gpte)(vcpu, sp, spte, gpte)) @@ -545,16 +543,7 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, pte_access = sp->role.access & FNAME(gpte_access)(gpte); FNAME(protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte); - slot = gfn_to_memslot_dirty_bitmap(vcpu, gfn, pte_access & ACC_WRITE_MASK); - if (!slot) - return false; - - if (kvm_prefetch_pages(slot, gfn, &page, 1) != 1) - return false; - - mmu_set_spte(vcpu, slot, spte, pte_access, gfn, page_to_pfn(page), NULL); - kvm_release_page_clean(page); - return true; + return kvm_mmu_prefetch_sptes(vcpu, gfn, spte, 1, pte_access); } static bool FNAME(gpte_changed)(struct kvm_vcpu *vcpu, From patchwork Thu Oct 10 18:23:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830910 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4A8FFCFC5F9 for ; Thu, 10 Oct 2024 19:13:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Q/hohgjh6yrJlZ3SghbJ2GQxy58R3QDiQB1iDQ7epI0=; b=FV4xLbHH9Hh3js U4M6mLNJiOfG8DSkT83IJrVBua4ngQb40H7iekyB3na6BvBTQfUCiZGq7SZZRCqYlII+leqKYgoYN Bzb2cPV9KmgcxWsTc4gdGN3PUKc6bissv0vz7mi46ctrKfkhJBaVgMx894CsUEsV5VmUqIfFG341A eYEKpPP6VTi9XM6oqaDZDzq+nYOX7NruNj+CKbowB7ZNQYg72PLfzO2fta8RKJkdoUo0YQvR2dCQ8 igJCusluQXAfo30Pj9EmkE0MxcKj8A1Fq5CeGJyYT88q/mH7+8HMSuSp6dwqHqo08V+FLEmnXLJGM NOBxTcMhlrotJTMpiN2Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyb6-0000000E4SF-0wmH; Thu, 10 Oct 2024 19:13:16 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrd-0000000DqhR-11Xo for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:18 +0000 Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-71e0228d71dso1225410b3a.0 for ; Thu, 10 Oct 2024 11:26:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584776; x=1729189576; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=GAJaAoo95YXsHy4joQlb4gblEl3cKLpcYsG4EH0XckI=; b=agto7fsVBUrNPJE79ZqKdKpL56Mwp4ggAufpi3aobdfUMqqDLiZePIEDWdZdbEiu8K t8r+5OhPjz9fgq16P3VlDF8eKhXvyvPklbZFKypYRoc2ct12HZqqWkaTgun7QUkTKvO0 dNyxvyCqVBkkl2G1Bynvde5QJxv86Iskrfv+necqUnTYkwOORvwd+LUfpSQyTp0tj4Sn 95J2lIFDA5QKqrewiHle9ipChsR0tw7IP8b/yIlAW8KtmlUhkkMxFmnnZ46aKVXuB3E5 DiwGOd4HYAa8c2P85NmCinCOH65of45ssCv3+8jc/ZLu/J3Rq2BD6TCXI3hQ2zqYzUeN Fidw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584776; x=1729189576; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=GAJaAoo95YXsHy4joQlb4gblEl3cKLpcYsG4EH0XckI=; b=fAj0VHZ6O7hFRvs9qCZ7NhX8AU2Cvz8imjecXb1ahWrW5qbgiBMuofPgXn4EW19KC0 ISro3i7PySTEK7FYiP2wNQQYn1sq2Md11OHbnmlBC9kdt3dC8H88X8W8uiWb/RX2NUSV E4MC4YntX0oG+UUw89UuKV2jKs4zgoz1CtzqlA5pKV1JlMNFfTKY62jPuRSJCmuWZVYS QNkTpZBFgy8dEaI/Kj9bu+Mi2H9WKyaSnjlWfqz9Wo9XuUjLSxPoHn2DAxvyR75x2QH5 CvKJGXqgDnAHDqYTvtgLqUiEQAQMg7ne+IsE2REunrWoQC1tHHII9wEk8KzaK5l2ezdI NuRw== X-Forwarded-Encrypted: i=1; AJvYcCXGl70ehNd8s0rdHhEPy5odhwgnRLszgw6vzfsDlUpAxxopHIwLyol9LRRxZ+V7QHbCRHSzOG8Kj1z7wg==@lists.infradead.org X-Gm-Message-State: AOJu0YwA/ume4Z9DG7AxK47mCl6qccpxoHthOupddBvdad2D8q7XZoTi rp88CkwoX66oVZrnVRxnsTe6zR0tHNASZwuFRngZyHrIeHBi8GqjbNpcPe9kPh7+3LfSSFjGe7Y 8fw== X-Google-Smtp-Source: AGHT+IEXxr/4vsCtgUsUHHkctm5xGRqb6tsNipBtkL7vgMjj75AzVZauqGsQqnffIf336Hhi8e9e6vZrhMQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:aa7:9184:0:b0:71d:f7c9:8cb3 with SMTP id d2e1a72fcca58-71e1dbf1ab5mr7792b3a.5.1728584775458; Thu, 10 Oct 2024 11:26:15 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:42 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-41-seanjc@google.com> Subject: [PATCH v13 40/85] KVM: x86/mmu: Add helper to "finish" handling a guest page fault From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112617_336025_0F2270AB X-CRM114-Status: GOOD ( 10.21 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Add a helper to finish/complete the handling of a guest page, e.g. to mark the pages accessed and put any held references. In the near future, this will allow improving the logic without having to copy+paste changes into all page fault paths. And in the less near future, will allow sharing the "finish" API across all architectures. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 12 +++++++++--- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 65d3a602eb2c..31a6ae41a6f4 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4357,6 +4357,12 @@ static u8 kvm_max_private_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, return max_level; } +static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault, int r) +{ + kvm_release_pfn_clean(fault->pfn); +} + static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { @@ -4522,7 +4528,7 @@ static int kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, * mmu_lock is acquired. */ if (mmu_invalidate_retry_gfn_unsafe(vcpu->kvm, fault->mmu_seq, fault->gfn)) { - kvm_release_pfn_clean(fault->pfn); + kvm_mmu_finish_page_fault(vcpu, fault, RET_PF_RETRY); return RET_PF_RETRY; } @@ -4598,8 +4604,8 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault r = direct_map(vcpu, fault); out_unlock: + kvm_mmu_finish_page_fault(vcpu, fault, r); write_unlock(&vcpu->kvm->mmu_lock); - kvm_release_pfn_clean(fault->pfn); return r; } @@ -4685,8 +4691,8 @@ static int kvm_tdp_mmu_page_fault(struct kvm_vcpu *vcpu, r = kvm_tdp_mmu_map(vcpu, fault); out_unlock: + kvm_mmu_finish_page_fault(vcpu, fault, r); read_unlock(&vcpu->kvm->mmu_lock); - kvm_release_pfn_clean(fault->pfn); return r; } #endif diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index a476a5428017..35d0c3f1a789 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -836,8 +836,8 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault r = FNAME(fetch)(vcpu, fault, &walker); out_unlock: + kvm_mmu_finish_page_fault(vcpu, fault, r); write_unlock(&vcpu->kvm->mmu_lock); - kvm_release_pfn_clean(fault->pfn); return r; } From patchwork Thu Oct 10 18:23:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831316 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 69EC4D2445D for ; Thu, 10 Oct 2024 23:31:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=EGSEH70eynaya5Nylvfrr4A0ElttLtbokZmntMH+qJY=; b=Umjj3XHoHFeVo6 Atk4B40eGNCGJILndHdWMERBfo2pjltKJhboWgt1XbgboRx4NIT2YaLlBYVvENCv9ikYnFBMx/viJ xfc+jwUbjoyPHBvW9QT+zlYPvhSm2SkxSFoH12+5r45LF98KShWDJRbQra0wZegSqi/eG+Ia/BjYd PmuY3D9SgawQbPmsGvXEu2jdjSJEqJP0w+IsvqeO6F+U21GNLEymao5iFuvrKOmzUCbxY+J5M4nia MWfpFexPuKpVXGvJKBTEegSrX3kcqcvWZDnR/cqOU6ZDamM85sm3ySjvZqPJkHVOFHu8UG7B4sKkL hGuzVf1u47NwVi0VVDJA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz2cz-0000000Ef4h-2L8f; Thu, 10 Oct 2024 23:31:29 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrf-0000000DqjR-05Yr for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:21 +0000 Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-206b912491eso23155655ad.0 for ; Thu, 10 Oct 2024 11:26:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584778; x=1729189578; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=sgK+YcjRpEQ+YhbfYv/o4akq/yPA2qNOXCMP41gc9EY=; b=iFkBAzBXYFBUziXH1DPHFwz0QuJbVZY9yzLrch8ot8iJUnZFwu0Ch7La+ANEyti4nR AiDSevGke+MAd+fF5+8281cC7OvrCxYOzA8Jn/wVBWgRyeNM1C+8Y4ZOM9b7hYkXpmDz rSxcKYrmZHnrNvGHVOgV5h+nWW41K7iFAYTqArL5ncvyqN2Nrg8M2ksNxs489+ODUdI5 RkxO0PgCZ3IqNb8GULWvgAuWQDd7DbJqF6gmz7/xpgXrONOBNZ8xnCYXKQDivmKtwi++ SkbfWdozQu9XNTOmoT67SyrcEA3ebbCiwMyOSypgPQ2bTRLYaltBc5xINem/udZ5gqOM R5OQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584778; x=1729189578; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=sgK+YcjRpEQ+YhbfYv/o4akq/yPA2qNOXCMP41gc9EY=; b=USjtn95MZ9+Dyizs+NGALTTbp4XAVvruFTpJe2XGoMZs2v62MsPcXH8LR/wMJMBCTE LoovBt78gKxoWPOI886u1AfG5SNg1sDyp5XDX5ggO2pSWrrcQDAtX8BUmf9mS5mjprjU IOCqbAy86Gl9eVaSlpn1iuTMIAI00JYNg0k//yzfLgGvI2mUyDyTEwCheRriXRpD77De wB8WBpHufcx1o8RUbGWpP1yZsa6fuokxmoECOsDgPBdPLx6+ASYac91WRJvR6wkXyLZw NQPC4WVCMtAdeLzJ/aokbgLqw1NoVklv4pjxmWQc4tlTmAGYyNMyGOS+tGluq9XIN1FF Xa9Q== X-Forwarded-Encrypted: i=1; AJvYcCXEDOxs4MppAIszHqBFE7U7KUZqMhirhkRGbQEdQJ+y5+Ax/kbu/9EwjtxvSsWJHrNEHSaug4NXomnfjQ==@lists.infradead.org X-Gm-Message-State: AOJu0Yypk6VYUviU9gECaZptvhAJ7/DkPValLj/3tY5pp8bKeyM++R2y A+EVm4+rw0/tniGgMRVfUd+urVTJmUt5ejrxiAvkuaBgPuCfRQrBUSDGn3AwnQqlz6+M97uJCbE fFg== X-Google-Smtp-Source: AGHT+IHgES02wxBmUBuw4MXImaMxOgBvLS6HzbS699NHinso/VI3SIlimd+hdhb5F/AgwxmTSyUtUzYKtBo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:ec85:b0:20b:7bfa:ac0f with SMTP id d9443c01a7336-20ca037f212mr485ad.1.1728584777475; Thu, 10 Oct 2024 11:26:17 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:43 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-42-seanjc@google.com> Subject: [PATCH v13 41/85] KVM: x86/mmu: Mark pages/folios dirty at the origin of make_spte() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112619_199281_EC287716 X-CRM114-Status: GOOD ( 17.21 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Move the marking of folios dirty from make_spte() out to its callers, which have access to the _struct page_, not just the underlying pfn. Once all architectures follow suit, this will allow removing KVM's ugly hack where KVM elevates the refcount of VM_MIXEDMAP pfns that happen to be struct page memory. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 30 ++++++++++++++++++++++++++++-- arch/x86/kvm/mmu/paging_tmpl.h | 5 +++++ arch/x86/kvm/mmu/spte.c | 11 ----------- 3 files changed, 33 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 31a6ae41a6f4..f730870887dd 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2964,7 +2964,17 @@ static bool kvm_mmu_prefetch_sptes(struct kvm_vcpu *vcpu, gfn_t gfn, u64 *sptep, for (i = 0; i < nr_pages; i++, gfn++, sptep++) { mmu_set_spte(vcpu, slot, sptep, access, gfn, page_to_pfn(pages[i]), NULL); - kvm_release_page_clean(pages[i]); + + /* + * KVM always prefetches writable pages from the primary MMU, + * and KVM can make its SPTE writable in the fast page handler, + * without notifying the primary MMU. Mark pages/folios dirty + * now to ensure file data is written back if it ends up being + * written by the guest. Because KVM's prefetching GUPs + * writable PTEs, the probability of unnecessary writeback is + * extremely low. + */ + kvm_release_page_dirty(pages[i]); } return true; @@ -4360,7 +4370,23 @@ static u8 kvm_max_private_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, int r) { - kvm_release_pfn_clean(fault->pfn); + lockdep_assert_once(lockdep_is_held(&vcpu->kvm->mmu_lock) || + r == RET_PF_RETRY); + + /* + * If the page that KVM got from the *primary MMU* is writable, and KVM + * installed or reused a SPTE, mark the page/folio dirty. Note, this + * may mark a folio dirty even if KVM created a read-only SPTE, e.g. if + * the GFN is write-protected. Folios can't be safely marked dirty + * outside of mmu_lock as doing so could race with writeback on the + * folio. As a result, KVM can't mark folios dirty in the fast page + * fault handler, and so KVM must (somewhat) speculatively mark the + * folio dirty if KVM could locklessly make the SPTE writable. + */ + if (!fault->map_writable || r == RET_PF_RETRY) + kvm_release_pfn_clean(fault->pfn); + else + kvm_release_pfn_dirty(fault->pfn); } static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 35d0c3f1a789..f4711674c47b 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -954,6 +954,11 @@ static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, int spte_to_pfn(spte), spte, true, true, host_writable, &spte); + /* + * There is no need to mark the pfn dirty, as the new protections must + * be a subset of the old protections, i.e. synchronizing a SPTE cannot + * change the SPTE from read-only to writable. + */ return mmu_spte_update(sptep, spte); } diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 8e8d6ee79c8b..f1a50a78badb 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -277,17 +277,6 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, mark_page_dirty_in_slot(vcpu->kvm, slot, gfn); } - /* - * If the page that KVM got from the primary MMU is writable, i.e. if - * it's host-writable, mark the page/folio dirty. As alluded to above, - * folios can't be safely marked dirty in the fast page fault handler, - * and so KVM must (somewhat) speculatively mark the folio dirty even - * though it isn't guaranteed to be written as KVM won't mark the folio - * dirty if/when the SPTE is made writable. - */ - if (host_writable) - kvm_set_pfn_dirty(pfn); - *new_spte = spte; return wrprot; } From patchwork Thu Oct 10 18:23:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 57AA0D2445F for ; Thu, 10 Oct 2024 23:31:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=dW+Fh8FAoNei6ky5ikVWn0yswrmuSqK7sgohu7H5ROw=; b=jykd/8ISz/djVG InzZ7LxBbT5jIPvfa6g0eGXzHO/DU+UchFh2XlbN74qf4+A4oOl9FXfwmrFGvaxeRwBB3XcsmQYWU GbB7Eqjeq2PjYVzYJFPiUz72pS8Rd/zoowlew+oqMmYkzMozogdW1GRVwClDlxIMZbmvNx+lM82Ix nKmnaygEC6y/IsVDDKBgDXdRHhhb4zwI0gIO22rJ31xGX0evaqHCi+hbtkVvh4k3CNTkh6F9hfHB2 A15CaQTgv6olIF5kSMBSK/pD2Rmf7pe3BjGP17xngXmhHVJd2NPPoXSmu0Lj+j7N1KkhZAbMghjvy VHW2dtKnzgkiMCk63Dbw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz2d0-0000000Ef4t-0f1L; Thu, 10 Oct 2024 23:31:30 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrh-0000000Dqko-0FBc for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:22 +0000 Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-71e026caf8bso1466238b3a.2 for ; Thu, 10 Oct 2024 11:26:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584780; x=1729189580; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=8DwvYXI8CmE5CO0Ycvk+EgdgF52vTMi74FFh8TONwA8=; b=4QzPzbH+y3h+qIy3AOczqs89mCTm26A8Xmw0N0dOgewGXg9A6oNEwweMsZb+ojs0A+ /oaXqPla6lblPPbLsCeAkMWE0ipPJUeX7/BMJCeUmSAFOBCsBqR9tEnyG6dtvMhnmUoa icT7EjStSrVntpraIZTjudeYgVzDZgDSb8Ozn3GE9fd2lwS76/qNcWlYT5VV3157N2U3 boorlU+iazI5EA8U+PS7+8dAzQPprqHAb+bDJ23cP+mz4Fe98+JO8wzksaoGlcC6sRxS GoaOsto/Zr0ovDqlpn2iqeKujfmzR03yyd8aPIXxNhya6wLqUx9xXzeU/hX0itI/QzVu Is8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584780; x=1729189580; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=8DwvYXI8CmE5CO0Ycvk+EgdgF52vTMi74FFh8TONwA8=; b=a4hQ8heLfDq8ElEvHefSYuedvWWh/5wQp0och4xeWa/JN/hLq4I04BOkD5CbPfpvev +2jnYDCxzV0yysd2GayPonB+xiIA5ZATui9mjCH5hDFsGsHDjNXLepZ0o7Li+Lf7Cc/B OxZ6l4q89PfwveIVEBJDB//UIJWGIyrZA8LPUkwda6f1sKv1aq3lAjsWyTCo3LO7SbfA VlEypaVVBzmr7OpgSN3f7nvvJD9bFHxn0cInb6T1oa3MiAdjp93Ht2topFEAZeZgTKeX xzoB0OFLyygy9XgrNKhFvE/uRjQtSrAIb4aKaxtbQ1/3dtHaupA35Y5kRjpzInllyV4t 1YBQ== X-Forwarded-Encrypted: i=1; AJvYcCX/imTQpYySRpBBo0fkHvdSXXLS/r74MlKfgj7TdO2EY9GJN+mSTBHO2sY1kMpkqpXGA/3NBOVX81wgPQ==@lists.infradead.org X-Gm-Message-State: AOJu0Yy1yo2hLPfSv9J9rOd8YS3qDAardAqZgOS2XfyLWk6Gj2EdbFzn hgkbRBeG0HEd2xfBaqMISkDqq+ak3KJcO48VScCJyOw0hYim0mI8D5ZMdpQEPLxQFFp5+7TAQZK Ngg== X-Google-Smtp-Source: AGHT+IG3pPvsekcqYnotExeZVZUXEYlJLq+6e3a1eutYJX4gtVKcKQbhJKMNrxmpaRIGMBxtyW/kZ/ByL14= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:91a3:b0:71d:fb06:e79b with SMTP id d2e1a72fcca58-71e37c53abamr28b3a.0.1728584779496; Thu, 10 Oct 2024 11:26:19 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:44 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-43-seanjc@google.com> Subject: [PATCH v13 42/85] KVM: Move declarations of memslot accessors up in kvm_host.h From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112621_190530_612F6579 X-CRM114-Status: UNSURE ( 8.26 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Move the memslot lookup helpers further up in kvm_host.h so that they can be used by inlined "to pfn" wrappers. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 9263375d0362..346bfef14e5a 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1168,6 +1168,10 @@ static inline bool kvm_memslot_iter_is_valid(struct kvm_memslot_iter *iter, gfn_ kvm_memslot_iter_is_valid(iter, end); \ kvm_memslot_iter_next(iter)) +struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn); +struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu); +struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn_t gfn); + /* * KVM_SET_USER_MEMORY_REGION ioctl allows the following operations: * - create a new memory slot @@ -1303,15 +1307,13 @@ int kvm_gfn_to_hva_cache_init(struct kvm *kvm, struct gfn_to_hva_cache *ghc, }) int kvm_clear_guest(struct kvm *kvm, gpa_t gpa, unsigned long len); -struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn); bool kvm_is_visible_gfn(struct kvm *kvm, gfn_t gfn); bool kvm_vcpu_is_visible_gfn(struct kvm_vcpu *vcpu, gfn_t gfn); unsigned long kvm_host_page_size(struct kvm_vcpu *vcpu, gfn_t gfn); void mark_page_dirty_in_slot(struct kvm *kvm, const struct kvm_memory_slot *memslot, gfn_t gfn); void mark_page_dirty(struct kvm *kvm, gfn_t gfn); -struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu); -struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn_t gfn); + kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *map, From patchwork Thu Oct 10 18:23:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831078 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1D7FED2444D for ; Thu, 10 Oct 2024 20:31:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=NDYXAdS3XYiprcTc97f/OScbt0M0P0wPG4OeYtKRENE=; b=QY3CPqvkQW8hXo 46IrZnoY7H/F6va7ZhnqYQgB9EBTQ35n7H9wUlFZ1fUd4AsGK9We7Em9HilLdgaTGTBUsBn1OVfNe wmjaT36yCWVGOGJv2tvTQ+61ZDjQBthJDatpqzwNT5l4z3RmkV7tVxq3I0HXzyFVAbX5ul7ZgKELE FyiaJXB+AcjSHdBaiyuU/DpAxZsdmvr9N2tD6yl9HU+V61StMCrmvMkosymU3VoTN+epDIWbrBkdc hv3WOeVQvqKUnVseYDERYgw1T+lfwTn8iLABW7K1AfGlR6xf+M0DF1oPbE2Dl4/5bTZkxWriDSrAB TK4z/3u99bg+sly/RGxQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzoh-0000000EGlq-2vOJ; Thu, 10 Oct 2024 20:31:23 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxri-0000000Dqm2-3iyL for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:25 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e29b4f8837so18944097b3.0 for ; Thu, 10 Oct 2024 11:26:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584782; x=1729189582; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=CHy2j9VNwFKd5G/Ubc3O4QkXvNoIm5ox3M6XZ4oswcY=; b=PxBQb6V+W9jxog8EtEIgeGjMXgX3DoQlH9csUz21WWACHY5TzBY6Zb0NYKAEDEvhCL EEORQUDWODB5Zlh+y65IhXmZaI61GUcrVHjyExWDVVnYWImh+IDn5v6Rn0iYUtOApd27 lRgtb7DyZK3i73ZvBLazXX+S80VsUBJVrvIewB0whu96yv3ay3gWL2sKc8xdagcNXSsC B5F89XVKv1Hf4taRgzl222K8qjxQHiSGWVj8Yi577akb5HVrn6V1Cuc+beMKFlqGzfAA 9iXv89Gftqp9NkxCMeqn6A/sH+3xcYVnYTU5K9D7AKyhHG0S1pgbqIsLN4HiKAti+VlZ OOVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584782; x=1729189582; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=CHy2j9VNwFKd5G/Ubc3O4QkXvNoIm5ox3M6XZ4oswcY=; b=poffuGcsPYZxmNUvbyvci8g0KjIuC7WQXIN5Am+4NoBkQ7o5TkA8lZvq1iUWd6KsKa AkYev799cRraqz+QUn3pDmZS+OnXZ+2XUBUDQp3LMHoDlJm/GTju7A7i+AjYUb7hhHUz Di+PyvaG3+rKPq7BO7lwS36go2WJTbd/8nOQUSCbvs4KNLV/+VXT2CPreZ3eGOrc8PLL YVcvdrA4+q3RxIX1y2N5QEoGBoVJzoIEXsmgykK02HzNZOXaQU8w9EI+iwmfj1FVdqaq PvC/z4YSB/3qWhbJB68hBcUC/zqQfwUWejdENZaauv85NsQD/hqfqvGIkPd6Piyosydo Lrgw== X-Forwarded-Encrypted: i=1; AJvYcCXIYmvylauDmBNepFzoqoqrkO8sicZI2f9Rd60Cs9Ol2zye8T/bXxYkseGZf5VZ0KKgIgerxC6Wgluksg==@lists.infradead.org X-Gm-Message-State: AOJu0Ywj/5yXRQ1usfLhXo50vKXVAQ1ovy9g7YOe+rFzltVyqJNOIPS6 I7JGImpCjdEsjP5byhZeuRB5YmyAE7CH1zSk7+x+kfObHekgaGMmoC70lRaKD9FnK/Nwu4O84g+ GAw== X-Google-Smtp-Source: AGHT+IEBXjEpYcXfPYRm3Nf4G9ZN1ucR55aqjscHr2p9QPsSrcRZYI0185ucK64AXquh+1QVcruWQv+kNDM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:5605:b0:6be:523:af53 with SMTP id 00721157ae682-6e32f2f4e28mr919077b3.3.1728584781575; Thu, 10 Oct 2024 11:26:21 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:45 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-44-seanjc@google.com> Subject: [PATCH v13 43/85] KVM: Add kvm_faultin_pfn() to specifically service guest page faults From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112623_267989_DFFF55F7 X-CRM114-Status: GOOD ( 11.95 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Add a new dedicated API, kvm_faultin_pfn(), for servicing guest page faults, i.e. for getting pages/pfns that will be mapped into the guest via an mmu_notifier-protected KVM MMU. Keep struct kvm_follow_pfn buried in internal code, as having __kvm_faultin_pfn() take "out" params is actually cleaner for several architectures, e.g. it allows the caller to have its own "page fault" structure without having to marshal data to/from kvm_follow_pfn. Long term, common KVM would ideally provide a kvm_page_fault structure, a la x86's struct of the same name. But all architectures need to be converted to a common API before that can happen. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 12 ++++++++++++ virt/kvm/kvm_main.c | 22 ++++++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 346bfef14e5a..3b9afb40e935 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1231,6 +1231,18 @@ static inline void kvm_release_page_unused(struct page *page) void kvm_release_page_clean(struct page *page); void kvm_release_page_dirty(struct page *page); +kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn, + unsigned int foll, bool *writable, + struct page **refcounted_page); + +static inline kvm_pfn_t kvm_faultin_pfn(struct kvm_vcpu *vcpu, gfn_t gfn, + bool write, bool *writable, + struct page **refcounted_page) +{ + return __kvm_faultin_pfn(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn, + write ? FOLL_WRITE : 0, writable, refcounted_page); +} + kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 6dcb4f0eed3e..696d5e429b3e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3098,6 +3098,28 @@ kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn) } EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn); +kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn, + unsigned int foll, bool *writable, + struct page **refcounted_page) +{ + struct kvm_follow_pfn kfp = { + .slot = slot, + .gfn = gfn, + .flags = foll, + .map_writable = writable, + .refcounted_page = refcounted_page, + }; + + if (WARN_ON_ONCE(!writable || !refcounted_page)) + return KVM_PFN_ERR_FAULT; + + *writable = false; + *refcounted_page = NULL; + + return kvm_follow_pfn(&kfp); +} +EXPORT_SYMBOL_GPL(__kvm_faultin_pfn); + int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, struct page **pages, int nr_pages) { From patchwork Thu Oct 10 18:23:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 33CCDD2444F for ; Thu, 10 Oct 2024 20:31:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=RG7hhddPRSH0dwciXHH8gfzMg7L6eFvrYNkkp1ctLlI=; b=se3bkDwE9j60IJ QvnOF0VtKL1NWVfP+y7nPMaxgpYlK7uV097OjDGpDUbBuzKRDmyNBgvEqoMM7pXS+vi2WaNAKggbn xk9wvFpqxcNOGMGYZXjt9aB9YO9Obd7T6KUGp4n0CdME9+QAMQsd3l0HRROKPpXaG0SYSA//DmbiJ rKP1M6KocN07OnW2UtexcUBlKK0HxGmX2FZcz/qvZR32St8ijqkq2Cr2IXob1/nWsStD360M9WTvE 747qNqoSMAjcJQmCkXwPhCoYdAvTFcjhN9ZsVl5RNxmbmhScYBKeHEsOZ77OcjP594xTm4EQBcAzE dtotOIMu+fm8TULs8IJA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzoi-0000000EGmQ-48lw; Thu, 10 Oct 2024 20:31:24 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrk-0000000Dqnp-32Cn for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:26 +0000 Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-7e6e98892e1so1288241a12.0 for ; Thu, 10 Oct 2024 11:26:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584784; x=1729189584; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=O48RmE1hmLPs0qNEdPIh+k/rpCPu+e8A6GC4+tMXVMw=; b=tUX4W6o8HRVDNg2p6vKzVqKv4QtBGye1jr9W70enPs3WfdHdg/DAX4SGK3PXyvt/PU b7AYI/CL/BlJ+quKCLDQWnwM3l+DwmoJ38wzIZ0AUR/5qG3XbJ6uKyGqIGs7af000tzu Yw5mLr5FCn/tAEl9m+rbmGRIlmBjOVsDrIb/34VVROnkoIFIZoc77UW/hjwsWuFMrKWu /akxlANMKK8Zni3hn6A1as9i+vAkfXKoaefSt+xnuNgimIQ5h/U2840t3CpBs4rUac96 3v9uJ9pPCbO+EtZ7MI416Oy4vh2RW3JLQnyfGXSyS7FkCqEOQLvRezcMeq0XiOM0Uyrt IWbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584784; x=1729189584; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=O48RmE1hmLPs0qNEdPIh+k/rpCPu+e8A6GC4+tMXVMw=; b=Bq7D6UBMGklSijpwdIeacKsnPnU34c9sMHXlo9Es/Xe55MiM/E1LvHb+PkfpHE5sUl Z0aLmfq63DwzrWBq332t9sHYidveTIEC19I3nsulsnnJ2uhx/bcvWFCmKVhb7/PvaWtF xC7nrR+SrYMWTwO6tFPRYJT/35OF5c6h6dLKHmVDJgmWcXkf9a1X0rZjLdg9TlbBoj2m t56tDsA+Hd6jh0wxyA+8kmvtYcvUmAj9zibg8mrBjAnp98kMip9hUjYSpDUp1rHiSDCh EC3OvbAZIwXnKnYvcZNOpFbIJJTZmgG8m6jiU9t9hVPiWI/W8Xz4/VXjd5XljmHQh1hM AfXA== X-Forwarded-Encrypted: i=1; AJvYcCWbYXbSB37C/qel0ToL4NzJGFRiHb+G8m+QR2SUW/uICgYryilhysYCzQPAEx0fJRIbZDcRaxywgO8qLQ==@lists.infradead.org X-Gm-Message-State: AOJu0YxXc3HZ8PjHlGJgmNtpL7RnRJAPCAjFc1xq68jNKmyD+5cqOBCP Chp9EF9HEmYM5oe33Ej0Ihq6iaL8UY6J5pLLeYIVl0woyjNvzi60R2cfzzLp2iykXWHoBY+tZmA SmA== X-Google-Smtp-Source: AGHT+IGY3J0AFIsOlJbVDrqn6majLeEUOkcp3LAakwsbgSwDXT3SnAGgA9L20idMLdLGQvBeoRsHZDJxo50= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:2445:0:b0:75d:16f9:c075 with SMTP id 41be03b00d2f7-7ea5359cd2amr19a12.9.1728584783356; Thu, 10 Oct 2024 11:26:23 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:46 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-45-seanjc@google.com> Subject: [PATCH v13 44/85] KVM: x86/mmu: Convert page fault paths to kvm_faultin_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112624_810591_D76A17CA X-CRM114-Status: GOOD ( 12.96 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Convert KVM x86 to use the recently introduced __kvm_faultin_pfn(). Opportunstically capture the refcounted_page grabbed by KVM for use in future changes. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 14 ++++++++++---- arch/x86/kvm/mmu/mmu_internal.h | 1 + 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index f730870887dd..2e2076287aaf 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4416,11 +4416,14 @@ static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { + unsigned int foll = fault->write ? FOLL_WRITE : 0; + if (fault->is_private) return kvm_mmu_faultin_pfn_private(vcpu, fault); - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, - fault->write, &fault->map_writable); + foll |= FOLL_NOWAIT; + fault->pfn = __kvm_faultin_pfn(fault->slot, fault->gfn, foll, + &fault->map_writable, &fault->refcounted_page); /* * If resolving the page failed because I/O is needed to fault-in the @@ -4447,8 +4450,11 @@ static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, * to wait for IO. Note, gup always bails if it is unable to quickly * get a page and a fatal signal, i.e. SIGKILL, is pending. */ - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, true, - fault->write, &fault->map_writable); + foll |= FOLL_INTERRUPTIBLE; + foll &= ~FOLL_NOWAIT; + fault->pfn = __kvm_faultin_pfn(fault->slot, fault->gfn, foll, + &fault->map_writable, &fault->refcounted_page); + return RET_PF_CONTINUE; } diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index 59e600f6ff9d..fabbea504a69 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -238,6 +238,7 @@ struct kvm_page_fault { /* Outputs of kvm_mmu_faultin_pfn(). */ unsigned long mmu_seq; kvm_pfn_t pfn; + struct page *refcounted_page; bool map_writable; /* From patchwork Thu Oct 10 18:23:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831080 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1D5ACD24450 for ; Thu, 10 Oct 2024 20:31:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=MPaz6imQoa1VkT+pK3kB9rJlmjGBgz55q5ibS634j14=; b=pCoqnHLZIoPtAj QVuGj88r0+NU5K8yOboIts1xh0Ew9A2eZJCwEH3DVCtSk67TvfzkQmenNASe5rbgxrMtbFz3VNzyi FzGk1fG8OE7UxoqCNf5g2nugzeQUP1hFpY6c+z5MPsoSIGCB62ZJ6ak9lUDks4OnpFS2owhXLY/wB e3KKAQdU7gyzh8//cdLb2iRa569OsOzd7Mm4zPsTYPmBd8dCDVY5mSjtBwchMjoXdjvKMKILcf6G2 KISO1zlv/zYMDOsgJ2gb5nBToKmtExQGKFYRT5GoZ0UG2Wb7M1iNKs5U1V7WVkAVEq2kQJPphisTE hGgB7QedI72YOVz8bXIQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzok-0000000EGnW-22j3; Thu, 10 Oct 2024 20:31:26 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxro-0000000Dqq0-01in for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:30 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-71e0228d71dso1225534b3a.0 for ; Thu, 10 Oct 2024 11:26:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584787; x=1729189587; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=uRLwC7za7TRhyOe/CfUzhvpHD8RlodmrAsAEh9/RrLY=; b=PKoD/wJAb5njzUtpsWhaiNM2r+OCsHfWtiV/apSkhgK99vTei83ZnneTKYniaGobtS J45i/TolbRzdp5c0H22Luy6Rs4kKlLJaRGcpt/l3Ev+zc3SHfJ9EHEt2Wsh5CiYgVmOL SGYSWmYIMPQNWdysyBr7Fl4AZNrhGBJq/a7zAXoeKDXdLH9srkAQl32RTPc6myO/6pT+ +U1xZ7bie0Oj+11nv8QY/+JxDm6Kylusn84ikqXYk1Ql5f+RfVxJAVfljbZTD77+jbRg HvSzJHmdhjQLWIuP1P/nY+XLd0JF6+Ao5kyRK9GS8cjA6e6dQHUvz3M5OWWBXCiJa34T JjjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584787; x=1729189587; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=uRLwC7za7TRhyOe/CfUzhvpHD8RlodmrAsAEh9/RrLY=; b=uWM+bqWBNbIyJwUhotCbdX1PCYMMOKgPUAe/qH+8gB8l9SH2I2kv8Ua6JVuHD6soh5 td9I1i9TWyR7TpFGke2EtCHZ6QMoyyrdQIn6ilTOEfOVhLVIh9B4HsNd025C0HV/rTSK uKRlufQVartrl1+QvC5Kjgby/6d93iaMxEhfiATnF7jCclAYdx+axn3IcBOn2+N/Z+5k G4q2jumtaBt6x2twcg4cFKyenqb4WSsx1ha3AUkGrVFmqQz2l3CENdylHWhMQhCME+TV qlixBSp0nk9XL0kNUYeq/FThFtX1VogdGbPp8/rmVXU3JqF/PJBFzgIdBhNgOh+jO+A1 D7Rw== X-Forwarded-Encrypted: i=1; AJvYcCUrW7mPdbg/dclLL6pRsj+tv1pZFScU/436ByESqCANbl8ll5tNafphBA5fJE/P8cZXj+NlJZvZlF+kTQ==@lists.infradead.org X-Gm-Message-State: AOJu0Yxe7FuRM8PMhyFkGbGE3OeNzHzP8Ri9AMbITOSZ/jAfB5dOYcEU lvTC8OpYlS6uwOa3okVbHVuJBCqOC45STru/4irVQy0F/g6nrBkZt+FUxShuyD9ZwK3nPATv49h ZxQ== X-Google-Smtp-Source: AGHT+IHljZntO7iRsDkdAW1llbbIMc26iesy/7PCA9rtG6130gQpp6naB/6JGeYrnZXaXdo41CPgD6F3FKs= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:aa7:9184:0:b0:71d:f7c9:8cb3 with SMTP id d2e1a72fcca58-71e1dbf1ab5mr7795b3a.5.1728584785442; Thu, 10 Oct 2024 11:26:25 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:47 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-46-seanjc@google.com> Subject: [PATCH v13 45/85] KVM: guest_memfd: Pass index, not gfn, to __kvm_gmem_get_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112628_103491_9E1FA2B7 X-CRM114-Status: GOOD ( 14.86 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Refactor guest_memfd usage of __kvm_gmem_get_pfn() to pass the index into the guest_memfd file instead of the gfn, i.e. resolve the index based on the slot+gfn in the caller instead of in __kvm_gmem_get_pfn(). This will allow kvm_gmem_get_pfn() to retrieve and return the specific "struct page", which requires the index into the folio, without a redoing the index calculation multiple times (which isn't costly, just hard to follow). Opportunistically add a kvm_gmem_get_index() helper to make the copy+pasted code easier to understand. Signed-off-by: Sean Christopherson --- virt/kvm/guest_memfd.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 8f079a61a56d..8a878e57c5d4 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -302,6 +302,11 @@ static inline struct file *kvm_gmem_get_file(struct kvm_memory_slot *slot) return get_file_active(&slot->gmem.file); } +static pgoff_t kvm_gmem_get_index(struct kvm_memory_slot *slot, gfn_t gfn) +{ + return gfn - slot->base_gfn + slot->gmem.pgoff; +} + static struct file_operations kvm_gmem_fops = { .open = generic_file_open, .release = kvm_gmem_release, @@ -551,12 +556,11 @@ void kvm_gmem_unbind(struct kvm_memory_slot *slot) } /* Returns a locked folio on success. */ -static struct folio * -__kvm_gmem_get_pfn(struct file *file, struct kvm_memory_slot *slot, - gfn_t gfn, kvm_pfn_t *pfn, bool *is_prepared, - int *max_order) +static struct folio *__kvm_gmem_get_pfn(struct file *file, + struct kvm_memory_slot *slot, + pgoff_t index, kvm_pfn_t *pfn, + bool *is_prepared, int *max_order) { - pgoff_t index = gfn - slot->base_gfn + slot->gmem.pgoff; struct kvm_gmem *gmem = file->private_data; struct folio *folio; @@ -592,6 +596,7 @@ __kvm_gmem_get_pfn(struct file *file, struct kvm_memory_slot *slot, int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, kvm_pfn_t *pfn, int *max_order) { + pgoff_t index = kvm_gmem_get_index(slot, gfn); struct file *file = kvm_gmem_get_file(slot); struct folio *folio; bool is_prepared = false; @@ -600,7 +605,7 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, if (!file) return -EFAULT; - folio = __kvm_gmem_get_pfn(file, slot, gfn, pfn, &is_prepared, max_order); + folio = __kvm_gmem_get_pfn(file, slot, index, pfn, &is_prepared, max_order); if (IS_ERR(folio)) { r = PTR_ERR(folio); goto out; @@ -648,6 +653,7 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src, long for (i = 0; i < npages; i += (1 << max_order)) { struct folio *folio; gfn_t gfn = start_gfn + i; + pgoff_t index = kvm_gmem_get_index(slot, gfn); bool is_prepared = false; kvm_pfn_t pfn; @@ -656,7 +662,7 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src, long break; } - folio = __kvm_gmem_get_pfn(file, slot, gfn, &pfn, &is_prepared, &max_order); + folio = __kvm_gmem_get_pfn(file, slot, index, &pfn, &is_prepared, &max_order); if (IS_ERR(folio)) { ret = PTR_ERR(folio); break; From patchwork Thu Oct 10 18:23:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830921 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BA088CFC5FE for ; Thu, 10 Oct 2024 19:21:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GQ+uhhN85wkDZ7a/8YspSeLrDnBklEzur8mbZv8hsEw=; b=21pKPdFWvFs0r+ C2VqwWaXHU+CQEVXiWuKt3CjGY/VqCJXoYSsyApM/LPrMWk1jMcuE0GeAKcuT2yEBndumg+/3jsnX clqNn7RsxHWJXKEK22DXMPyjWWbul9dQSoN0o9/sxdtWqVnf3Eso73mHbBFGGmtSRLI3y8OaNjCzQ aypL2+QVP8lT+y3ZT2L31NcQVU5CmzW7tOPAn0R8ARv8ItTczU8p7UfKYulCJEBOSWg0qnAMnUk78 g6mI8VWpELoV3gjFt7qKrg+ea1b+Cg9tNN0mqwrxq3gsZTutWBSomdu6eFJtFLx9KkEK/+Lrl3t7o 8onl8lTU/rBE4/6czExg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyjE-0000000E5VP-3cuI; Thu, 10 Oct 2024 19:21:40 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrq-0000000Dqs2-1Wry for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:32 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-e165fc5d94fso1877585276.2 for ; Thu, 10 Oct 2024 11:26:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584789; x=1729189589; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=/DfuS3cqNKQbdjipc1jp6gNPynmr8H/oDbYVrbrij3Q=; b=bQ9AW2IX1XkoVL4gLFimXEPTP1Mhxriw/jA12X4nUWJqxirBw2m1oV21TXm/Q54OeR 3xmOsY2+fN14ugnmdPBfXy10JZiNvnidlIpFjwC2GEgPzc3S1L2f1a7G1sLYsbXGtr6i 9XRCsT2/Cc4kodHq8DEyH1VOHP5NEgfR88ObjV0h7fThBsu7UaMqpD7iHSjQ0uhYnbpX UTjnPR9wQqePEmjiseiBgcqy85260s+ZN+LEAB38tRq3kQI0DwyghI6k3Keuhi5xIiqt IyWVuhf8iqzQLYFlCsp4W/nAMsCB+r6tSjgae+dSuOCaKKvoIZGnzvWO9SINB0Uprnwn bh7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584789; x=1729189589; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=/DfuS3cqNKQbdjipc1jp6gNPynmr8H/oDbYVrbrij3Q=; b=Rqw0OWy7C7JoyediKMPKXFtHSsgBhDkxyrDAmV+1qP0gY68CbcK2aUvECBHdMtpLtX qjManC/pmKXCwXAVVulSLRGQ8M1eG6rYL79D1xFvvYTQhZkOWBIT0o+rDoQ+f4Jz5qC0 HovobjLPKeexkwarQa2kNhS1JwSIAUIG5gGWJdIB8PXICiCGGnNlOE3cyRnCrMExQ0nO KFTqztx5Ugohdse5p1zteKkQZVZehC1tp/a4XhMxR4YTrrQMDeqRcDB7LJc4THTsOObw Rtdu4yZp8jXX/nmE1vQ6WFvNqqfrdz499UBXk/EgWt7r49qHI8OYtv5HVcUfHXbrFtJL aUqQ== X-Forwarded-Encrypted: i=1; AJvYcCXgP27xmxf3cJnALwlty9wkqjpRvBxXGo3CMZlWHN/B+BDNiOh9EzbBPmOIpL7b3O0/XVypX2b4OQDQ+g==@lists.infradead.org X-Gm-Message-State: AOJu0YzezKkJ/rTi12jspeYsJsBJ7ZmkI+a1nwMOADnWJehQV/luaxkA 5v1iONL7hyBmkq7pGL5IA3HPz1rK9bNevSuAbUtaIpjvKy76ksFfGepUAh+mVJ2fHbnGjmFhghT FsQ== X-Google-Smtp-Source: AGHT+IHLQFn8WLOIHu4PHZ3K+TrLKBMF9JlmWl30kTXK43TD90b+TNbg3ZepRFQQdmJeel5bU/EhDzobpdQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:b191:0:b0:e1f:eaf1:2254 with SMTP id 3f1490d57ef6-e28fe41acaamr82536276.8.1728584788687; Thu, 10 Oct 2024 11:26:28 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:48 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-47-seanjc@google.com> Subject: [PATCH v13 46/85] KVM: guest_memfd: Provide "struct page" as output from kvm_gmem_get_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112630_564258_F36E0A06 X-CRM114-Status: GOOD ( 14.39 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Provide the "struct page" associated with a guest_memfd pfn as an output from __kvm_gmem_get_pfn() so that KVM guest page fault handlers can directly put the page instead of having to rely on kvm_pfn_to_refcounted_page(). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/svm/sev.c | 10 ++++++---- include/linux/kvm_host.h | 6 ++++-- virt/kvm/guest_memfd.c | 8 ++++++-- 4 files changed, 17 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 2e2076287aaf..a038cde74f0d 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4400,7 +4400,7 @@ static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, } r = kvm_gmem_get_pfn(vcpu->kvm, fault->slot, fault->gfn, &fault->pfn, - &max_order); + &fault->refcounted_page, &max_order); if (r) { kvm_mmu_prepare_memory_fault_exit(vcpu, fault); return r; diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 4557ff3804ae..c6c852485900 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -3849,6 +3849,7 @@ static int __sev_snp_update_protected_guest_state(struct kvm_vcpu *vcpu) if (VALID_PAGE(svm->sev_es.snp_vmsa_gpa)) { gfn_t gfn = gpa_to_gfn(svm->sev_es.snp_vmsa_gpa); struct kvm_memory_slot *slot; + struct page *page; kvm_pfn_t pfn; slot = gfn_to_memslot(vcpu->kvm, gfn); @@ -3859,7 +3860,7 @@ static int __sev_snp_update_protected_guest_state(struct kvm_vcpu *vcpu) * The new VMSA will be private memory guest memory, so * retrieve the PFN from the gmem backend. */ - if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, NULL)) + if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, &page, NULL)) return -EINVAL; /* @@ -3888,7 +3889,7 @@ static int __sev_snp_update_protected_guest_state(struct kvm_vcpu *vcpu) * changes then care should be taken to ensure * svm->sev_es.vmsa is pinned through some other means. */ - kvm_release_pfn_clean(pfn); + kvm_release_page_clean(page); } /* @@ -4688,6 +4689,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) struct kvm_memory_slot *slot; struct kvm *kvm = vcpu->kvm; int order, rmp_level, ret; + struct page *page; bool assigned; kvm_pfn_t pfn; gfn_t gfn; @@ -4714,7 +4716,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) return; } - ret = kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &order); + ret = kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &page, &order); if (ret) { pr_warn_ratelimited("SEV: Unexpected RMP fault, no backing page for private GPA 0x%llx\n", gpa); @@ -4772,7 +4774,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) out: trace_kvm_rmp_fault(vcpu, gpa, pfn, error_code, rmp_level, ret); out_no_trace: - put_page(pfn_to_page(pfn)); + kvm_release_page_unused(page); } static bool is_pfn_range_shared(kvm_pfn_t start, kvm_pfn_t end) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 3b9afb40e935..504483d35197 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2490,11 +2490,13 @@ static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn) #ifdef CONFIG_KVM_PRIVATE_MEM int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, - gfn_t gfn, kvm_pfn_t *pfn, int *max_order); + gfn_t gfn, kvm_pfn_t *pfn, struct page **page, + int *max_order); #else static inline int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, - kvm_pfn_t *pfn, int *max_order) + kvm_pfn_t *pfn, struct page **page, + int *max_order) { KVM_BUG_ON(1, kvm); return -EIO; diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 8a878e57c5d4..47a9f68f7b24 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -594,7 +594,8 @@ static struct folio *__kvm_gmem_get_pfn(struct file *file, } int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, - gfn_t gfn, kvm_pfn_t *pfn, int *max_order) + gfn_t gfn, kvm_pfn_t *pfn, struct page **page, + int *max_order) { pgoff_t index = kvm_gmem_get_index(slot, gfn); struct file *file = kvm_gmem_get_file(slot); @@ -615,7 +616,10 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, r = kvm_gmem_prepare_folio(kvm, slot, gfn, folio); folio_unlock(folio); - if (r < 0) + + if (!r) + *page = folio_file_page(folio, index); + else folio_put(folio); out: From patchwork Thu Oct 10 18:23:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830922 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0C91FCFC5FD for ; Thu, 10 Oct 2024 19:21:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2UDOASqnWKvY+O1IJzNBfWk/0kxnO2gOVfgqkAW7SRI=; b=rrvZ7mqID3xUy8 U94Dvry2aaCZWbEk1C6MmnEcS4qW+6KE/qobud64+7b4x+gvQotTMhvG26NnoyKIW4fzkpQAvkt7P RCgPJzYTijDnW9ZlJukKU4gsbRkLMF6uEJdwO38YrtwaEzVPF30IMi2qwuvsPjIf4B6K9G9phQbP5 qQ+JbryvWFvV3F7z2eEIZrB3CV7GS9HrRT+w0u9RpzUVTkOF6zTVjvtqttsD7GLW7Ml80hjE3n1hY e6q69BYHHdN4byqt7FJxXTmxjCYlJ6eSE3rJSiKmqxiVpbJu40VrIVYJQBJrB3rFxW+O/1WvcaEsS a58ngP5DYWFXKMYz0R+g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyjP-0000000E5ZL-2gdS; Thu, 10 Oct 2024 19:21:51 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrt-0000000DquZ-1Bno for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:35 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-71e0228d71dso1225581b3a.0 for ; Thu, 10 Oct 2024 11:26:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584792; x=1729189592; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=l4aL9ye2ZxrCnqTpLqQlinT65BUul16gfmDzid4NN5Y=; b=4NJxd+23djSQD5IrKrOM7+QBFAb+zUEzLAelhcpuCsN/jNgk8WGR2zJ21WF08csT9V yhMLGsZmfSz963iR9mrnBU4cwLKvwl5BPrO7GoCUIghqK86NVrPcw9z4snYKKkabblTO z/jBEzoAkqypTekJV+lLgO+n/EPna95q8QOQmqUodoUob08Ymw8w1UizJ3J5xwCC3ymz wTklvrSW+SKTZSOpfYN6lllfuRcZhijDV7ltjf7exYNEDFDq4/QVOcZQfFIaQqokMloc 9m29lmfaoY7ZsvkPtfUGCgdPTiTz7Ji2mxd/cOFOBuo50KHj5VvP2JR6hGeFnLweMZll 9c8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584792; x=1729189592; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=l4aL9ye2ZxrCnqTpLqQlinT65BUul16gfmDzid4NN5Y=; b=Enezv8aElsEkDAqc1V28ReeA9A/k760+LGzhEQ6dg+0KAiIpG+YRI53UoYrwhEwWoz Ii7emlBWClTGFX9nl7LPaB6eyDIXMDOtamf3USztnU5FpO6+Np4kiNzgkatrNjOuxM8I 4jXN8mB5zPe/jbwBRLNedOC2U31uRIeOeoqqPZlTIFGrGm1tornCDj5mAMRAR6zuNCAU j/ZUIE/XsAZo4ToVxbSTCEKf6zWOc5eAo8BXpJVXrXmA+fmTtLzPaNZnpHsw82b/zPqs d28E5qpHusGQ9JT94ggOafm1GExDlyUn/bLwt/c6au0MsFpDmM0NK2f+7wYcq/bCPyho oKOQ== X-Forwarded-Encrypted: i=1; AJvYcCUkwfetjZpbGyx4YwKeFUan8YdUkitBG0hvRV7L5517JvJ/QeC91s0YEUWtez4Ze92w+jLxrJQnL5Wgkg==@lists.infradead.org X-Gm-Message-State: AOJu0YziC7UAyWHMhI3PEZsAWGya2extaDifP7oeSCMJgGwIe8Uu9xOP teydmAxPaIJRZu6aFfN8ltoXmOTSw20RuTjK1VysLN73Dpeglx6N3vXjyDV9EC7u38I/ygK8pvU XMg== X-Google-Smtp-Source: AGHT+IFunsAHL0GYDEyxrwNnSExkBiXOv88aJkEgjVxw5ZDXOBsEr4Eha3ByEK4QIBQ4extfr7SnAz+Fl/s= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:aa7:9184:0:b0:71d:f7c9:8cb3 with SMTP id d2e1a72fcca58-71e1dbf1ab5mr7799b3a.5.1728584790633; Thu, 10 Oct 2024 11:26:30 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:49 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-48-seanjc@google.com> Subject: [PATCH v13 47/85] KVM: x86/mmu: Put refcounted pages instead of blindly releasing pfns From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112633_467639_D7832FED X-CRM114-Status: GOOD ( 10.71 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Now that all x86 page fault paths precisely track refcounted pages, use Use kvm_page_fault.refcounted_page to put references to struct page memory when finishing page faults. This is a baby step towards eliminating kvm_pfn_to_refcounted_page(). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a038cde74f0d..f9b7e3a7370f 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4373,6 +4373,9 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, lockdep_assert_once(lockdep_is_held(&vcpu->kvm->mmu_lock) || r == RET_PF_RETRY); + if (!fault->refcounted_page) + return; + /* * If the page that KVM got from the *primary MMU* is writable, and KVM * installed or reused a SPTE, mark the page/folio dirty. Note, this @@ -4384,9 +4387,9 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, * folio dirty if KVM could locklessly make the SPTE writable. */ if (!fault->map_writable || r == RET_PF_RETRY) - kvm_release_pfn_clean(fault->pfn); + kvm_release_page_clean(fault->refcounted_page); else - kvm_release_pfn_dirty(fault->pfn); + kvm_release_page_dirty(fault->refcounted_page); } static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, From patchwork Thu Oct 10 18:23:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830923 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DE8E5CFC5F9 for ; Thu, 10 Oct 2024 19:21:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=H+kfG9DgsKWE1QG4Eq/byBdu8SNPmXP5QO3fNGCa8Ok=; b=hhekYX7zpBd1Af 6618tZY1LDH8KOtwZRv5soFinsrUcLDAv2Zv7Ndx12g+Lsy5nRmgPp1khj4rIdcWsOPordB9M9hPC ZAoWOw9lNG+H0fD3CBDpriGhXmX+wNP8UxfI9R5dri08uQGkYc+Q+t+5M+xAUFnm56PDqsf33aGi+ XQ1Hj3P0+oflSCPKq9JEOFQDtRMCAoS+5SZURzrWarojvUFK6CwcwKILFTaqD9een8eWVrXByXM05 7b3WkliwHPLM2eMD7L4MH7M/7IErd8VZrt3Q6oRqwt9nrorOPH/tnumMesLotWGelYEmFBpmhqLqJ f7k5OSDIGgXRli9jZfZg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyjQ-0000000E5Zy-3Gfl; Thu, 10 Oct 2024 19:21:52 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrv-0000000DqwN-1o6d for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:36 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e2e4874925so21251437b3.1 for ; Thu, 10 Oct 2024 11:26:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584794; x=1729189594; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=BN+hsHae4h8wzhzshUJRGKM30XxzO0F311sC9z4rUGc=; b=mFEQ1pk0GSblJlbvVxjSomWrM6IE+btPApHqV5NrilXRXoFJJ2QtCA8C3DRzZU638K fGwnvg2LFw4lj/o2cvU/G8+4i1rkmzmCYJ+K8kAtgcwfRhJdswGO4ZjYFmc0fOpnGAaO qM8T1NUyZqa09HXc5T1Ol762RIpMtB9T2BxUQSneDHsTS4aPWVdavwKvZRgVyE0BRFlK J2q6XJotNBexnIBUs1cGiNgPYYBQ8CryEKe7aIWrWBYO5N0YgHcBVCFW6MVxRz/ypuMy GI2HAbh6rArHdBxq4xWdbR4NHv2mRM+UDfic7XIfgYAex/UlvCHyl6arPi0UFMRFcmya 8iOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584794; x=1729189594; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=BN+hsHae4h8wzhzshUJRGKM30XxzO0F311sC9z4rUGc=; b=kwLtYtPH9kC97pnEJWI8/Hk8NT5idk5ghw716bD0fLNT1xmNsANU3thZrIhcShdKBV kLH797qai++XZhII5Xe2Tbm8rPXmdMB405NlUdI6rkJEpBcv0I4AVUpaimTvpwBVs/rW ZrF7iBTg7L+JWPd3J6X+hL45CsYfieTnsD/8/vO9NYl8jaF3xEjwUS6ytHmPDXz9Jf6O HRTNFwwuG5SeiS5lAr5YoawwKBlHNqwNg9YuWtbg9QvatZVAZJ/dLT35/ZW8qWnyD+m+ ucTe38h8uDJ2AZ6QJSJmJu8iZTah5ZpMVmnEz6xeiEUZe4kcBiQO3oZ567+J4yAc2kbd iNNQ== X-Forwarded-Encrypted: i=1; AJvYcCW6oSmZkZfHJ099hCZuA/6pljZQLmZLYScv50R1hGlY/M+Gm5bnOdzfC6bXrS3JQSrSqZp59nygI2M4PQ==@lists.infradead.org X-Gm-Message-State: AOJu0Yy+smQTg9AYa38NZjynDuOk+Fu6KDzBptl0W5JK+xo5uuRwm6n+ 9qQpMzrgpClr073Ns/DggzN9pJeEgpMANT0Bg4KFr2qzCpwsrVpRedCvs/99K2ecEryJf+2zA5u 92w== X-Google-Smtp-Source: AGHT+IEjw5s9ln5Mmco0bHKHDYNNIWwRc71SIREcZZABOz/fcRYPRAU5+BeNEqASF/uZrX6+QTap8C7eoHE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2c01:b0:6e2:3e17:1838 with SMTP id 00721157ae682-6e3221407b0mr440167b3.1.1728584793806; Thu, 10 Oct 2024 11:26:33 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:50 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-49-seanjc@google.com> Subject: [PATCH v13 48/85] KVM: x86/mmu: Don't mark unused faultin pages as accessed From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112635_500860_87311F51 X-CRM114-Status: GOOD ( 11.86 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org When finishing guest page faults, don't mark pages as accessed if KVM is resuming the guest _without_ installing a mapping, i.e. if the page isn't being used. While it's possible that marking the page accessed could avoid minor thrashing due to reclaiming a page that the guest is about to access, it's far more likely that the gfn=>pfn mapping was was invalidated, e.g. due a memslot change, or because the corresponding VMA is being modified. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index f9b7e3a7370f..e14b84d2f55b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4386,7 +4386,9 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, * fault handler, and so KVM must (somewhat) speculatively mark the * folio dirty if KVM could locklessly make the SPTE writable. */ - if (!fault->map_writable || r == RET_PF_RETRY) + if (r == RET_PF_RETRY) + kvm_release_page_unused(fault->refcounted_page); + else if (!fault->map_writable) kvm_release_page_clean(fault->refcounted_page); else kvm_release_page_dirty(fault->refcounted_page); From patchwork Thu Oct 10 18:23:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 84CA4D2444F for ; Thu, 10 Oct 2024 20:31:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=WDJzg/rExKHDP4gATntrLwP0JYFHE9qDcQgQGlLeHZU=; b=fQLTFQbXhCiHEy G68EYmu2ztytWZlJPpwSp5nkBcVArT2PFaHrI/GxKzyWmr+yPHDx+48WV6xmg9mEsI+uprp+bz896 QMGavCqinGHotPyQzPnObQjtcmZfxkyOuGcE6KlDa2scAuM7j0bdlY0ETBLLw/QrT4Zq4aC5QE2HJ lFEzeWHReNfZGpsUniMOvSzsYZf81QYoxMue/+JokTw6i5iHQp8orWA10GBLRqJ9tL2BYAYy2l+XI hkVM+ahn1L608zAoVncvQYXuokz1KqygV/e/1SsDU2o9HdQyiOFBI4nlwOdQlbcoku5hiKcWDbDYx metSg9/opIBp3HwJLKOw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzom-0000000EGpR-29MQ; Thu, 10 Oct 2024 20:31:28 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrx-0000000DqyO-39Em for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:41 +0000 Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-7b696999c65so863532a12.3 for ; Thu, 10 Oct 2024 11:26:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584796; x=1729189596; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=Ndpfmsd07O3XHkbROvy9SWySHpmtrF0LUOG5EHCQjAE=; b=4HGW8f7tjownBfYqPr3g4q5rap4pEjc4NDH2XoQtWUJmNugApsKx93exzBPNkaqmo7 22HQMvd+Q4rL+NUcId2bgBlZd7JTUfOuTG+AaGGp5/MieBVOiymMRBah3+emzZqc2o47 EaCg48mEguoYCSN3ZQZnwHaOSKoILqP3AFoA8aYxTJ9yw2yQTyuvbig92YQxp/aOz8iq L7zSmYRy7UmWkkcy15/lY4+BC2929WmoIEeZ+DilgVBJARphsvEY2zJXni0FF0XUYOSc OlRUp7qqzox27rKuo+WQQnNLhRVDeqO+GFbPztsLp2jp1XWndoR4wUm2paLbo9oPpwYg FXkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584796; x=1729189596; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Ndpfmsd07O3XHkbROvy9SWySHpmtrF0LUOG5EHCQjAE=; b=hQ1RrE2mkjQY1GyDcZBZorC7kvBG3334cpmBiwDyznojzkmYIvvDSySuOHuv2s/IW7 zLGB9TjlSIDc4cT7ag2xxMUU1V1xFiRgm62ruUSfHkPupYkwsLVu/WLCkGZKN5H+/hBt 7QDKXxLdlHX9XtVzM2N4YlVsARa2b5GxYpUIKCVtHZtvJ8ZA7cbT00/DAfmqjqVkgrE3 cU5qcwSzZs7nEabucmssshonPF+k6GrJENT4jwFt1q44fQuHbz0oHQa/V1sb5sgwdm7B fsmsskY/hOxrss9cZRFbX4ZN/Q8j3jwzmlLRw9VRjKgBMMPBaeb5BCr7CoYL8IBS0VD/ 2LAQ== X-Forwarded-Encrypted: i=1; AJvYcCVMQosOI81BWnT0iwy1gIXnQaJkMKErDlKejYbigCtEM5HDzGYQAWUkGRicByVsQJOMpIX8ME2VQ67xcw==@lists.infradead.org X-Gm-Message-State: AOJu0YzKwiHvY4kU4oTW5OoVf7lNP4C6dttCmIZZ8B5UtgI1PB45Al6K zbCtPfg8GsbQENMFyRYYn3uRj2aF4LNitPbLHM5IAauh7G/zZRBn4ZIBM+p1NnSOarBlpIov4pg lMw== X-Google-Smtp-Source: AGHT+IGJc8hgBZq6lOovV4Dgly5ctIXEs7jDnJfEupYxVqninaLorgqTg5WNW0BPJZYnJnHdEtp0yWaf7UM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a65:450a:0:b0:717:a912:c302 with SMTP id 41be03b00d2f7-7ea53525e73mr49a12.1.1728584796119; Thu, 10 Oct 2024 11:26:36 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:51 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-50-seanjc@google.com> Subject: [PATCH v13 49/85] KVM: Move x86's API to release a faultin page to common KVM From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112637_877617_873E8817 X-CRM114-Status: GOOD ( 14.78 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Move KVM x86's helper that "finishes" the faultin process to common KVM so that the logic can be shared across all architectures. Note, not all architectures implement a fast page fault path, but the gist of the comment applies to all architectures. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 24 ++---------------------- include/linux/kvm_host.h | 26 ++++++++++++++++++++++++++ 2 files changed, 28 insertions(+), 22 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index e14b84d2f55b..5acdaf3b1007 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4370,28 +4370,8 @@ static u8 kvm_max_private_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, int r) { - lockdep_assert_once(lockdep_is_held(&vcpu->kvm->mmu_lock) || - r == RET_PF_RETRY); - - if (!fault->refcounted_page) - return; - - /* - * If the page that KVM got from the *primary MMU* is writable, and KVM - * installed or reused a SPTE, mark the page/folio dirty. Note, this - * may mark a folio dirty even if KVM created a read-only SPTE, e.g. if - * the GFN is write-protected. Folios can't be safely marked dirty - * outside of mmu_lock as doing so could race with writeback on the - * folio. As a result, KVM can't mark folios dirty in the fast page - * fault handler, and so KVM must (somewhat) speculatively mark the - * folio dirty if KVM could locklessly make the SPTE writable. - */ - if (r == RET_PF_RETRY) - kvm_release_page_unused(fault->refcounted_page); - else if (!fault->map_writable) - kvm_release_page_clean(fault->refcounted_page); - else - kvm_release_page_dirty(fault->refcounted_page); + kvm_release_faultin_page(vcpu->kvm, fault->refcounted_page, + r == RET_PF_RETRY, fault->map_writable); } static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 504483d35197..9f7682ece4a1 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1231,6 +1231,32 @@ static inline void kvm_release_page_unused(struct page *page) void kvm_release_page_clean(struct page *page); void kvm_release_page_dirty(struct page *page); +static inline void kvm_release_faultin_page(struct kvm *kvm, struct page *page, + bool unused, bool dirty) +{ + lockdep_assert_once(lockdep_is_held(&kvm->mmu_lock) || unused); + + if (!page) + return; + + /* + * If the page that KVM got from the *primary MMU* is writable, and KVM + * installed or reused a SPTE, mark the page/folio dirty. Note, this + * may mark a folio dirty even if KVM created a read-only SPTE, e.g. if + * the GFN is write-protected. Folios can't be safely marked dirty + * outside of mmu_lock as doing so could race with writeback on the + * folio. As a result, KVM can't mark folios dirty in the fast page + * fault handler, and so KVM must (somewhat) speculatively mark the + * folio dirty if KVM could locklessly make the SPTE writable. + */ + if (unused) + kvm_release_page_unused(page); + else if (dirty) + kvm_release_page_dirty(page); + else + kvm_release_page_clean(page); +} + kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn, unsigned int foll, bool *writable, struct page **refcounted_page); From patchwork Thu Oct 10 18:23:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830924 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 27EC8CFC5FD for ; Thu, 10 Oct 2024 19:22:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=AM40cmduuKQ+NYae/zEfKYDNmevDcOhMyFqI1Q/YOMQ=; b=Ptt05d/qhNmpbN C2Vvj+UMB4DEf/Qg8nnh2uWE75XCKsz8nHK05CDkJ2EX4msin9xtpfTUBgHPxawZ8tAe77I2W1yUR MlnKgVcNdUmi7srQzaZSOxaCq+lSGbTteAu8xsGxtI1t7fMpNXrAo48z26KAv4jrUzM6JhMlU6aLJ AGkVnGwFTI28kCQ7vgZABwJnfTTdgT20+CZYDiWAr43MMAO+rAshIZUd5HsDJpHAqeuNes896acvT BJA1J5JUoLWjagtDe8z5EUFhSj2cEi5OQjFwb25S/9gFlLh8ghOD+RRb3bxhl1PEtPRWtGAgANbCY j9O02G409eUFX6C/YZzA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyjv-0000000E5ke-3Fm0; Thu, 10 Oct 2024 19:22:23 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxs1-0000000Dr16-1bUF for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:42 +0000 Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-5e4df21f22dso1173234a12.0 for ; Thu, 10 Oct 2024 11:26:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584800; x=1729189600; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=iay4RI/uPKTQE0By5qqA//dP+sG5w7++36T/RHxbCNM=; b=elSopkBGV/LG/XHB/FpT5FQ1XT7+s8gQjY51jYXfFfnx0SB95Fq9HirqWw0fsoteYW Bffj+ulNayw15y81/0D2n6Yv0IDMpZGIOP92+zOPG2I+hWzmnSL6j82Db+Nnuc+If0F+ jRQFR5zcK+C9bg3YuwgbQ6dacmJfuU9Yzf5hjvl0BdTAcz0oc/hsnwzTI6VTwzB1N359 yYxZs5N9ir8CnrpandHwL87esEKL7WNWT1uDtaC+el7gvgzBd+WfI6Ag1K5EEQ/8XTqN hmhMLcy6QLvFtNVMU8MuYmQBYmGdxbPvlTqSq9ZDA868gww6BrArZnM7UTRqLXqpeVdC 26wQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584800; x=1729189600; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=iay4RI/uPKTQE0By5qqA//dP+sG5w7++36T/RHxbCNM=; b=vEBROiZ4WjfNGIYBqpociiCFfY8rGcMIyAsn/9PeAGT05WhOrTgmNBHQWzmsc8Hele +PpU7hO7/Hzp8RWAd6tadtfZBA5yWdWcivD78cEG+R2tFbs9EhBFTwmxP1tBch5bdv9R V+Wk3Dqxkxp0HXtFB+S3xjm620bAywwuiYxuKW0nHYnM+PBBuDijirEct49+s8bcWNZZ z5/qqLgPia8E/2ldIzQLS/CofMf4dUwQsWkbP/w2XP6roJN2VsLPoazKeCemzANfGWXe DghcNAWnhs0tgNxEALPd/MxdEcewDMCCMj9AcJg8JeN9VQPIR/mbr+iqsORcn4aJLQ2y XNgQ== X-Forwarded-Encrypted: i=1; AJvYcCWjG4oqmsTYtqLd8NDirLXt7xmxrgZFeAujEC+Gp0m61n3pAFCzZ2w5x4bOqMUEs/A32U6+9ivnBXCtnw==@lists.infradead.org X-Gm-Message-State: AOJu0Yzj3lh51eDrS+WIDDDmIE08niWrcLNAAcLre5/UOd4pX+/1dqvO AV87YLovfEFKcgAbdSgs5bGFNhobEYp5cPYYe104Oi3FuuNuT4ubQNGNRXDgQGI437GrOYlWpXS OSg== X-Google-Smtp-Source: AGHT+IEd4uxjZrUyEQQM0MU+jkiJqh3LQboji5/ZXtrCgOTVgrRv0yMO5ln1NKE0y5PwnoqkXxjnsp8xFDc= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:fe86:b0:2da:872e:9ea4 with SMTP id 98e67ed59e1d1-2e2f0d7e9d7mr54a91.3.1728584798284; Thu, 10 Oct 2024 11:26:38 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:52 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-51-seanjc@google.com> Subject: [PATCH v13 50/85] KVM: VMX: Hold mmu_lock until page is released when updating APIC access page From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112641_505693_98A2815A X-CRM114-Status: GOOD ( 12.67 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Hold mmu_lock across kvm_release_pfn_clean() when refreshing the APIC access page address to ensure that KVM doesn't mark a page/folio as accessed after it has been unmapped. Practically speaking marking a folio accesses is benign in this scenario, as KVM does hold a reference (it's really just marking folios dirty that is problematic), but there's no reason not to be paranoid (moving the APIC access page isn't a hot path), and no reason to be different from other mmu_notifier-protected flows in KVM. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/vmx.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 1a4438358c5e..851be0820e04 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6832,25 +6832,22 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) return; read_lock(&vcpu->kvm->mmu_lock); - if (mmu_invalidate_retry_gfn(kvm, mmu_seq, gfn)) { + if (mmu_invalidate_retry_gfn(kvm, mmu_seq, gfn)) kvm_make_request(KVM_REQ_APIC_PAGE_RELOAD, vcpu); - read_unlock(&vcpu->kvm->mmu_lock); - goto out; - } + else + vmcs_write64(APIC_ACCESS_ADDR, pfn_to_hpa(pfn)); - vmcs_write64(APIC_ACCESS_ADDR, pfn_to_hpa(pfn)); - read_unlock(&vcpu->kvm->mmu_lock); - - /* - * No need for a manual TLB flush at this point, KVM has already done a - * flush if there were SPTEs pointing at the previous page. - */ -out: /* * Do not pin apic access page in memory, the MMU notifier * will call us again if it is migrated or swapped out. */ kvm_release_pfn_clean(pfn); + + /* + * No need for a manual TLB flush at this point, KVM has already done a + * flush if there were SPTEs pointing at the previous page. + */ + read_unlock(&vcpu->kvm->mmu_lock); } void vmx_hwapic_isr_update(int max_isr) From patchwork Thu Oct 10 18:23:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830925 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 61681CFC5F9 for ; Thu, 10 Oct 2024 19:22:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=hHJavT1gA0jzeJpBGEK8S0nGxH5Uz8WSzr+n7zheO1k=; b=CyN0QEvj4Rseoe MT22tbdbjRWIzkKgEXqagxxHai2Pvv/iqjMX+RSnA9yWofY/dgDEzBt0dbuDrXSuyZx1UBVzoGinH yK/YNqIqVoqPlmoz+QQl1oRVti1NA5PgTBbx6vreqzO1VuPEj8Cda9YepUCDhXldiWXbCx22ymy/k Bah1UZZtBxPjzNjKc2yz2iRr1hw36LuRBowmTeTHivk2wJMKfurpUGJEI/p+pnBAZEaHWJqU7rbdy yzj1cbtTwcRyA9wLeS3W1tBFrdtnIL5lOwa2oVdtCEYWqBY4nNdrxKPdmbEsnOpyWTMhUs2NVu2Io aM8YvQ744/cVv06Rt5Kg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyjw-0000000E5lU-3frB; Thu, 10 Oct 2024 19:22:24 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxs2-0000000Dr2U-3vNi for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:44 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e284982a31so22453657b3.3 for ; Thu, 10 Oct 2024 11:26:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584801; x=1729189601; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=W6Zzl9AmyqedpEIwNpLXiwPcmrnHw6ZPa/9pyFpLcZ0=; b=OrgrVnTtrz+qiD9I0S55GdwguFxumnKv3WwSP9lTvNcek28RXofi/P4YgBMLPPYMhb zCOiCCo9fd6Nk8bAoy3uGkg5aB+LFL9zzvsIoGmtZJWHk7S3XH6jGl4UgTdmy0jdzTRe uigdTw+2nDTBogSg6xzYhiat0dWmlOkE4NK4fdBCXO6eLIjWbfaMMyYAgIx+4I9JXs+q YYlcvGdNTOyx4Od36oRcYng2RADTfSTd5NrCF3iOCvOKsNAyx+6m8RHLPIhQGep99KBK 95kPdxMBUQVU3/eq0UM2UXAuETUAQK8pz5gNiTU0JXjKfe390eNJWy/Ucn6yEkU139JE isgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584801; x=1729189601; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=W6Zzl9AmyqedpEIwNpLXiwPcmrnHw6ZPa/9pyFpLcZ0=; b=k6uKLd8yFAM4qd0iGB/fw8AGvAdYa4Xw1R17c+RWAN0CPdNCP6qmN0yA/I9R9zliJx Mw956GsyFwe4Ya1dd/XBAOBOtGiqXFO5ndGXB0Cb6DVbVGHvuRNmnuo2zUoA28z++5YM aijZV9K/FrN/7u7nXbcPdrFoBpIwCznxjglkm6Fqa3me9mQCH02Uuh6B+kvGta7bYP9F 3ZH80cakaPAwiGnFTXPR9jaEpbYR6Tb8ptwEjH/jMA3Uc2CiJ7xiHyn2ygJq6QdgMWw0 /7LFfccLTCcYMH6fl0xt9gc0fn0mvs1MyGLQkxqky98QzolqMN4ez2F0T3MQeFYQTKZO 38+g== X-Forwarded-Encrypted: i=1; AJvYcCWIUGI7ENtjtG05XUJxfFn2s4kSNh3yuuV8N535Ley7dHrwtO5Z+iwHgWi0KBjQmUTrmRfv1A/iuMcMDg==@lists.infradead.org X-Gm-Message-State: AOJu0Yyg4raZe1MZ2/s8V+i7TfP2sq/anUKc6FyQ04rtX0BfcMj4KXX1 ObcUKqHyO86zjj57l/hpkyQt2b28menr+7/xXgMvf141nUSbbjiX3uSEfAIcpJo1qOBuanHK1r9 r4Q== X-Google-Smtp-Source: AGHT+IGFUTZueI+Si3awRWj8jA+YGlhXMEzUcm/2nm7iA7dHF8070ZlvI1MSF0nUJUhFBhXPPz9v/uVg5Fg= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2f08:b0:6e2:2c72:3abb with SMTP id 00721157ae682-6e3224db667mr816567b3.7.1728584801239; Thu, 10 Oct 2024 11:26:41 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:53 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-52-seanjc@google.com> Subject: [PATCH v13 51/85] KVM: VMX: Use __kvm_faultin_page() to get APIC access page/pfn From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112643_100237_CE83C25F X-CRM114-Status: GOOD ( 14.22 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Use __kvm_faultin_page() get the APIC access page so that KVM can precisely release the refcounted page, i.e. to remove yet another user of kvm_pfn_to_refcounted_page(). While the path isn't handling a guest page fault, the semantics are effectively the same; KVM just happens to be mapping the pfn into a VMCS field instead of a secondary MMU. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/vmx.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 851be0820e04..44cc25dfebba 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6790,8 +6790,10 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) struct kvm *kvm = vcpu->kvm; struct kvm_memslots *slots = kvm_memslots(kvm); struct kvm_memory_slot *slot; + struct page *refcounted_page; unsigned long mmu_seq; kvm_pfn_t pfn; + bool writable; /* Defer reload until vmcs01 is the current VMCS. */ if (is_guest_mode(vcpu)) { @@ -6827,7 +6829,7 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) * controls the APIC-access page memslot, and only deletes the memslot * if APICv is permanently inhibited, i.e. the memslot won't reappear. */ - pfn = gfn_to_pfn_memslot(slot, gfn); + pfn = __kvm_faultin_pfn(slot, gfn, FOLL_WRITE, &writable, &refcounted_page); if (is_error_noslot_pfn(pfn)) return; @@ -6838,10 +6840,13 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) vmcs_write64(APIC_ACCESS_ADDR, pfn_to_hpa(pfn)); /* - * Do not pin apic access page in memory, the MMU notifier - * will call us again if it is migrated or swapped out. + * Do not pin the APIC access page in memory so that it can be freely + * migrated, the MMU notifier will call us again if it is migrated or + * swapped out. KVM backs the memslot with anonymous memory, the pfn + * should always point at a refcounted page (if the pfn is valid). */ - kvm_release_pfn_clean(pfn); + if (!WARN_ON_ONCE(!refcounted_page)) + kvm_release_page_clean(refcounted_page); /* * No need for a manual TLB flush at this point, KVM has already done a From patchwork Thu Oct 10 18:23:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831082 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 40F58D24450 for ; Thu, 10 Oct 2024 20:31:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1PaIjMym6rlZ7pDV/m1+7CNUf46wqjgd8B44fc6rrUg=; b=WjrcMUtEdY3SMm tFrRkfMxbSHix5D1JqM3s3v+b2SzdXJZmUh7uzJSeGUjqnPzZ3m8kAFXEOHPb7muBWLuze4e/UKGS hSA8hFBHJFshcV2BO0pD6cfyc5Ba2XlOJYQzTF8VgSPHlfgZSuc3rzlvptFLYIUhCN69/6/rG5wli ueO4ORl7f1RLOnGdn1KVxl8gkDJGSJFwFe5IY/fwXjvnqKCp2Q/XwROT3qsMhNHpXXUeRQf2885TD u9S97jZjqN9X3N0POsCKmuUwRSN4zCRbTsp8BPK1HU8zNrKnjL/JvAqoW6ZN0LqPVLEFCT3JO1BXn e3IJVFlEP5NGptE0qYUA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzoo-0000000EGqc-1X2S; Thu, 10 Oct 2024 20:31:30 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxs4-0000000Dr49-3wCe for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:46 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e29b4f8837so18952217b3.0 for ; Thu, 10 Oct 2024 11:26:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584803; x=1729189603; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=/fC7L3Fchvvuu9GyGKk9VEzuiyLOIeBtO2SWEaXDNWE=; b=BnLmoeZUgDZeTfvLJFFaz58gmWaTmR+ISKXM1NM9+8Tz6u4K97yuHNDt/+ETUMaj+3 fbCJPJ11scsCf1SP+PIrpg0K/mVzzcYJSLqADVfzCIHdWlRu3FqR50x4ezGvzQsafiwI V/afjWOZ/n1EQinWq7bWXdQ4rQ6WU70i1MUJGIwMRhIC/nDA9OKg2xkV95XZg4jVhnIq MCaeu9xDMI7eJlCThHlmOwEzeN/sh1eYMUjy5l3EXfzuPM2unnM2PEBaD7dRrjT1aG9O 8Z7SFpLghppE2P8lmROc1IplRfOtIfNZcuJoAatjBuBbu51DS2IopbOgls6I3ZHBiyPh LPFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584803; x=1729189603; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=/fC7L3Fchvvuu9GyGKk9VEzuiyLOIeBtO2SWEaXDNWE=; b=Dm0kAWP5xXqGTftce5t2xgTGUj1dXoxBeVY2PlxfL9ob2k42g1R8yZdw+5a3UKZPe3 Fc3GvWhqNrEkWVDFlEx3jNJATz0+Ls1hlKaKpEZLPZ9KqeOmL/MVMrEJXKRRHkLvWg4k Y617mV/E5REH20UbzCCY0oSPX0E3HmaEktX3yW27XenqBGEDleuPkxoznO72TowLqGAw /BcSrmBaPCHre6oa6i+2I6E12fQI2fpEiG/lB80LPD8L/uGB0eol9wsJ1dH33i7n+FZ+ Eb6pHa3cC4g7K7HfRXLHF0CNnCDdopVsKQZXr78FBbXlilvdBn6zxfG630BriclepZcO EHNQ== X-Forwarded-Encrypted: i=1; AJvYcCVd+44H5Ud5s3W4WpXxgc19wJnNabudVgMB2C5U28WljzHsbsr3Q61S1UzOW6SRPUk7On5nBXiUAHUlrg==@lists.infradead.org X-Gm-Message-State: AOJu0YxMuwIWclQlyKOv9coLqdrjuVenk5Zdl+v7aZp2HFvg/FKqS+Ju aPwZAigy4CyFNIR3SA2W4x7DI0h2VHWW0P+yqjKgeovVhL/QKMnatStVsrY9zKJwizaJjDJ6J07 CjQ== X-Google-Smtp-Source: AGHT+IF0bQSeBV+k7XDKFBXYzNvC7N99ueMTlOEEH5KNk6JZHzu/00W1ndtLnT51eyBFoV94Bmz3yNyB+P4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:dd1:b0:6db:e464:addc with SMTP id 00721157ae682-6e344ce9308mr1337b3.4.1728584803335; Thu, 10 Oct 2024 11:26:43 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:54 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-53-seanjc@google.com> Subject: [PATCH v13 52/85] KVM: PPC: e500: Mark "struct page" dirty in kvmppc_e500_shadow_map() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112645_112435_6D23657D X-CRM114-Status: GOOD ( 11.69 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Mark the underlying page as dirty in kvmppc_e500_ref_setup()'s sole caller, kvmppc_e500_shadow_map(), which will allow converting e500 to __kvm_faultin_pfn() + kvm_release_faultin_page() without having to do a weird dance between ref_setup() and shadow_map(). Opportunistically drop the redundant kvm_set_pfn_accessed(), as shadow_map() puts the page via kvm_release_pfn_clean(). Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/e500_mmu_host.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/powerpc/kvm/e500_mmu_host.c b/arch/powerpc/kvm/e500_mmu_host.c index c664fdec75b1..5c2adfd19e12 100644 --- a/arch/powerpc/kvm/e500_mmu_host.c +++ b/arch/powerpc/kvm/e500_mmu_host.c @@ -242,7 +242,7 @@ static inline int tlbe_is_writable(struct kvm_book3e_206_tlb_entry *tlbe) return tlbe->mas7_3 & (MAS3_SW|MAS3_UW); } -static inline void kvmppc_e500_ref_setup(struct tlbe_ref *ref, +static inline bool kvmppc_e500_ref_setup(struct tlbe_ref *ref, struct kvm_book3e_206_tlb_entry *gtlbe, kvm_pfn_t pfn, unsigned int wimg) { @@ -252,11 +252,7 @@ static inline void kvmppc_e500_ref_setup(struct tlbe_ref *ref, /* Use guest supplied MAS2_G and MAS2_E */ ref->flags |= (gtlbe->mas2 & MAS2_ATTRIB_MASK) | wimg; - /* Mark the page accessed */ - kvm_set_pfn_accessed(pfn); - - if (tlbe_is_writable(gtlbe)) - kvm_set_pfn_dirty(pfn); + return tlbe_is_writable(gtlbe); } static inline void kvmppc_e500_ref_release(struct tlbe_ref *ref) @@ -337,6 +333,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, unsigned int wimg = 0; pgd_t *pgdir; unsigned long flags; + bool writable = false; /* used to check for invalidations in progress */ mmu_seq = kvm->mmu_invalidate_seq; @@ -490,7 +487,9 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, goto out; } } - kvmppc_e500_ref_setup(ref, gtlbe, pfn, wimg); + writable = kvmppc_e500_ref_setup(ref, gtlbe, pfn, wimg); + if (writable) + kvm_set_pfn_dirty(pfn); kvmppc_e500_setup_stlbe(&vcpu_e500->vcpu, gtlbe, tsize, ref, gvaddr, stlbe); From patchwork Thu Oct 10 18:23:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A8D53D2444F for ; Thu, 10 Oct 2024 20:31:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=c3LckTz2M1CucM/bD60OGn9zknespQVe4quZthjDaRc=; b=WcC473x15ZR/d2 OTP0Wu9BKnRblB/2viFZrnUvduuHVnSb5X3Amo9WuR79vaAYXaGL361QcySUKEYuUjvjYc9Trkp1x 5ezYWtHqM5xmpT58HxROuS7AHRK9RmGljqeVq9KYRKH/+EzkG//OTOTAHNEzBkW4/wxNSFZ2wp3v+ hubC6XrdoBFhC+DCON8Vhzw0q36KDM5iz0Y/eJjLr1wlDuDcCWptSLOOjF0bAlTRCjFcf7xPO3/O/ St6TPyKSt9FPPwJXxvHB155pdgXeSe4cb/pMU8TlL3HCbFaqN4A0S0KxbbmCXvXs/Wdhiy+4GdoHI MQwpIup9I741T9y0q4gw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzos-0000000EGuI-2gna; Thu, 10 Oct 2024 20:31:34 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxs6-0000000Dr5y-2QWU for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:48 +0000 Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-7c6a4d99dbdso297879a12.0 for ; Thu, 10 Oct 2024 11:26:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584805; x=1729189605; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=RAEBOjiR86B8MKio5svj+gL/GJb0VtWq7ImNxArS0H8=; b=M+42XwPeNm2oSLyAg3Vl9WyPEJno5UFR5cRhnp+qZDz+0cqpTMt6YrirrABxaPv4FV X5sXr42pqnv6G1sQWDVLbO442GXAXQG2xyM3M6hcjelKEbRIy6LZO34fqlWHgsRi7Dn5 pe3x97wUmaoCQM1ZXS/3Vl1pXCclzZ79Oao4ekCGwyg+ntTW5kORcLutbnMz2yqINPCW e6N8R1w5Gma7o0kQmT4SxmZY4nm22nwQ7NlqJ5SdPUEv2jT5Vm9fPfwj7eK0gmVTDy6p /bUqBoiqtFPjelGUsNSKXHemm2iWSIzBddpPfZJioxZ+Ky5Ja03rjPcof5VsFg8VXqyr vjOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584805; x=1729189605; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RAEBOjiR86B8MKio5svj+gL/GJb0VtWq7ImNxArS0H8=; b=AP2xdsWPhr25Qw7/1r73MVwDx6tRhFcXmJ0OXm+3v2i2oxxbCBQG4JfpJXj/nfUuCh gB7g9LI7sd2t4PgbWGvsXjsAXtzqcRyLYT7ELjYIxx1njeXiK5yhS+V8bKRD1F71BMzK KPkCjYB/M6h6GoheI0O3NMCz24CN6z/MgkMoai/N2DDhiG1IiiPLaxbgyM95LLMz+EIO uz7o0eRL3hw5Qz/6q9ir/950ek2gJJYYKMO8Ve1leTm5c6o3PXnkgbUeDqBggZn5O3QQ 78HRmOjUNq3ZHVgb9DJsfMDWQ5zy1Jy32DCEVc/mVgpDpNeed9XBgn9iLU3bYpyNmKeb vkKQ== X-Forwarded-Encrypted: i=1; AJvYcCWlZlEuePTjArCWdC+n3UGRtmiVnbexWnJFaeByrxR1KzYb/NbZkd/zK9doijtcSC63NufSprQoe7gBTQ==@lists.infradead.org X-Gm-Message-State: AOJu0Yx1xB6eGo7izxxGU3tUz5/hFtkBKpNKIvWij7BmwGgIjBUGU4Rc 2bQfj01yKZRtOBVNHbhSJ6F7u/auzti1XYJ2d5ZVTgCa6RhTDxKJeTkzYSYCu4JK1VeASH/Np/G /fg== X-Google-Smtp-Source: AGHT+IH3g+or1uGKI8zaqA7g6VjAId1uToioip8sNGZo/KyXggeZoODfUOK1q5z450FXHERnWYgwR7EwX2I= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:5107:b0:2e2:ca3e:10fe with SMTP id 98e67ed59e1d1-2e2f0f88a54mr24a91.8.1728584805241; Thu, 10 Oct 2024 11:26:45 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:55 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-54-seanjc@google.com> Subject: [PATCH v13 53/85] KVM: PPC: e500: Mark "struct page" pfn accessed before dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112646_752411_B05B90BD X-CRM114-Status: UNSURE ( 8.22 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Mark pages accessed before dropping mmu_lock when faulting in guest memory so that shadow_map() can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Marking pages accessed outside of mmu_lock is ok (not great, but safe), but marking pages _dirty_ outside of mmu_lock can make filesystems unhappy. Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/e500_mmu_host.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/arch/powerpc/kvm/e500_mmu_host.c b/arch/powerpc/kvm/e500_mmu_host.c index 5c2adfd19e12..334dd96f8081 100644 --- a/arch/powerpc/kvm/e500_mmu_host.c +++ b/arch/powerpc/kvm/e500_mmu_host.c @@ -498,11 +498,9 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, kvmppc_mmu_flush_icache(pfn); out: - spin_unlock(&kvm->mmu_lock); - /* Drop refcount on page, so that mmu notifiers can clear it */ kvm_release_pfn_clean(pfn); - + spin_unlock(&kvm->mmu_lock); return ret; } From patchwork Thu Oct 10 18:23:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831084 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C3AA7D2444F for ; Thu, 10 Oct 2024 20:31:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=mTdnpZ7JGQitYNiafNKXELF+PspIaiBa0jHtVVqySk0=; b=wjcvm8U2pSUlEK yZEF/BgFiq9NT/l8uYcdeyUsqpxNQkv0+TcCeJrPuLwB7S099KR2GH8ImXUUa8piXsU91jDft5NOP vzn+DNO9QSBu/Bgndlk80oY4QT27CkX1alR6QHvKCrEKJ8mevdO/sN+3aLsbGv8JEPKB+QX55CuIW QQ3+D6uwF95A8a/gtZpyOUcjjNaJ/hn/mjO/P61xDCKpkZ26ntOojeoOzGvJpr7l2XzM8DM2r2mPG YPOLbwtkjWhSy+3XpqRKIirBrdRo7UmdXOV82B5SOG957cwCBOaQ06TYxOgL9OnW2P3vnYS3IlzNU WWpTdNddFNNxtaWH7a1A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzp1-0000000EH26-2nsS; Thu, 10 Oct 2024 20:31:43 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxs8-0000000Dr7L-3HbD for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:50 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e28690bc290so2031250276.1 for ; Thu, 10 Oct 2024 11:26:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584807; x=1729189607; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=nHP38EtPz8K1+ebGaPqvgAKvgaQP36cKGSrn8JpxRdw=; b=uPMYi0jlcTO7wI+Xm2S76hfStVln/w8HUwai+nZMA8xWjcjtyTkYO5BLvyQK9HVnVf IoC4lxHelaOpDAbOwDoB+oedr7QlY1s0rbnaO+yPdv/IKAl3sf4qGFiPJRU3CPMHE1Ts bmeAWdy0d19eYiKgbHnbIWmxF0u1ugZLdKpLZO2jEwuBE0mZanlwFxa/r776lqNmdndR 7aH8b9KOMLsBIOlzOWkLUDwWW1S1geoFO5SEFRbl3u07z8W40g/J3NQFWAUGK5Zf9guL qrxNClMmqikhJpVN2wcLFUeTcVHJZSytmG4budlvJ+haM4VLpLTbPtFg47vTPcWmRRDU xtzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584807; x=1729189607; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=nHP38EtPz8K1+ebGaPqvgAKvgaQP36cKGSrn8JpxRdw=; b=vKy7ct3HWJgohfguumJYkUHHUL5zhVIlb6XDtTs9sqhwrhcwtVYwWaK+TtiO4AnZXa jUKBtBJKo/UQdXl1PbCOBk4A49Vlv6dAE0nnJw3GFmPUqhKmmMTgAPrdla7DZ04Tgyr0 4cVszaOd71S/7oYWeWV7dJWQ3cXNxvUDi4YRChidFII9pNiskIUh3T+0WxO1L4eIjnb+ yF0SJdn6QOj9GKIATc3S5Kx89Kj0gcsUSWPDBX7Qr8TNH2wib1KglgnzQ9qZt27jE86U yqo8XX08pLwEMwvKBg36R7nlQNFe4RzJ/Hm+a+csr8h60GhxywUPHGEakP4AVlYh9MsP u2Vw== X-Forwarded-Encrypted: i=1; AJvYcCUAOnpFFSGS/Ft/6I4i8kxOpbKcIzEG7z6nNTg3X+gEmjKlPpkwgCmZheUfK6sYohIaqO45M+aLvcNKHw==@lists.infradead.org X-Gm-Message-State: AOJu0YwmLNiRrfnuLp6tJMtnyfRBBRQcI+o5MJ18RErjNnhEaLNImFkt KRBeArEz8TtbpJDtvr1VepOuomKIcSY2Q/lPs73fIXdgvhbTWJ2Yb6W2PAc4kNapue+jsgB8waf Yqg== X-Google-Smtp-Source: AGHT+IHQvsj0FcPHwm2v4Ks73fb0XY1uO3QmFFCKoUyfp7OKRYByAKtc8uXQ4anZ8QaSR/IH5oavOTtruEI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a5b:b86:0:b0:e0e:8b26:484e with SMTP id 3f1490d57ef6-e28fe516b5amr5188276.8.1728584807403; Thu, 10 Oct 2024 11:26:47 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:56 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-55-seanjc@google.com> Subject: [PATCH v13 54/85] KVM: PPC: e500: Use __kvm_faultin_pfn() to handle page faults From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112648_860470_81F950DE X-CRM114-Status: GOOD ( 11.77 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Convert PPC e500 to use __kvm_faultin_pfn()+kvm_release_faultin_page(), and continue the inexorable march towards the demise of kvm_pfn_to_refcounted_page(). Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/e500_mmu_host.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/kvm/e500_mmu_host.c b/arch/powerpc/kvm/e500_mmu_host.c index 334dd96f8081..e5a145b578a4 100644 --- a/arch/powerpc/kvm/e500_mmu_host.c +++ b/arch/powerpc/kvm/e500_mmu_host.c @@ -322,6 +322,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, { struct kvm_memory_slot *slot; unsigned long pfn = 0; /* silence GCC warning */ + struct page *page = NULL; unsigned long hva; int pfnmap = 0; int tsize = BOOK3E_PAGESZ_4K; @@ -443,7 +444,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, if (likely(!pfnmap)) { tsize_pages = 1UL << (tsize + 10 - PAGE_SHIFT); - pfn = gfn_to_pfn_memslot(slot, gfn); + pfn = __kvm_faultin_pfn(slot, gfn, FOLL_WRITE, NULL, &page); if (is_error_noslot_pfn(pfn)) { if (printk_ratelimit()) pr_err("%s: real page not found for gfn %lx\n", @@ -488,8 +489,6 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, } } writable = kvmppc_e500_ref_setup(ref, gtlbe, pfn, wimg); - if (writable) - kvm_set_pfn_dirty(pfn); kvmppc_e500_setup_stlbe(&vcpu_e500->vcpu, gtlbe, tsize, ref, gvaddr, stlbe); @@ -498,8 +497,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, kvmppc_mmu_flush_icache(pfn); out: - /* Drop refcount on page, so that mmu notifiers can clear it */ - kvm_release_pfn_clean(pfn); + kvm_release_faultin_page(kvm, page, !!ret, writable); spin_unlock(&kvm->mmu_lock); return ret; } From patchwork Thu Oct 10 18:23:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831208 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 006F6D2444F for ; Thu, 10 Oct 2024 21:46:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=6KJedHEf2SytfhI8hto1yN7Wlp2EDp2GRBQ3t9n3P2o=; b=CSVZxc7rMPz9wi ZSfavjfIh2Kmm4wmplgaMnd+R8+37AAewIscq+qsyNUhsugd61bAS7963hEYdB0gc/shpjc4okevc eUluUA/1aRVREDtwCs5AmhBeUihq0F8uGS0bFUbIlHLMWBJGIkZynwzdB7g4ahNJOs6EToHxvplpw kLY6yGJ8sFfC6vh3KCfLGOVQ6QUDBbL7QgP6bYUcXcx3l5JLw3cT3qBv/GHRH3viIUjIUpKWOZbi7 53wZLUiicZiyb1sdHOyz2vHWaRgcPhYmE4a2hno8JK+Oc1CYXIAv/rnMfuvocnBOqLy7/Gn8/TG4D 5tTmEZUMXpgpJcINl80A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0zK-0000000ESTE-3mgv; Thu, 10 Oct 2024 21:46:26 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsB-0000000DrAM-1Lff for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:53 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-71e019ab268so1589445b3a.2 for ; Thu, 10 Oct 2024 11:26:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584810; x=1729189610; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=K6YUqJX41UvLftiTjNafBx3/10X+6KBTIzQjXduU2xw=; b=0NCiVq29rFGnhx5H88IoL3ATW2C2rUWOjXZSdIu+X2IO47lSMoJniHN5TNSeqAXiCE vdYQvKFrx3oDn1gRyQMyhpHTz+6quCptZkRRo3o7mi+i280e9CsMWrw12MOxLkLf/z0d n9S3M2ueKS5c0w86AOpvJSx6WYvwwYWNDLTNWHkWQHmHyk73I3u+hhvSuESEc9jxPtu9 I8RiTJ2ti60nNLRKX10QJc1F/7mmAIhzBpW5L7gTAasZ32ELbTbyr66jwUWqpuYSli8q TzZYiRXhNgW10vDG12czBFzP40DQAV1W9mSZKLrtG3DwUE5f8tQuEUoHd9oyl6g+1FqZ QTyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584810; x=1729189610; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=K6YUqJX41UvLftiTjNafBx3/10X+6KBTIzQjXduU2xw=; b=mAhgD5o2g2O/wh3Hxq262yuzToEgSiwkVF3A7+U/f8yfPch29PuWgx+A23j270cIWe BuIs3cC4ZQowGA9oBor5mb9U931Cho/szi8urpHOI++JEWaPYM80MtF699WD98tf8FcX JiJsg5PGq9EZsvizY4xvT1JdyWFgSB2d7K6/6RYDErgyKkZamz+jo592HnG85vwbETXb VGLTITTdrLQLtGKL5I1UEdLd2HmvWfuE5qIjL+lQVRhJqvUmV/jDKRdZQpS3IYHE2MfK XDy7zOhLkbV8YJVx1sZobI1d+lvZ6CsTXkbVpVtZgwxGjiSyImmEz4xcP6MdteNROgbw 47Nw== X-Forwarded-Encrypted: i=1; AJvYcCWsn4jVc1tS6wk7Eke5p+Czev8ZpuHfMn9mqqdH2v7RJ3UOhUJfK7YoFul7R68zqHEHwF8974CwdZbsbw==@lists.infradead.org X-Gm-Message-State: AOJu0YwkqnWEqeP1pA8C5LjkRoPfFiwrEy8iNaXu6I65TKClUsnMkjHm y8y3W0fSlom+/pIG1seJKyzlG//PZmwH0nnPvSusqI6yLkg7yP3Dg/yXMig0NZj2t9Z52/zCJ8w v1g== X-Google-Smtp-Source: AGHT+IEO1ECBC98fGvB+GRqYHAQaBdxJcqvY9DmP3QI4gZm0KdOyij1fLmFl0bPXgfvra8JJdQe8JNFxBHM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:9199:b0:71e:1e8:e337 with SMTP id d2e1a72fcca58-71e1dbe467fmr8496b3a.4.1728584809106; Thu, 10 Oct 2024 11:26:49 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:57 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-56-seanjc@google.com> Subject: [PATCH v13 55/85] KVM: arm64: Mark "struct page" pfns accessed/dirty before dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112651_491535_34900D27 X-CRM114-Status: GOOD ( 10.36 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Mark pages/folios accessed+dirty prior to dropping mmu_lock, as marking a page/folio dirty after it has been written back can make some filesystems unhappy (backing KVM guests will such filesystem files is uncommon, and the race is minuscule, hence the lack of complaints). While scary sounding, practically speaking the worst case scenario is that KVM would trigger this WARN in filemap_unaccount_folio(): /* * At this point folio must be either written or cleaned by * truncate. Dirty folio here signals a bug and loss of * unwritten data - on ordinary filesystems. * * But it's harmless on in-memory filesystems like tmpfs; and can * occur when a driver which did get_user_pages() sets page dirty * before putting it, while the inode is being finally evicted. * * Below fixes dirty accounting after removing the folio entirely * but leaves the dirty flag set: it has no effect for truncated * folio and anyway will be cleared before returning folio to * buddy allocator. */ if (WARN_ON_ONCE(folio_test_dirty(folio) && mapping_can_writeback(mapping))) folio_account_cleaned(folio, inode_to_wb(mapping->host)); KVM won't actually write memory because the stage-2 mappings are protected by the mmu_notifier, i.e. there is no risk of loss of data, even if the VM were backed by memory that needs writeback. See the link below for additional details. This will also allow converting arm64 to kvm_release_faultin_page(), which requires that mmu_lock be held (for the aforementioned reason). Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Signed-off-by: Sean Christopherson --- arch/arm64/kvm/mmu.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index dd221587fcca..ecc6c2b56c43 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1692,15 +1692,17 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, } out_unlock: + if (writable && !ret) + kvm_release_pfn_dirty(pfn); + else + kvm_release_pfn_clean(pfn); + read_unlock(&kvm->mmu_lock); /* Mark the page dirty only if the fault is handled successfully */ - if (writable && !ret) { - kvm_set_pfn_dirty(pfn); + if (writable && !ret) mark_page_dirty_in_slot(kvm, memslot, gfn); - } - kvm_release_pfn_clean(pfn); return ret != -EAGAIN ? ret : 0; } From patchwork Thu Oct 10 18:23:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B8D63D24454 for ; Thu, 10 Oct 2024 21:46:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Yf6LdUuiXd5bHyMNW6zRVTvFx4hHgFJmXjR2s95IMWw=; b=U78Kji7OYbt8Nt LamJn/9JAFxqbHLPb7x7gZVkPzRNxL4iWvHQuJSN5yr1AiytX7VzvvFCr3pLTqfspwnKkn8Q1w0DC s3y2MScscG2Z0eEAaE6l3QKS1b+u6DPhLL6te4H4ExYazyldW1ae0ZFlnxsgIOmf0VMc/+wxIcobU KrKYDPGwEnJyaWZzNZIAtJ+bL57eN2Yiem4pHmrOWRueFLxTzWmq58Dfuc7xfOPzptae32fN32Uq9 0THsB7j2KMLVT6MaK+gFtwxaOBUy+Wl545bDvxJapfpupLUfqTZMpi+axUlNZWXlZ/AmQT8nSsF5j 7Rdyr/VL6ibsbUpoBihw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0zM-0000000ESTy-0iAq; Thu, 10 Oct 2024 21:46:28 +0000 Received: from mail-pj1-x1049.google.com ([2607:f8b0:4864:20::1049]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsD-0000000DrC1-4AVD for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:56 +0000 Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2e2bac0c38aso1505569a91.2 for ; Thu, 10 Oct 2024 11:26:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584812; x=1729189612; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=L6S7ecEpHeP2pNYK74CcyfNjVxKIOVPI1u0giMU3AdU=; b=Tuq8jkdyupHP826P5bLjfoLYxHZeD/rX8rWQA7Q8ZEMCeDRAxuIwjVqa40D00ztFy+ D2mzL2FwoGdy9Y/Zmc9BOowAx/frGlZMdf9np48O+Abjk+HSQ8te8uSbkGaO+iXzsKte 5kwzt6k+5tFWim+CNFlXXNaVBlBDWjqVnLWP+hMDO9SIN+dnSIgvcfkYQUAyfOB42RJ+ gnVrEAThIhugh/52odDRt13phaIvxB1EuFKBb2MgPPMXJDRyW7wjwfkJnPK/O46Cls14 kti4++RxaaPlgzg9f/fpqZUZf4UJe/oCupqUIDVWRpDygUJokX5Sf1I+q/Rq2nmvNSE+ rlfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584812; x=1729189612; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=L6S7ecEpHeP2pNYK74CcyfNjVxKIOVPI1u0giMU3AdU=; b=m6oyKg5tryATnzChwDtBFpXyEJN03J2hxE4TaIG2aUjFrfFEqNI8RWnp7PVMNLDUPU GOEt32DcJg/2Ou2n3UdaKVwgO3thHLbl+gIJ+1UBfXBR1pHG1x4IK896Dgk3ylrTzA9s BzHkSXrjU52SusHGYWOacQGNArKan2CMH2jSmtONGoJPKD47Z2oTZzxxdn1PN/c99Prb TBH4zSR78LrsB46IKIhvKIbnU02TKNwVWi8nMTotOycvMxLeSpfiXZylD7BFgX++/HlP 5YQ4gZleryqhYveSwIk6mEz7q9s084t5xeLdymFNLqTVn29ZM7TwRnI30mpD36VB1ZhQ FE3Q== X-Forwarded-Encrypted: i=1; AJvYcCVbgItaBnN17vPw4x/h/tKFg0Bd1YTErIlrVq9a2dztCs49zGot0+r4ovP4iEFKmOdzS8uz1V1Lyr6xyg==@lists.infradead.org X-Gm-Message-State: AOJu0YyKJIIm/Anc+q+4bBRZt0kauGBCyUmasiQbgKwbsZrNEeZFyPG1 aOl0UKplMWB9yLjM1BgVx1ae+QSB5AbxAunGUe7LhPktT3H86FUbb7+lkiTaPksS5bHEeY0H3WH R6A== X-Google-Smtp-Source: AGHT+IFNqFi+zU38KeaNCnXXdZA3xW66mqPsBP1LkkibZO//JxtU2HWKMaWXgt9tHH1pSDOFY+ZWY6Gk4GE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90b:4f4b:b0:2d8:bf47:947c with SMTP id 98e67ed59e1d1-2e2f0c456f8mr35a91.3.1728584812033; Thu, 10 Oct 2024 11:26:52 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:58 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-57-seanjc@google.com> Subject: [PATCH v13 56/85] KVM: arm64: Use __kvm_faultin_pfn() to handle memory aborts From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112654_122325_09D79196 X-CRM114-Status: GOOD ( 11.04 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Convert arm64 to use __kvm_faultin_pfn()+kvm_release_faultin_page(). Three down, six to go. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/arm64/kvm/mmu.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index ecc6c2b56c43..4054356c9712 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1439,6 +1439,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, long vma_pagesize, fault_granule; enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R; struct kvm_pgtable *pgt; + struct page *page; if (fault_is_perm) fault_granule = kvm_vcpu_trap_get_perm_fault_granule(vcpu); @@ -1560,7 +1561,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, /* * Read mmu_invalidate_seq so that KVM can detect if the results of - * vma_lookup() or __gfn_to_pfn_memslot() become stale prior to + * vma_lookup() or __kvm_faultin_pfn() become stale prior to * acquiring kvm->mmu_lock. * * Rely on mmap_read_unlock() for an implicit smp_rmb(), which pairs @@ -1569,8 +1570,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, mmu_seq = vcpu->kvm->mmu_invalidate_seq; mmap_read_unlock(current->mm); - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - write_fault, &writable); + pfn = __kvm_faultin_pfn(memslot, gfn, write_fault ? FOLL_WRITE : 0, + &writable, &page); if (pfn == KVM_PFN_ERR_HWPOISON) { kvm_send_hwpoison_signal(hva, vma_shift); return 0; @@ -1583,7 +1584,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, * If the page was identified as device early by looking at * the VMA flags, vma_pagesize is already representing the * largest quantity we can map. If instead it was mapped - * via gfn_to_pfn_prot(), vma_pagesize is set to PAGE_SIZE + * via __kvm_faultin_pfn(), vma_pagesize is set to PAGE_SIZE * and must not be upgraded. * * In both cases, we don't let transparent_hugepage_adjust() @@ -1692,11 +1693,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, } out_unlock: - if (writable && !ret) - kvm_release_pfn_dirty(pfn); - else - kvm_release_pfn_clean(pfn); - + kvm_release_faultin_page(kvm, page, !!ret, writable); read_unlock(&kvm->mmu_lock); /* Mark the page dirty only if the fault is handled successfully */ From patchwork Thu Oct 10 18:23:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831209 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6FD15D24455 for ; Thu, 10 Oct 2024 21:46:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=h+EXa58M5OB2otP+DWaTk0pt464NBWamj3eXsytYiGU=; b=xl88Ueg18CgjRR Et49NicxLUZS88GWQS96We0/UCUzxj/Eh/zZ8myy76kJxvi23xRXJZu8TwOjTuW1W+umjJYX3LBQg xiLT49GqHZgdk5ioEwVKEdSeL0taEJK03bfp3eBv7WYDpZMq4PRYYiCc8Cn0kba5MSVZ3Ke6imjny 5UWbTq0mIHwRLTuGnez+B109RfGUWjgWdDHH1ogNMi6E9uggawnu9nuiQhYQj4Y5VxJMkNusb/llb JX5jztq1rejQLFvj9RBmnqbCoIVLOhRbghXPDnUrcuRSHVSrMsAYlkSk6z6JOxHJvblHinSi9pPon 9mtrLBG5/6GVS50b6fsw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0zN-0000000ESVa-3vBj; Thu, 10 Oct 2024 21:46:29 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsF-0000000DrDa-2uqg for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:26:57 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e2baf2ff64so22506637b3.0 for ; Thu, 10 Oct 2024 11:26:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584814; x=1729189614; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=9PLNR7J7/StxhsadOj2FvapRbIqIfrDfQ6nFmhEUvk8=; b=oxXZstpB7bZForNibrdtUdgMDhKCKO5NoyWz/FZhmXi1O/5zBMkdNGoOBmZUnuHPL5 G5xj1yNE8SP9DIK/5R+FSLX1z3xFvSK7JT9CV/kxO/TU+0fFWjpeJqmMwLlYFO0Q+Tv4 I858qQIY6Wc0qtONrNlC8gbObQgUKRGiuFgn7O1Kkb3dzs2Qp7EmMuvNXtmlilk77dG6 00Dp5QMLKTvno0LnGfKXj09dJixXH9wsYC90KdPrw/8w/aMV7XGhbAVdXGuP5cSdmKEJ /2emmDAUPw3bLmVyf1eaRDVyfGHUrBZvZno+DFT7U8iIof4e8MNFX2e2pMznO8WrlsKE bLEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584814; x=1729189614; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9PLNR7J7/StxhsadOj2FvapRbIqIfrDfQ6nFmhEUvk8=; b=FApOMGNuKm3WKOu97vfcyWi2XMYzvdGbQZ/1pw+iSHZdnRmpf3fVL2sEsOzZfkj9Zz hhllv90PihfSSiC+rVsbit6O255depZZhyUMmVG7xYlOWLqBRMXLn2YWqnCZFBxlUge0 wkpv8XnSmmfTh21bAF9+zcTvqFX1Xvevx8LlrS+IawTNtXC52GSRn1ZphQKOxukxtRLv p5ZKLEcI+NpFvy0VtmS08BbqDTkIfhkkjrMAs09BR++70LsF+nQgHv5g/PBC33lp1bvD Nbr2vjkYgkfy7vY4tN+JJkRGXrJPjcbPlHqGOWMs/cLRZMCKbq3B+6nOq0Q7gGKLbeiO 77fw== X-Forwarded-Encrypted: i=1; AJvYcCUIRuYtHX60MRodAYoNwPkmRwArfmexYnYYRgJlcyA7ovYmSVPGLBK4LG24xSbZxxLc/VJ7qDoK/wyGvg==@lists.infradead.org X-Gm-Message-State: AOJu0Yz5oFiL/KfoEwrre50ZdPdpPbuhO1es2nv/JuL53V3160tmiT3Z UpX3BfhLandTU3pJZSI0P2fRKqUu5eO5RApbptZLBj4CVeq6soANgHRvxj8ipMtgpFD65uq3NnE yEQ== X-Google-Smtp-Source: AGHT+IFTz5/NgoL1lG2XSkf5pquU6CJl4M8l3x57mh4U7FbzzS84rW+oHIk+OXb87gKu92fZB4cgNqwOitE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:46c3:b0:6e3:21cf:a67f with SMTP id 00721157ae682-6e32242fb5emr1006557b3.7.1728584814138; Thu, 10 Oct 2024 11:26:54 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:59 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-58-seanjc@google.com> Subject: [PATCH v13 57/85] KVM: RISC-V: Mark "struct page" pfns dirty iff a stage-2 PTE is installed From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112655_826230_744ABC6A X-CRM114-Status: GOOD ( 10.32 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Don't mark pages dirty if KVM bails from the page fault handler without installing a stage-2 mapping, i.e. if the page is guaranteed to not be written by the guest. In addition to being a (very) minor fix, this paves the way for converting RISC-V to use kvm_release_faultin_page(). Reviewed-by: Andrew Jones Acked-by: Anup Patel Signed-off-by: Sean Christopherson --- arch/riscv/kvm/mmu.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c index b63650f9b966..06aa5a0d056d 100644 --- a/arch/riscv/kvm/mmu.c +++ b/arch/riscv/kvm/mmu.c @@ -669,7 +669,6 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, goto out_unlock; if (writable) { - kvm_set_pfn_dirty(hfn); mark_page_dirty(kvm, gfn); ret = gstage_map_page(kvm, pcache, gpa, hfn << PAGE_SHIFT, vma_pagesize, false, true); @@ -682,6 +681,9 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, kvm_err("Failed to map in G-stage\n"); out_unlock: + if ((!ret || ret == -EEXIST) && writable) + kvm_set_pfn_dirty(hfn); + spin_unlock(&kvm->mmu_lock); kvm_set_pfn_accessed(hfn); kvm_release_pfn_clean(hfn); From patchwork Thu Oct 10 18:24:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830928 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 12949CFC5FE for ; Thu, 10 Oct 2024 19:28:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+tDuIMJX0wZNZAXVJOPVJ/1k0HI+QyWHalGcFGK7WYw=; b=Rz1glFPeJhEQE7 L0dDGBCP6ee2noI8htIgKFp6TQ3wmRFy1mZBcazf3iV+B4WA2OXnh6qNfWer/BPzqsw+geDPb/nPB CgIB0gkMrg4iov/Osf/dmzZyD7Y+fpTCP0tvH/h1ujiv/XtVf3W3KEdGiGPjNJZxGuDz+vksU8y94 P6muLw5JdRvsgwv6W+jTwiMT9VUfK9jEHt+J4iJypbwlMCjL7jauSE/267+aBD5wJu0Zd22/trVCR zXp5623LlBSFlqf9kp0bmhIw8+9p/FBhsoCCetJKORR0ThsOzkp/o6fe/cYq9C4c4yF59+B/6NSS6 CP1EDtVjmCVb1QRvmkvQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyq0-0000000E6kf-1haH; Thu, 10 Oct 2024 19:28:40 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsH-0000000DrFd-1e8S for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:00 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e28eea1074dso1984131276.0 for ; Thu, 10 Oct 2024 11:26:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584816; x=1729189616; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=h7pIaRprT8i6iodquNsVL5vPZSNT9KsZLdIQsfz/EL8=; b=SlOntVtK7MfXmxpZt9qlDfPDhXBRuKbOOqW7IqQ0P93vwPs1FWZo8d/fPJW4iNnsl9 WaFEojLUv/xsVJmZwP672jadiAn28zabsdyG4luGg8vBEBqdfYs+GmhJTKS7MdDeavEu IIS/1rUscUKoFp5AiL5O+Hix5GdZV6jM1pdKwvb8MlIpTYXGrXcWMrPmxpoPEpHJVb9n zpWK/On5Nqt1OS/n+NtmVJUQ9bqeF9a8ZoOJNBl3NwrmW1rSb5CDX6JCZIt8XI8kpm9c 5W0lnYFPhUi4nG7xC4RgfzO65Hjb/3a1HK1I7N7Vy96v150QVnYn24O+JGrFNRZCzfq8 7meQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584816; x=1729189616; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=h7pIaRprT8i6iodquNsVL5vPZSNT9KsZLdIQsfz/EL8=; b=a0xG9Y10vL1ydiuudZ+kl+dOWiD09RxOOwVDeN3qlJXm/c0x6JSfMoHEm1OWtCAz/l soiV7FnUvdh3PqByELnT4U3luC2GyO84I2H13Yqn9Dcykyv2Ldm6Jzitx+eiuCDyY4De LkIAzU0lkOYOhzGq00voNZ6TZgFGF0XCEFpZA2Fz7LQhkQOuMrslCK/izMLwAM5QvUgI N97zfH3QeNxOeMjDIK3YFH8+s1Buwx17KelnTVcWEXfBtASMIGy0Xj2ZWTE2KP8UZ4DH a+FjkBchHxHos/7HtDtcpGGHKy+MaBmGiSHGGlP6nX6OYFvYv/pTK3xmiyIwjuJEB2Pr M6sw== X-Forwarded-Encrypted: i=1; AJvYcCVzO/y3ynEr+YMaXzh3QM9+B+LO5Hp6jdpVvxRT/fvYqS5ZO6L7S/+rr4VPAs85tKtmiWW3D4m3QPq1ew==@lists.infradead.org X-Gm-Message-State: AOJu0YwYszTTjgsrBAwAWSbI3W4dLOBqWtCBTJeJDzB6Vdndt/Ibq3QZ OqQ2bKzr5IM7X4RY/z3KWfuzzWyyGZrAgrMX4Gj2x43wlHwXzL+hTk4UZF2DmO/XNJR32fyjlLC M4Q== X-Google-Smtp-Source: AGHT+IER2ThMFqRrPlrH2YqE2Mm6yZZ3YtFM0VnOojG5BgkcM2HHrRnMjpl0ZvMfhTqewlD6nF2BR0rNKwA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a5b:c45:0:b0:e0b:f69b:da30 with SMTP id 3f1490d57ef6-e28fe41d0e6mr89144276.9.1728584816073; Thu, 10 Oct 2024 11:26:56 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:00 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-59-seanjc@google.com> Subject: [PATCH v13 58/85] KVM: RISC-V: Mark "struct page" pfns accessed before dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112657_537809_D27398BD X-CRM114-Status: UNSURE ( 9.62 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Mark pages accessed before dropping mmu_lock when faulting in guest memory so that RISC-V can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Marking pages accessed outside of mmu_lock is ok (not great, but safe), but marking pages _dirty_ outside of mmu_lock can make filesystems unhappy (see the link below). Do both under mmu_lock to minimize the chances of doing the wrong thing in the future. Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Reviewed-by: Andrew Jones Acked-by: Anup Patel Signed-off-by: Sean Christopherson --- arch/riscv/kvm/mmu.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c index 06aa5a0d056d..2e9aee518142 100644 --- a/arch/riscv/kvm/mmu.c +++ b/arch/riscv/kvm/mmu.c @@ -682,11 +682,11 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, out_unlock: if ((!ret || ret == -EEXIST) && writable) - kvm_set_pfn_dirty(hfn); + kvm_release_pfn_dirty(hfn); + else + kvm_release_pfn_clean(hfn); spin_unlock(&kvm->mmu_lock); - kvm_set_pfn_accessed(hfn); - kvm_release_pfn_clean(hfn); return ret; } From patchwork Thu Oct 10 18:24:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830933 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 45266CFC600 for ; Thu, 10 Oct 2024 19:31:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=snHx4DOeu9juEonyMWzeYUj38qx1Nei+kioJahDL24c=; b=zKhofoke+UVPab kXcSKA1duqUVHbnohwqxoDbLMaYyj3u6tJiBPnz9R7fQFjsuPnlZMzKPCmguTAoarKWWopt2Akaio iEb1laR2YoBhNjbktiXDlyhcRKlpupYeGWfbff7F0YdNlgF/ilcueJNb5GyAkCiwMWaTsTEdTsMZh nLOFWds2MgkbCn5OSQOpZto481HXdgcVbY4ZYF7wQSDne2iez8SS0IqYZ5hYA+J4TKNULnAf6wqEa TeTj8jJt8iOv29UR0oE8+CbjvZsa2RwqwzlTf0NUgzm6w20D/BZWY1NNoy++HVKdQfSISK5fw4ch8 CrGAZNisZRaw0bVD32fw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyse-0000000E7ED-3Z5N; Thu, 10 Oct 2024 19:31:24 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsJ-0000000DrHK-2WxR for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:02 +0000 Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-7e6c40a5795so1041175a12.3 for ; Thu, 10 Oct 2024 11:26:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584818; x=1729189618; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=OPPvMTYiUGb6pAXW02Q14XiUF2+KvArosM2sRK6VjAA=; b=jHZE+5uWuKTW/XclrN7CxgU01R867qap0OT1d1SQAxMutGogOns2m7h4OczE0MMOAh 1bOC4OAreKrsxWwPn6Lq4yUn6ww5VGN/P/6dCQu7Wz1aJXS6RA5MlDKU8blhQdAK3+oJ w7Hui36VpO0Ua+GpakXkDzGrrZtAOQ4i+bpjS2LZB15ep1oIi954CEVd8VSCUEpuTEmH uMVRVR+qn2o1f5I/7taWb50bvQjkE/GagwpScYZSMNiGtslKvTiZXEYVCdh7kpLhPoVU OeNf3Txm2Xav5Rvn23+I0GhmOXaelr06WRlnGWvEKKd7xtGeetms/WSDotPLATORh2a1 urMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584818; x=1729189618; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OPPvMTYiUGb6pAXW02Q14XiUF2+KvArosM2sRK6VjAA=; b=RIQ+srId1xRRyDojPuLjavuP8Lut3i/+aGmVKox9LLOpNj4HuF64ZXqJFcJcEM0fxa Ci147OB61mlTx4eLPuezVPNz2jsxYkDcjAKm8Zuw/uVvxxqxe1t+QGOLmDDQsr4KCVit gqEdGfAWF6DcGaZ5Q6sPUZTi0xptS8moGQy+DdJD9L85ShH0fNxlxQVROlyZxVLbwiP5 MfsK4jo+1v5HWM512mxBmjso/fo+MmC6S5RzdIsNMT1GPjqFUaj37yV1Ar51NRkBBUhj /I/+ZuWx5j7Vc04f2koBwy7LsrpogROME/i21V+VGJg9T9mUWLrR8iA+uJZ8aqH1s1Vs FEAw== X-Forwarded-Encrypted: i=1; AJvYcCXM5jcBzFFtv/MLyosV/Qkd3gEGBvDLJ1CGLw432PawhHUaz1qbqYg2x908BQMk/ZU+D5Lj4siH/y9EtA==@lists.infradead.org X-Gm-Message-State: AOJu0YxDjP1yXKW3gvgWGqyfKwR4thEYK9SnnzhvDK2QumuoEmGOqDCa oRK//y67gb5seLeWWtDCpzCcMD5LRcWC2zv859DM/wYuvVCnM8lW1L5jzt1bCMXYEkrF8ll6uM9 8Uw== X-Google-Smtp-Source: AGHT+IHVZkXcNH8eDb3GIQubykYaXKWHdrpGe/zHTLQHIjPMwQpAN0ra/QncFunLfkL/hgYZuPxk0qgLkH8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:2343:b0:20c:747d:ec15 with SMTP id d9443c01a7336-20c747df618mr290175ad.1.1728584817934; Thu, 10 Oct 2024 11:26:57 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:01 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-60-seanjc@google.com> Subject: [PATCH v13 59/85] KVM: RISC-V: Use kvm_faultin_pfn() when mapping pfns into the guest From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112659_735731_B18F0A71 X-CRM114-Status: UNSURE ( 9.97 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Convert RISC-V to __kvm_faultin_pfn()+kvm_release_faultin_page(), which are new APIs to consolidate arch code and provide consistent behavior across all KVM architectures. Opportunisticaly fix a s/priort/prior typo in the related comment. Reviewed-by: Andrew Jones Acked-by: Anup Patel Signed-off-by: Sean Christopherson --- arch/riscv/kvm/mmu.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c index 2e9aee518142..e11ad1b616f3 100644 --- a/arch/riscv/kvm/mmu.c +++ b/arch/riscv/kvm/mmu.c @@ -601,6 +601,7 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, bool logging = (memslot->dirty_bitmap && !(memslot->flags & KVM_MEM_READONLY)) ? true : false; unsigned long vma_pagesize, mmu_seq; + struct page *page; /* We need minimum second+third level pages */ ret = kvm_mmu_topup_memory_cache(pcache, gstage_pgd_levels); @@ -631,7 +632,7 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, /* * Read mmu_invalidate_seq so that KVM can detect if the results of - * vma_lookup() or gfn_to_pfn_prot() become stale priort to acquiring + * vma_lookup() or __kvm_faultin_pfn() become stale prior to acquiring * kvm->mmu_lock. * * Rely on mmap_read_unlock() for an implicit smp_rmb(), which pairs @@ -647,7 +648,7 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, return -EFAULT; } - hfn = gfn_to_pfn_prot(kvm, gfn, is_write, &writable); + hfn = kvm_faultin_pfn(vcpu, gfn, is_write, &writable, &page); if (hfn == KVM_PFN_ERR_HWPOISON) { send_sig_mceerr(BUS_MCEERR_AR, (void __user *)hva, vma_pageshift, current); @@ -681,11 +682,7 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, kvm_err("Failed to map in G-stage\n"); out_unlock: - if ((!ret || ret == -EEXIST) && writable) - kvm_release_pfn_dirty(hfn); - else - kvm_release_pfn_clean(hfn); - + kvm_release_faultin_page(kvm, page, ret && ret != -EEXIST, writable); spin_unlock(&kvm->mmu_lock); return ret; } From patchwork Thu Oct 10 18:24:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830932 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A41DCCFC5FE for ; Thu, 10 Oct 2024 19:31:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Dhd/DztSMRsR8kdPc+AsBfX5TvLAkDZ/Vw2uUxTpMOQ=; b=fdbxI7sPdw3mhQ XquwArOjbQTQzjESx5NI+Ij42OpnpNh+M3H7cljNkmbNf/ZZiYFz7S9a4pbwAjroWXn63npMf4+Kp FVqDKX904TCeG6McP0dZ523u7kg8PGnxjm1PJwZBkG9Eb+P8BQiwSYFkXmI0o6y1hS5zuVeExz6JM u5cvB4gEZmu2OfUJFb2e5NfGSBlT3B+BCuxapdHxIvzupxRuhUHAvKESgA1M47+NflgFvd5KGGfxD XJchieU3ftmrWxqwAPFPp7nOhzpRM37+a3hiwFdHZjwN0IEBRpSgIHaNEEK/0AfrlVD/agV3v8mol Z3YZf4uvE5OM2OK4tNcQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syysf-0000000E7Es-2kMA; Thu, 10 Oct 2024 19:31:25 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsL-0000000DrJT-1Ahv for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:04 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e294f38bddso22119737b3.3 for ; Thu, 10 Oct 2024 11:27:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584820; x=1729189620; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=SD6mqWUZeYJtl6GnT7U0LdjoV3gPIs/AdF7iNpavyUo=; b=aBDkusJdX7g5BtG4H5fZwYumgK8uIRaVDx6oulzlDnAE3QYd/mM7atYX1oex5UM4kp 36HYJ8PnDzc4AfK9p/87+lpLgfHVh0ndquokKS44VTbMF2Oucv/TNfFgaSF5koQ84Xx+ eB1oBqtktOPbKyCDITEKMRpMt/zheIQaROymC4XkoClTL+P0IlaWL4HGnExqhsEfvMNX Grcu4zqEHuWTwdwnkzgTXcXDWS+EhrV5yrg6rh0PIUpcmQja2DfSNNDcjEgoXjjL+DOb 4f1UgnylDz6N1QWZCTJX94ziWXvHKu5BciXd0kjBuyWLffYTsruMTc2z38kb5Ir5V3zW suxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584820; x=1729189620; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SD6mqWUZeYJtl6GnT7U0LdjoV3gPIs/AdF7iNpavyUo=; b=l5xSUZrpHvKdeGoLhgVP5TvWhBPzLXlBwad6VgWFuTygx0o7K2QA1mU18SmFHD2Q3t cnWW8qj+gAZZ63LNO42ZLUY55365PN/HMtW8hcK9teNNncoIIjmcfgKDQnjeAc/bMT5J jqBoeD+sQAi1BHk3bZR3KMrfcWGNrfOnlOKBcs5TRQ85uXx5B+luQDXLDCgSuoO6/Sn7 j5R0cB3GsnTjf19GqedrKH/0p89qnapKlhfLOCFHpTw8v7N3xdzUMPD5HGJByAmP4NaU sgjWYfiaefa71HXkMXFDAHK7EdKY++TcgGdM9/rXtFF+PvXY6eWTD3j8z9fLiFOaGzk6 eTww== X-Forwarded-Encrypted: i=1; AJvYcCV8oLtpTzP+UufuOFLUtpVOK9AQuzzkuTC3QOjlrbCRbuO5CtZ4bV7FLAne+hO6TBuE7HXH7/u9i5CuhQ==@lists.infradead.org X-Gm-Message-State: AOJu0YxKP7DkrZvUu48zwJGnpR2KX36RykGEpnATIDHUu0aXnFTNVI/T GTO4XC+e1DgvBJgDdhmETMd+ABo8t7jD/2KUJGCiv4RBTx9j3hy1gEoFJmaVzkGpD6EzE/E1dHz eTg== X-Google-Smtp-Source: AGHT+IGRQvxFZI4ySDHPZ8aV1dJi8vCYkQEt4dLiT8ndc3AvxflLIddldJBCv79vdNFr7/DaBU5iasgDdOk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:ce11:0:b0:e28:fb96:d0f4 with SMTP id 3f1490d57ef6-e28fe33505emr40781276.2.1728584819828; Thu, 10 Oct 2024 11:26:59 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:02 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-61-seanjc@google.com> Subject: [PATCH v13 60/85] KVM: PPC: Use __kvm_faultin_pfn() to handle page faults on Book3s HV From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112701_453553_EF965D4E X-CRM114-Status: GOOD ( 12.49 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Replace Book3s HV's homebrewed fault-in logic with __kvm_faultin_pfn(), which functionally does pretty much the exact same thing. Note, when the code was written, KVM indeed didn't do fast GUP without "!atomic && !async", but that has long since changed (KVM tries fast GUP for all writable mappings). Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_64_mmu_hv.c | 25 ++++--------------------- 1 file changed, 4 insertions(+), 21 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c index 2f1d58984b41..f305395cf26e 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c @@ -603,27 +603,10 @@ int kvmppc_book3s_hv_page_fault(struct kvm_vcpu *vcpu, write_ok = writing; hva = gfn_to_hva_memslot(memslot, gfn); - /* - * Do a fast check first, since __gfn_to_pfn_memslot doesn't - * do it with !atomic && !async, which is how we call it. - * We always ask for write permission since the common case - * is that the page is writable. - */ - if (get_user_page_fast_only(hva, FOLL_WRITE, &page)) { - write_ok = true; - } else { - /* Call KVM generic code to do the slow-path check */ - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - writing, &write_ok); - if (is_error_noslot_pfn(pfn)) - return -EFAULT; - page = NULL; - if (pfn_valid(pfn)) { - page = pfn_to_page(pfn); - if (PageReserved(page)) - page = NULL; - } - } + pfn = __kvm_faultin_pfn(memslot, gfn, writing ? FOLL_WRITE : 0, + &write_ok, &page); + if (is_error_noslot_pfn(pfn)) + return -EFAULT; /* * Read the PTE from the process' radix tree and use that From patchwork Thu Oct 10 18:24:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831210 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F0CB1D24454 for ; Thu, 10 Oct 2024 21:46:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=pLzJht7NpGvV8rI508iLgvyDdSmfvsn8YAHHxH84ie8=; b=UpbNQPgiGf3cyV 6LpVZN5lojL4IcX+zvsRC2GShJuQeGwnrA0qasTMA+OwSkYuM8/Md2f3KiNzMXz+dqh9KJXYauOhL CP+xoHzq1JAT6HTJbmcEZOTdiBNzmNtDZVvw8YjNW4Y2Zu6Dltxlbearsll2lhrHIpeuS0fYtJyww aCXZE0Glm9gpYOwEde1otjMQjRWrDRzuF8FIusZvderABsGYb7zhLNlWu13KmJWZZZQfSs/GCmYSx U9WXy+b/oBuZzS9vwHkp6UVty5yHJAjzzPvy2hHT9V4Z835Lo9BLm6oPc5611YEURwLaoakqJGj2j RaqgJq3g+0EAU+/no3mQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0zQ-0000000ESXY-0OmA; Thu, 10 Oct 2024 21:46:32 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsN-0000000DrKM-1oYQ for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:05 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e290b8b69f8so2139789276.2 for ; Thu, 10 Oct 2024 11:27:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584822; x=1729189622; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=YGpJ+GsXH4oG8p6vIrNjfshF8svSSAFECmaA34wp4no=; b=AFXrQf91aXR1KTzaINYKc11bKcpAL/FBJRU5jqINr+Nhzi0EHvlki0oC9dCnjEGnWx 0zVm9mSBj2AcgW4LR34T//6/HY2HcASslxa2iNw7wwXCbTgZ8oxdMI635Z5mVbucf5i8 v2DE+6sqEAB9v5KGBcqRVR6xzDXFBq5pkHblKYkHjr3+BtsbZCDKCmyrCZD1LW0IdcUM 1pD9rayn4EnfsTjku1tz91kdoDjX74SAWAwF9d2bm028TG3RB/P064MIKeodw8ez2JK+ zBjIC/kVsK1IkoUFRaArozf068vnlLw74GKO4rSiTlL8f/D9lFwS307IYBCFbQOTHiVU JxKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584822; x=1729189622; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YGpJ+GsXH4oG8p6vIrNjfshF8svSSAFECmaA34wp4no=; b=SV4ar1Q+0LKmeEJh6wt8jAcJzGi+toaxdKpGhcmL9zouJocc+jOryZ9JzzNIHRiJze t58WQrszkdbRjwOinFsdmgwUplDZ03WagUXCGLi5xn1sW4fbtRMkqIAJukLoVazgJLkU cuiCQG1q2UZNKskhGMGIOzEauSsWiCSgFlraNt+lkpjn1sozRK+ABnIOsiL52n5LwAUO qTKm7G4x1vbLVWWLPleaJQ1x+32K2EIgwpcrnKgZ65fz5D1JaGYjefp8VK+W8kpQ43El mkfHxUXthZrqyd0P3PZKHiU2uiMBvbaTdWrS1aVzfKOUS178ZdRGja9WPtTF9GDchlP4 X+Vw== X-Forwarded-Encrypted: i=1; AJvYcCXWfiNZP/GznYR2nU20QNE1HKkaY1iJupVXWNByF8T2Yu+Gm/CBfRe1d2uZangtpBTREHB8iQbVLtGRbA==@lists.infradead.org X-Gm-Message-State: AOJu0YxX/826cEwC0PKcyAlMIykeQBaw9jCoY1zjiEPvU3+SeMJ/NHzK Yb/7haPXDdCJrDFUCdAIUH0R+EYnxlNC+p3caGXq+wojAddkObc/6id7A+rbGHQQOaovyge9w3b hlA== X-Google-Smtp-Source: AGHT+IFyzgxXPfejJoY0SZCQ3T+Vo8hzb1HJlB5l94FFDfZID39IeT9MHN/ZgR4beqIvxQ/gO8qC3c4Qafk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6902:4ac:b0:e28:fdfc:b788 with SMTP id 3f1490d57ef6-e28fe4426b8mr4979276.9.1728584821558; Thu, 10 Oct 2024 11:27:01 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:03 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-62-seanjc@google.com> Subject: [PATCH v13 61/85] KVM: PPC: Use __kvm_faultin_pfn() to handle page faults on Book3s Radix From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112703_564754_90B32420 X-CRM114-Status: GOOD ( 12.96 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Replace Book3s Radix's homebrewed (read: copy+pasted) fault-in logic with __kvm_faultin_pfn(), which functionally does pretty much the exact same thing. Note, when the code was written, KVM indeed didn't do fast GUP without "!atomic && !async", but that has long since changed (KVM tries fast GUP for all writable mappings). Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_64_mmu_radix.c | 29 +++++--------------------- 1 file changed, 5 insertions(+), 24 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index 8304b6f8fe45..14891d0a3b73 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -829,40 +829,21 @@ int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, unsigned long mmu_seq; unsigned long hva, gfn = gpa >> PAGE_SHIFT; bool upgrade_write = false; - bool *upgrade_p = &upgrade_write; pte_t pte, *ptep; unsigned int shift, level; int ret; bool large_enable; + kvm_pfn_t pfn; /* used to check for invalidations in progress */ mmu_seq = kvm->mmu_invalidate_seq; smp_rmb(); - /* - * Do a fast check first, since __gfn_to_pfn_memslot doesn't - * do it with !atomic && !async, which is how we call it. - * We always ask for write permission since the common case - * is that the page is writable. - */ hva = gfn_to_hva_memslot(memslot, gfn); - if (!kvm_ro && get_user_page_fast_only(hva, FOLL_WRITE, &page)) { - upgrade_write = true; - } else { - unsigned long pfn; - - /* Call KVM generic code to do the slow-path check */ - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - writing, upgrade_p); - if (is_error_noslot_pfn(pfn)) - return -EFAULT; - page = NULL; - if (pfn_valid(pfn)) { - page = pfn_to_page(pfn); - if (PageReserved(page)) - page = NULL; - } - } + pfn = __kvm_faultin_pfn(memslot, gfn, writing ? FOLL_WRITE : 0, + &upgrade_write, &page); + if (is_error_noslot_pfn(pfn)) + return -EFAULT; /* * Read the PTE from the process' radix tree and use that From patchwork Thu Oct 10 18:24:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830965 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 93028CFC5FF for ; Thu, 10 Oct 2024 19:35:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=HYOoS2BUHdSHMNjQKhyiQ0OXZIUdsaQULqLxi2r25aE=; b=qWJKko8+BoudKP aEBX2kTNCza8nFMjefJYoMU2xFoWbPUvMPJugvlFADD1FQ2w7C3yHwolwX2tprrfUKlf/tMW+blJT /OC2cX3SMw8iCUTy9e5zyC5xNnghekS0XI4nyFMEhlDWCKqoaR5KRplvwbJx6OFugfzfE3TVoohQW 2fEFkg9ncQflsxQ77tTf+Lpqi1ck5Ga1hD+IwgYUsK8VoPatzijovy2vaRmTzaHBVch07fq4BcPLY lGMYO/68AuejOgPze/Onxkn1A6FTXONT0xUnlYsqEfc1NPia6+oifr9vDtz15crVW0vAUOFBQ++7G dVMTimarIIA/rufiqOBw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syywe-0000000E7s6-0ipq; Thu, 10 Oct 2024 19:35:32 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsO-0000000DrML-49yW for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:08 +0000 Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-71e1ed58e9bso1479112b3a.0 for ; Thu, 10 Oct 2024 11:27:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584824; x=1729189624; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=8JV5lSZeo1By1ou6uwDGqoH8kmaYenoeIxSHz9El/d0=; b=tfVvytWhY08fj6I/CIwHTgEx2WklGedcKVKQGHSPBJOoelJUycx/EZ/LKn+/zj318X +kIiEPwDv/ASCm+gLQ84XPKhqTSK5qjUYu2ciwEDod/zaHs8mN1gvJCvKbI5NpfvZypx qOwChr4uhvzgYifTcfVHYeIKAb8EfGqtCxyKhKoDtd+i42wQEeJvNFUrcS+w2JiMkh07 w1qh6lYrF02WLFCBkElXAmzCsdAdd/czW6ZFQBo27MM6UJCiVnyAhbX+hCaS0deCq7BJ +zOPhFnwvwq357J5qFvUs1Y2+q1JPAAp++JCFyYOWMGcoALF2Atj2zMDguXp+4l7N4RV RY+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584824; x=1729189624; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=8JV5lSZeo1By1ou6uwDGqoH8kmaYenoeIxSHz9El/d0=; b=wYvFPFeYHAbUXs6BemxaXSB7T9j2eZgRoLTMB1cUIDYMbx5aq6eBtomWgBG4D5sOdq cRLhD1Z+UpInj0NqpF0UHBe/IhCbTeRT7RP9cfCqiyNXdkxXR8+NUoK1H3OxaFWJmAhy xJcORMdb9/UZQbogxTlQVyGEfjF9JpyqAQiAQjGwOHoMmlH3uFfA1L8jKpC21JHiUC+q XRGi2Bshw9U7eTuWfQs1uVc94NYATd2cjr4r3GHUxQZ+yKG91g8KpqaZMetLSuTK2AHI 2CdrumYP8vwhwaczDKcufL/FnZLOFJwso/Z9aBsax9vyTYQHwor8U6ApctVelnrMxkWn 66Zw== X-Forwarded-Encrypted: i=1; AJvYcCXO9LN3ZGpJQSF0xmrR+h8nQeSw/wuoDL+e0RnFWO9pjM+e0E6L4ciBzluLFo8Y5VMjJKkZNVYrojN6Iw==@lists.infradead.org X-Gm-Message-State: AOJu0YwMwL8DTCeEl9JdXxzeAkveKc2eL8lUIGiVuxzsZYz6jG2Cw1Mx RXAcQX8P/PsuORofCHA3uHE7UcOQpNrfl0PG6ISPEkIIP+KeSo9qvCBEdYnGQEwfOAkDDixspsx sXw== X-Google-Smtp-Source: AGHT+IFev0i08WnhWXOHYVCiDa8hDSrBgilu3q+XI+BXP7ns5qqyHbYBrSKz+aJVjoCmfpAuiTEXvCWN4KQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2daa:b0:71e:401:6580 with SMTP id d2e1a72fcca58-71e1dbf2587mr27202b3a.6.1728584823552; Thu, 10 Oct 2024 11:27:03 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:04 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-63-seanjc@google.com> Subject: [PATCH v13 62/85] KVM: PPC: Drop unused @kvm_ro param from kvmppc_book3s_instantiate_page() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112705_239206_6B0F06D8 X-CRM114-Status: GOOD ( 12.92 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Drop @kvm_ro from kvmppc_book3s_instantiate_page() as it is now only written, and never read. No functional change intended. Signed-off-by: Sean Christopherson --- arch/powerpc/include/asm/kvm_book3s.h | 2 +- arch/powerpc/kvm/book3s_64_mmu_radix.c | 6 ++---- arch/powerpc/kvm/book3s_hv_nested.c | 4 +--- 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/arch/powerpc/include/asm/kvm_book3s.h b/arch/powerpc/include/asm/kvm_book3s.h index 10618622d7ef..3d289dbe3982 100644 --- a/arch/powerpc/include/asm/kvm_book3s.h +++ b/arch/powerpc/include/asm/kvm_book3s.h @@ -203,7 +203,7 @@ extern bool kvmppc_hv_handle_set_rc(struct kvm *kvm, bool nested, extern int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, unsigned long gpa, struct kvm_memory_slot *memslot, - bool writing, bool kvm_ro, + bool writing, pte_t *inserted_pte, unsigned int *levelp); extern int kvmppc_init_vm_radix(struct kvm *kvm); extern void kvmppc_free_radix(struct kvm *kvm); diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index 14891d0a3b73..b3e6e73d6a08 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -821,7 +821,7 @@ bool kvmppc_hv_handle_set_rc(struct kvm *kvm, bool nested, bool writing, int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, unsigned long gpa, struct kvm_memory_slot *memslot, - bool writing, bool kvm_ro, + bool writing, pte_t *inserted_pte, unsigned int *levelp) { struct kvm *kvm = vcpu->kvm; @@ -931,7 +931,6 @@ int kvmppc_book3s_radix_page_fault(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot; long ret; bool writing = !!(dsisr & DSISR_ISSTORE); - bool kvm_ro = false; /* Check for unusual errors */ if (dsisr & DSISR_UNSUPP_MMU) { @@ -984,7 +983,6 @@ int kvmppc_book3s_radix_page_fault(struct kvm_vcpu *vcpu, ea, DSISR_ISSTORE | DSISR_PROTFAULT); return RESUME_GUEST; } - kvm_ro = true; } /* Failed to set the reference/change bits */ @@ -1002,7 +1000,7 @@ int kvmppc_book3s_radix_page_fault(struct kvm_vcpu *vcpu, /* Try to insert a pte */ ret = kvmppc_book3s_instantiate_page(vcpu, gpa, memslot, writing, - kvm_ro, NULL, NULL); + NULL, NULL); if (ret == 0 || ret == -EAGAIN) ret = RESUME_GUEST; diff --git a/arch/powerpc/kvm/book3s_hv_nested.c b/arch/powerpc/kvm/book3s_hv_nested.c index 05f5220960c6..771173509617 100644 --- a/arch/powerpc/kvm/book3s_hv_nested.c +++ b/arch/powerpc/kvm/book3s_hv_nested.c @@ -1527,7 +1527,6 @@ static long int __kvmhv_nested_page_fault(struct kvm_vcpu *vcpu, unsigned long n_gpa, gpa, gfn, perm = 0UL; unsigned int shift, l1_shift, level; bool writing = !!(dsisr & DSISR_ISSTORE); - bool kvm_ro = false; long int ret; if (!gp->l1_gr_to_hr) { @@ -1607,7 +1606,6 @@ static long int __kvmhv_nested_page_fault(struct kvm_vcpu *vcpu, ea, DSISR_ISSTORE | DSISR_PROTFAULT); return RESUME_GUEST; } - kvm_ro = true; } /* 2. Find the host pte for this L1 guest real address */ @@ -1629,7 +1627,7 @@ static long int __kvmhv_nested_page_fault(struct kvm_vcpu *vcpu, if (!pte_present(pte) || (writing && !(pte_val(pte) & _PAGE_WRITE))) { /* No suitable pte found -> try to insert a mapping */ ret = kvmppc_book3s_instantiate_page(vcpu, gpa, memslot, - writing, kvm_ro, &pte, &level); + writing, &pte, &level); if (ret == -EAGAIN) return RESUME_GUEST; else if (ret) From patchwork Thu Oct 10 18:24:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830966 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6E975CFC60B for ; Thu, 10 Oct 2024 19:35:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kojyu0UfWOChY23ElAAPS5nr2EbxjFYzmV8MDMz+GGY=; b=pt7pZQVYyTPbrA qSs7e43brvuLn/TCJO3Hia3tN8S7pNwzhhSCcyVESyZ8bKqvvh3Bdj8Hcz8gDa5lhTvLlcf4xb3C8 rjnlWf/SgqBmPvurqqBCtMxZ/55COE3IzFeQs4Dl3EOeE+K40+j6b7QchUhdHk0xFHBiqKsWt607n xCX4z8TD/dZFiQx9QeWnJFinZ/txtbb47Q9FNLauivzTnM6mw2Isw8MOH1VfgOGYMyhe9KX/FLXmB fj5Xq1WQnVdERLOSfNIG4UqdWC+QyybeygMXQrI5i3fjeQZYnSa1YU4iuv3qToxS5e3u0nWe+xTWh +RMnhJuAjwcA7LpPa79w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syywe-0000000E7sI-3ks0; Thu, 10 Oct 2024 19:35:32 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsR-0000000DrNg-0YjC for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:08 +0000 Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-7ea0bf14523so1033550a12.1 for ; Thu, 10 Oct 2024 11:27:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584826; x=1729189626; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=I+2gp5vLlKRJWMtv7YF+IvP9h9YjJXEpCIK4AMigCis=; b=ovwcCNiTe5u04NlAVbvO4xK8yzOXNx3Ter+QhNXl3VOJCQEzI/gEvnC3H1V7Pqxq5A sOkzivM5rCRz91Y8yeZwvNqvIZLyWI/UTPOKy44Uv53XB221DIga2phGSlSM+PBCmaOl bgVg0eim/0V6+Ct3L/8ml5n/a7aAynz40Pzgot6a8ms+YupfuzIQ2Mh4wpNXQzF1nicw 60opQHHdJC1LDVjxEqhBrglkeQMOhacGkNexNdqMNHeRff1T8NraOFVlEAgUebACjRAr 6UmULf0Wwv971DDjdvY+42yiNd3AMS3OUiyNgkJ4jpH03Q1sbY6akeGf+wKLHMkEHkZ9 eKlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584826; x=1729189626; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=I+2gp5vLlKRJWMtv7YF+IvP9h9YjJXEpCIK4AMigCis=; b=o7CiF//kxvSuCCYNJfVTwoZWpnYIo7Gm4+1N6jusxuZnpfyxwq5UvMZFu1u5njVB1p 0dfNdnIs/XzZwILbWW8cMiCuWFd3inx5K++nhPGwp3D1dxf1EWkJbpP+EqMg8mbxkfZr OPwsshvidySmOf3z1tgF3W/dfUHCcHWlPRTTfwC4sW2fLQnvCRVQ1cjZP/k0Vg73Ou7g 0SzgGT41/Lyu+nUjdIamOpsp/U7T7nzRFTabsLVQ/S4UIBAxUMRaxL2ICrwkKk2czT3Y 28ip6dTVI2W1pssyS0zEnpMF+2BP+3VG0alCMj9zQNDx3rDyC8shfL4vsS3tdjxmxHKj sKvw== X-Forwarded-Encrypted: i=1; AJvYcCVYQggpoakjy5vMlj7cJCuJa3B3hBY48ETAbpUwCCh3ZClngWzzage/Ei0M2ALlzWYL27tfnZjnuIKM4g==@lists.infradead.org X-Gm-Message-State: AOJu0Yy/5/diePmxRTelA3h4jy/MHbNDszAlw89GngW3xcivu9MbuIdc XGChPogBVzOENV2b6Qg5cAt4F4Ea9Jcn1M594B7ARqaOQXeU+Hcn836NRuUf+7u+2LqzdS7XNZ9 lSw== X-Google-Smtp-Source: AGHT+IEBzxfSdV5URAWtipQSxApJSdWqJAcjcFibIBQs32g4G2eVPMYjVJgo5zFwBPuTEySHq4oumcdrCOE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:e546:0:b0:684:6543:719 with SMTP id 41be03b00d2f7-7ea535307afmr40a12.4.1728584825522; Thu, 10 Oct 2024 11:27:05 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:05 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-64-seanjc@google.com> Subject: [PATCH v13 63/85] KVM: PPC: Book3S: Mark "struct page" pfns dirty/accessed after installing PTE From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112707_322319_10E566AB X-CRM114-Status: UNSURE ( 9.69 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Mark pages/folios dirty/accessed after installing a PTE, and more specifically after acquiring mmu_lock and checking for an mmu_notifier invalidation. Marking a page/folio dirty after it has been written back can make some filesystems unhappy (backing KVM guests will such filesystem files is uncommon, and the race is minuscule, hence the lack of complaints). See the link below for details. This will also allow converting Book3S to kvm_release_faultin_page(), which requires that mmu_lock be held (for the aforementioned reason). Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_64_mmu_host.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c index bc6a381b5346..d0e4f7bbdc3d 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_host.c +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c @@ -121,13 +121,10 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, vpn = hpt_vpn(orig_pte->eaddr, map->host_vsid, MMU_SEGSIZE_256M); - kvm_set_pfn_accessed(pfn); if (!orig_pte->may_write || !writable) rflags |= PP_RXRX; - else { + else mark_page_dirty(vcpu->kvm, gfn); - kvm_set_pfn_dirty(pfn); - } if (!orig_pte->may_execute) rflags |= HPTE_R_N; @@ -202,8 +199,11 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, } out_unlock: + if (!orig_pte->may_write || !writable) + kvm_release_pfn_clean(pfn); + else + kvm_release_pfn_dirty(pfn); spin_unlock(&kvm->mmu_lock); - kvm_release_pfn_clean(pfn); if (cpte) kvmppc_mmu_hpte_cache_free(cpte); From patchwork Thu Oct 10 18:24:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E3B4CD2444F for ; Thu, 10 Oct 2024 21:46:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=u8gagoViXj7KP57Fs1L5Ny+JN3aowNSsP1ZsgYPo54Q=; b=3ROGeGpfMUC42F l6vBFsrP2Isl6GUBICdP7fkxPz+64hibJunATtCL44w2zScjHqV2+C+0be6iFcKnq3eHov0KSj6Kx q7T/pXxxqzCAy+yRScQ19zfPIM0xREuFZfS/z8fuswr32dwPAjNSPD6pwxLzuA7/kJhlpzp3lTO3e k2j8tvPS+GtuKFKidRRhyNylS6ampufZ5QpeOcA3MNBXDuZrTEDZoPW6cqNnfwMKNCKmJV9k6t1aP 7p7wFo9LdW31BUDyibd3Oud4J/dmMgh7OuxaAgyCA/ZArqDolAwitVSAQOLo/RD0IGZQXjBIVPM1n MvbfoLwidrhrKinsaHVQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0zU-0000000ESbo-2SUd; Thu, 10 Oct 2024 21:46:36 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsT-0000000DrOv-0nqt for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:11 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e29088a6942so2105909276.0 for ; Thu, 10 Oct 2024 11:27:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584827; x=1729189627; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=dJnHBDyCwUFZKgDYTVV2s9CsyA2Ll45yxWv/Qp8Ijpw=; b=Yw/fbnOtwrVcGPcy7qGySNhOJV6AqYCCkEKx6aIGv4uXkgcnVc6Upnl4oTw3sw05cf ecnC/qDQZE1PvYAcnxlOaiW2ZxoaFFflqxfT2puOfCO2/dZTjmot+SvaEYsNUrqoKIT3 KSm4OecDD/WjXsqjbMLjbi/q8iWRulvNi8fwIFJ4z2fUXcpfDNfPh9w7CE6vNgk6v5GV H6SrX7iC/DfPfR1P1mAo9mLwfPCXNBZJBPZunlcp/21jgbdSa4XBRWGHtBGNsbusOSTR OysSez9Nn6NNAIPq3O/DHUBoflUdQV3WEbYocMt1Y38AgKAFlFuz0TfBg5z4Z99m71DM IVTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584827; x=1729189627; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=dJnHBDyCwUFZKgDYTVV2s9CsyA2Ll45yxWv/Qp8Ijpw=; b=G5nOjxmGZ/NdaON8j0O6o104pGHvfUObMdh0Dgdz5OmwSAFljUzB1ViOd8oFE174Pb q9+S6345NPRFG77C7TEhh848yvNoS7SfNe7bTcQvqjxvZJOVGfRXkAxo/jS2vtFb5z4L 8D6PQPt0ShZy3XJwnM1rVRdVBl/y/UAXhEmczmjCiCj0niwOz9ONIf74bGHtqy5XdMn0 Lod1M0aSayZOQPwwlOTocg7VWPNHGWA+AqTVq41m7WQzkQlsnrRb+qwgTt/dNH2SK0Jq lB7tyh+N6TkuL5EbJLpvDmVxwVDGNfKvnBaNn+skbNxdOcZsJv0+h5fb33NPY1IV61Bq Ur+A== X-Forwarded-Encrypted: i=1; AJvYcCU28zdkF/8UnV6tsPHYKcrntHF+zEj0/NCEVPB6N4l9jN1IvFlf0RUgnqZqAL0eloGodVFJPvMMccz16g==@lists.infradead.org X-Gm-Message-State: AOJu0YzwrHuLoBnTRo33obZmaY5veGvEPnhbKfc7wLjQtEU4BLN/yRpT iD98j0iP9YQnkGyWA3j0MFP1h4sMs3z5g6nIn9X5cewxwAYcynS6PyZRDMCSxU0NOTh+mPbeixa 6mg== X-Google-Smtp-Source: AGHT+IE3wQG+AmAWHFVOEkWtTHcZkQX/AbMzk25RszCXWWBD0mfEZbS2aMxfU1imj+f8RQ2cZBD1mpHhlQw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a5b:a07:0:b0:e28:ef8f:7423 with SMTP id 3f1490d57ef6-e28fe355de0mr48920276.4.1728584827639; Thu, 10 Oct 2024 11:27:07 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:06 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-65-seanjc@google.com> Subject: [PATCH v13 64/85] KVM: PPC: Use kvm_faultin_pfn() to handle page faults on Book3s PR From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112709_347956_B4AFB36B X-CRM114-Status: GOOD ( 13.11 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Convert Book3S PR to __kvm_faultin_pfn()+kvm_release_faultin_page(), which are new APIs to consolidate arch code and provide consistent behavior across all KVM architectures. Signed-off-by: Sean Christopherson --- arch/powerpc/include/asm/kvm_book3s.h | 2 +- arch/powerpc/kvm/book3s.c | 7 ++++--- arch/powerpc/kvm/book3s_32_mmu_host.c | 7 ++++--- arch/powerpc/kvm/book3s_64_mmu_host.c | 10 +++++----- 4 files changed, 14 insertions(+), 12 deletions(-) diff --git a/arch/powerpc/include/asm/kvm_book3s.h b/arch/powerpc/include/asm/kvm_book3s.h index 3d289dbe3982..e1ff291ba891 100644 --- a/arch/powerpc/include/asm/kvm_book3s.h +++ b/arch/powerpc/include/asm/kvm_book3s.h @@ -235,7 +235,7 @@ extern void kvmppc_set_bat(struct kvm_vcpu *vcpu, struct kvmppc_bat *bat, extern void kvmppc_giveup_ext(struct kvm_vcpu *vcpu, ulong msr); extern int kvmppc_emulate_paired_single(struct kvm_vcpu *vcpu); extern kvm_pfn_t kvmppc_gpa_to_pfn(struct kvm_vcpu *vcpu, gpa_t gpa, - bool writing, bool *writable); + bool writing, bool *writable, struct page **page); extern void kvmppc_add_revmap_chain(struct kvm *kvm, struct revmap_entry *rev, unsigned long *rmap, long pte_index, int realmode); extern void kvmppc_update_dirty_map(const struct kvm_memory_slot *memslot, diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c index ff6c38373957..d79c5d1098c0 100644 --- a/arch/powerpc/kvm/book3s.c +++ b/arch/powerpc/kvm/book3s.c @@ -422,7 +422,7 @@ int kvmppc_core_prepare_to_enter(struct kvm_vcpu *vcpu) EXPORT_SYMBOL_GPL(kvmppc_core_prepare_to_enter); kvm_pfn_t kvmppc_gpa_to_pfn(struct kvm_vcpu *vcpu, gpa_t gpa, bool writing, - bool *writable) + bool *writable, struct page **page) { ulong mp_pa = vcpu->arch.magic_page_pa & KVM_PAM; gfn_t gfn = gpa >> PAGE_SHIFT; @@ -437,13 +437,14 @@ kvm_pfn_t kvmppc_gpa_to_pfn(struct kvm_vcpu *vcpu, gpa_t gpa, bool writing, kvm_pfn_t pfn; pfn = (kvm_pfn_t)virt_to_phys((void*)shared_page) >> PAGE_SHIFT; - get_page(pfn_to_page(pfn)); + *page = pfn_to_page(pfn); + get_page(*page); if (writable) *writable = true; return pfn; } - return gfn_to_pfn_prot(vcpu->kvm, gfn, writing, writable); + return kvm_faultin_pfn(vcpu, gfn, writing, writable, page); } EXPORT_SYMBOL_GPL(kvmppc_gpa_to_pfn); diff --git a/arch/powerpc/kvm/book3s_32_mmu_host.c b/arch/powerpc/kvm/book3s_32_mmu_host.c index 4b3a8d80cfa3..5b7212edbb13 100644 --- a/arch/powerpc/kvm/book3s_32_mmu_host.c +++ b/arch/powerpc/kvm/book3s_32_mmu_host.c @@ -130,6 +130,7 @@ extern char etext[]; int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, bool iswrite) { + struct page *page; kvm_pfn_t hpaddr; u64 vpn; u64 vsid; @@ -145,7 +146,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, bool writable; /* Get host physical address for gpa */ - hpaddr = kvmppc_gpa_to_pfn(vcpu, orig_pte->raddr, iswrite, &writable); + hpaddr = kvmppc_gpa_to_pfn(vcpu, orig_pte->raddr, iswrite, &writable, &page); if (is_error_noslot_pfn(hpaddr)) { printk(KERN_INFO "Couldn't get guest page for gpa %lx!\n", orig_pte->raddr); @@ -232,7 +233,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, pte = kvmppc_mmu_hpte_cache_next(vcpu); if (!pte) { - kvm_release_pfn_clean(hpaddr >> PAGE_SHIFT); + kvm_release_page_unused(page); r = -EAGAIN; goto out; } @@ -250,7 +251,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, kvmppc_mmu_hpte_cache_map(vcpu, pte); - kvm_release_pfn_clean(hpaddr >> PAGE_SHIFT); + kvm_release_page_clean(page); out: return r; } diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c index d0e4f7bbdc3d..be20aee6fd7d 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_host.c +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c @@ -88,13 +88,14 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, struct hpte_cache *cpte; unsigned long gfn = orig_pte->raddr >> PAGE_SHIFT; unsigned long pfn; + struct page *page; /* used to check for invalidations in progress */ mmu_seq = kvm->mmu_invalidate_seq; smp_rmb(); /* Get host physical address for gpa */ - pfn = kvmppc_gpa_to_pfn(vcpu, orig_pte->raddr, iswrite, &writable); + pfn = kvmppc_gpa_to_pfn(vcpu, orig_pte->raddr, iswrite, &writable, &page); if (is_error_noslot_pfn(pfn)) { printk(KERN_INFO "Couldn't get guest page for gpa %lx!\n", orig_pte->raddr); @@ -199,10 +200,9 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, } out_unlock: - if (!orig_pte->may_write || !writable) - kvm_release_pfn_clean(pfn); - else - kvm_release_pfn_dirty(pfn); + /* FIXME: Don't unconditionally pass unused=false. */ + kvm_release_faultin_page(kvm, page, false, + orig_pte->may_write && writable); spin_unlock(&kvm->mmu_lock); if (cpte) kvmppc_mmu_hpte_cache_free(cpte); From patchwork Thu Oct 10 18:24:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830967 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5F989D2444D for ; Thu, 10 Oct 2024 19:38:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=JEPlshgRWRRqhP7abXt+CqAihZdROvyf+AKZQkYdANQ=; b=2lCrrj9jla3uK/ H0m9XVtqo5AWrnwyVn35cO2NceEm3IpMau4PwERor9JUWqOVr14EjWwZEQKf/VRGTKJkX0JAQz0Iu Cw+qDOTXp7XIexib6sb3EBF+i5M57O6bSO9FYVIMtjKeXhdug34JUGmWo7vdScixkZyfwQI/IWCMZ bAhgkTBiaJzjCZMvKhBGR05DWZd6+fVjzktKtdtIhlwCcO2mvmYz09lz3cJJ5I09pCLXysdlzt6UQ wRzDWd9PDvXnx0OAbt1D8Mw5GJAkV+tcugS4H8wrlAFzu0MPusPldgKRLmWsbTgMUBstNFpzgBJWn vihOXf2Wn/11/HK9sZVQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyzH-0000000E8RF-3MfW; Thu, 10 Oct 2024 19:38:15 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsU-0000000DrQM-2PBY for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:12 +0000 Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-7ea0d46d1f0so1514631a12.2 for ; Thu, 10 Oct 2024 11:27:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584829; x=1729189629; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=0kIT1cNFK/xt59MDB/Io88ATgyMybLemEwY+DfOgUM8=; b=ncGV/c+AluGiB5Wub5u8UvSkBS0krkBKkswJl6TMY+O1XuP40I467d8ozAcdtO7hSX bztAb+6FlXzcvwIsAQ6TTFHDKDi7VhRVvgGcklZgclHPJPmQIJUmDxuuKMxJyiWV4Us9 Ml94yw8Oe73armjC74hfiiSHJHgPJIO7CSlAJKylZc6klQTjK+b7jKsxlkI+sKtaU9qd cifr07vNF2dIWwlBAwdWJ4VSg4uRAO8nMuyrn3L4ExIpZkQBXLiKPMlqGK55FpGn/Tqj k6bkx9fc/VgXfATlBMbhoK+qvMiheT5pGQJ8Qkty2qp7noqUc5YtxHoCd3kwzYuiRXs3 y4BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584829; x=1729189629; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0kIT1cNFK/xt59MDB/Io88ATgyMybLemEwY+DfOgUM8=; b=fDsgSswpDw+ZGjqjGGVVd/p4hv2yIK1qdkEJRIhUQbEyj9oVf7H1h3WwBexlttkb1a V5WHfDWph8GJvKDDarhpSAE4YzTi/jaiABhWnXUQeYTPZ0z5Ms+bNCiWUXGj5cY+6Zpo Y+5qrhPFUERNgLt3aXsLgBl+nQAloCVW+aE3qlb8n+2ppBWAFt2pixWUf5gmtiIgM275 /YoY1Z1qD1OEz6TpREfzzAl/2wkw2X7WKVT2u6BAygj2yd1KPsp9yLJ9NppToftgeaTn XqY65kYoOgOFa+7m8IGt6LpnEkdmP+7ZDRzKxg+4v523aNeUzgKdn3LumMVifO7jsREN AkMQ== X-Forwarded-Encrypted: i=1; AJvYcCU+WJR7pL9mkAMVlz5cMHUrKBHa/roD9huKgqze0ZcHrC5IQfsK2KRfmoKjkLqyW2A7l3Uo+tlXm+nxsQ==@lists.infradead.org X-Gm-Message-State: AOJu0YxDjaiBpHOVNZzn6ah7WtxGG7HmJJj9bcZeOnecyHrEom/xKlQx g0RPWzGcb9/X0eXM4o9cvmE1LXT3FRZE8T4irCVoeOfrPuHN3WoeqW14qixUlM1AKryuS71HpM3 8fw== X-Google-Smtp-Source: AGHT+IEVjHGN/ZoxQo+Z413816spxD1Vig97FmVkGQ57WYZx6k/7CY2I8TFVs2SPVm+Oq4UkGx7TgXf3+Fw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:192:b0:20b:bd8d:427a with SMTP id d9443c01a7336-20c6377ab7bmr1062205ad.5.1728584829203; Thu, 10 Oct 2024 11:27:09 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:07 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-66-seanjc@google.com> Subject: [PATCH v13 65/85] KVM: LoongArch: Mark "struct page" pfns dirty only in "slow" page fault path From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112710_656332_3380CD48 X-CRM114-Status: UNSURE ( 9.71 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Mark pages/folios dirty only the slow page fault path, i.e. only when mmu_lock is held and the operation is mmu_notifier-protected, as marking a page/folio dirty after it has been written back can make some filesystems unhappy (backing KVM guests will such filesystem files is uncommon, and the race is minuscule, hence the lack of complaints). See the link below for details. Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Reviewed-by: Bibo Mao Signed-off-by: Sean Christopherson --- arch/loongarch/kvm/mmu.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 28681dfb4b85..cc2a5f289b14 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -608,13 +608,13 @@ static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, bool writ if (kvm_pte_young(changed)) kvm_set_pfn_accessed(pfn); - if (kvm_pte_dirty(changed)) { - mark_page_dirty(kvm, gfn); - kvm_set_pfn_dirty(pfn); - } if (page) put_page(page); } + + if (kvm_pte_dirty(changed)) + mark_page_dirty(kvm, gfn); + return ret; out: spin_unlock(&kvm->mmu_lock); @@ -915,12 +915,14 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) else ++kvm->stat.pages; kvm_set_pte(ptep, new_pte); - spin_unlock(&kvm->mmu_lock); - if (prot_bits & _PAGE_DIRTY) { - mark_page_dirty_in_slot(kvm, memslot, gfn); + if (writeable) kvm_set_pfn_dirty(pfn); - } + + spin_unlock(&kvm->mmu_lock); + + if (prot_bits & _PAGE_DIRTY) + mark_page_dirty_in_slot(kvm, memslot, gfn); kvm_release_pfn_clean(pfn); out: From patchwork Thu Oct 10 18:24:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831212 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 148A4D24455 for ; Thu, 10 Oct 2024 21:46:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=FLK2cRfsEDkaGn3/Q3Ka2N2Qce9ALCnRdot2Df9vcNo=; b=p5j5zAN/VxT2B9 USmALwm5Z0k+gmWM94E6VO3MZjExKTEkNKYeum+L8NzgIfRSrRNX1x9DPBMRMTGLr1eSQ4kt5kpKd jAonqX+maIa6XtVjObatuvOaRslAHiRQUfSj5pHm4w6nyMElmymgMdbfk+XUEYhoogBP3w9SX1a0y /Y/oA70X7z13Ju6Ox5NwYPL8akXfwwmSgLnmpFLf99Fhq4EzVutB4qO7+OPa066KysreAs6/uYStT ldhFeQ9smcXi5J+uYq7kGw0eqkstL3QuZRvogatchWXDTO3DgvLvGVV9Zky/G4m6QmfvAIs4J69l1 llCR6eZAr0VK8cyKc+DA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0zb-0000000ESiG-1LMY; Thu, 10 Oct 2024 21:46:43 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsW-0000000DrSM-2ojI for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:15 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e28eea1074dso1984587276.0 for ; Thu, 10 Oct 2024 11:27:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584831; x=1729189631; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=2XctyNuj5rmp2glWM4MaDBrVp36o24kp0nhTjUz4HGM=; b=YAw0UCrfTR8yZSyO2C++/akkY9F8ImVdU+QHywnyEzDNoziQZcTnFyWLoTIQpJhvbV Y8Pdn3YxYXd0G6JVJDHuHfdxdQ9m+Qk7S6GzmJ9qM85akpQuGxzBDKtJPWDI2IUxznCD vq6DmPEgcnDV0SF23KJrvt4oUa9M6w69GM7oSp9H743VLbGqcPfYLAjnqaTw6lH5kL5I AeSEU2al1IUe8APYV1qkZosGuvEU/HUkQXE+1awQrmJRkpjT8XqM9EaKYUWt37X5qHY/ OQdNCE4r4uavOa2sXQyoineMlK5JoEa5HC2eFTf5e4O7iVUYru4c1B0gd0vQezhVxbT8 DApg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584831; x=1729189631; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=2XctyNuj5rmp2glWM4MaDBrVp36o24kp0nhTjUz4HGM=; b=GkXGYfGsJBfOBS+sypJFmITN0jJsB8HzyVQugJVIZbeFPUhziyCdZB3+o+SCpLFgZy J4a3Fz4VY8Vd8QXm3OtGJZTxzNkZEug+tNR22K1MEHy6VlQE9ekP1GvgR5dhRzawrZPr PW4eMNSGctrXJWwDSQgrI7BWdmwOsgSHoKJWi3w5ytvB49hp14LjVuOFqA/TIBR/8wUB HLrxkFwaJCzT98J1OT9t4bGHoUT9ShKVWrYY7Ec0fAo5+Lt5B4YZa8xSJe8XAkGgpuc5 K268dOPkXS7P3yuH16slSwQvqOvC+7hgAhhNk1PSjEKNcFV5K+9e6pIatlpWSzEe7dym rDhg== X-Forwarded-Encrypted: i=1; AJvYcCXiHJEUiw2UPoEdgxO1eBROjHj+i/wiMPJdZ9KDa/LDYAYoNdYKap++EVTpEZjvfcivuZkHyTp7EsaEmg==@lists.infradead.org X-Gm-Message-State: AOJu0YyG3j0AX+ERYg6gYS2x6G9BAPEXQpQ7CYRKJ8JNkvSyC+wmN0AF c4KD+BN+2aTx58zsDMJQywcd1hNzNcBlF2CXOoWqvhrKFfS4uefKGg9LdY1O5vbHlOffs3wYayJ Ljg== X-Google-Smtp-Source: AGHT+IHdtHq1a5g5QmW1TlBMuTKUOZ7R4d6aOVSlzi7r2JK/pfzCqaKRixPsJw6iRhPElP6sWQ73LfqmPps= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:d353:0:b0:e28:e74f:4cb with SMTP id 3f1490d57ef6-e28fe0df614mr96115276.0.1728584831124; Thu, 10 Oct 2024 11:27:11 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:08 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-67-seanjc@google.com> Subject: [PATCH v13 66/85] KVM: LoongArch: Mark "struct page" pfns accessed only in "slow" page fault path From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112712_844863_13A0150A X-CRM114-Status: GOOD ( 11.87 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Mark pages accessed only in the slow path, before dropping mmu_lock when faulting in guest memory so that LoongArch can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Reviewed-by: Bibo Mao Signed-off-by: Sean Christopherson --- arch/loongarch/kvm/mmu.c | 20 ++------------------ 1 file changed, 2 insertions(+), 18 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index cc2a5f289b14..ed43504c5c7e 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -552,12 +552,10 @@ bool kvm_test_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range) static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) { int ret = 0; - kvm_pfn_t pfn = 0; kvm_pte_t *ptep, changed, new; gfn_t gfn = gpa >> PAGE_SHIFT; struct kvm *kvm = vcpu->kvm; struct kvm_memory_slot *slot; - struct page *page; spin_lock(&kvm->mmu_lock); @@ -570,8 +568,6 @@ static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, bool writ /* Track access to pages marked old */ new = kvm_pte_mkyoung(*ptep); - /* call kvm_set_pfn_accessed() after unlock */ - if (write && !kvm_pte_dirty(new)) { if (!kvm_pte_write(new)) { ret = -EFAULT; @@ -595,23 +591,11 @@ static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, bool writ } changed = new ^ (*ptep); - if (changed) { + if (changed) kvm_set_pte(ptep, new); - pfn = kvm_pte_pfn(new); - page = kvm_pfn_to_refcounted_page(pfn); - if (page) - get_page(page); - } + spin_unlock(&kvm->mmu_lock); - if (changed) { - if (kvm_pte_young(changed)) - kvm_set_pfn_accessed(pfn); - - if (page) - put_page(page); - } - if (kvm_pte_dirty(changed)) mark_page_dirty(kvm, gfn); From patchwork Thu Oct 10 18:24:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830994 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C690ED24441 for ; Thu, 10 Oct 2024 19:41:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Ird9RK1nMnxvx5f3Q8pk6B2t9fiC5bgi9PKxFKvrKsE=; b=CDEjPvBRAorM3G P6ZrS/SukI8emDwQOFQuLA9gA81R+/YYleQ648xOE88deOltMp+3QkHxJTS7kywE+3N8FQc0Gw8m1 V6/VaWRHAEoS1HMedYbcWzSlywlim4YgXXBEQizVq6F6Mo4LMPqHYj+wGSpYQcKrwMCj5Vew1N6ir tnw4TnNqFcwkq6mLt+MFJBf8JfwsqrQo9jeTU7RKqhE3Jq2BsFk1tl3utwsWnQvp8b0FNmwytPJat G6iSWu+uBKHBwRym+SlxANZoJTHu3BBrPA1Xb4aG5mBHXDRcqRiikgvg1aP7dBlfxzioecW97P8HI BmrZ8f5rlgLdn974Uhjg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syz1v-0000000E8ti-3OKc; Thu, 10 Oct 2024 19:40:59 +0000 Received: from mail-pj1-x1049.google.com ([2607:f8b0:4864:20::1049]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsY-0000000DrTq-3Np3 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:16 +0000 Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2e148d4550bso1323671a91.1 for ; Thu, 10 Oct 2024 11:27:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584833; x=1729189633; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=3FFUjrr9Bcuid6sLVD5OEsWVTb8VG25P9bGjkNYbAzk=; b=jRyJBHlowWWe5rLfr7phIk02+Dz1smiC9B+8+l8fjItytQHfaPqfwXS9kwAda+9AmE shexSqSJbO2Rx5qii1tALAryS4pGnWvtVZCfeOEjmFWSO4HKZPxS8nkGfwDKwQymv5/V wfkt5jyod4tGS84UNkBF5H1yJop/76JLfNyM1euymRTGgEs/dQq38usgpogljBsjQeGI Dm9WN/3n4aZGT70TBOP644TkbWftIVyZDtlLOtZt6iA+3KaFCeUyJfaSP6mtPhXMMbZl celjnP4Ov3sMYoEohXSA+AP8wA6JsEeZNgXH6S6QNub5U4W5uofyo/ebOoZSoyqeVbK2 vrZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584833; x=1729189633; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3FFUjrr9Bcuid6sLVD5OEsWVTb8VG25P9bGjkNYbAzk=; b=bYQD+DD5aMfZ7efz0rFpNR3oqzGIve5q4W+Z0avdxBvsq3MenBLVwBuloQjasOsk5Y V+Yy5rnRPLnmJ+QzHWqChtuvmkuL0VzgyLIKRYko94fZ+B7DVYb2kN22+9w9JjRZiiGG m+wufA15ioVC4WzMp+ky8bkrImvNJQiSNXi2p9EJXKp/0f7IKjvbXHfIG6TVbprirlMG D5/j3Fq05AToWarrb7x/mXPc/QQhrLabaAPwi7jnjYgR/sDYu3ZATO6/sITzS1KxE4px oOogVYWOVRaJxsHju9JiAcVH6cYg7x3T1zjKv1fwFevpzggPLBFZm7LNaDPl1vi0+tWt jN1w== X-Forwarded-Encrypted: i=1; AJvYcCVJpbmT/aqr3anOUT39sBJ3WdayJ2Y3aEUXDfpqGNTmK3f5RuaIALMs7Oe/bAva73MbOWtEzSt2PW2WxA==@lists.infradead.org X-Gm-Message-State: AOJu0Yw1e9k0ILK1gUcBqf2gE2Icyh7LiV4TyhNSASWe5pAMndaBQg2i jCMc6n0D6YO+lmVno88b1KA8JAfQUpo9GeQOIjP+MlwqNKymLrNU/pNKBuccAaCbrUagNSF9XlH L7g== X-Google-Smtp-Source: AGHT+IFpwqxWKAWfrUvKF1EatvnBv5g5CrzcpgT/ZTuiTODJnzoCoYK053S1MFwF0WyL9wI3LVhDYDAiv74= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90b:46d4:b0:2e2:e148:3d37 with SMTP id 98e67ed59e1d1-2e2f09f1a56mr94a91.2.1728584833102; Thu, 10 Oct 2024 11:27:13 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:09 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-68-seanjc@google.com> Subject: [PATCH v13 67/85] KVM: LoongArch: Mark "struct page" pfn accessed before dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112714_964461_624660B8 X-CRM114-Status: UNSURE ( 9.77 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Mark pages accessed before dropping mmu_lock when faulting in guest memory so that LoongArch can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Reviewed-by: Bibo Mao Signed-off-by: Sean Christopherson --- arch/loongarch/kvm/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index ed43504c5c7e..7066cafcce64 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -902,13 +902,13 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) if (writeable) kvm_set_pfn_dirty(pfn); + kvm_release_pfn_clean(pfn); spin_unlock(&kvm->mmu_lock); if (prot_bits & _PAGE_DIRTY) mark_page_dirty_in_slot(kvm, memslot, gfn); - kvm_release_pfn_clean(pfn); out: srcu_read_unlock(&kvm->srcu, srcu_idx); return err; From patchwork Thu Oct 10 18:24:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831213 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6528CD2444F for ; Thu, 10 Oct 2024 21:46:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=AGU/GqKu5gmitGlqNZL0ex+Vjx34GEE9NoP6G2B4ALA=; b=wEbu/iUTxrizKW cK8/DQ03MdchjX8e8bS0AyjkUd9dhqm0Qbt5tzhPk2BCYTWWPolYw67+9eUqTRzvhcHM+qcADnqPP 6iQNDjN61n4iAZD4MmIjeJzgGWpHU3pOSiLQ8vibW1qauUn93x3zRWImt0/4qgxIo+VK3Wrba8gH+ FZI+YCcD/Fu0MTLAboWxi7nL14vbKj5g7ZsfRd6Wk5c3iubJDPZ3bkC6IOoK+riO+kqBc2RenBspi 4/xO+Lpw/RfKv1I9rcQrJ6NJ5LZ+wDm/fwzl30CfaISOLYmIZbwGS6/q4496h8Yrz6YrKvYhhh46h neAH8cbHgvoZxU1GIsYQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0zj-0000000ESpJ-2pR6; Thu, 10 Oct 2024 21:46:51 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsb-0000000DrVO-30Ey for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:19 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-71e019ab268so1589828b3a.2 for ; Thu, 10 Oct 2024 11:27:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584835; x=1729189635; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=9LTMkDJZxM2S5Qh4t0DfHllWvhtFq1rJ9EqXHTYHy/A=; b=Ezg7/CMWQDMLN8YEdrgAzjhCIggY01A7M2zQRTVFeivnUP9xNYMnDgSn08tTNAQswa kqk7p2xGoJBvlWCJY2F295EpfrTh6Gs382v094DWZ6fCATf7c3raAMlaHVsWLbMuZX8q lhxCBrETU5/+2nt/FLnhY0WsswBeH/2OGGYCpGMOpF9BEDb2/zwz1gDzgmFNd4XSF4Jc yE5+qnvk/nYcYh7E+904AvUOhyHPUho7ahqpJEfAu8ClaWnh4qMEVzY/715tDYQ7YiPf TRld3kwCAO1N9Z5t2zjmuJFVNJrxILimSxqerNKmR1ypmopxODnWp9w7ztgiyY+zhKy6 q3iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584835; x=1729189635; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9LTMkDJZxM2S5Qh4t0DfHllWvhtFq1rJ9EqXHTYHy/A=; b=esiW/Bc6+LMpM0xGBsek8715lS3Xc9X60tbK3ttWnxJvOgpXq/XoVRQ3Cayy2pjSm1 sfl0NJZTnn3XldqKFfGk+SlhrQZ5wltvf556yyyVAmcOIuISlvLWpaCZVJ/xFipvpuNd VYwSXCMc3XWfBiziCopxK+I3TiIZQr4K7eWjKNPLoG7FqXkZazNyXuXjoJQ5L6UJLZs3 iI21VMijSTONV0ISsxvlCfJY1xJBXHXzq4u5aX6a3NPa5hTWTBEP/X8FE6H6GLHtYjJO c73/WUnqqbo2O5HXQEnfpbW5bnKZcvPco4LR7FIWNZA2oBxDLZFE974bDQGMXF81hg49 soDg== X-Forwarded-Encrypted: i=1; AJvYcCVFNA+mxsv18vYd8jb0gybm8pxk04ZEoUFXTIgQGTEVIqKPlE6Al/ZyTaoQ0rNoNlddEqoxYPDkduoe9Q==@lists.infradead.org X-Gm-Message-State: AOJu0Yxcl2wy4fTS6q/CzulhNdlnZzNv7R8idicwps1PrjMftXWoiiyB COwvwddaiGDXHAAaRB8VDHVpRZm+oeqX2LDuaYzo4wz/Mvj7OgtxGfVR8ksuGSb1TVIPItZx2C8 h6Q== X-Google-Smtp-Source: AGHT+IExt7obORK9hFSNzpsBdJKygVAVx/dPvrUuCl2bsyTFt9B1GPfq0Q1aHG/lNRl/yht5gsZcWFEXy1g= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:8614:b0:70d:fba:c51c with SMTP id d2e1a72fcca58-71e1dbd032amr6898b3a.3.1728584835380; Thu, 10 Oct 2024 11:27:15 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:10 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-69-seanjc@google.com> Subject: [PATCH v13 68/85] KVM: LoongArch: Use kvm_faultin_pfn() to map pfns into the guest From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112718_441417_D536197B X-CRM114-Status: GOOD ( 12.62 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Convert LoongArch to kvm_faultin_pfn()+kvm_release_faultin_page(), which are new APIs to consolidate arch code and provide consistent behavior across all KVM architectures. Signed-off-by: Sean Christopherson --- arch/loongarch/kvm/mmu.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 7066cafcce64..4d203294767c 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -780,6 +780,7 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) struct kvm *kvm = vcpu->kvm; struct kvm_memory_slot *memslot; struct kvm_mmu_memory_cache *memcache = &vcpu->arch.mmu_page_cache; + struct page *page; /* Try the fast path to handle old / clean pages */ srcu_idx = srcu_read_lock(&kvm->srcu); @@ -807,7 +808,7 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) mmu_seq = kvm->mmu_invalidate_seq; /* * Ensure the read of mmu_invalidate_seq isn't reordered with PTE reads in - * gfn_to_pfn_prot() (which calls get_user_pages()), so that we don't + * kvm_faultin_pfn() (which calls get_user_pages()), so that we don't * risk the page we get a reference to getting unmapped before we have a * chance to grab the mmu_lock without mmu_invalidate_retry() noticing. * @@ -819,7 +820,7 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) smp_rmb(); /* Slow path - ask KVM core whether we can access this GPA */ - pfn = gfn_to_pfn_prot(kvm, gfn, write, &writeable); + pfn = kvm_faultin_pfn(vcpu, gfn, write, &writeable, &page); if (is_error_noslot_pfn(pfn)) { err = -EFAULT; goto out; @@ -831,10 +832,10 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) /* * This can happen when mappings are changed asynchronously, but * also synchronously if a COW is triggered by - * gfn_to_pfn_prot(). + * kvm_faultin_pfn(). */ spin_unlock(&kvm->mmu_lock); - kvm_release_pfn_clean(pfn); + kvm_release_page_unused(page); if (retry_no > 100) { retry_no = 0; schedule(); @@ -900,10 +901,7 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) ++kvm->stat.pages; kvm_set_pte(ptep, new_pte); - if (writeable) - kvm_set_pfn_dirty(pfn); - kvm_release_pfn_clean(pfn); - + kvm_release_faultin_page(kvm, page, false, writeable); spin_unlock(&kvm->mmu_lock); if (prot_bits & _PAGE_DIRTY) From patchwork Thu Oct 10 18:24:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9B7B9D2444F for ; Thu, 10 Oct 2024 23:01:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LVs9qHt/eGF3EiceM99W6MXvNO9gbtKsbHgCn2n2Uk8=; b=RpvOzn63CONoEN ecFOSNr+4mnOT6YY9Hj637NmGA2zen6afFypFJQrRNVR8ejL/GTlGUlnEOsLXi8Me09V0l2mNb3P5 F1eui1QtFLoo+pjkYzoJ1f7NeReJUr07t4S9YdXDsAr3cc9kJmOqvmUPiRvEOup9uFA+QGBUZXTeg zgdg5H0edZOQ+SqhOIbaAV/ewbgXfx0JO/gSHraf3uc0qnRQS0Z79p+xKZkzsX2LhDuIk3GIs130I QTJyCZYtpTWKns38MwaGm3MqF+AVpQnQfteDQixEsAhRgocILSUHZuVID8l+v0rZqLwHxdxRO6aL3 DzBuN+KRxDMN8JeEuggQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz29v-0000000Ec5r-2KvK; Thu, 10 Oct 2024 23:01:27 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsc-0000000DrWC-43OE for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:21 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-e2906efc0eeso1521807276.3 for ; Thu, 10 Oct 2024 11:27:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584837; x=1729189637; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Sag0fj8OwucY2qRogRu5ELvjWktNcy71jZ/+qgEdHaE=; b=E2zHzaRZqpx7PmmOp3LUhoHU9Dr1MkBl5G0WTprILSNA1mq0tHxEkTsU10v77+QvpN 0zFwtgeL7W3gLusd7cDl6EFytNV+h4xDtKwQ84UL7e0cR2cdNgj3Dnms8wEJu/GQeCw2 rt7R7dMOceITVKPIgq5JWL/qwK3SfMqFQ2pgXkciOlQiuO16Q7TJPnLvr3j8HJGzk/g/ PNbNvTkYqJyNtEyeLClT7pwiry+CiUZP34/omc1IrYUyQZyp2bxaEbPY4j91F8t3KOyK XOI9XrCQ/PvTIRVwt9Rf9AqfV0T7By8bPGHJYXiy5nTIL84TTZYDMKUFvEYpQPHXGEzJ 9snQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584837; x=1729189637; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Sag0fj8OwucY2qRogRu5ELvjWktNcy71jZ/+qgEdHaE=; b=tQyVMPph9HIfDGEETOJnmPNZ+LKOHR6ChP88PMVGDK5eWOpGWf8ZOKLheFoSiW0v8X jawKkxzf/OrA0a6lXs0iU6D6GGMHR4QikjG58dJbnpc/+rKYUHt65Z2ProfkuTiCCHy6 FFjjv/a6Ci/3vd4gXz9GM6sN+BrmrubbnGzX78cqjcRd7I0Y1JbafwkFcG2mGKw6pXIQ jJFSCplXxT4QE78w70oeK4RC6JbvJ2u52fdtdwERJo13qo3zBZ30Z4Ejlw3ZgWteMFOM fHx5wFAwEYdJlbnXF38O6GS49KV3EKJaK7DD3jEIa+hWx/QiXGPb6heWECwSRPHbG7UC DMgg== X-Forwarded-Encrypted: i=1; AJvYcCWZ4CdPO6/SS76F28oNQ85Kgtd/6CkAt+YZ4/ShNAI2RFDtYB/6CleAU/hhZkppS+vIjPXEh7Ps3OrwzA==@lists.infradead.org X-Gm-Message-State: AOJu0Yw1GkFr0386J/f+OzP4MT/RAni2LByWZgVxw1w0azQNZNvYdiPo 66/B7nMwjSGgUDM/lNRw5DKKyOrhYY1Omgm3bOnQux/DpsN2X2meK26oeunk+gDdQoEBYU9dTkR LFw== X-Google-Smtp-Source: AGHT+IEjJriLMGICq9yty21Fx3Z3FQyTG0XCg9lS5qhc+MvTVoRASG8drmw0OxSoWZyAtrQmM3pJDo8jQYk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a5b:a4c:0:b0:e28:f454:7de5 with SMTP id 3f1490d57ef6-e28fe52740fmr75826276.6.1728584837458; Thu, 10 Oct 2024 11:27:17 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:11 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-70-seanjc@google.com> Subject: [PATCH v13 69/85] KVM: MIPS: Mark "struct page" pfns dirty only in "slow" page fault path From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112719_106173_80506684 X-CRM114-Status: UNSURE ( 9.57 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Mark pages/folios dirty only the slow page fault path, i.e. only when mmu_lock is held and the operation is mmu_notifier-protected, as marking a page/folio dirty after it has been written back can make some filesystems unhappy (backing KVM guests will such filesystem files is uncommon, and the race is minuscule, hence the lack of complaints). See the link below for details. Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Signed-off-by: Sean Christopherson --- arch/mips/kvm/mmu.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index c17157e700c0..4da9ce4eb54d 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -514,7 +514,6 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, set_pte(ptep, pte_mkdirty(*ptep)); pfn = pte_pfn(*ptep); mark_page_dirty(kvm, gfn); - kvm_set_pfn_dirty(pfn); } if (out_entry) @@ -628,7 +627,6 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, if (write_fault) { prot_bits |= __WRITEABLE; mark_page_dirty(kvm, gfn); - kvm_set_pfn_dirty(pfn); } } entry = pfn_pte(pfn, __pgprot(prot_bits)); @@ -642,6 +640,9 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, if (out_buddy) *out_buddy = *ptep_buddy(ptep); + if (writeable) + kvm_set_pfn_dirty(pfn); + spin_unlock(&kvm->mmu_lock); kvm_release_pfn_clean(pfn); kvm_set_pfn_accessed(pfn); From patchwork Thu Oct 10 18:24:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831214 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CF2F5D2444F for ; Thu, 10 Oct 2024 21:47:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=fS5i+fG9tyTxF5pXq+KCcTf8aPtRMdI7NiKzsmzeS9A=; b=KMvDSOhQO0v8nD e6FdxeYlcJom4PsKnwL8A9nAa4lmsv+RS7k9rzMuSvVFEenobM/qf/+aiwbfby1fCFxBwPtSUAdAd GmZiHIMja1gVCq7hu7THSgXHgR2ETQlOYykAbHZjd1/b2WXfXQMWretgDFGoPqtJlQOis9HjLQzKA MMC6S/Da0JMSYB+u7Tyd9E2aLJvQc49HTeyF32nlpn8DEsuJNQWq5AKTq/JbSIXOLAs31d3IO6AYX W1xUcsujCjSJYpbeyRMzNcvHVDuS574GXlU7ib5RQC3NnFYwC4NobeFSIqsLO8YMLSP/ZyezzOZoM 9TVIU4wq05AhoeNIi2FQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0zp-0000000ESut-3waN; Thu, 10 Oct 2024 21:46:57 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsg-0000000DrZ3-1dQg for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:24 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e25b39871fso27072847b3.0 for ; Thu, 10 Oct 2024 11:27:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584841; x=1729189641; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=KbunjaDKwfDzESX7bCyZnVXI4Jj60WAsli6cgjlK44o=; b=d36EgFFD42cTaH04fi/y4xFv+qp2ifFMxUrP1SQnXseIpwdmnbEzTQ8DSdY5otquib Fe4VayuL6/R0aasi9i7aCCPV4Gv1zlkR0JHNaWNpB/Uu253T6upc+xiprDa9B/mJybKL 77xr20zI26cg8Mq8sN8ZrtHXoNuERM9pAq2+ZDWuy6zeA9+GuTt7xnLiNbaJwQKdPD2U 8iRnxJv76mYNSesvHPOFSlccB6NnI7KLu7OUcwUJa63Q2yECwrEM4nizOHw+DX60vWLd /6jJWIBZt18OZ6zN0L0nZtncrUpAsn3nxdPUEj281wfWOhAwbAmyb2k5lD1F9+UTfMxZ W5xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584841; x=1729189641; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KbunjaDKwfDzESX7bCyZnVXI4Jj60WAsli6cgjlK44o=; b=B2gXxbh6Pf0WoheCTZtPL1MV9XCkJboQ9pdJtvQJ/9okY4YK7LpRl8Q/Ys7rGIJbBO bYj+0jecWh9EmfFJsCyucwwjjy/vJW5BuntbUcZq+Gx17dQ9L7Np4qYEDwnLUTkW62M9 TGoqVB+J1sFMjvGZqDRaOxDBDADh2NDFQ24CyBDKOUt89Xr9oRWHlC4N+feqhH7m6S0k oHN79fQmCyUYYzTqpyeD+xLZdx9l+L/qSFai0AT7xvVfovc8wnbm0XFn203Ib2vpDkDH 0X3gmasMeIgLepP8HrHXkPNAcGlJmYL691OpidhRy+LsWmmL3LjbOpTV0UEHCLHlvwF4 nu0w== X-Forwarded-Encrypted: i=1; AJvYcCUiuSBSr25pEel3Ghs3jmwNAJtnW5bvSNVMxoHztszz3MOuu9EkelkCioAAEc/X30wAV6lmQk2kIUZ/kw==@lists.infradead.org X-Gm-Message-State: AOJu0Yxc4olneZevXUNHth4LqMrJgMBSorKrJ7N+yIAkpbKnQhomB/mw q17ut0pCmK/y3d4ZgLGQ/CdQBBPRx9K2e5P7QpDxD+Xr31OsdSgUm6CXsgRShErSMdfgZGbJaOF Qog== X-Google-Smtp-Source: AGHT+IEbamZE/KJDJ4kw63fZFZN0PThnG54xKd82AQ09AwFkx3KIodEPvEqsDBHZyT6DCHkY/GuTD4px7aQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:6c05:0:b0:e24:c3eb:ad03 with SMTP id 3f1490d57ef6-e28fe540170mr1289276.10.1728584839383; Thu, 10 Oct 2024 11:27:19 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:12 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-71-seanjc@google.com> Subject: [PATCH v13 70/85] KVM: MIPS: Mark "struct page" pfns accessed only in "slow" page fault path From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112722_521795_7BF8D00C X-CRM114-Status: GOOD ( 12.14 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Mark pages accessed only in the slow page fault path in order to remove an unnecessary user of kvm_pfn_to_refcounted_page(). Marking pages accessed in the primary MMU during KVM page fault handling isn't harmful, but it's largely pointless and likely a waste of a cycles since the primary MMU will call into KVM via mmu_notifiers when aging pages. I.e. KVM participates in a "pull" model, so there's no need to also "push" updates. Signed-off-by: Sean Christopherson --- arch/mips/kvm/mmu.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index 4da9ce4eb54d..f1e4b618ec6d 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -484,8 +484,6 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, struct kvm *kvm = vcpu->kvm; gfn_t gfn = gpa >> PAGE_SHIFT; pte_t *ptep; - kvm_pfn_t pfn = 0; /* silence bogus GCC warning */ - bool pfn_valid = false; int ret = 0; spin_lock(&kvm->mmu_lock); @@ -498,12 +496,9 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, } /* Track access to pages marked old */ - if (!pte_young(*ptep)) { + if (!pte_young(*ptep)) set_pte(ptep, pte_mkyoung(*ptep)); - pfn = pte_pfn(*ptep); - pfn_valid = true; - /* call kvm_set_pfn_accessed() after unlock */ - } + if (write_fault && !pte_dirty(*ptep)) { if (!pte_write(*ptep)) { ret = -EFAULT; @@ -512,7 +507,6 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, /* Track dirtying of writeable pages */ set_pte(ptep, pte_mkdirty(*ptep)); - pfn = pte_pfn(*ptep); mark_page_dirty(kvm, gfn); } @@ -523,8 +517,6 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, out: spin_unlock(&kvm->mmu_lock); - if (pfn_valid) - kvm_set_pfn_accessed(pfn); return ret; } From patchwork Thu Oct 10 18:24:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831335 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5DFD9D24462 for ; Fri, 11 Oct 2024 00:01:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=e2yS4EjTXSUuHl8zP7Dlq8rnQGVfvAkDWYpjRt9thxw=; b=4ZMso+pFg5fYa3 IomO0fmcOs27D+Pc0XwSLU59NroqJ/4FZjmd2kJpdG8k5q6hUvDC0HNJHrP2HnClNh9jfGzdvjRsi 0m2QbWFGsIDf3vULrEyG7WcH/qCzu/4XdKsIEnMqsfP/28+LDr+BH2jFYm3zbcX+kuqGfGPdT+KPe u2yzx0dfRHnkuoOo348AdSx72gZ3XxphB0DznuY1ApAz9ewzsulKX5AYNGVjD21cOMX2LWRKwbeWq mJmyAin8M2dM0pM00nWtQisdn7Ej4fzgXUfE8jRWGbOnqAXPd2jDKI/inOtPgeoOU7DQKS2xsRHm5 FC05rr/OYbRQ9bNp2B4g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz362-0000000EiKM-0GPl; Fri, 11 Oct 2024 00:01:30 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsi-0000000DraI-0ICc for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:26 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e31e5d1739so25061577b3.1 for ; Thu, 10 Oct 2024 11:27:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584842; x=1729189642; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=OZ/c8QPJoSxQXGCWT5aWp1nMy1t7EEzE7+N8fqq+BFY=; b=v6KPJlqYJR4mGmcmL/AsMF84scs4V0oyq2QlyNfixKKzIATDSNUV+0XlnkYG0wXdfG 1TGlx8Q3aEM7vHjZJ7j9Aff5MlUcPnC5NpHxe2emoEMK6I3xeIS+wrEnMFX0gGMrKMd6 3o+8Tcuf41G1Rd4uU3ujqDhOcby0NH+Z3wn6chCUf85Q1DZyOLbb/v5kRcBVgBP2tXFX wAs/sI95wXa968+EifVDV5rU72Gj02WH3yR+CxwZG+eE0WG8Dd7khjpUpzuT79kEnRBO BRi/5p+4SSJNd/ZT54cjZaN3WBY1yijvzhZ2QmbtJmCAtVIifH/3gMdwG9OHl/0vDIL0 j49A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584842; x=1729189642; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OZ/c8QPJoSxQXGCWT5aWp1nMy1t7EEzE7+N8fqq+BFY=; b=FEQNMDQU/h1MK4L8oD/hJWVJY4jVDtrvxD5QwGoq8un//NDK6C3LXYMCn/qU934QOF NkxkJIXoAHxVhYiMPB/0uxEV1BxN+bBKbZnjT6cq71SXVzm0ajZ8yd3wbH9qtuincuXQ Tf/nXT1vnSb6pZ55Ax0yRNbnhITjqtkMKza87S6LtvEiYhEwvpvcD8W1fWBhh1+blIlp xCvSLCfEuXC/2ttR4YBGiLFL21ElChYbQhfIzy3CaYWbyDKBqna/TI/81xAjQU3ejSxd RAUYQ3cwyEnbM9ehcDs8kV4Yd/rfVV4OkS8IUNduhVHwqrTDIW859o6Xorqli5H1YnkL vhBw== X-Forwarded-Encrypted: i=1; AJvYcCXRLGs4JK32yLVeZWrUOLPqoeB51bXuJ0gihNmfJJFiLcUT0z0k1q0vkKWZUKeqGi3pKKCAE0ba0+uHnA==@lists.infradead.org X-Gm-Message-State: AOJu0Yzs0MRnGTCTFG9G2Sm8wL+UYr7J4HXMKxFfT7Kr5HckPDCjB7B/ OoGLxM+Z8IJKeBcQY/wWIufhU49o96p3lQ3yxZRjOpaUl0F3keu5FiB+hXZtowr5J6FypMtr8rZ wGg== X-Google-Smtp-Source: AGHT+IHhD+Vk6TwFIS5v8fA24SNPRWU4vCcQM+1g+zVw6L+6pfrMpIdoNti+PehtHCeQDGmk8Jps/9lQ5Ek= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:d30a:0:b0:e28:fe07:9cc1 with SMTP id 3f1490d57ef6-e28fe4a3a81mr68085276.3.1728584842321; Thu, 10 Oct 2024 11:27:22 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:13 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-72-seanjc@google.com> Subject: [PATCH v13 71/85] KVM: MIPS: Mark "struct page" pfns accessed prior to dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112724_262725_85FB71E1 X-CRM114-Status: UNSURE ( 8.31 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Mark pages accessed before dropping mmu_lock when faulting in guest memory so that MIPS can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Signed-off-by: Sean Christopherson --- arch/mips/kvm/mmu.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index f1e4b618ec6d..69463ab24d97 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -634,10 +634,9 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, if (writeable) kvm_set_pfn_dirty(pfn); - - spin_unlock(&kvm->mmu_lock); kvm_release_pfn_clean(pfn); - kvm_set_pfn_accessed(pfn); + + spin_unlock(&kvm->mmu_lock); out: srcu_read_unlock(&kvm->srcu, srcu_idx); return err; From patchwork Thu Oct 10 18:24:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831336 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0757FD2444F for ; Fri, 11 Oct 2024 00:01:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LKBr5my2ABOVRAAPoUjiooXv82BYEKKboYYPDphBtAI=; b=SUCTQ6Ldfn+oF4 q9x4FhVf2WNI/rs+AAP4RDsEFmuAKztyYsIstB+u7yJJ1xf92OiE1IsHFthy4a2g+bezh0pecRrV9 dBpA1eldVDmgyJcgrYcqIuqJHrxdG9po3uP8Hcw8NNpT8qt2x9EN9RSpBQVvFDTji+TynuARyJOeS PCcSeGvnLtITtEqPDl8gAEBxMH00WNOnXJ4EFS8bQWi+04OWRGpj3e+kGGWB+zY0kvD/KEsLcR58C adVOVNJANvyNqfwaN0TIrRFTxdnr3FKRW091DSojAaajQb4dreyVCDdvE0hkmDuhGWYf60mAF+1b5 GMH18z0cBdEDS9fjm2DA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz363-0000000EiKd-0FOR; Fri, 11 Oct 2024 00:01:31 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsj-0000000DrcI-3xFi for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:29 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e1159159528so3623547276.1 for ; Thu, 10 Oct 2024 11:27:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584844; x=1729189644; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=ojyPU0tzKIRRbswoqL6JjpEgqRgu1FFXpokE5rqa8U0=; b=OAZSNh4NZXQ/Zipvt47P/0efWGRR16annLUzUsTQgPtiN5G+fdWQUGTrVxyFBmqnhB 5AwMcO0uoBvpg7YhIua7lfEp5t/Wdpj3/l3GdjJ7VGRuLVwzna5mFAu5gcMnBfIGaUJv J6ZtjS5V8iY78UnacO6paUXR+XjxiokqAg6xpYBPOjRZbJJ2/OlFErl6lNLT3KX2S1eP I8AQ8GkmS6CRQMk3isu+HpdKOIFg7PcvPLchk2swDOijcWMRHLjAwION6+UAevFGARbl XMgILnZ8FZ3qaVR2BJLppR7eXDvA4spUXJRz/lQJetwc+AO7yz1h7BktBMonH7m3/0Ta 6SNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584844; x=1729189644; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ojyPU0tzKIRRbswoqL6JjpEgqRgu1FFXpokE5rqa8U0=; b=jFIVHVp0D1UiUuQdDg2k3NKntzF2MFxVn/2tycnf/63vv4+Rs58vcvPr6qY4jGwja8 YQxUVjWJBttmYCJquRe4btfrCy/YEZOESi/VtQQE1fuIVAQ7f5/7Bz3jIryewvHjUcGb Hu/u2i33J+FEmCdTi4SMTIa2V+NrihtXLw9Ez5j/XFc2sChHBSy3KuTyatGkSXS7Z1YC sPEv/dvvfyXpawZXWsyXM3g8gX+Njikx+sciWvYR5j7O/C2DQr9WiLVkgJBXBQLUxSez n/MAhGtNCL/V050vM4Q/CmgWQgGwYov6kzWRt95ez/xfV7LJhgRwdqXx9E4j4tTXef3X +ymQ== X-Forwarded-Encrypted: i=1; AJvYcCXiLPNeShuZAUZhdBG2monmYwrMgPJkvdfsq/1g955YPAGh6VF4mwjdfeEPxTVWx+yXzVn+XcQWgs9w3A==@lists.infradead.org X-Gm-Message-State: AOJu0YwSgJRIsyvRoyEUCKSlECeDWf8q/xCyrWnCvnln2JxTeHVRA2yq 31diAKUkAPt86uQ2JTm4PG5rjQH6YaKjeWwhkqAMPC/T3NtzngpsA/KNAGuu4yVLjDIpKcr8msG Fxg== X-Google-Smtp-Source: AGHT+IHEyKrdGSoNbMWcRNiKhgC/jPgVf/TjauaPCrq7yqhpXyuJQagleroMC0w40O7lZsOojHLr7EwrZqI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:6805:0:b0:e20:cfc2:a326 with SMTP id 3f1490d57ef6-e29184333b9mr26276.6.1728584844184; Thu, 10 Oct 2024 11:27:24 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:14 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-73-seanjc@google.com> Subject: [PATCH v13 72/85] KVM: MIPS: Use kvm_faultin_pfn() to map pfns into the guest From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112726_233136_C2F933E3 X-CRM114-Status: GOOD ( 12.04 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Convert MIPS to kvm_faultin_pfn()+kvm_release_faultin_page(), which are new APIs to consolidate arch code and provide consistent behavior across all KVM architectures. Signed-off-by: Sean Christopherson --- arch/mips/kvm/mmu.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index 69463ab24d97..d2c3b6b41f18 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -557,6 +557,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool writeable; unsigned long prot_bits; unsigned long mmu_seq; + struct page *page; /* Try the fast path to handle old / clean pages */ srcu_idx = srcu_read_lock(&kvm->srcu); @@ -578,7 +579,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, mmu_seq = kvm->mmu_invalidate_seq; /* * Ensure the read of mmu_invalidate_seq isn't reordered with PTE reads - * in gfn_to_pfn_prot() (which calls get_user_pages()), so that we don't + * in kvm_faultin_pfn() (which calls get_user_pages()), so that we don't * risk the page we get a reference to getting unmapped before we have a * chance to grab the mmu_lock without mmu_invalidate_retry() noticing. * @@ -590,7 +591,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, smp_rmb(); /* Slow path - ask KVM core whether we can access this GPA */ - pfn = gfn_to_pfn_prot(kvm, gfn, write_fault, &writeable); + pfn = kvm_faultin_pfn(vcpu, gfn, write_fault, &writeable, &page); if (is_error_noslot_pfn(pfn)) { err = -EFAULT; goto out; @@ -602,10 +603,10 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, /* * This can happen when mappings are changed asynchronously, but * also synchronously if a COW is triggered by - * gfn_to_pfn_prot(). + * kvm_faultin_pfn(). */ spin_unlock(&kvm->mmu_lock); - kvm_release_pfn_clean(pfn); + kvm_release_page_unused(page); goto retry; } @@ -632,10 +633,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, if (out_buddy) *out_buddy = *ptep_buddy(ptep); - if (writeable) - kvm_set_pfn_dirty(pfn); - kvm_release_pfn_clean(pfn); - + kvm_release_faultin_page(kvm, page, false, writeable); spin_unlock(&kvm->mmu_lock); out: srcu_read_unlock(&kvm->srcu, srcu_idx); From patchwork Thu Oct 10 18:24:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 38CD5D2445D for ; Thu, 10 Oct 2024 23:01:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Bq2nCUdNUenhn+0qwMkfOevNMnhTN1K8S3VFw3uS1p8=; b=vLSzu6BgAKW88k PD02gLbiUen5L8xfD96sJ5brepd7MF9JealPEc/iJFZ1opTBwDZA66vA9R8HJmaUNaPmpmr43FJ9s JfAcpqIw/9wzuGyFvJkjKbgjlUpjrLkQ7REH8ti2MnIHUdyoHPs+0xLsHzcC9NMCwap+MNnkdvD1s LKbFKE88xdAGB9ZZUrjz4ceNTwM8OZr7TPzaaAg7Qo6yjLOk0LRYk9HCYWdqkhPH/32taXowBUixO 62iiagjuZ2aN5OZs2EBaZ/0UAvSv8UBuCnZ046bpRZ2lRbFNtbGCmPLxtUzjDuFcVoNB7QaPUtpp1 gMktmMAqQJ5RzY/9j4Yg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz29w-0000000Ec6I-2S8k; Thu, 10 Oct 2024 23:01:28 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsn-0000000Dren-2Rj6 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:31 +0000 Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-7ea0bf14523so1033825a12.1 for ; Thu, 10 Oct 2024 11:27:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584848; x=1729189648; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=t4DEWxyHdS8FH8bmPlxV35CavnLWtw4MDWneo+GIfaw=; b=3qI8Gw0xZEjRCMSlShvB9CxPYVE+sQUqWY+2lQ3x/hsrogq9AlmlMY0W+xIiiyB2+G ivumflW0xs/76E0l8I7fd7bbDdyPLteemqZYYQ+ZKK/YPblG/xVoZuzgyJoRX97tQfwU YIGnX9P9pbtCRUkjnKrNiuDGenR/Sjm4DPbwNWZAbMadx+iMNdYcHOHirra/2+DNgg7J SV89gZ2MqD3S2NzVbA5KhNlU7zbQKrI9AUvsp4uNOzuM55xfH+GZz5OMQp029NBhBRbT 1kBG/UEPxC0x7ucQl28rHGPkGDnEsOsIC7omLOZgZouSoX8ld20Qe3dCRqoJaXRUVo6j ZO5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584848; x=1729189648; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=t4DEWxyHdS8FH8bmPlxV35CavnLWtw4MDWneo+GIfaw=; b=WaFBbUo6rmP4OVo6P8ffFTv3oFmhi3u0brDlquvHr8pS8e28BG5hfn82m8vB3AM4os XYPXv51NauUbhaBjzM6hj+Gtc6lFpnueMP10UcxKh6xl1fe0qZ/VrDHj75+3aNBEr/Sc 1rSsX4dGLwdlK2kMAfn6gbrlLKsImtR0HrQ/CcP4pMcvZ+9qVBmwYmJ1U3JxW8XzFDEr Da/V4gwHWlaTC1kTpLi9MV1/crJPR+W0NWvE0lUeAn+R4KDaHzDKwv8pE83uAcxXMUtZ lf6AIS/ypxnQ28E0Dq8zB2aFd6t5kyJ1FgsEJ6lPfHG5bXKqpcAjwJCWOZZItBTLcV3t lYMg== X-Forwarded-Encrypted: i=1; AJvYcCUegWK3cZViVLaLWwIxIrRF81vdlAEEqCaxFv4SC0zl9aNjl/uT/Lrw/gT8YuZ5yGGmp/yeC0+YFthHsw==@lists.infradead.org X-Gm-Message-State: AOJu0YwKM5wFOn3roG4zovA48qi7bO0OQPuhHGVcEFoMGo1Hed2sGqHa HAU92pWlYEDtJ2+sxr3N4K7+3TK+OVueo9o0Pe1YeFMy5sjqv4lQ6UuLt2MHotO4Rj0sZhjgw2L RTQ== X-Google-Smtp-Source: AGHT+IHU1NaHt28BJAfE9+Eihx21gIRaHWD3wJUzNDiwOfeaS4l1n7K6hdxyKTp49hAFCJqVbhS/fDDFwRQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:2545:0:b0:7e6:b4dd:fc0d with SMTP id 41be03b00d2f7-7ea535b0baemr20a12.7.1728584846518; Thu, 10 Oct 2024 11:27:26 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:15 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-74-seanjc@google.com> Subject: [PATCH v13 73/85] KVM: PPC: Remove extra get_page() to fix page refcount leak From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112729_704341_5A8C6D23 X-CRM114-Status: UNSURE ( 8.71 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Don't manually do get_page() when patching dcbz, as gfn_to_page() gifts the caller a reference. I.e. doing get_page() will leak the page due to not putting all references. Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_pr.c | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c index d7721297b9b6..cd7ab6d85090 100644 --- a/arch/powerpc/kvm/book3s_pr.c +++ b/arch/powerpc/kvm/book3s_pr.c @@ -652,7 +652,6 @@ static void kvmppc_patch_dcbz(struct kvm_vcpu *vcpu, struct kvmppc_pte *pte) hpage_offset &= ~0xFFFULL; hpage_offset /= 4; - get_page(hpage); page = kmap_atomic(hpage); /* patch dcbz into reserved instruction, so we trap */ From patchwork Thu Oct 10 18:24:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831276 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 66294D24459 for ; Thu, 10 Oct 2024 23:01:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=V8OpRdYDYNYMg+v3Q68IlYlCm51rOaXUgBZzUEytETQ=; b=eMCvMppkf4B8XT px36inpNzTyG/WNJ4Ob2uleYvaFrbE6ZP0VRd1/vVsbPeJyV37UMbnqQ8HaE+8YSIk+jTdPDQf9iG fFSsxB8I4jluMezqAmmzcN5NtMIm8Oerz4Fnyj7BTW5zE6UKnLjabpSpp6dcp5ioIJMeKZKapUkqX VXral9SzTN1LH3mKegekx5XhmA7LQlnaEWB/udfAymhQwHm+y4u8XTkFKpEmKAP4477Q9kSwm/eVN SoQYiY2WyIoAGBbujzhNdl3WHG2DALVISoORJHUCiessD1iWaPdRKIsJmu8l/N6uTur9K/mN5s58C sOtX2cy1DJZRdZCdjyTQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz29x-0000000Ec79-2AvH; Thu, 10 Oct 2024 23:01:29 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsp-0000000DrgR-0elV for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:32 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e2d287f944so24113597b3.3 for ; Thu, 10 Oct 2024 11:27:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584850; x=1729189650; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=DOlbrv8SiXT4cHz79KGvl4RMUSE/a2Kzzqa4ZWuWNYk=; b=WG3oPKOSidsrkm4ZH+Q4EkkXjyWwVnO477YpGgUaQXbbCDGKf93WFRJzL3OwJpofqf p8bdC1jeQCqxkvH9UEPZCI4TUBUrl11FjPcF0Jg+56xJraONLFpIh4p8QB77QEpzyASg WVsvq1N9adyKBs7THN8PQgi2x2Rkj4Fo63J7UR0df8h9znJyqu5xPmc1cvtbMBbD4Xom up1kKmglzcsU9NWDYJmdhKbHEZKKESlhu0NvtbrxieUxie5gb9cciF1QAZfHOGbJGhuE kLWFwIdmEyWnDLNui2wg1ranKldu2obulk+W9zlMSWf7qQsfQ4M5Jphmr1wnhyoiFjiT ia4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584850; x=1729189650; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DOlbrv8SiXT4cHz79KGvl4RMUSE/a2Kzzqa4ZWuWNYk=; b=i75SNND1dAhLsiLkW3uWuls3mE9TEw38zgInBpXri+1g7mqNZlWSLlwNLQO0q6jt6H FxRuqVm0Va8BTRvqz/tP8S/0wcIslGPgIDLisYRbvFM0+biGayGkd7i01+yBogygIHK1 yRloDhHtuFTsqEpVMKU0h35nIgzezMY47NNcBVly4jhLshlqavHDoaHD6hfdTzscj64g hIh/7ocCXRSZvu/6pSPynDtlAyPKY1aTyknYNNxUqvGbwCUeqm8p/2JX9AqrmIvbxVU9 /9WOiekUQLNRaLXcXCN034FK+mLMunSeY3D3XdkmjaFyLGFueRntHQ2SWJiHTegP4oAh Ro+g== X-Forwarded-Encrypted: i=1; AJvYcCWXtG4eHw4Mj/eLofV/lsLZ4jYNnxroObddn0Lap+6q6KakIurQ9ANKd315IIOjN+VeUFvEsHer0jy/lw==@lists.infradead.org X-Gm-Message-State: AOJu0YzxhbTInYQZYNyB3HFpr0RSSclJxkYYeRRzkcAqvrM3lYhK+IYM k/OlKC0+reEIAXGiWu8TFlfFVSfnsPU/v5QnpCCqQRSsEaxSkkGxbIkRcFfPy49d3gFPqlfXqyD OFw== X-Google-Smtp-Source: AGHT+IFYivPtvbNeJBK5QJ/MFgPKbd4XQ4GDAEMeTnfD7MuZnOh5ZwxKPEpX2RNQ0xKTZSwXcPzTQuagswA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a0d:f043:0:b0:6db:c6eb:bae9 with SMTP id 00721157ae682-6e3220dbc31mr798327b3.2.1728584849896; Thu, 10 Oct 2024 11:27:29 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:16 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-75-seanjc@google.com> Subject: [PATCH v13 74/85] KVM: PPC: Use kvm_vcpu_map() to map guest memory to patch dcbz instructions From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112731_241789_5EC0106F X-CRM114-Status: GOOD ( 11.69 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Use kvm_vcpu_map() when patching dcbz in guest memory, as a regular GUP isn't technically sufficient when writing to data in the target pages. As per Documentation/core-api/pin_user_pages.rst: Correct (uses FOLL_PIN calls): pin_user_pages() write to the data within the pages unpin_user_pages() INCORRECT (uses FOLL_GET calls): get_user_pages() write to the data within the pages put_page() As a happy bonus, using kvm_vcpu_{,un}map() takes care of creating a mapping and marking the page dirty. Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_pr.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c index cd7ab6d85090..83bcdc80ce51 100644 --- a/arch/powerpc/kvm/book3s_pr.c +++ b/arch/powerpc/kvm/book3s_pr.c @@ -639,28 +639,27 @@ static void kvmppc_set_pvr_pr(struct kvm_vcpu *vcpu, u32 pvr) */ static void kvmppc_patch_dcbz(struct kvm_vcpu *vcpu, struct kvmppc_pte *pte) { - struct page *hpage; + struct kvm_host_map map; u64 hpage_offset; u32 *page; - int i; + int i, r; - hpage = gfn_to_page(vcpu->kvm, pte->raddr >> PAGE_SHIFT); - if (!hpage) + r = kvm_vcpu_map(vcpu, pte->raddr >> PAGE_SHIFT, &map); + if (r) return; hpage_offset = pte->raddr & ~PAGE_MASK; hpage_offset &= ~0xFFFULL; hpage_offset /= 4; - page = kmap_atomic(hpage); + page = map.hva; /* patch dcbz into reserved instruction, so we trap */ for (i=hpage_offset; i < hpage_offset + (HW_PAGE_SIZE / 4); i++) if ((be32_to_cpu(page[i]) & 0xff0007ff) == INS_DCBZ) page[i] &= cpu_to_be32(0xfffffff7); - kunmap_atomic(page); - put_page(hpage); + kvm_vcpu_unmap(vcpu, &map); } static bool kvmppc_visible_gpa(struct kvm_vcpu *vcpu, gpa_t gpa) From patchwork Thu Oct 10 18:24:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 216D0D24447 for ; Thu, 10 Oct 2024 19:46:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=VLGDBnmINk9iNIHfMt/nQmLojOk5Man65yV0msS6bCA=; b=L7aP0wFrYwTcXI E8fd97bHqAfXDOTUwP5XfbmXy/PDIiMpBOP8sducQuA554pp6A72Ul1q+KRVCKGYI8mjJMNPoLtjP t4E4YpzBrCbMwTMIyUyhNpUcTDlph6YYF9Y93UjP2lZ4Ic7Ser7XgqS9y4yi2sOo3BAc88PWmOq5L D6P0uy/KHBnbFElL7xd3yMF3sFKm2W8mDUYrRJKOeTl+t2pVeJPDxZHVeIv4ba+E0aivcH2yYYs9X amzZ5KsjoXuK8pLSYBMQ1vin8iGrvwnnIQtWA9X9q81dT3vbjIpp00B4Kcxx+111VszX219pLgIhQ 0uqnSWXlt99G+FxoVTnw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syz6v-0000000EAeX-0og4; Thu, 10 Oct 2024 19:46:09 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsr-0000000Dri6-0f4k for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:34 +0000 Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-71dff575924so1544123b3a.1 for ; Thu, 10 Oct 2024 11:27:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584852; x=1729189652; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=fZpnWf51XalE5JTpUzeqj9xBy85lPuBTlHLH/3crpzM=; b=cLJk3d4tBFRZOiFs+6bQQUgL9+5XLv36ZxgTPI3j7N9jPbVYQn5UYEHXG30q5cZAtp iLoVd863iI6QslfhR4bSkprz1688JE5LF2FO2gSOhcM+IGJX0QpjdBnRDdMQ62ouC3oJ bplSy/qMe21l7Xy1xG1IlCZN2yro1FWFNRNmv6sS9NMfZSUFAnJgg0EuPch5URNPcMG1 EWbkSmQF0LU/UoEnI63VnExMYr/1qn9aDn18jIArJhfw+WuPYh6KtbBICRCY7K4lBcNN jSLMr/sBO077LKQZMCJI/BZ+oDjzP6wnkJ4048tQ5WDcVuMAxNcfuoW2taX/zjS5bjPT jaEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584852; x=1729189652; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=fZpnWf51XalE5JTpUzeqj9xBy85lPuBTlHLH/3crpzM=; b=SSzFQKsrTT52kN18edqmrZYJFfZrNwjQqLcXpb7C2tTcYfwLNrnUNZZQcZMWhbpOLj YgWUDJLnDmh1xaXd1o7gZVz+9ZzPyJFWVWvM4pNVY/Ls0PfAdgeLWHkCYbYqS6YTqR9F rYXydc0CW0iZyGcSko9Rr086ugMKud9hdsDL37UoET9IpHb1qJg+Na1B4WMIGints38S T+A8XRaYW5F1IG2D6fI4P5EAeC9I3agaETdOqmA3TXwi6nca+megCOBtXNFmDi/dLggu VWqcgg9g37P2Jo7PWZ9twesBsd4iZTG2MnK67IiFALJ4yVHbvh+QstCH9Fi9pBoChAeA 8QtA== X-Forwarded-Encrypted: i=1; AJvYcCVX5J7yNABiJpAaVJfG+9k5SjqJ72XS3AtxGQ3VKLsRRUGrE1E+3hl/2CPS8N8OC/fv0bRykef9uCWVww==@lists.infradead.org X-Gm-Message-State: AOJu0Ywuz19H/V2unOyEhg2xz5WTz6iaiLbFp2wfNOHSuEOrlaFZawym KF4mUqFs47hSDKO44ICLwufaZdev3OX5GQwEtl9xou5rdRngL4qgGGZpV0/at8BO1Iu6/IhhQdU 1kA== X-Google-Smtp-Source: AGHT+IHVUYgAS8Gi1OfHaumBjOVs/68U//Xc3NGlK2wEye9l2w1SuCdDQ/WQfsMbBzqgGqsCESzcy3AUxv4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:aa7:93b4:0:b0:71d:f64d:712c with SMTP id d2e1a72fcca58-71e1db65a16mr7365b3a.1.1728584851459; Thu, 10 Oct 2024 11:27:31 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:17 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-76-seanjc@google.com> Subject: [PATCH v13 75/85] KVM: Convert gfn_to_page() to use kvm_follow_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112733_323638_D767D447 X-CRM114-Status: UNSURE ( 8.42 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Convert gfn_to_page() to the new kvm_follow_pfn() internal API, which will eventually allow removing gfn_to_pfn() and kvm_pfn_to_refcounted_page(). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 696d5e429b3e..1782242a4800 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3145,14 +3145,16 @@ EXPORT_SYMBOL_GPL(kvm_prefetch_pages); */ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) { - kvm_pfn_t pfn; + struct page *refcounted_page = NULL; + struct kvm_follow_pfn kfp = { + .slot = gfn_to_memslot(kvm, gfn), + .gfn = gfn, + .flags = FOLL_WRITE, + .refcounted_page = &refcounted_page, + }; - pfn = gfn_to_pfn(kvm, gfn); - - if (is_error_noslot_pfn(pfn)) - return NULL; - - return kvm_pfn_to_refcounted_page(pfn); + (void)kvm_follow_pfn(&kfp); + return refcounted_page; } EXPORT_SYMBOL_GPL(gfn_to_page); From patchwork Thu Oct 10 18:24:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831243 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6B414D2444F for ; Thu, 10 Oct 2024 22:16:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9LljG5y5OtePdz6zv/Y9FrxpXXIp4krs/rdLlRD98ww=; b=Wa3Z8EI/NOKuBw 3tERcKwT+o88p2QTfiO4UQ6ejRljtzekPL++ExWjjj2OsWi9avouaX6bv6Gf+doZA5ZTgvKPgC4yP F6ogmb/JlTINxTJamRJM2UCWEpa2cYGGzxcaF5NGWDCCTi6yjFuFQsSPCcXiwS2WMfEM8Umlq++eu g/rxrtblfO7B8KZkPXdVl3B0BLNISeXtuOKjOGm5Eaa/Dx7WTd/32QLorvnUHEIycgXp1A/LMkPfH 50nIvlf9rKhL5uAZI9PpZ3V1wgEP+7paVTgSWVwhn9rsoOuRctmd1MqFaAUJwrRNrfQWgnxcoK6di Je6FqhNEr08jv3t1ZK6Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz1SS-0000000EVvs-0rDP; Thu, 10 Oct 2024 22:16:32 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxss-0000000DrkA-41Nx for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:36 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e28b624bfcso20418157b3.2 for ; Thu, 10 Oct 2024 11:27:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584854; x=1729189654; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=fQZoDf0NIFdWK0Ory3IxzF5D2dmoMJWboRGGHV/nWyI=; b=aLG1ERGsB5NvGh5NeKe3MCWim49sfOkO/B4LSwLQ0B6zMt1Z4zcHaGKAQHVba9OrFU QUTVJ4caJJgKcEln7aMvhzkYAClxH4agKnVmAQUlEtIcIbWcMEx6nPQMfPsnIsjywymV H6URPqrZmddDnbO3fB+DaERHSqNHQzuWZ0yDyezoaY8+rRHdLQ4Ga8EmDXO1zEkwK9s3 g+YEQ2EL99QEKhplKwek9Qwbsoh5C+rUXoK63HH2nyy1uU/moUakBBPGG28zmfGGFcjz h/DZDDZHkaZcqSHfipSgnfEpXCOx/kjd+vWOiVgJh82gxS0bYE1WVYdtDM+oPtbZzxJT x46Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584854; x=1729189654; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=fQZoDf0NIFdWK0Ory3IxzF5D2dmoMJWboRGGHV/nWyI=; b=S1ziD+oKTBa32EJevU/76io/bxT+g3l0oxZLZWbTD8kzFOXRaXgI/uYlQzEZ31qVp8 218/cUetURG1ffDplE3dHuFeDrW/Dw7UIoq6eNzo8WMiNyEpRIaTPbI9x7drZowjTqnc AGy7uT0PsLg1rxfq0nzG2zCre96w5A+DHCEiM2k44WduxPU+FrmuJHVQ28ho9u4oFjmA Atmql6ZvGoI7wrnCgJq0XAuENiDAiDzA/URnCPbxQVhELp5tx48rnv2Kc17CcdIbiLoM axILFMmJ2Q5vp99g6vsymYVkq1rQnbAfPgSgnPvfTAbqd4yCYFTeR7z1hhGeU+hLkBcg /P2g== X-Forwarded-Encrypted: i=1; AJvYcCUEdrcI2JULXn2Tla3zzRS6jNvdzipWeo0IpeIotFE6LzseF1Llmb+nLGKsTsjpB3g2hbap2OQ7oYxFyw==@lists.infradead.org X-Gm-Message-State: AOJu0YxTDJQgp03sXTcjoO+eqRK2TGV7CswJT9psJzV3XZAhPunFk4SV DrSXn0pb8qex5S7XFBsnYxkhaNBEaNA0yWB2u8nGhqZrreRhS6YBBwkSNaV7VnvoSs4aDQumKdL Z1w== X-Google-Smtp-Source: AGHT+IGkMjnJcd2oWHVLgC5xfQJb4mHLQjnUvb6XgW9ZV8VH9lzrMuhcRfl3e0TBTVqXdJBO9V/Z2ttoh2Y= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:4091:b0:64a:e220:bfb5 with SMTP id 00721157ae682-6e32213fbacmr192077b3.1.1728584853563; Thu, 10 Oct 2024 11:27:33 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:18 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-77-seanjc@google.com> Subject: [PATCH v13 76/85] KVM: Add support for read-only usage of gfn_to_page() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112735_109685_FE26BD8C X-CRM114-Status: GOOD ( 14.01 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Rework gfn_to_page() to support read-only accesses so that it can be used by arm64 to get MTE tags out of guest memory. Opportunistically rewrite the comment to be even more stern about using gfn_to_page(), as there are very few scenarios where requiring a struct page is actually the right thing to do (though there are such scenarios). Add a FIXME to call out that KVM probably should be pinning pages, not just getting pages. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 7 ++++++- virt/kvm/kvm_main.c | 15 ++++++++------- 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 9f7682ece4a1..af928b59b2ab 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1213,7 +1213,12 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm, int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, struct page **pages, int nr_pages); -struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn); +struct page *__gfn_to_page(struct kvm *kvm, gfn_t gfn, bool write); +static inline struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) +{ + return __gfn_to_page(kvm, gfn, true); +} + unsigned long gfn_to_hva(struct kvm *kvm, gfn_t gfn); unsigned long gfn_to_hva_prot(struct kvm *kvm, gfn_t gfn, bool *writable); unsigned long gfn_to_hva_memslot(struct kvm_memory_slot *slot, gfn_t gfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 1782242a4800..8f8b2cd01189 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3138,25 +3138,26 @@ int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, EXPORT_SYMBOL_GPL(kvm_prefetch_pages); /* - * Do not use this helper unless you are absolutely certain the gfn _must_ be - * backed by 'struct page'. A valid example is if the backing memslot is - * controlled by KVM. Note, if the returned page is valid, it's refcount has - * been elevated by gfn_to_pfn(). + * Don't use this API unless you are absolutely, positively certain that KVM + * needs to get a struct page, e.g. to pin the page for firmware DMA. + * + * FIXME: Users of this API likely need to FOLL_PIN the page, not just elevate + * its refcount. */ -struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) +struct page *__gfn_to_page(struct kvm *kvm, gfn_t gfn, bool write) { struct page *refcounted_page = NULL; struct kvm_follow_pfn kfp = { .slot = gfn_to_memslot(kvm, gfn), .gfn = gfn, - .flags = FOLL_WRITE, + .flags = write ? FOLL_WRITE : 0, .refcounted_page = &refcounted_page, }; (void)kvm_follow_pfn(&kfp); return refcounted_page; } -EXPORT_SYMBOL_GPL(gfn_to_page); +EXPORT_SYMBOL_GPL(__gfn_to_page); int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map, bool writable) From patchwork Thu Oct 10 18:24:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831244 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7A7B7D2444F for ; Thu, 10 Oct 2024 22:16:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9NRpCwQ37hwv4iVTTa5kooGUJM7bH9iwLl4FRi7YOwY=; b=zIYdPHbW/Od83v btMmgITd8JDzkHJHedCi/iK1z19bak364RPgWiMypaOtWlvFpY7m5t8/1kGjr5JJ6bAvpdteVuHT3 /dwXJOHRP2cTDXX27uKEALtbgCRegkKKoGhV90E8drNWhHVbr8ao1GDfSI2w68ekF/9BdQ1dCozmj 20IHPGPAi2Z0RclY5jUXkvokbbAbE4R3juCem8GbpiCx7YT6O79uiaY058uyrO8j+QgUrL74Lla6t FhiI2GYm4jfjddyA2ViSrtj5CWNpawIl2Ih5GuXBHLzmdqIJRmIccjT5LRKfme8ibow4qpbIsUzmy sP9Y8GrslxDnomJK86iA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz1SV-0000000EVyX-1QYe; Thu, 10 Oct 2024 22:16:35 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsv-0000000Drlw-0ia9 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:38 +0000 Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-7ea07766e09so1070446a12.2 for ; Thu, 10 Oct 2024 11:27:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584856; x=1729189656; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=04eHUk6AClz9cRMQcjlWjE46ZR1uXceY2ogFB00LGDM=; b=alq1Gi8IKHXoRenVXYXecNt3KC/lj7CRLd4Ivpw77cRV2hhaqmorMgi3dwBXJZi2Z3 +oSIHdym3Kp6z1HFNs3xH8wtQWXWauU8ROxoDYovdX651koTj7r/xxKLyCYmTJRKZ2Q7 SZ6WMuDGYO+bEyQTNVfOCYgEzHS+3cAAeAuzf6XXFt3BezmXoMSk1NB74uRtHh2g67pb m2KGWKyP5u4SgLIVf5Unq6dz72HfCgCIZ6QIcRV/fMvo9IuyY/oxw9iWwhx0/1tvNyg4 NQoXsHYuYADad7JGxE3yi6t0dOyKX2LqcGYeBRRgvvWIXq3TkwikALhUOeoR75a/tGED uyZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584856; x=1729189656; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=04eHUk6AClz9cRMQcjlWjE46ZR1uXceY2ogFB00LGDM=; b=cZPtGXYhZY1RT4CZ8EzYbhOASqIfPNEJAMGovMsk+2Q3T8X+/nAn5UGTud6Wquev5V hHqbGjldSueKn902keaUNQpIWHrUBr6cfMRobiiFEKNifXt/k0WJMZip76SnCmvw7qi5 /sQaCl8D2+pNA+JH5jHcEus2MOoinjeF2neMqO0G+pBJkSnIklyQigzCGmMsuoXHnieA S6plj1hQ5U3aWH/fab2xy0pFkyJchfw9e+6SzG/czrsI+cH/rqmH+UYJ5LnjJFFhdvLD DvjuztQPQO8XZbBRB09NqSn++i46OktgIekgpqZnPD0+WWy/i8lwz/rkw1msfv79uZxT 2LRA== X-Forwarded-Encrypted: i=1; AJvYcCULxR/h/fadtts/nDOJousGL9gXbLplGxze23MhrVQ71mXhNsw5qvC1mitXS83PICLHNF1KUrGAh0KN2Q==@lists.infradead.org X-Gm-Message-State: AOJu0YyKJexfdffXvd2JirHzEQMmtEYTRtethiZkR5LETni5MfEVlE5M WR4CJZMG+iPNCiv1gj8+uVIfJnfiU7uA73gZtueYp2n1+mxnspPm0ekzHHbiUgiKLKzRHH2BQzX ZGw== X-Google-Smtp-Source: AGHT+IHGiwNd3UbnZUzPre1tk98dNYoM10q7gFGG4lq0PldsJuDm1rxMHI+g8Yc3UsVnmkzWnOurUMFmqIQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:41c4:b0:20c:8f78:67c9 with SMTP id d9443c01a7336-20c8f786946mr26385ad.5.1728584855618; Thu, 10 Oct 2024 11:27:35 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:19 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-78-seanjc@google.com> Subject: [PATCH v13 77/85] KVM: arm64: Use __gfn_to_page() when copying MTE tags to/from userspace From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112737_306723_62F8600E X-CRM114-Status: GOOD ( 10.54 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Use __gfn_to_page() instead when copying MTE tags between guest and userspace. This will eventually allow removing gfn_to_pfn_prot(), gfn_to_pfn(), kvm_pfn_to_refcounted_page(), and related APIs. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/arm64/kvm/guest.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index 962f985977c2..4cd7ffa76794 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -1051,20 +1051,18 @@ int kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm, } while (length > 0) { - kvm_pfn_t pfn = gfn_to_pfn_prot(kvm, gfn, write, NULL); + struct page *page = __gfn_to_page(kvm, gfn, write); void *maddr; unsigned long num_tags; - struct page *page; - if (is_error_noslot_pfn(pfn)) { - ret = -EFAULT; - goto out; - } - - page = pfn_to_online_page(pfn); if (!page) { + ret = -EFAULT; + goto out; + } + + if (!pfn_to_online_page(page_to_pfn(page))) { /* Reject ZONE_DEVICE memory */ - kvm_release_pfn_clean(pfn); + kvm_release_page_unused(page); ret = -EFAULT; goto out; } @@ -1078,7 +1076,7 @@ int kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm, /* No tags in memory, so write zeros */ num_tags = MTE_GRANULES_PER_PAGE - clear_user(tags, MTE_GRANULES_PER_PAGE); - kvm_release_pfn_clean(pfn); + kvm_release_page_clean(page); } else { /* * Only locking to serialise with a concurrent @@ -1093,8 +1091,7 @@ int kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm, if (num_tags != MTE_GRANULES_PER_PAGE) mte_clear_page_tags(maddr); set_page_mte_tagged(page); - - kvm_release_pfn_dirty(pfn); + kvm_release_page_dirty(page); } if (num_tags != MTE_GRANULES_PER_PAGE) { From patchwork Thu Oct 10 18:24:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831246 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EF7A9D2445D for ; Thu, 10 Oct 2024 22:16:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=J7hsaTlJW2pBR7xTxpMBuMIP79NssEbzzlCsAgl2vrA=; b=AcbPi/lxe59UNw b1LsZcP1p28jHk8vc5zgDSgbnHpK4QvzEXF2nZm1UXCXCHsaKuZxjMjc7l3mMCVmw4DZ3eHrkatFg vBy4RRHZ981lw5Pg8WpdnZnowZOipc6VZbl9i4VcyU5/kiOAnEmNXSvpne0EPZzkcxYfw9rVAhZIe L9q0BnHj+DO+aQP1vFGwFqFtaOJM5jcQmQYy9yYyYIfV62FFneVYA9kSruAItsn8jup4x6fWcfEcz OwgC+ENfGxEKD0C8fqM5TiGtr0ZhB508j9PYsCgLDtSlnR1EIdXWUPKvHGAc1BUpS1Sy7h1yNQe4K 6B/xYhRhE+0dzbw85lKg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz1SZ-0000000EW1L-0yv2; Thu, 10 Oct 2024 22:16:39 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsx-0000000Drnx-1opc for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:41 +0000 Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-778702b9f8fso1040813a12.1 for ; Thu, 10 Oct 2024 11:27:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584858; x=1729189658; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=43Dd+lWRPu/QQcFLGDLhCyyiH7uAqu5vbmltTcj5okg=; b=ge/KVFe5o7m0CrYItpSkaRrqEs6HCvjuf4dtPzfv98KYlQGh/QLeT5Z5fSRGccyiyM L5Z9adg+js6Zw4rVdQ2gjoah1thpDjUSIdwA5BEg90iA0A65+u/GFFXrzuRt0RMsF8nc FjEi9xlv7xPIBxHYIHGj0TGsieacYRgZvJkfGXimSosPbUmJj+3ajkP7+MzdBEKWOnvp SHbcEfX0hrL5Pr7biZ1dmkKwIXNL+Zbo7SPItkvjjlFS+KCoBI062lzohEowRpVeDVTa 7NPcGhOmHlAyjnfSrXiHN7Vs52TomwHW0GC81PbeGkG4HVA6M8JwCgLDg1dR+4rNISfj iZjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584858; x=1729189658; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=43Dd+lWRPu/QQcFLGDLhCyyiH7uAqu5vbmltTcj5okg=; b=FhqclAo2MWr2xIQpJsydhnwhh8N0F9Sy7Q3rd9HdMvLaI2Tpp7krIEt24KWtlXK3/m ceaRUi4st/UVR7dX+Tbd8bGLTHS/NB8Vyt05gyxjnpU9fS3YwXi6Q780RbMzqj0T2fqj vGdrv+nq9VHN6/0tft9Pg4UtbrYiKusTwPb+BnxoPbewbP7JfJ3BxeNbpUhZU1nv6Vcq nUz3m8kqCIiXn1Hpaw2jzjB65SNToH4dZfJV1+gxs+9T14PrQA1H/4PYjJb0qiTa4Xia nfFNbzWzX44lNib+1eTcH/9LAdgL9zrj32jG9C+XFh3gwLWog32Bb7p3Bbr6OBF9x1l3 ftQA== X-Forwarded-Encrypted: i=1; AJvYcCUIYVsSSvE0xS1nRDOPoDiI9pu5pnvfwgcT+2kN5x5AvVL2PFJhUzXlxrjeZbw/t9yyU9rjQ7GhvW/xhw==@lists.infradead.org X-Gm-Message-State: AOJu0YzPD9hRZhqliu9sO0lkkxu9pHTWDdN/2iJqQV3bUHGn5fjfYnPD x+qclDWe+/XSA9C9yNRP5w9eU/InDDW2BcykW/s5U5tXdM0e+SOcz9qlf5wVUp+0A6M8TVvZuAo Bog== X-Google-Smtp-Source: AGHT+IGjs5y0lK7jvR8/zbNWJNDHfDWRUWSPa98vwOnU2bZ6sNQJgoH/6j7HDiYxyC+lHu033MhghQMujcY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:b208:0:b0:7e6:c3f2:24c2 with SMTP id 41be03b00d2f7-7ea535dce6cmr30a12.6.1728584857741; Thu, 10 Oct 2024 11:27:37 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:20 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-79-seanjc@google.com> Subject: [PATCH v13 78/85] KVM: PPC: Explicitly require struct page memory for Ultravisor sharing From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112739_656492_24ABB1AA X-CRM114-Status: GOOD ( 10.30 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Explicitly require "struct page" memory when sharing memory between guest and host via an Ultravisor. Given the number of pfn_to_page() calls in the code, it's safe to assume that KVM already requires that the pfn returned by gfn_to_pfn() is backed by struct page, i.e. this is likely a bug fix, not a reduction in KVM capabilities. Switching to gfn_to_page() will eventually allow removing gfn_to_pfn() and kvm_pfn_to_refcounted_page(). Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_hv_uvmem.c | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv_uvmem.c b/arch/powerpc/kvm/book3s_hv_uvmem.c index 92f33115144b..3a6592a31a10 100644 --- a/arch/powerpc/kvm/book3s_hv_uvmem.c +++ b/arch/powerpc/kvm/book3s_hv_uvmem.c @@ -879,9 +879,8 @@ static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, { int ret = H_PARAMETER; - struct page *uvmem_page; + struct page *page, *uvmem_page; struct kvmppc_uvmem_page_pvt *pvt; - unsigned long pfn; unsigned long gfn = gpa >> page_shift; int srcu_idx; unsigned long uvmem_pfn; @@ -901,8 +900,8 @@ static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, retry: mutex_unlock(&kvm->arch.uvmem_lock); - pfn = gfn_to_pfn(kvm, gfn); - if (is_error_noslot_pfn(pfn)) + page = gfn_to_page(kvm, gfn); + if (!page) goto out; mutex_lock(&kvm->arch.uvmem_lock); @@ -911,16 +910,16 @@ static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, pvt = uvmem_page->zone_device_data; pvt->skip_page_out = true; pvt->remove_gfn = false; /* it continues to be a valid GFN */ - kvm_release_pfn_clean(pfn); + kvm_release_page_unused(page); goto retry; } - if (!uv_page_in(kvm->arch.lpid, pfn << page_shift, gpa, 0, + if (!uv_page_in(kvm->arch.lpid, page_to_pfn(page) << page_shift, gpa, 0, page_shift)) { kvmppc_gfn_shared(gfn, kvm); ret = H_SUCCESS; } - kvm_release_pfn_clean(pfn); + kvm_release_page_clean(page); mutex_unlock(&kvm->arch.uvmem_lock); out: srcu_read_unlock(&kvm->srcu, srcu_idx); @@ -1083,21 +1082,21 @@ kvmppc_h_svm_page_out(struct kvm *kvm, unsigned long gpa, int kvmppc_send_page_to_uv(struct kvm *kvm, unsigned long gfn) { - unsigned long pfn; + struct page *page; int ret = U_SUCCESS; - pfn = gfn_to_pfn(kvm, gfn); - if (is_error_noslot_pfn(pfn)) + page = gfn_to_page(kvm, gfn); + if (!page) return -EFAULT; mutex_lock(&kvm->arch.uvmem_lock); if (kvmppc_gfn_is_uvmem_pfn(gfn, kvm, NULL)) goto out; - ret = uv_page_in(kvm->arch.lpid, pfn << PAGE_SHIFT, gfn << PAGE_SHIFT, - 0, PAGE_SHIFT); + ret = uv_page_in(kvm->arch.lpid, page_to_pfn(page) << PAGE_SHIFT, + gfn << PAGE_SHIFT, 0, PAGE_SHIFT); out: - kvm_release_pfn_clean(pfn); + kvm_release_page_clean(page); mutex_unlock(&kvm->arch.uvmem_lock); return (ret == U_SUCCESS) ? RESUME_GUEST : -EFAULT; } From patchwork Thu Oct 10 18:24:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831313 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 31F18D2445D for ; Thu, 10 Oct 2024 23:31:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ALzKC+A6D1n7k+pmdsoUauQFjVVfrRt+6Hrs89CnFQ8=; b=jJpwE3N4AeXzdW dV2ZrHcGZWpRffO4F+Pcsk9nrTmnfIXz36S7aZfySeLAyMfTcPn7iG33Lrc0365wbS8vybBcEt8N2 pQ/NW/WN/p9lYkzOhxJ2MyycIKS8cXe5rgk3tTgDjTGHFum79C1A53+sJt5xhNfHA2ZBcci+eqF3V /AyzXCnebJPbeS6X/JzM3Qqyirxx52T2G8yknj7mS2AyM9KyV3+iVUyTfKYMvMtv6ABFy5MLXdezd lMAzPZrWJyp50VZpN4rXlr41MOEb2AB27lIHS2kCJ3I7wMKCmSm1K9zUFXV49i4ETYjaxPTaPdh/Z g0+xtkZ64OLxOwJCzbDA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz2d3-0000000Ef6x-28rv; Thu, 10 Oct 2024 23:31:33 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsz-0000000Drpp-2b4s for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:43 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e30fe2a9feso16561857b3.3 for ; Thu, 10 Oct 2024 11:27:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584860; x=1729189660; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=Q1MnEbvSXNK1DBYHKiiPeGTlNBdM3PqCgrnaMRmX38Y=; b=i5UNoI6Kh250+uksVmaSRP/cHhMpWg5vufAjEq5qaf4KW2tpEbQBEnUAxe8WtIPzVD RJtBDpF/jSRcHx7V05sRWdvewtg7NDONoenoJgCZkES3b9pZIwJplK/ea4WgKsJKtwN5 vMqotF/eVBLeHPF6N8JBMZWeQP/i8pY0rcsG2bjOsA3Ak+wU161wXH9k9YYRw5nCULcN Q2W/gCFxPgQdHaPAphdCHemVDWrBZIyWZIh98QKAKROgiWi1Q9z6KLI7U/yyFI2iRCjS 6yq/rY9hUeyZBGeU+ENe+RD5wX0ilZR4YZoGuleOjUADwYxAX1LH7/JJjIHIfRbA8QTS JeXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584860; x=1729189660; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Q1MnEbvSXNK1DBYHKiiPeGTlNBdM3PqCgrnaMRmX38Y=; b=EvJETgYB/QL4O9zn07lG9B5AWsEjg7q24E5yyOKOafs0ofGQIIP686ctUycZbuqPWK jjGn6uxtaGV8n1sv2rIbp+tEa3f/Sdid/Ai96kF+zVpYrxXYGhS4J2QodukUHY7jo3Je ifC6aP9hGuIBwQLf8xn4c0IAH4CT1NUQM+9KZBmBmTozdPlaxamQWSgSVnZhTPcbgKmi y9TV5k51ToJbKxnV2FrYzSncZ8OpJMmfBSUMPgKx2E/Rv6+0KhY3IF9QAsmkXWi+OxuR EXy+vad4KNNbWM3S1GmTDPw0QskQFH7GpQRf9UKPZHpMnF3nAF1PmJqh0YC9d91tA2w5 PH6w== X-Forwarded-Encrypted: i=1; AJvYcCVibQtN0KPQgMnLA5/GwDl0RaI6+0u2s3uteP3qj5QsVE0TdVgn3WDPfEcHcWGz93uvkAI8JCh2oAM3sg==@lists.infradead.org X-Gm-Message-State: AOJu0YzPwnvUsQhnP1YNrii5AnPUYfgnYhkmMjRfjZRcQgylFIAXjFZr gfJ96Y7QUlNaNVfTbcCL5V7Q0AHOml6vO8KOh4QvA1Qu55GZBaqlx7fpdxkDK/9dMmjrqwCJY/i VAg== X-Google-Smtp-Source: AGHT+IGvHgo281GUU728l2kpqfRNdoQ8M4AtWxtMdwK+Sbd/Nmo7aGxeBXrCmxWS+HWzNnhFIgaVqhxkocM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:f20e:0:b0:e11:5da7:337 with SMTP id 3f1490d57ef6-e28fe3505c2mr129032276.3.1728584859805; Thu, 10 Oct 2024 11:27:39 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:21 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-80-seanjc@google.com> Subject: [PATCH v13 79/85] KVM: Drop gfn_to_pfn() APIs now that all users are gone From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112741_733603_60689322 X-CRM114-Status: UNSURE ( 7.66 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Drop gfn_to_pfn() and all its variants now that all users are gone. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 11 -------- virt/kvm/kvm_main.c | 59 ---------------------------------------- 2 files changed, 70 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index af928b59b2ab..4a1eaa40a215 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1274,14 +1274,6 @@ static inline kvm_pfn_t kvm_faultin_pfn(struct kvm_vcpu *vcpu, gfn_t gfn, write ? FOLL_WRITE : 0, writable, refcounted_page); } -kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn); -kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, - bool *writable); -kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn); -kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool interruptible, bool no_wait, - bool write_fault, bool *writable); - void kvm_release_pfn_clean(kvm_pfn_t pfn); void kvm_release_pfn_dirty(kvm_pfn_t pfn); void kvm_set_pfn_dirty(kvm_pfn_t pfn); @@ -1356,9 +1348,6 @@ unsigned long kvm_host_page_size(struct kvm_vcpu *vcpu, gfn_t gfn); void mark_page_dirty_in_slot(struct kvm *kvm, const struct kvm_memory_slot *memslot, gfn_t gfn); void mark_page_dirty(struct kvm *kvm, gfn_t gfn); - -kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); - int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *map, bool writable); void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 8f8b2cd01189..b2c8d429442d 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3039,65 +3039,6 @@ static kvm_pfn_t kvm_follow_pfn(struct kvm_follow_pfn *kfp) return hva_to_pfn(kfp); } -kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool interruptible, bool no_wait, - bool write_fault, bool *writable) -{ - struct kvm_follow_pfn kfp = { - .slot = slot, - .gfn = gfn, - .map_writable = writable, - }; - - if (write_fault) - kfp.flags |= FOLL_WRITE; - if (no_wait) - kfp.flags |= FOLL_NOWAIT; - if (interruptible) - kfp.flags |= FOLL_INTERRUPTIBLE; - - return kvm_follow_pfn(&kfp); -} -EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); - -kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, - bool *writable) -{ - struct kvm_follow_pfn kfp = { - .slot = gfn_to_memslot(kvm, gfn), - .gfn = gfn, - .flags = write_fault ? FOLL_WRITE : 0, - .map_writable = writable, - }; - - return kvm_follow_pfn(&kfp); -} -EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); - -kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) -{ - struct kvm_follow_pfn kfp = { - .slot = slot, - .gfn = gfn, - .flags = FOLL_WRITE, - }; - - return kvm_follow_pfn(&kfp); -} -EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); - -kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn) -{ - return gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn); -} -EXPORT_SYMBOL_GPL(gfn_to_pfn); - -kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn) -{ - return gfn_to_pfn_memslot(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn); -} -EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn); - kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn, unsigned int foll, bool *writable, struct page **refcounted_page) From patchwork Thu Oct 10 18:24:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A3BD8D24447 for ; Thu, 10 Oct 2024 19:52:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kfZFV3tO6AJp6TJ9UzO/p+dXCTe/4rQdYXXIixUroxE=; b=kpCFeMNzc1Moxg RkgqGZrO4UdH9AIuHpjts7zVEdV27AhHGOy9hnl8Id4xdDlEHa2HvyHAwlVRzFha6qUVBIQAJExGp fRrw3RomVV+EwGQmx2srY42hcGsTJ2W55sgSKhkk7rWJI6I5fF6oNmcxpivh/4DCibJEECYCfXN8J 3TX7DGDcGsDs3Pd60sFt7gJMtNmZhuyVc3cZZA6TQXkVRV2WgjGZYmpshSBz0iMiMKZ89MajAXkW1 6KalT6ZPepzMg6PjI+9+jB7WmD9wDiSIwQETkWjRxrYuK1tCYlTOnIv49hT5N5n60h1wmndpPmPEM BviJwoa70bUKtuwcSm2Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzCk-0000000ECES-3hgO; Thu, 10 Oct 2024 19:52:10 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxt1-0000000Drqz-1UFS for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:44 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-71e1e989aa2so1656603b3a.3 for ; Thu, 10 Oct 2024 11:27:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584862; x=1729189662; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=A6YyPrl/aVt4zvDZFXUjzF/Zy9evtaTkH7tjbHPMI5E=; b=ls6EkiR4KuhBz6/sYUk1bFoSMivDvkbu3Ml1zJw7Cvt4EQSBWmD1o8EtK4HJvzfTGw hC730xjyAX2jlRhTdUDbZ4yQM03Vvp+wRCa3voXldfXjYz/y3KRfiPSfPQvoEMkhWV9M jMqxVQrr9LdJzhtKSrtfqCOm56lhRocISlwa60nFFZzbQAAoHnHDKtjqmghany/Exgs/ pP+fhdk2VmvD7o+2ZQYsc8z/6o5klCeCgp8pjFTTL1t3EhZkhvKN0EWKDORfwXvvtHYi wt2sjPeF1qKNEIryfkK05p9i2SDvhfBiXnxLot1wvYGKZsMKwYYZHhhjIXzBzWpa4BmA 1MmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584862; x=1729189662; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=A6YyPrl/aVt4zvDZFXUjzF/Zy9evtaTkH7tjbHPMI5E=; b=DrVfGomWfDbj/kT9qtyoXZPeKtBl1JIYDDzcFqTz4xaDO9f1+wixlxaDfdl9IInGXQ ndCfumJL5HG5LThFkuM10azcj8JU3l/N31CnfclY9RwMWwHa3VW0Ext5KGXq/IsbHbxS JIp4K96P8jJG4mCBuXYsD20r24C0aUjs0P7PEwVGtVPiUTcKXJ60YsBeoVdkh2SW7MAD J5I2cIpwidYmCSrvLKCgCp/klIxRPnFMc8vk/wYQSvO7/v3T4Pi+e3VnF9t7bx9gkKM9 FwU3nRB1BDyG5rOGyg0HQxdTMTRUxInX5T6TKp3K6DH8HpoBok7JK+OoGxZzx9/tfb40 4xzg== X-Forwarded-Encrypted: i=1; AJvYcCUtqP9t57k1HHEGLelc6b8varACqZ6cjX4aLtKsKmfVOd4+9AgeKXTDmK9gop8kaa5IZDqIY1nv26t39g==@lists.infradead.org X-Gm-Message-State: AOJu0YxdGHGSO6/2/jer6fRz7arC4jx1VIx9DgKuLURqjcSnbLWVoikD 5x2sRfWqQ/HGg4ghfjCnnvyvg9HYvlEENeb7V2/V1aIZx3T5/PIW8D3sdMXRxyXVK1WIkCaBdV6 n/g== X-Google-Smtp-Source: AGHT+IFRVsKTyvnlBUrQtNNg8zvYT0OU1qw5Av8a7Gq/P6vdrCxit4uLl8sEcLrSKXXTWA/vla3myJTtPpY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:9443:b0:71d:ff10:7c4 with SMTP id d2e1a72fcca58-71e1dbd1d0fmr15196b3a.4.1728584861919; Thu, 10 Oct 2024 11:27:41 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:22 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-81-seanjc@google.com> Subject: [PATCH v13 80/85] KVM: s390: Use kvm_release_page_dirty() to unpin "struct page" memory From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112743_484534_CC6EB510 X-CRM114-Status: GOOD ( 11.21 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Use kvm_release_page_dirty() when unpinning guest pages, as the pfn was retrieved via pin_guest_page(), i.e. is guaranteed to be backed by struct page memory. This will allow dropping kvm_release_pfn_dirty() and friends. Signed-off-by: Sean Christopherson --- arch/s390/kvm/vsie.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/s390/kvm/vsie.c b/arch/s390/kvm/vsie.c index 763a070f5955..e1fdf83879cf 100644 --- a/arch/s390/kvm/vsie.c +++ b/arch/s390/kvm/vsie.c @@ -670,7 +670,7 @@ static int pin_guest_page(struct kvm *kvm, gpa_t gpa, hpa_t *hpa) /* Unpins a page previously pinned via pin_guest_page, marking it as dirty. */ static void unpin_guest_page(struct kvm *kvm, gpa_t gpa, hpa_t hpa) { - kvm_release_pfn_dirty(hpa >> PAGE_SHIFT); + kvm_release_page_dirty(pfn_to_page(hpa >> PAGE_SHIFT)); /* mark the page always as dirty for migration */ mark_page_dirty(kvm, gpa_to_gfn(gpa)); } From patchwork Thu Oct 10 18:24:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5012CD24447 for ; Thu, 10 Oct 2024 19:52:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vujKWdpU1hzMQuz/3/RdIFLZDPxpas/j4aWDZCLPqxs=; b=nZTrK8jHN4guCW jD+l5LUa42oxstAL4UWLxSYFy0TAELnETDjecuoI6uAbb9BRgNATZTOsXbNbcw03d2peyGnb9lpI+ Dg0LTszxyece6dzCuKRGryMkFX2DQoA6eEZtlvms+pxbkTEk6oyTPmhnpv7y7CRGbNyPwH1IH/Ngg feJNS5+lTjnOIxev3nHhc8r7/khLLS8toPZ6jCp1wtyA5ULJdJdenSfn+u6H5a1jbkuDhVnoNJqaL G+map1LgvkOV7sOeGdSDa071SVIgDZIDpO0KmpDcZBQqpPZ9YBC4jXjHy7QpqsEvyJhnwqave4TtB LJ39uyAR6SZn1bmxTCFQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzCw-0000000ECI3-0FoM; Thu, 10 Oct 2024 19:52:22 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxt3-0000000DrsT-1OnB for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:46 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e290222fde4so1382412276.1 for ; Thu, 10 Oct 2024 11:27:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584864; x=1729189664; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=7IfrRBwnhP/ARkGggvkJ//bNWbTHtvwsCnjZUdYSoz0=; b=J8taJzN9t4s6HfK9u69vl1xjw/eLAz6KM1QsCV7wHhxGzLMHMqrB4B7KQn2Pb8VPjz X3Gysx6a10sADIsjtWJMpLpSugHV+E/l0HLp7kiFs/dOcFljnIA5sZLidASSdLrQfZX9 9X5eNMsraY6t+GseYThGg9Rn6iAOYfdqVDtIGerSU6Y30fee7uv030SPw6meaPF0Uc25 sTllS9cUGwDgPbQWSiu7DoEP//VL9LoKzRHmp/xUnHXTWsskxPUFohgwPbENzH6G18dh kT0KCIrvOHsjeeiF5DVSyAG2QYB7RrwDv/Ob+GGkK35pg1Tfd+LtAI8McZJFCLPRBXTk NZ6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584864; x=1729189664; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=7IfrRBwnhP/ARkGggvkJ//bNWbTHtvwsCnjZUdYSoz0=; b=hAfN/iCVYwwwSJiGy01qQVPAklbP+hBSnmQMM6aA3jvVzR+T6fk7csqYNYt6MRNTdd LGCUNTTUk2xvRBTBrAxmIbZQRRWJoJjGvYwu2RYDG6InTWM4PwNUnhdvqd1VcYzuWd3B cZvDuAd+b+Ig/wJBd5RjKEIUyKiurH/9FHz63G7tNmPOBDK9cldhXy+fWkpeNomp25tb iI8PktNBpLCfKxQCO+838H6gIbFlvZdF/jsBfoPcXCWT81on6FvBsUDq71lLdDIgRYWz 6VunQPDBcWdSB/Jymsl+4OTQzP4RUtufrZgtxhOr1M760cn37sBpmMeVvGAH8tpAxtBg cRog== X-Forwarded-Encrypted: i=1; AJvYcCXfrnO2K27FLGyS2wuPP7bfs9dJfB/Cm/9Mpl6WxiekXKq+y11B+uwSLX2P3CFlVc8OTQmZ5VTAKqaOGA==@lists.infradead.org X-Gm-Message-State: AOJu0YyoJLR42Pct27fiQOPd0t+8B9RuQct2/C2P/lCtyCN17jwPRdC2 53opoK8+N797+1fc/CieyELc7lOcf6Ea0IueG9X3qLhkFQueld6/+U05nnKsH5dz8jEWh+dGdpE xgg== X-Google-Smtp-Source: AGHT+IEHmTL+CF0lwT1L0/srW8AumAgUZ3z+xXpT6s6F1mY7sAC4x3PdYh5uuuYsI/CKxaNz7ipzx99kqDg= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:ed0b:0:b0:e17:8e73:866c with SMTP id 3f1490d57ef6-e28fe421ddfmr136730276.10.1728584863963; Thu, 10 Oct 2024 11:27:43 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:23 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-82-seanjc@google.com> Subject: [PATCH v13 81/85] KVM: Make kvm_follow_pfn.refcounted_page a required field From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112745_440203_B4E22432 X-CRM114-Status: UNSURE ( 9.77 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Now that the legacy gfn_to_pfn() APIs are gone, and all callers of hva_to_pfn() pass in a refcounted_page pointer, make it a required field to ensure all future usage in KVM plays nice. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index b2c8d429442d..a483da96f4be 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2834,8 +2834,7 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, pfn = page_to_pfn(page); } - if (kfp->refcounted_page) - *kfp->refcounted_page = page; + *kfp->refcounted_page = page; return pfn; } @@ -2986,6 +2985,9 @@ kvm_pfn_t hva_to_pfn(struct kvm_follow_pfn *kfp) might_sleep(); + if (WARN_ON_ONCE(!kfp->refcounted_page)) + return KVM_PFN_ERR_FAULT; + if (hva_to_pfn_fast(kfp, &pfn)) return pfn; From patchwork Thu Oct 10 18:24:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831314 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C84CAD2444F for ; Thu, 10 Oct 2024 23:31:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=K4mU+rJBNREgJ+iSiWEKUn3a18lRER5QyPSy3ptsneA=; b=x/pFPWd05qbumm JMRoa60uIqOwBs6WTliBof4TqIVlumoqFrWXjhEpYpDvyAKg9oWYlJ3LVEPY/8XhbavNLOmHfGQ1J BAnC9W+aZr+Joy1uAvyMXBYEnti1+UJ3luWUmBGOGUTfj8k9PuSxCY2IVB/FpRn0ijsZQPw/Dgcaj TGv9YYhsALqVa6ZZxnEr843rTJe2DRXe740b27Cl7D4g/V+yJneHtoim2jpj8GUMMjTNyEWe3u39r e0w3JCMRLHGYgVxX7dbHUaCukwX9zsziS6rR2UVM6+WkZTIYj9jV12s4/Apm7GbCyPSPV3vgYVFTE vMTuA4rgRdezIjtax+aA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz2d4-0000000Ef7y-3MrB; Thu, 10 Oct 2024 23:31:34 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxt6-0000000DruZ-3DuK for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:50 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e2d1860a62so23490327b3.0 for ; Thu, 10 Oct 2024 11:27:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584866; x=1729189666; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=l0R/snR1lBsefu+Gd9v26W9gBjpskR/lINZXMlcZUSs=; b=oW/VXzAGHJr65LOPS7f+dxkMLtFUFWnBWUdwJwkoJynjNOMQA6FyT/YEbpKkDQB8Y1 XeIzl5K+t84icDZE9u4ugYhCQHyhHKiuRnoiR2Lm7C1CCatKcksv+5I0FiTm4qTRbBb0 cvFh3Pt4jkcYCbXegCrQTtJpKleedb6mm9BKTQq01qUz3KYjfrjunAPBGh+J9kARPELz TBCYbi38ooUbc2ZMOdoZYQZumPdBuR5OB6mAmUFQ49qCz1YIqoFGVX+FsfjgnF+fQ1da U7+/aWlz9istv976pUh+e8rfrPsTAHulSo8C7s5jKqn07pvvsJtIBFi3eS1ZDtNdKBDa b4bQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584866; x=1729189666; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=l0R/snR1lBsefu+Gd9v26W9gBjpskR/lINZXMlcZUSs=; b=OgdiaI3RWSitKvYVg+3s72k8bd47/njiRlvGyasm+BIGFhhIsySN6H4jHuzfFZU+97 DsDusZFV/9IPl67JRfx1fZGt1DOZkXhGjIyhvCC+wSquHxJIYc8F+TNlIof71VF3FJew 3kqVa5qhfy+vd/jHgi3ptMJgXl/dRwTxGWL7pZglSceb9wyypGvtlyvgAB/V9ki5eJfi Ue1Ep2eXYNLOHhB+W0tC6ydlZHWpQOq32flw1x6eUrXv068ErnylRma88O4Bpc1rvYYm du0oJ3pY3KbbbleHmthkKsMqH7d/AOt/IWF9cu9cK9XtWilfUiPXWJiYmeUtG6kNgohC Ab8g== X-Forwarded-Encrypted: i=1; AJvYcCUeBmAcVBimsRa+KwQC0EgflHLHITugocQXkdEnLpdZaSb07FrJjbUoCSJ588UCMXy6Ej9XRtU40bWwYg==@lists.infradead.org X-Gm-Message-State: AOJu0YxorrpZWAoFa1DLMl2WlRDLFsCkno8f+uSVfar3jlfvpxZf3SCx HJ4oHNjPWqwVMNOCwXcfROUTVMOaYspQDl+1TaJL3vggMmjHoUDls3PCbBlYSsSEGoSe4ykDnnX CUA== X-Google-Smtp-Source: AGHT+IE49fRlqLzz+ddXUJ4fW719WQNIEaF+exc+KxAZEodOOT6YKCh59uZdX+xzyPwnVHJ15Mb6QwFZ+jU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:470f:b0:6e3:39e5:f0e8 with SMTP id 00721157ae682-6e339e5f461mr119227b3.6.1728584866570; Thu, 10 Oct 2024 11:27:46 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:24 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-83-seanjc@google.com> Subject: [PATCH v13 82/85] KVM: x86/mmu: Don't mark "struct page" accessed when zapping SPTEs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112748_879376_098E52FB X-CRM114-Status: GOOD ( 13.34 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Don't mark pages/folios as accessed in the primary MMU when zapping SPTEs, as doing so relies on kvm_pfn_to_refcounted_page(), and generally speaking is unnecessary and wasteful. KVM participates in page aging via mmu_notifiers, so there's no need to push "accessed" updates to the primary MMU. And if KVM zaps a SPTe in response to an mmu_notifier, marking it accessed _after_ the primary MMU has decided to zap the page is likely to go unnoticed, i.e. odds are good that, if the page is being zapped for reclaim, the page will be swapped out regardless of whether or not KVM marks the page accessed. Dropping x86's use of kvm_set_pfn_accessed() also paves the way for removing kvm_pfn_to_refcounted_page() and all its users. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 17 ----------------- arch/x86/kvm/mmu/tdp_mmu.c | 3 --- 2 files changed, 20 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 5acdaf3b1007..55eeca931e23 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -559,10 +559,8 @@ static bool mmu_spte_update(u64 *sptep, u64 new_spte) */ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep) { - kvm_pfn_t pfn; u64 old_spte = *sptep; int level = sptep_to_sp(sptep)->role.level; - struct page *page; if (!is_shadow_present_pte(old_spte) || !spte_has_volatile_bits(old_spte)) @@ -574,21 +572,6 @@ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep) return old_spte; kvm_update_page_stats(kvm, level, -1); - - pfn = spte_to_pfn(old_spte); - - /* - * KVM doesn't hold a reference to any pages mapped into the guest, and - * instead uses the mmu_notifier to ensure that KVM unmaps any pages - * before they are reclaimed. Sanity check that, if the pfn is backed - * by a refcounted page, the refcount is elevated. - */ - page = kvm_pfn_to_refcounted_page(pfn); - WARN_ON_ONCE(page && !page_count(page)); - - if (is_accessed_spte(old_spte)) - kvm_set_pfn_accessed(pfn); - return old_spte; } diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 8aa0d7a7602b..91caa73a905b 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -861,9 +861,6 @@ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct kvm_mmu_page *root, tdp_mmu_iter_set_spte(kvm, &iter, SHADOW_NONPRESENT_VALUE); - if (is_accessed_spte(iter.old_spte)) - kvm_set_pfn_accessed(spte_to_pfn(iter.old_spte)); - /* * Zappings SPTEs in invalid roots doesn't require a TLB flush, * see kvm_tdp_mmu_zap_invalidated_roots() for details. From patchwork Thu Oct 10 18:24:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831020 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6531ED24442 for ; Thu, 10 Oct 2024 19:55:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=hX0QUqeZJsHdtAw/9DPu7MHUY+uukdsb22WsFohr9yo=; b=VIAwnoAvDS3W3K CyoKkdwkAhNGtL6vKbFPlnHEmp+Eh3M1ELA8NfXyxEusbCndiJ1U7cOhcc66OczFh5pQsG2wtbNEq CcDbSFYFC2m6/zxSKfcc6FbsbCXiuiBOyW9c+hjf06lFePH5ZT/uQN6Tx/KOdGCbidujOsc5ALFmO N/IJGHXTaWLCBfjK1kBVSpGyqHBEUrQuvRGohkXQVDhkd5noovtbK0iL8WnC9J8nF4C4AkwBgd0kt GJmbGplXd/YRsWlz9B+DmNZ1tdJH6zHNe+xAMTCgXmOku+mKWS2BPltDor/Al9enuiyZYu52HJVge QunGmwmdSrYpB/jBpIqg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzFp-0000000ECie-0gXE; Thu, 10 Oct 2024 19:55:21 +0000 Received: from mail-pl1-x64a.google.com ([2607:f8b0:4864:20::64a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxt7-0000000Drv0-1WeK for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:52 +0000 Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-20c8a637b77so13075005ad.2 for ; Thu, 10 Oct 2024 11:27:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584868; x=1729189668; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=qYRk8t9w2U8aQ/ID4AWjouVrA1tdB4hWvcNmB/sB3HQ=; b=MUIvCWd3OKL3bG2LnW491oxhTPr2o7/fCOnjUOkW6ycXy+5uHrA31BINqdPJuJ3C3/ 5DXZ4T8bTuWTd5pncs9YLc3/n2uR8x54Vs/bYASESllzopfA6VCT9SAWgifi3sSY6Qkl RJ0aG8HQgu0RB6tNsUB6isyfkRPB4kYBT4bcNCJEtYN4QGGG6Gb5tjcu36I2dXx7LaSe eG9yeEcdUT1bz6iz6MXsGzWuvXb823sD9bk5OtzpOhbQRpZvKKKqxXMfUSPh3lkWrLvQ 63KzXC9l1Jx6YZ1qfhtiZQ9PLgmDhUFzpW+u6EFMfvHpAsOXJ5IxILDg0Et7pSJqVR/l vVbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584868; x=1729189668; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=qYRk8t9w2U8aQ/ID4AWjouVrA1tdB4hWvcNmB/sB3HQ=; b=nEUdZ9lQG+ehXvP8JAbkztGFOq+vO+ewZ4qFv5rVUbzdf+Z1cslZ5DQr6MwB7nYtv4 2rxHunLdlW9XcjKMmqQxvcFKW0dDUQfwAH00sp9i3q8Gn+PLzf4VOYMiX84NxnyPsOZq 7HV+AwjbbbRXNDVtrVMnaApI7p2RPz07zzAiGTz7RR7V3x2xXuCP5NLD1i8rJXBe/3CY o/Uuwz/BxHNRobZQktOmJGfttYNHG0oIZ+h9jIaS8JIF2pQktCVdWuTuBpuK0Gv2ODcy kzoQadVkL+FDdPdGGrrQ993dhjT3immfE7UmjfuqKRH8OV4I2oFAqwElfRX6ljy4TvB2 9/gQ== X-Forwarded-Encrypted: i=1; AJvYcCVAtOZbxQMYrudntHSmI3/hckBlt6VDQJmM8ezAf3A3zkLnvQhdkHrQYwemSBh2oWR5Ve1jU3vEYk7WSg==@lists.infradead.org X-Gm-Message-State: AOJu0YznrQZZ/quDUmcZEYnJGxTa7oQcVB1yg8UXpD50rvjBXT8vcnVJ 2FPlgQu8N3JV+mtKsXKUVs1yZSq9YMW1P2SRwCG8RtTlRotgiyYLI1Tgz0cHx9kUpN35i22lvye 7+A== X-Google-Smtp-Source: AGHT+IHfwq/leOSDpGNmElHsEevN2NLhktzxL5WWYsNV7s51yqV4fIBatX1NQsHzbv29AfLNcoyCuLVeClM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:e745:b0:20c:857b:5dcb with SMTP id d9443c01a7336-20c857b62ffmr639275ad.4.1728584868183; Thu, 10 Oct 2024 11:27:48 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:25 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-84-seanjc@google.com> Subject: [PATCH v13 83/85] KVM: arm64: Don't mark "struct page" accessed when making SPTE young From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112749_457075_65034457 X-CRM114-Status: GOOD ( 12.09 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Don't mark pages/folios as accessed in the primary MMU when making a SPTE young in KVM's secondary MMU, as doing so relies on kvm_pfn_to_refcounted_page(), and generally speaking is unnecessary and wasteful. KVM participates in page aging via mmu_notifiers, so there's no need to push "accessed" updates to the primary MMU. Dropping use of kvm_set_pfn_accessed() also paves the way for removing kvm_pfn_to_refcounted_page() and all its users. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/arm64/include/asm/kvm_pgtable.h | 4 +--- arch/arm64/kvm/hyp/pgtable.c | 7 ++----- arch/arm64/kvm/mmu.c | 6 +----- 3 files changed, 4 insertions(+), 13 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 03f4c3d7839c..aab04097b505 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -674,10 +674,8 @@ int kvm_pgtable_stage2_wrprotect(struct kvm_pgtable *pgt, u64 addr, u64 size); * * If there is a valid, leaf page-table entry used to translate @addr, then * set the access flag in that entry. - * - * Return: The old page-table entry prior to setting the flag, 0 on failure. */ -kvm_pte_t kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr); +void kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr); /** * kvm_pgtable_stage2_test_clear_young() - Test and optionally clear the access diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index b11bcebac908..40bd55966540 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -1245,19 +1245,16 @@ int kvm_pgtable_stage2_wrprotect(struct kvm_pgtable *pgt, u64 addr, u64 size) NULL, NULL, 0); } -kvm_pte_t kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr) +void kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr) { - kvm_pte_t pte = 0; int ret; ret = stage2_update_leaf_attrs(pgt, addr, 1, KVM_PTE_LEAF_ATTR_LO_S2_AF, 0, - &pte, NULL, + NULL, NULL, KVM_PGTABLE_WALK_HANDLE_FAULT | KVM_PGTABLE_WALK_SHARED); if (!ret) dsb(ishst); - - return pte; } struct stage2_age_data { diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 4054356c9712..e2ae9005e333 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1706,18 +1706,14 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, /* Resolve the access fault by making the page young again. */ static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa) { - kvm_pte_t pte; struct kvm_s2_mmu *mmu; trace_kvm_access_fault(fault_ipa); read_lock(&vcpu->kvm->mmu_lock); mmu = vcpu->arch.hw_mmu; - pte = kvm_pgtable_stage2_mkyoung(mmu->pgt, fault_ipa); + kvm_pgtable_stage2_mkyoung(mmu->pgt, fault_ipa); read_unlock(&vcpu->kvm->mmu_lock); - - if (kvm_pte_valid(pte)) - kvm_set_pfn_accessed(kvm_pte_to_pfn(pte)); } /** From patchwork Thu Oct 10 18:24:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831021 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ADCC6D24447 for ; Thu, 10 Oct 2024 19:55:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=j2mib9Krnlty0WR2io2FDu1dtHPSSd8KCFg9XVSs6Hw=; b=r78lKikDJnvZIP Y6L1MS04FCKXSOQN+7V6lun2IyzOPxnTM6LARbgN8TMnkriM8/LQOs/7fm3oYfOY7ZxGbJ4fXUpmA fjDzqNxVuYhVcJyB0V7koy0aqgTBcQNaVEC3MLT5uErsNFDEWvb0nJGxV3ec7QJvVMHgHK4+h/wCg wTgQoui1zswvRWHuvfFfrLSacvCRt96Rogt5YrwVS2VF9J5gh6MwxTVBQdaMvMlTPZFSDmkZ3KdLn DmpgKmKUclsah8C+lb8aBf3NTLvO/mQau2SASDpjXdj24ip/qLwt9YLvwYKtUF/g7a2wOqzLCd8PW bZN50zZ4/jD3ZeCdMEVA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzFp-0000000ECiz-3PVK; Thu, 10 Oct 2024 19:55:21 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxt9-0000000Drwm-1pz6 for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:53 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e2b049b64aso20338017b3.3 for ; Thu, 10 Oct 2024 11:27:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584870; x=1729189670; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=BeNSr3VxopvO4QfIOfKviYSPGRm8vVX1K3YR0sLhejo=; b=QrJKmIjvSEAhTIlwVFJOmQ5I7I1VQhdnUVXCZMq3P4JJusmgXnFbWJC6rS25qcKrc9 PN5Xs1EwDCBnS6Jfegln//omwt2zD+qAgL/jptfn/AmjxhsbXA9WzAJnKDAbX142AR6h RNYj2DIR2JEFC5wyhzG4jhM0r+T/Z/aReqH/PvQPn9B40wk+edj62VBTntkLj/5FBB25 QYk85cbXR2hELgL/DYbNsXGMlrv7GaJaMXWXi7F9kTmNYdUQHst26FT47gHsPnKmbLt7 75NSePZV/v2zlVeBCcHeWFDP2qwe6gTBBEdY9I3NyXl+0CYfK1SourHBzLUeeikafN8J vQPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584870; x=1729189670; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=BeNSr3VxopvO4QfIOfKviYSPGRm8vVX1K3YR0sLhejo=; b=oL3XICJJMFYjPyN8jaH3bvvE0Sze90u3ZuZpdOphcXU5ZhngnyLPEJyxtqQ/nI2NUc QicYVD9C5OvdJB7S04Zg2+Zm2fhiyag7QlzfVZ//fEdv/fTte4snHV9zi6zcP5ivaRTI 8DOOK9qc37SjNdEHlYnRRUecPnm7mpVXIsNL7p4L6TymZmvrXB0cZPZq1sikhmwEnTQI OUwS9hp57UNTwNdTfjrydgWkfvGCY2IjufEnYNEkqKdIfLkEiXEGFjiVH9m1w2tgW/qC DXeKVXKvQT7kercqX6pNvZg19avJwe6wBbrxh/AoQxnZK+gU1WwyWGy7KUIOCw9P6UVg 74xg== X-Forwarded-Encrypted: i=1; AJvYcCWYUejLIXmrmNwC5WnVyKNPq4XJs/b0Qihy91IKWbj4RGgtxzm7t4vd62iKMS1BXhGTrr1uEPz2Gjp1LA==@lists.infradead.org X-Gm-Message-State: AOJu0YxcngQq625FYDOoTSSt0Bc2pib9s78nhQOUfASFFdEZzbIBcDFB cgzVeMdTqhqbkpTuTRp4fGUHSZ6uhu8DPCOQKfBxCxd1FyOpKr/oohWBKCYjSa5DLDey4MQp2JG SkQ== X-Google-Smtp-Source: AGHT+IHhmEpfdjhYs/g96WpbaMl3fao5EtfU192MAf3yyGbJNoRiDL4cTPbNYR1fq37SA+x5liddjIzSKiU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2e86:b0:6dd:bcce:7cd4 with SMTP id 00721157ae682-6e3221404ddmr516647b3.2.1728584870148; Thu, 10 Oct 2024 11:27:50 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:26 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-85-seanjc@google.com> Subject: [PATCH v13 84/85] KVM: Drop APIs that manipulate "struct page" via pfns From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112751_595633_081E9FE6 X-CRM114-Status: UNSURE ( 9.36 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Remove all kvm_{release,set}_pfn_*() APIs now that all users are gone. No functional change intended. Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 5 ---- virt/kvm/kvm_main.c | 55 ---------------------------------------- 2 files changed, 60 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 4a1eaa40a215..d045f8310a48 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1274,11 +1274,6 @@ static inline kvm_pfn_t kvm_faultin_pfn(struct kvm_vcpu *vcpu, gfn_t gfn, write ? FOLL_WRITE : 0, writable, refcounted_page); } -void kvm_release_pfn_clean(kvm_pfn_t pfn); -void kvm_release_pfn_dirty(kvm_pfn_t pfn); -void kvm_set_pfn_dirty(kvm_pfn_t pfn); -void kvm_set_pfn_accessed(kvm_pfn_t pfn); - int kvm_read_guest_page(struct kvm *kvm, gfn_t gfn, void *data, int offset, int len); int kvm_read_guest(struct kvm *kvm, gpa_t gpa, void *data, unsigned long len); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index a483da96f4be..396ca14f18f3 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3164,61 +3164,6 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map) } EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); -void kvm_release_pfn_clean(kvm_pfn_t pfn) -{ - struct page *page; - - if (is_error_noslot_pfn(pfn)) - return; - - page = kvm_pfn_to_refcounted_page(pfn); - if (!page) - return; - - kvm_release_page_clean(page); -} -EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); - -void kvm_release_pfn_dirty(kvm_pfn_t pfn) -{ - struct page *page; - - if (is_error_noslot_pfn(pfn)) - return; - - page = kvm_pfn_to_refcounted_page(pfn); - if (!page) - return; - - kvm_release_page_dirty(page); -} -EXPORT_SYMBOL_GPL(kvm_release_pfn_dirty); - -/* - * Note, checking for an error/noslot pfn is the caller's responsibility when - * directly marking a page dirty/accessed. Unlike the "release" helpers, the - * "set" helpers are not to be used when the pfn might point at garbage. - */ -void kvm_set_pfn_dirty(kvm_pfn_t pfn) -{ - if (WARN_ON(is_error_noslot_pfn(pfn))) - return; - - if (pfn_valid(pfn)) - kvm_set_page_dirty(pfn_to_page(pfn)); -} -EXPORT_SYMBOL_GPL(kvm_set_pfn_dirty); - -void kvm_set_pfn_accessed(kvm_pfn_t pfn) -{ - if (WARN_ON(is_error_noslot_pfn(pfn))) - return; - - if (pfn_valid(pfn)) - kvm_set_page_accessed(pfn_to_page(pfn)); -} -EXPORT_SYMBOL_GPL(kvm_set_pfn_accessed); - static int next_segment(unsigned long len, int offset) { if (len > PAGE_SIZE - offset) From patchwork Thu Oct 10 18:24:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831022 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9B690D24442 for ; Thu, 10 Oct 2024 19:58:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=0mRJU0IiKvwPrZuW+mfN4ZAM/nax/283V5R79NFNZYA=; b=o8RGYp4mrgn/m5 YSgTeXCH5qft8E3WTsV2T+QqEO7jBkJ/oX+WejRxLIGqQM7sv8+8KN2Ue+FCxkzpKlcZxSexc0Yi6 1WGpf83sHI/w61oIS5U8X0S3IyQaTg0PZ/HiY70HEC4Wb3N39PmK9iloWVY2yBcmoYcTthH7zxW4/ iN8qTw5Bnd8MAQHvVfj1qTwVALMxNw+ANObrKk7RVXgKxBjpBphQKriUA2NRsY7gDSbaXPB4kDuSF WmX0fKwdObfAzKl6uWAoaENYoICEX6c0FFAyN1qrNZCCJjNI+7UFjWH6fUXBZ8krscDufH2oLsNPx XvKQfsmgMwkHs7qSHynw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzIS-0000000ED79-2l0f; Thu, 10 Oct 2024 19:58:04 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxtC-0000000Drz7-2hYd for linux-riscv@lists.infradead.org; Thu, 10 Oct 2024 18:27:56 +0000 Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-7b696999c65so864046a12.3 for ; Thu, 10 Oct 2024 11:27:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584873; x=1729189673; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=nwXbeHTlA0ocP8s6EMi2R3V8WkxZqKgY3fo4QQ/WBXk=; b=C91LwOthEGXkxiPQx1j9UVtQEFjUoPUaMYYqnTNP0TCswa9+hhVSEARBcovA5nnV2c 32Zbt1YX9VZVNqA/MIhC1E9BJoIzGjcTF5OAV0U6hy0FCdm9R8+piH4EntBUT5Flzs0g 7XRT8akLUDK2TQ7w5LYG7bObbSDKy6EuyXBiGjndcjvGJU4XUeHMGrzHCRWi3yzoHhuS sjBWzkfEmw+kj5JGm+gKX2+ScM8kNPIL2wd94uiV/uBTA9RB+YhiZeF2TIqolgG7sF1b +fe9fpJERCCgWOqYOz0sVLDF1YIWhDfhuugU+/HTW/vPaD/KWNLvNxRYp2hahT2HA9dL ndpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584873; x=1729189673; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=nwXbeHTlA0ocP8s6EMi2R3V8WkxZqKgY3fo4QQ/WBXk=; b=LRuvJKS+Y5Gprny14z/YD5okOvwQErz369BdU6bmCj8Peajf/NADN2Q9UZptiZ/+/J YwYwAVS20pQdEljoBw1ZUNipDsKrF4HHwBc1iU/SodYRZYGvUx6DC0UEPlZifknbF7r7 8HFrGWWGn3ib/3BfIOfoqIDQjvkk/q8o7/AtGYE2ik9gdhxdj635aAbetBbiqgsxW2/R QIseW00gedVjfwtrLoO3lg7iYuuTsTqdk4JG4gTj6ETgqq0M7HgTUNCn2Act1593qYxL lZephJA41SfPEqDiGhKC/s2+3htAozTJw/VyI9ahaKpkkmoR0gEZ125GmKGtJV42sJNZ Q6aA== X-Forwarded-Encrypted: i=1; AJvYcCVhvOAANITUS8rDTZliIcW7DfFaWjdk30bBc+HkOHerjFgApbAJYg34DPEf/fPT88j+gzfZvX6nygf/fA==@lists.infradead.org X-Gm-Message-State: AOJu0Yy4WjZRccNbgb5RNjDHouozw2tMH3N5VUrxOWsHVBfpXfADI4j5 clBzXE9dFv7os6d3q5spUlkpwoVqeqxNXlHoRHxtNlJXdUtgBKAa6J7Oiak1Ma8mmnnP+BCqg/T Klw== X-Google-Smtp-Source: AGHT+IHZp/jcJ6VF/U4r8Ds8cBqCm7oOG+WyWj8b167iLxGpwwfpynwEnbVm0Ou2T4AqluEw0PW/YyS/yKE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:e715:0:b0:7d5:d088:57d4 with SMTP id 41be03b00d2f7-7ea535b6b4fmr26a12.9.1728584872016; Thu, 10 Oct 2024 11:27:52 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:27 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-86-seanjc@google.com> Subject: [PATCH v13 85/85] KVM: Don't grab reference on VM_MIXEDMAP pfns that have a "struct page" From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112754_780379_3048782F X-CRM114-Status: GOOD ( 20.24 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Now that KVM no longer relies on an ugly heuristic to find its struct page references, i.e. now that KVM can't get false positives on VM_MIXEDMAP pfns, remove KVM's hack to elevate the refcount for pfns that happen to have a valid struct page. In addition to removing a long-standing wart in KVM, this allows KVM to map non-refcounted struct page memory into the guest, e.g. for exposing GPU TTM buffers to KVM guests. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 3 -- virt/kvm/kvm_main.c | 75 ++-------------------------------------- 2 files changed, 2 insertions(+), 76 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index d045f8310a48..02f0206fd2dc 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1730,9 +1730,6 @@ void kvm_arch_sync_events(struct kvm *kvm); int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu); -struct page *kvm_pfn_to_refcounted_page(kvm_pfn_t pfn); -bool kvm_is_zone_device_page(struct page *page); - struct kvm_irq_ack_notifier { struct hlist_node link; unsigned gsi; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 396ca14f18f3..b1b10dc408a0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -160,52 +160,6 @@ __weak void kvm_arch_guest_memory_reclaimed(struct kvm *kvm) { } -bool kvm_is_zone_device_page(struct page *page) -{ - /* - * The metadata used by is_zone_device_page() to determine whether or - * not a page is ZONE_DEVICE is guaranteed to be valid if and only if - * the device has been pinned, e.g. by get_user_pages(). WARN if the - * page_count() is zero to help detect bad usage of this helper. - */ - if (WARN_ON_ONCE(!page_count(page))) - return false; - - return is_zone_device_page(page); -} - -/* - * Returns a 'struct page' if the pfn is "valid" and backed by a refcounted - * page, NULL otherwise. Note, the list of refcounted PG_reserved page types - * is likely incomplete, it has been compiled purely through people wanting to - * back guest with a certain type of memory and encountering issues. - */ -struct page *kvm_pfn_to_refcounted_page(kvm_pfn_t pfn) -{ - struct page *page; - - if (!pfn_valid(pfn)) - return NULL; - - page = pfn_to_page(pfn); - if (!PageReserved(page)) - return page; - - /* The ZERO_PAGE(s) is marked PG_reserved, but is refcounted. */ - if (is_zero_pfn(pfn)) - return page; - - /* - * ZONE_DEVICE pages currently set PG_reserved, but from a refcounting - * perspective they are "normal" pages, albeit with slightly different - * usage rules. - */ - if (kvm_is_zone_device_page(page)) - return page; - - return NULL; -} - /* * Switches to specified vcpu, until a matching vcpu_put() */ @@ -2804,35 +2758,10 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, if (kfp->map_writable) *kfp->map_writable = writable; - /* - * FIXME: Remove this once KVM no longer blindly calls put_page() on - * every pfn that points at a struct page. - * - * Get a reference for follow_pte() pfns if they happen to point at a - * struct page, as KVM will ultimately call kvm_release_pfn_clean() on - * the returned pfn, i.e. KVM expects to have a reference. - * - * Certain IO or PFNMAP mappings can be backed with valid struct pages, - * but be allocated without refcounting, e.g. tail pages of - * non-compound higher order allocations. Grabbing and putting a - * reference to such pages would cause KVM to prematurely free a page - * it doesn't own (KVM gets and puts the one and only reference). - * Don't allow those pages until the FIXME is resolved. - * - * Don't grab a reference for pins, callers that pin pages are required - * to check refcounted_page, i.e. must not blindly release the pfn. - */ - if (map) { + if (map) pfn = map->pfn; - - if (!kfp->pin) { - page = kvm_pfn_to_refcounted_page(pfn); - if (page && !get_page_unless_zero(page)) - return KVM_PFN_ERR_FAULT; - } - } else { + else pfn = page_to_pfn(page); - } *kfp->refcounted_page = page;