From patchwork Tue Jul 18 23:44:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13317870 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65D60C001DE for ; Tue, 18 Jul 2023 23:48:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D5B778D0020; Tue, 18 Jul 2023 19:48:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CE48D8D0012; Tue, 18 Jul 2023 19:48:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B5D6F8D0020; Tue, 18 Jul 2023 19:48:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A41CC8D0012 for ; Tue, 18 Jul 2023 19:48:36 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 71A811604FA for ; Tue, 18 Jul 2023 23:48:36 +0000 (UTC) X-FDA: 81026374632.11.518DD7E Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf28.hostedemail.com (Postfix) with ESMTP id 7EF5AC001B for ; Tue, 18 Jul 2023 23:48:34 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=1t0P5wmE; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf28.hostedemail.com: domain of 30SS3ZAYKCCQSEANJCGOOGLE.COMLINUX-MMKVACK.ORG@flex--seanjc.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=30SS3ZAYKCCQSEANJCGOOGLE.COMLINUX-MMKVACK.ORG@flex--seanjc.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689724114; a=rsa-sha256; cv=none; b=3Bme0lWsUHeJMN0F4KEEgOonGA9fe9ZW2LCxcpr922ie64MaRNr+dJsG33UoqZNWumcraT pLrRV+uO9wn/HsP5RAIX3IU2CIMWUa7fIw0JKkeRwvJ1ycJ0hwRHuunvwkPowvNOAmq3zW 8kUuRLbRxuUmORhoknodf4zuMNOnIz8= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=1t0P5wmE; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf28.hostedemail.com: domain of 30SS3ZAYKCCQSEANJCGOOGLE.COMLINUX-MMKVACK.ORG@flex--seanjc.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=30SS3ZAYKCCQSEANJCGOOGLE.COMLINUX-MMKVACK.ORG@flex--seanjc.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689724114; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LxdpiWTpX3B7gMKbPihLQ6FRzF4V55oVGzPxSGe/+4U=; b=odIU6E+8o1h0Q+EWUTFd3/3LaQ1Y7Wvtdh0ZQsR//GAZt7Oy5NWcX08n2XeCVJ8qcALzxc jzzXS0Gq0g5sYKvouRD5G3hJ455LKJj5HW13E5Vhl4BEs2NaOznwZ8TuE+fzQ4n27FrXC4 bvW5d7Avcd/aCfz0U3xgC9YWF7T0MKY= Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1b8b2a2e720so32164375ad.3 for ; Tue, 18 Jul 2023 16:48:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689724113; x=1692316113; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=LxdpiWTpX3B7gMKbPihLQ6FRzF4V55oVGzPxSGe/+4U=; b=1t0P5wmE/XmWdqU7J55641EABrTeVarxWaQR4bJ8HE5kvt+bT2Zvilx6zvBC6IhCug lxtaG0ZNNeNxKJwNRyofNcY/3UY8SXaGscGhXtOBqjuhZKJLWAvQEL/Od2vXIHWkp721 m6ADzg8L3JrLCsaJYeIfX0G7ZaqvY3dbA4PpFF78/ivmcmih9CPggzA6FpbWCll6Rc7J s/iuth191O25UoKCd/SjRjFkh1FVoiOHo2ymxYD/iPBQpPOGMgrdqXeht4Yx5rEFGAGu E6MjDgW0gFgwwVKsMh4xUPmBsbIlsG5Mzs33eQjh6Yww/GhiF/1eZ4NrCcSo7R/KWv9Z IRkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689724113; x=1692316113; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=LxdpiWTpX3B7gMKbPihLQ6FRzF4V55oVGzPxSGe/+4U=; b=ZwaJww+icOryJKFPcQmuCTShjfNKsCWdZ1xjzC8ARfpeAjOHRIb4km3QyyAfZSyTPV nJ68Ox+Undg9XJngBtJzA8sXlofPT1CxypI2ESfHRSLTkOjZ4rI34XXoM878IkubLPC6 6e5bty/ngI1Yz/wZ6Jb31MUhkKbs5r3MXrFSS8KfaAzg3ixLZdXsQbl/Zh3ori64xRzP SdmkY9GSNDj4TTvEQ1/BWMMq7EC9zRkEICm3peMzaibvGfBAUBjpa/G2pFwiYdMaserF ON3FR2L/y+O4vFVW1DLPUOn7T726CoxsGLY3a/BhQH93Y7/nPmtyYiy+uj3lZM/4trAd C3TA== X-Gm-Message-State: ABy/qLanE6w4bAeTOw8cuMlVeYQqkvbRVHe4rrjQY+r0WMN6FOE1nDM5 NsJpF/DKeNHGalmwDTNlFPi/Q9QNEe4= X-Google-Smtp-Source: APBJJlECbx+0jj+7Hb3hGFg2pirZsYDF3lE8aX5+9Q/S485STdtV3M1qZMxLbtNxaVklqPoT+BTJIsFwzzk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:903:2349:b0:1b9:de12:475 with SMTP id c9-20020a170903234900b001b9de120475mr8788plh.6.1689724113231; Tue, 18 Jul 2023 16:48:33 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 18 Jul 2023 16:44:44 -0700 In-Reply-To: <20230718234512.1690985-1-seanjc@google.com> Mime-Version: 1.0 References: <20230718234512.1690985-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230718234512.1690985-2-seanjc@google.com> Subject: [RFC PATCH v11 01/29] KVM: Wrap kvm_gfn_range.pte in a per-action union From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Sean Christopherson , "Matthew Wilcox (Oracle)" , Andrew Morton , Paul Moore , James Morris , "Serge E. Hallyn" Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-security-module@vger.kernel.org, linux-kernel@vger.kernel.org, Chao Peng , Fuad Tabba , Jarkko Sakkinen , Yu Zhang , Vishal Annapurve , Ackerley Tng , Maciej Szmigiero , Vlastimil Babka , David Hildenbrand , Quentin Perret , Michael Roth , Wang , Liam Merwick , Isaku Yamahata , "Kirill A . Shutemov" X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 7EF5AC001B X-Stat-Signature: 6q5ojbqgcm6pdps6gy83u6s97icjxa7m X-HE-Tag: 1689724114-539381 X-HE-Meta: U2FsdGVkX19VyZrkTbmWELTfdIZOzkU6WcP6Hg5apLUayPEnY6/havpyUlsb3dde5xGyoLP44Xe0hU1T4R4+UHHW1FkKPyQY3ECKVw3TQHPNZGvX6KzuIa5jeI8rDk2jLbr/XULgMBFHsht8J7Tfydkz5tsSAe8Mp2kyPn+mzaj3LwrYDrt+An7DEtYQDz2d4CnYYIy5ZeIfq876lS2CfIjbrzaSSZFg6EWn76y+9aYXX2/0UAidNrF7bwlJGm2Gk0rvRnuJ15oXnpVeGBSF6fNTkGtEsGqlhuF9W9MdIEfNqQWyzPAAqsblS5Y81X+ZSzSaNHG2tvyRl21urb/Oncd+NjeYRKOBR+sdAkaeRA/OyDohiyfvDCcAlAXk4293OMGQoAVNFf4iWdmYY25BURFB2rZ3QaTKlMHe+iEP9amrs1GkeP6q0Z2yaJU2ka6qGHlAJsu5Oy9xSUtQo9nIx4tUpE4s+PlhXgT2dhxEp4yEi66XTdIWv9kmNGKaOyp9h/MANvO6qw8D2t04vMfG1e2lnxSrB26lz+nakDL0J0pAySwS28bf/NW8t41FNnVNT2ZRUK7Un+GX0MQj3VLL09dd2NmMI0JkvqBjDzqUbltXa70qA58uhe7yvYTKnQWa0Zfqt/PwBfcOnAV7T6QyowGWMlTMmYwR9xKMBMG90LyAu884Q1G3ZdM2N6z++XvC/OUxolHc2kmiLa4osD4JAAQcftFzOBKvmqzPuKKn5jPzWEvemvQs2WWkeDAvBcKYI4KdjlWn0eXmFChLfEfTiZnXwccCAsY3Ll10uaU4Oj6gqdm/STpmSy1tYljapOAtPNIJaGXXuXdNg1iy/2F+W19bMuoPyt3dJvUDVdZXXpR0hVHu0hZ+nO/dAkhYJVVhJFlzcw3QEdzPDDUef2FD7b1w7eV22d6TluFFu/9lzQChyvv675vgKIKgrvwN/xtrTdReqzalp3ZC5B2Unpc DapGY9Z/ UBpagHf2u4aMluRJMWcZFbQIXw2OyAxWpAyX/cEbjLyqgUqqYg4Nw8pMOV19NKAgvZXDZ1eCs0bs9oBAlPjnKZawoBSb1TiFkoIWEZhOl+MvT5WfYr0qtViOd/TA51FxSdimxc//qHLGebg68WJ6/ebbN8uM+Ntw2KE1Ed/5PtzK5g1K2eeyx8tYD5J1GuleQeZI6zcwmdZoD5ThrDXf9/aKXtWDEJxwysIdZWlz7XrCUScQVcUwpyhfpvUPLJmfi2CS3Ze3Z5jz1oK4I/t54HYTBI8DU3iLe69Www386yIGJbzFNGm5P4zUSFqDvx6XB3dXMs7fJtedimncTQYuMbkU/QuHW2KcEvorC2VwgjgxwrO1xUWiVhPqrhTqj03/g8AeFLfPV5mS++DBb1ql9WoB+uCHB0rHOyJ8rmsuU9RsrbtCtm4ZrbJdeuI+IP/gWjEMme7gK27M+4f8S8KU+twEhENORwTou6sSCJOANXArj88CJV9wT+wzc3SanCvD6tWFcRFahmGdFs42WFhf2ckfNYv4D/uEVDBo9OKStozKhHp3Tgc8dRQ2TzfEmkEz/ZwQVyaFdMcDdnZBEooi9sZUdo9mIL6GET/gEOt760eR05qpetXnvpfrkR10Xj+Pz9zjC7PgzrOPms6zYqCaDMYlFTdzttPC0XFcv7+qe6NHwO7HjsM7pC1asmyt/oufz4IRkO3p1cap4mYHJCmpR68aKtl9RhJ0K0D4ZnCGWhnTUcerlc1y//ZM+zF52ANgZYHap0bgmO7UGF09932ZhgPX50JhEFIvTIXtS4a9RjPKmsFE5ZxGirXm6zvRc+tN39+xZ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Signed-off-by: Sean Christopherson Reviewed-by: Paolo Bonzini --- arch/arm64/kvm/mmu.c | 2 +- arch/mips/kvm/mmu.c | 2 +- arch/riscv/kvm/mmu.c | 2 +- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 6 +++--- include/linux/kvm_host.h | 5 ++++- virt/kvm/kvm_main.c | 16 ++++++++++------ 7 files changed, 21 insertions(+), 14 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 6db9ef288ec3..55f03a68f1cd 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1721,7 +1721,7 @@ bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range) bool kvm_set_spte_gfn(struct kvm *kvm, struct kvm_gfn_range *range) { - kvm_pfn_t pfn = pte_pfn(range->pte); + kvm_pfn_t pfn = pte_pfn(range->arg.pte); if (!kvm->arch.mmu.pgt) return false; diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index e8c08988ed37..7b2ac1319d70 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -447,7 +447,7 @@ bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range) bool kvm_set_spte_gfn(struct kvm *kvm, struct kvm_gfn_range *range) { gpa_t gpa = range->start << PAGE_SHIFT; - pte_t hva_pte = range->pte; + pte_t hva_pte = range->arg.pte; pte_t *gpa_pte = kvm_mips_pte_for_gpa(kvm, NULL, gpa); pte_t old_pte; diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c index f2eb47925806..857f4312b0f8 100644 --- a/arch/riscv/kvm/mmu.c +++ b/arch/riscv/kvm/mmu.c @@ -559,7 +559,7 @@ bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range) bool kvm_set_spte_gfn(struct kvm *kvm, struct kvm_gfn_range *range) { int ret; - kvm_pfn_t pfn = pte_pfn(range->pte); + kvm_pfn_t pfn = pte_pfn(range->arg.pte); if (!kvm->arch.pgd) return false; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index ec169f5c7dce..d72f2b20f430 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -1588,7 +1588,7 @@ static __always_inline bool kvm_handle_gfn_range(struct kvm *kvm, for_each_slot_rmap_range(range->slot, PG_LEVEL_4K, KVM_MAX_HUGEPAGE_LEVEL, range->start, range->end - 1, &iterator) ret |= handler(kvm, iterator.rmap, range->slot, iterator.gfn, - iterator.level, range->pte); + iterator.level, range->arg.pte); return ret; } diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 512163d52194..6250bd3d20c1 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1241,7 +1241,7 @@ static bool set_spte_gfn(struct kvm *kvm, struct tdp_iter *iter, u64 new_spte; /* Huge pages aren't expected to be modified without first being zapped. */ - WARN_ON(pte_huge(range->pte) || range->start + 1 != range->end); + WARN_ON(pte_huge(range->arg.pte) || range->start + 1 != range->end); if (iter->level != PG_LEVEL_4K || !is_shadow_present_pte(iter->old_spte)) @@ -1255,9 +1255,9 @@ static bool set_spte_gfn(struct kvm *kvm, struct tdp_iter *iter, */ tdp_mmu_iter_set_spte(kvm, iter, 0); - if (!pte_write(range->pte)) { + if (!pte_write(range->arg.pte)) { new_spte = kvm_mmu_changed_pte_notifier_make_spte(iter->old_spte, - pte_pfn(range->pte)); + pte_pfn(range->arg.pte)); tdp_mmu_iter_set_spte(kvm, iter, new_spte); } diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 9d3ac7720da9..b901571ab61e 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -260,7 +260,10 @@ struct kvm_gfn_range { struct kvm_memory_slot *slot; gfn_t start; gfn_t end; - pte_t pte; + union { + pte_t pte; + u64 raw; + } arg; bool may_block; }; bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index dfbaafbe3a00..d58b7a506d27 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -526,7 +526,10 @@ typedef void (*on_unlock_fn_t)(struct kvm *kvm); struct kvm_hva_range { unsigned long start; unsigned long end; - pte_t pte; + union { + pte_t pte; + u64 raw; + } arg; hva_handler_t handler; on_lock_fn_t on_lock; on_unlock_fn_t on_unlock; @@ -562,6 +565,10 @@ static __always_inline int __kvm_handle_hva_range(struct kvm *kvm, struct kvm_memslots *slots; int i, idx; + BUILD_BUG_ON(sizeof(gfn_range.arg) != sizeof(gfn_range.arg.raw)); + BUILD_BUG_ON(sizeof(range->arg) != sizeof(range->arg.raw)); + BUILD_BUG_ON(sizeof(gfn_range.arg) != sizeof(range->arg)); + if (WARN_ON_ONCE(range->end <= range->start)) return 0; @@ -591,7 +598,7 @@ static __always_inline int __kvm_handle_hva_range(struct kvm *kvm, * bother making these conditional (to avoid writes on * the second or later invocation of the handler). */ - gfn_range.pte = range->pte; + gfn_range.arg.raw = range->arg.raw; gfn_range.may_block = range->may_block; /* @@ -639,7 +646,7 @@ static __always_inline int kvm_handle_hva_range(struct mmu_notifier *mn, const struct kvm_hva_range range = { .start = start, .end = end, - .pte = pte, + .arg.pte = pte, .handler = handler, .on_lock = (void *)kvm_null_fn, .on_unlock = (void *)kvm_null_fn, @@ -659,7 +666,6 @@ static __always_inline int kvm_handle_hva_range_no_flush(struct mmu_notifier *mn const struct kvm_hva_range range = { .start = start, .end = end, - .pte = __pte(0), .handler = handler, .on_lock = (void *)kvm_null_fn, .on_unlock = (void *)kvm_null_fn, @@ -747,7 +753,6 @@ static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, const struct kvm_hva_range hva_range = { .start = range->start, .end = range->end, - .pte = __pte(0), .handler = kvm_unmap_gfn_range, .on_lock = kvm_mmu_invalidate_begin, .on_unlock = kvm_arch_guest_memory_reclaimed, @@ -812,7 +817,6 @@ static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn, const struct kvm_hva_range hva_range = { .start = range->start, .end = range->end, - .pte = __pte(0), .handler = (void *)kvm_null_fn, .on_lock = kvm_mmu_invalidate_end, .on_unlock = (void *)kvm_null_fn,