From patchwork Sat Sep 16 00:31:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13388036 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 64558CD37B0 for ; Sat, 16 Sep 2023 00:35:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID :References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=YsGbkw8qJ/m1hkPOrLsVg9QoxC8i0esSSUXUhU6f/Vg=; b=esOaYeS7hlqRls M1Y+ujGCWe+SQzWdwZOoB+za1BHPFD6cVcZE+rSyAPJnSXvr2Lipgg3zkQvqrVnu7k+0n1SpgVX9q QnD885ksezGIsQ0yFdYfcq+UmFESNHz4BjdJMr39fE5R9DnqmWVwCqjcZgs4ZANb3VEtjok/pEcD2 Ree5KK0gfuuSWgsQdMvzwEid0tGcHQMwqAJzb8jhSoDmbL76CaArjEvo9B3aPS8uflivNVSrq4FwB aR4L1aDHdAXDhSs26n0STAYrCDEMOlZgmn0p2bJsdQ7RIzqAb/Vm2yorIZeGMrbGUY0saPlzz0jII siqY8x2wnlz4orusBYwQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qhJGx-00BffS-2M; Sat, 16 Sep 2023 00:34:57 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qhJEK-00BdCB-2X for linux-arm-kernel@lists.infradead.org; Sat, 16 Sep 2023 00:32:20 +0000 Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-1c43cd8b6cbso8825995ad.0 for ; Fri, 15 Sep 2023 17:32:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1694824331; x=1695429131; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=XQKt9ip81eNaA/ioPgL3dH9e9vr3TzqljSeTBinzyVo=; b=HKFaO/z1aovNw9JDYBNXAKNxuubxdqbOmFVe7xwzyP3z1uatkSJlDU9LyQ1+Sa0yO7 9FHtgvZ5wMLqwRL5cxScMSdJttb/aHRgzlXAgAY9FB9zz3wK2DZDXgSsMkI2D/GRrFUV IMQroFfyeYk8vUhlQ6AN2JMkBnZVMiIBlEiL6dXqBFKkD/fphjLlzBH6CdIxzzjJtuBb jGh5L73RSEiI6YFid5txS+B61re4kXxlew9P0kshNLmshx0+69g0ykZ47M1Bjy7yhrTm 9SmBWBodyxKJyyHXe4+32mMKZvWI5/XECvyhUGRe4Bnixz+i55wqhS1msJc0GvisFX9D Affg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694824331; x=1695429131; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=XQKt9ip81eNaA/ioPgL3dH9e9vr3TzqljSeTBinzyVo=; b=X1qXswx1Bvn8a9/BPgrN3oKo86otVmKjABYISSqA9vuBUkDalvHXcF3sByh3mJAbOb UUb0HeYi0UvzshmuFQcfhvkDR6z7HTw4ZPZ2hABLl0GGBVNOG3X5rn/6gATfdYXJflN9 btn65Itl1IfLvruQlDjR5MpwfRPxnlSSIfr2VqSTJr0RrSx6JCVHkZOHzJO5LhGfJSMi sMWlF7BNpyv6jUbRNKa7+L+0YSv3GBZAjyhPkU/MIlTjgpWB3T6C9kHcQ1XM1DTEm9xJ Uc1sEVmSEOQeMwBOBCC3LLd7UNkqz9KB8S5TCmepeXdAtu/2nIFK+DJeDaQtYoA+7VnZ gZAA== X-Gm-Message-State: AOJu0YzwKCZFLJrRcgxwWPO2XMRQ+pWyI1S599YxBNEfejiqufH/ST6e xg5uqd8UPcHeaOE2zXFNRFmPQ1HOn1U= X-Google-Smtp-Source: AGHT+IGF/tpivhdzzMxCzu4p2COJ/tsujORVDRvqtMyofX9ypz/OjlCrOXRz/aOj6KmZJ7FZTcTX3ecEuUc= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:d4c7:b0:1bc:5182:1de2 with SMTP id o7-20020a170902d4c700b001bc51821de2mr81466plg.1.1694824330917; Fri, 15 Sep 2023 17:32:10 -0700 (PDT) Date: Fri, 15 Sep 2023 17:31:17 -0700 In-Reply-To: <20230916003118.2540661-1-seanjc@google.com> Mime-Version: 1.0 References: <20230916003118.2540661-1-seanjc@google.com> X-Mailer: git-send-email 2.42.0.459.ge4e396fd5e-goog Message-ID: <20230916003118.2540661-26-seanjc@google.com> Subject: [PATCH 25/26] KVM: PPC: Rearrange code in kvm_ppc.h to isolate "public" information From: Sean Christopherson To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Peter Zijlstra , Arnaldo Carvalho de Melo , Sean Christopherson , Paolo Bonzini , Tony Krowiak , Halil Pasic , Jason Herne , Harald Freudenberger , Alex Williamson , Andy Lutomirski Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, kvm@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Anish Ghulati , Venkatesh Srinivas , Andrew Thornton X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230915_173212_871505_3C9BF561 X-CRM114-Status: GOOD ( 21.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move all declarations and definitions in kvm_ppc.h that are consumed by things other than KVM to the top of the file. This will allow wrapping the parts of kvm_ppc.h that are intended only for KVM, i.e. are intended to be "private" to KVM, with an #ifdef to hide KVM's internal details from the kernel at-large. Signed-off-by: Sean Christopherson --- arch/powerpc/include/asm/kvm_ppc.h | 302 +++++++++++++++-------------- 1 file changed, 153 insertions(+), 149 deletions(-) diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h index 72fee202d3ec..ead2ad892ebc 100644 --- a/arch/powerpc/include/asm/kvm_ppc.h +++ b/arch/powerpc/include/asm/kvm_ppc.h @@ -30,6 +30,159 @@ #endif #include +struct openpic; + +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE +extern void kvm_cma_reserve(void) __init; +static inline void kvmppc_set_xics_phys(int cpu, unsigned long addr) +{ + paca_ptrs[cpu]->kvm_hstate.xics_phys = (void __iomem *)addr; +} + +static inline void kvmppc_set_xive_tima(int cpu, + unsigned long phys_addr, + void __iomem *virt_addr) +{ + paca_ptrs[cpu]->kvm_hstate.xive_tima_phys = (void __iomem *)phys_addr; + paca_ptrs[cpu]->kvm_hstate.xive_tima_virt = virt_addr; +} + +static inline u32 kvmppc_get_xics_latch(void) +{ + u32 xirr; + + xirr = get_paca()->kvm_hstate.saved_xirr; + get_paca()->kvm_hstate.saved_xirr = 0; + return xirr; +} + +/* + * To avoid the need to unnecessarily exit fully to the host kernel, an IPI to + * a CPU thread that's running/napping inside of a guest is by default regarded + * as a request to wake the CPU (if needed) and continue execution within the + * guest, potentially to process new state like externally-generated + * interrupts or IPIs sent from within the guest itself (e.g. H_PROD/H_IPI). + * + * To force an exit to the host kernel, kvmppc_set_host_ipi() must be called + * prior to issuing the IPI to set the corresponding 'host_ipi' flag in the + * target CPU's PACA. To avoid unnecessary exits to the host, this flag should + * be immediately cleared via kvmppc_clear_host_ipi() by the IPI handler on + * the receiving side prior to processing the IPI work. + * + * NOTE: + * + * We currently issue an smp_mb() at the beginning of kvmppc_set_host_ipi(). + * This is to guard against sequences such as the following: + * + * CPU + * X: smp_muxed_ipi_set_message(): + * X: smp_mb() + * X: message[RESCHEDULE] = 1 + * X: doorbell_global_ipi(42): + * X: kvmppc_set_host_ipi(42) + * X: ppc_msgsnd_sync()/smp_mb() + * X: ppc_msgsnd() -> 42 + * 42: doorbell_exception(): // from CPU X + * 42: ppc_msgsync() + * 105: smp_muxed_ipi_set_message(): + * 105: smb_mb() + * // STORE DEFERRED DUE TO RE-ORDERING + * --105: message[CALL_FUNCTION] = 1 + * | 105: doorbell_global_ipi(42): + * | 105: kvmppc_set_host_ipi(42) + * | 42: kvmppc_clear_host_ipi(42) + * | 42: smp_ipi_demux_relaxed() + * | 42: // returns to executing guest + * | // RE-ORDERED STORE COMPLETES + * ->105: message[CALL_FUNCTION] = 1 + * 105: ppc_msgsnd_sync()/smp_mb() + * 105: ppc_msgsnd() -> 42 + * 42: local_paca->kvm_hstate.host_ipi == 0 // IPI ignored + * 105: // hangs waiting on 42 to process messages/call_single_queue + * + * We also issue an smp_mb() at the end of kvmppc_clear_host_ipi(). This is + * to guard against sequences such as the following (as well as to create + * a read-side pairing with the barrier in kvmppc_set_host_ipi()): + * + * CPU + * X: smp_muxed_ipi_set_message(): + * X: smp_mb() + * X: message[RESCHEDULE] = 1 + * X: doorbell_global_ipi(42): + * X: kvmppc_set_host_ipi(42) + * X: ppc_msgsnd_sync()/smp_mb() + * X: ppc_msgsnd() -> 42 + * 42: doorbell_exception(): // from CPU X + * 42: ppc_msgsync() + * // STORE DEFERRED DUE TO RE-ORDERING + * -- 42: kvmppc_clear_host_ipi(42) + * | 42: smp_ipi_demux_relaxed() + * | 105: smp_muxed_ipi_set_message(): + * | 105: smb_mb() + * | 105: message[CALL_FUNCTION] = 1 + * | 105: doorbell_global_ipi(42): + * | 105: kvmppc_set_host_ipi(42) + * | // RE-ORDERED STORE COMPLETES + * -> 42: kvmppc_clear_host_ipi(42) + * 42: // returns to executing guest + * 105: ppc_msgsnd_sync()/smp_mb() + * 105: ppc_msgsnd() -> 42 + * 42: local_paca->kvm_hstate.host_ipi == 0 // IPI ignored + * 105: // hangs waiting on 42 to process messages/call_single_queue + */ +static inline void kvmppc_set_host_ipi(int cpu) +{ + /* + * order stores of IPI messages vs. setting of host_ipi flag + * + * pairs with the barrier in kvmppc_clear_host_ipi() + */ + smp_mb(); + WRITE_ONCE(paca_ptrs[cpu]->kvm_hstate.host_ipi, 1); +} + +static inline void kvmppc_clear_host_ipi(int cpu) +{ + WRITE_ONCE(paca_ptrs[cpu]->kvm_hstate.host_ipi, 0); + /* + * order clearing of host_ipi flag vs. processing of IPI messages + * + * pairs with the barrier in kvmppc_set_host_ipi() + */ + smp_mb(); +} + +extern void kvmppc_xics_ipi_action(void); + +extern void kvm_hv_vm_activated(void); +extern void kvm_hv_vm_deactivated(void); +extern bool kvm_hv_mode_active(void); +#else +static inline void __init kvm_cma_reserve(void) +{} + +static inline void kvmppc_set_xics_phys(int cpu, unsigned long addr) +{} + +static inline void kvmppc_set_xive_tima(int cpu, + unsigned long phys_addr, + void __iomem *virt_addr) +{} + +static inline u32 kvmppc_get_xics_latch(void) +{ + return 0; +} + +static inline void kvmppc_set_host_ipi(int cpu) +{} + +static inline void kvmppc_clear_host_ipi(int cpu) +{} + +static inline bool kvm_hv_mode_active(void) { return false; } +#endif + /* * KVMPPC_INST_SW_BREAKPOINT is debug Instruction * for supporting software breakpoint. @@ -443,166 +596,18 @@ void kvmppc_set_pid(struct kvm_vcpu *vcpu, u32 pid); struct openpic; #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE -extern void kvm_cma_reserve(void) __init; -static inline void kvmppc_set_xics_phys(int cpu, unsigned long addr) -{ - paca_ptrs[cpu]->kvm_hstate.xics_phys = (void __iomem *)addr; -} - -static inline void kvmppc_set_xive_tima(int cpu, - unsigned long phys_addr, - void __iomem *virt_addr) -{ - paca_ptrs[cpu]->kvm_hstate.xive_tima_phys = (void __iomem *)phys_addr; - paca_ptrs[cpu]->kvm_hstate.xive_tima_virt = virt_addr; -} - -static inline u32 kvmppc_get_xics_latch(void) -{ - u32 xirr; - - xirr = get_paca()->kvm_hstate.saved_xirr; - get_paca()->kvm_hstate.saved_xirr = 0; - return xirr; -} - -/* - * To avoid the need to unnecessarily exit fully to the host kernel, an IPI to - * a CPU thread that's running/napping inside of a guest is by default regarded - * as a request to wake the CPU (if needed) and continue execution within the - * guest, potentially to process new state like externally-generated - * interrupts or IPIs sent from within the guest itself (e.g. H_PROD/H_IPI). - * - * To force an exit to the host kernel, kvmppc_set_host_ipi() must be called - * prior to issuing the IPI to set the corresponding 'host_ipi' flag in the - * target CPU's PACA. To avoid unnecessary exits to the host, this flag should - * be immediately cleared via kvmppc_clear_host_ipi() by the IPI handler on - * the receiving side prior to processing the IPI work. - * - * NOTE: - * - * We currently issue an smp_mb() at the beginning of kvmppc_set_host_ipi(). - * This is to guard against sequences such as the following: - * - * CPU - * X: smp_muxed_ipi_set_message(): - * X: smp_mb() - * X: message[RESCHEDULE] = 1 - * X: doorbell_global_ipi(42): - * X: kvmppc_set_host_ipi(42) - * X: ppc_msgsnd_sync()/smp_mb() - * X: ppc_msgsnd() -> 42 - * 42: doorbell_exception(): // from CPU X - * 42: ppc_msgsync() - * 105: smp_muxed_ipi_set_message(): - * 105: smb_mb() - * // STORE DEFERRED DUE TO RE-ORDERING - * --105: message[CALL_FUNCTION] = 1 - * | 105: doorbell_global_ipi(42): - * | 105: kvmppc_set_host_ipi(42) - * | 42: kvmppc_clear_host_ipi(42) - * | 42: smp_ipi_demux_relaxed() - * | 42: // returns to executing guest - * | // RE-ORDERED STORE COMPLETES - * ->105: message[CALL_FUNCTION] = 1 - * 105: ppc_msgsnd_sync()/smp_mb() - * 105: ppc_msgsnd() -> 42 - * 42: local_paca->kvm_hstate.host_ipi == 0 // IPI ignored - * 105: // hangs waiting on 42 to process messages/call_single_queue - * - * We also issue an smp_mb() at the end of kvmppc_clear_host_ipi(). This is - * to guard against sequences such as the following (as well as to create - * a read-side pairing with the barrier in kvmppc_set_host_ipi()): - * - * CPU - * X: smp_muxed_ipi_set_message(): - * X: smp_mb() - * X: message[RESCHEDULE] = 1 - * X: doorbell_global_ipi(42): - * X: kvmppc_set_host_ipi(42) - * X: ppc_msgsnd_sync()/smp_mb() - * X: ppc_msgsnd() -> 42 - * 42: doorbell_exception(): // from CPU X - * 42: ppc_msgsync() - * // STORE DEFERRED DUE TO RE-ORDERING - * -- 42: kvmppc_clear_host_ipi(42) - * | 42: smp_ipi_demux_relaxed() - * | 105: smp_muxed_ipi_set_message(): - * | 105: smb_mb() - * | 105: message[CALL_FUNCTION] = 1 - * | 105: doorbell_global_ipi(42): - * | 105: kvmppc_set_host_ipi(42) - * | // RE-ORDERED STORE COMPLETES - * -> 42: kvmppc_clear_host_ipi(42) - * 42: // returns to executing guest - * 105: ppc_msgsnd_sync()/smp_mb() - * 105: ppc_msgsnd() -> 42 - * 42: local_paca->kvm_hstate.host_ipi == 0 // IPI ignored - * 105: // hangs waiting on 42 to process messages/call_single_queue - */ -static inline void kvmppc_set_host_ipi(int cpu) -{ - /* - * order stores of IPI messages vs. setting of host_ipi flag - * - * pairs with the barrier in kvmppc_clear_host_ipi() - */ - smp_mb(); - WRITE_ONCE(paca_ptrs[cpu]->kvm_hstate.host_ipi, 1); -} - -static inline void kvmppc_clear_host_ipi(int cpu) -{ - WRITE_ONCE(paca_ptrs[cpu]->kvm_hstate.host_ipi, 0); - /* - * order clearing of host_ipi flag vs. processing of IPI messages - * - * pairs with the barrier in kvmppc_set_host_ipi() - */ - smp_mb(); -} - static inline void kvmppc_fast_vcpu_kick(struct kvm_vcpu *vcpu) { vcpu->kvm->arch.kvm_ops->fast_vcpu_kick(vcpu); } -extern void kvm_hv_vm_activated(void); -extern void kvm_hv_vm_deactivated(void); -extern bool kvm_hv_mode_active(void); - extern void kvmppc_check_need_tlb_flush(struct kvm *kvm, int pcpu); #else -static inline void __init kvm_cma_reserve(void) -{} - -static inline void kvmppc_set_xics_phys(int cpu, unsigned long addr) -{} - -static inline void kvmppc_set_xive_tima(int cpu, - unsigned long phys_addr, - void __iomem *virt_addr) -{} - -static inline u32 kvmppc_get_xics_latch(void) -{ - return 0; -} - -static inline void kvmppc_set_host_ipi(int cpu) -{} - -static inline void kvmppc_clear_host_ipi(int cpu) -{} - static inline void kvmppc_fast_vcpu_kick(struct kvm_vcpu *vcpu) { kvm_vcpu_kick(vcpu); } - -static inline bool kvm_hv_mode_active(void) { return false; } - #endif #ifdef CONFIG_PPC_PSERIES @@ -642,7 +647,6 @@ extern u64 kvmppc_xics_get_icp(struct kvm_vcpu *vcpu); extern int kvmppc_xics_set_icp(struct kvm_vcpu *vcpu, u64 icpval); extern int kvmppc_xics_connect_vcpu(struct kvm_device *dev, struct kvm_vcpu *vcpu, u32 cpu); -extern void kvmppc_xics_ipi_action(void); extern void kvmppc_xics_set_mapped(struct kvm *kvm, unsigned long guest_irq, unsigned long host_irq); extern void kvmppc_xics_clr_mapped(struct kvm *kvm, unsigned long guest_irq,