From patchwork Sat Aug 28 00:35:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12463071 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90DE8C4320E for ; Sat, 28 Aug 2021 00:39:34 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 59ED960C3E for ; Sat, 28 Aug 2021 00:39:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 59ED960C3E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References :Mime-Version:Message-Id:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=c/4vAiEDjS895EXwj63pJG5g7o5qlfArYkqdbnzCh4I=; b=Rz0GANNP98H9ui QFxBSa6hLL4U8vsnthAp+fyjx+fktWWGkUZV3E2X6BWjW3o2h2ezkPeRt0kpPbZs59Cc9P87n+93/ pLn6gISfOGueVGYNDtnLLgwu0x2qemySlsQizXWd5/MOeWgsGvh98gkpw1N4oVTt4HmGwry0bZY/1 opo9XoyrDeI7M0G0gSi6f8VuR7UjU6ILCUKAxseav68xQ2vLs9s/fL9wQZlHkPrEui0Bir7vIR+cj wEC567qy0MGu2DXAOdrcgXS75hwEj1XJuZxDrYypE6Jj5xJZfHQSh79FNldA5uK3Stx7NPEt7mnhx OKbZhRSJypWlvQJViz4A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmLR-00DRJf-DS; Sat, 28 Aug 2021 00:37:13 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmKl-00DQx4-5x for linux-arm-kernel@lists.infradead.org; Sat, 28 Aug 2021 00:36:34 +0000 Received: by mail-yb1-xb49.google.com with SMTP id v126-20020a254884000000b005991fd2f912so8359809yba.20 for ; Fri, 27 Aug 2021 17:36:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=F8q52qTonl1WgIN1PG0lAM7qDfA4FG00OZ18YHpgfY0=; b=rKb6cxam3N0QSWl05NjiPcqFq7uB1w8asRUlLzt1P8OmdbNPgCxdR8y1I45+etXAlZ 0RVL63FCf8Ww49CMPik+P9mVseYK41393u4LYlafNyLY3IPgDeFTnY1gY4emJbZm+BPJ +SWnaaj/FFKx/zjJGIoBkjo0dC6OLyfJSgixINqsfVC47CtpLnDtfCNytECI3vLT+x1V ToLjyGJ/HjVr47VjK4dAorhBp0qtlW6DQWxDJ3YrMKO/yi9UPbfQi+CmEU2M0g9SvoyX oanCUklyXb9rt9xfgQwAKtMreuulqqn6OVP3UtgwK2aC8UiJvpY5FUSIFWofRAPdn9kH yQCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=F8q52qTonl1WgIN1PG0lAM7qDfA4FG00OZ18YHpgfY0=; b=JZVFk8rtXezRU6mRyWkWBURRuf373ycWuFi6jxGRolPG59sMDUpw8a4W+36mqG/yrg 7acH6VfdbWNhCM/1KCrTbm5xXCvIIAdbdNzUS9bfUIVKb9AmeQGgECgf8L84wGaHZ5ij M8kbjCg/gqDpnNEJTjKYDVphkezxVrFXaJOcP5YhjhjWMgD/4xofcbdpH8vwG3ABKIdm HWfOwC0IeUGPnfzUL0fJtf/pUVDG5gqJGvddFPby9Rw55NKtgaAZczEj0SIHu1g2Hfo0 wzK5so06MerMa3Y3fEEn7U2qwQAWKdddHpXo4yeZJbiXzzifDp3DUIekCxz0Ha2HzFMF lW1Q== X-Gm-Message-State: AOAM531vPJComM2HDb/Gq7RzZJNFN8NHZUIHAwXJi0BxwBMpn72ER7fO KiAzGFLVRcVQqSdSGjvFFiz5nCL6hrg= X-Google-Smtp-Source: ABdhPJyqSSFifuN3wpXn7AfwWRiIK9qd/5mSNUdpOo4/WAgd92xLY/ddbhP+qZR6voGTLj2v1wyNcy83s+s= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:f66c:b851:7e79:7ed4]) (user=seanjc job=sendgmr) by 2002:a25:d892:: with SMTP id p140mr8889410ybg.406.1630110987803; Fri, 27 Aug 2021 17:36:27 -0700 (PDT) Date: Fri, 27 Aug 2021 17:35:46 -0700 In-Reply-To: <20210828003558.713983-1-seanjc@google.com> Message-Id: <20210828003558.713983-2-seanjc@google.com> Mime-Version: 1.0 References: <20210828003558.713983-1-seanjc@google.com> X-Mailer: git-send-email 2.33.0.259.gc128427fd7-goog Subject: [PATCH v2 01/13] perf: Ensure perf_guest_cbs aren't reloaded between !NULL check and deref From: Sean Christopherson To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Will Deacon , Mark Rutland , Catalin Marinas , Marc Zyngier , Guo Ren , Nick Hu , Greentime Hu , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , Thomas Gleixner , Borislav Petkov , x86@kernel.org, Paolo Bonzini , Boris Ostrovsky , Juergen Gross Cc: Alexander Shishkin , Jiri Olsa , Namhyung Kim , James Morse , Alexandru Elisei , Suzuki K Poulose , "H. Peter Anvin" , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Stefano Stabellini , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Artem Kashkanov , Like Xu , Zhu Lingshan X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210827_173631_284809_1DA0453B X-CRM114-Status: GOOD ( 18.54 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Protect perf_guest_cbs with READ_ONCE/WRITE_ONCE to ensure it's not reloaded between a !NULL check and a dereference, and wait for all readers via syncrhonize_rcu() to prevent use-after-free, e.g. if the callbacks are being unregistered during module unload. Because the callbacks are global, it's possible for readers to run in parallel with an unregister operation. The bug has escaped notice because all dereferences of perf_guest_cbs follow the same "perf_guest_cbs && perf_guest_cbs->is_in_guest()" pattern, and it's extremely unlikely a compiler will reload perf_guest_cbs in this sequence. Compilers do reload perf_guest_cbs for future derefs, e.g. for ->is_user_mode(), but the ->is_in_guest() guard all but guarantees the PMI handler will win the race, e.g. to nullify perf_guest_cbs, KVM has to completely exit the guest and teardown down all VMs before KVM start its module unload / unregister sequence. But with help, unloading kvm_intel can trigger a NULL pointer derference, e.g. wrapping perf_guest_cbs with READ_ONCE in perf_misc_flags() while spamming kvm_intel module load/unload leads to: BUG: kernel NULL pointer dereference, address: 0000000000000000 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page PGD 0 P4D 0 Oops: 0000 [#1] PREEMPT SMP CPU: 6 PID: 1825 Comm: stress Not tainted 5.14.0-rc2+ #459 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015 RIP: 0010:perf_misc_flags+0x1c/0x70 Call Trace: perf_prepare_sample+0x53/0x6b0 perf_event_output_forward+0x67/0x160 __perf_event_overflow+0x52/0xf0 handle_pmi_common+0x207/0x300 intel_pmu_handle_irq+0xcf/0x410 perf_event_nmi_handler+0x28/0x50 nmi_handle+0xc7/0x260 default_do_nmi+0x6b/0x170 exc_nmi+0x103/0x130 asm_exc_nmi+0x76/0xbf Fixes: 39447b386c84 ("perf: Enhance perf to allow for guest statistic collection from host") Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson --- arch/arm/kernel/perf_callchain.c | 17 +++++++++++------ arch/arm64/kernel/perf_callchain.c | 18 ++++++++++++------ arch/csky/kernel/perf_callchain.c | 6 ++++-- arch/nds32/kernel/perf_event_cpu.c | 17 +++++++++++------ arch/riscv/kernel/perf_callchain.c | 7 +++++-- arch/x86/events/core.c | 17 +++++++++++------ arch/x86/events/intel/core.c | 9 ++++++--- include/linux/perf_event.h | 8 ++++++++ kernel/events/core.c | 9 +++++++-- 9 files changed, 75 insertions(+), 33 deletions(-) diff --git a/arch/arm/kernel/perf_callchain.c b/arch/arm/kernel/perf_callchain.c index 3b69a76d341e..1626dfc6f6ce 100644 --- a/arch/arm/kernel/perf_callchain.c +++ b/arch/arm/kernel/perf_callchain.c @@ -62,9 +62,10 @@ user_backtrace(struct frame_tail __user *tail, void perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); struct frame_tail __user *tail; - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { + if (guest_cbs && guest_cbs->is_in_guest()) { /* We don't support guest os callchain now */ return; } @@ -98,9 +99,10 @@ callchain_trace(struct stackframe *fr, void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); struct stackframe fr; - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { + if (guest_cbs && guest_cbs->is_in_guest()) { /* We don't support guest os callchain now */ return; } @@ -111,18 +113,21 @@ perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *re unsigned long perf_instruction_pointer(struct pt_regs *regs) { - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) - return perf_guest_cbs->get_guest_ip(); + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); + + if (guest_cbs && guest_cbs->is_in_guest()) + return guest_cbs->get_guest_ip(); return instruction_pointer(regs); } unsigned long perf_misc_flags(struct pt_regs *regs) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); int misc = 0; - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { - if (perf_guest_cbs->is_user_mode()) + if (guest_cbs && guest_cbs->is_in_guest()) { + if (guest_cbs->is_user_mode()) misc |= PERF_RECORD_MISC_GUEST_USER; else misc |= PERF_RECORD_MISC_GUEST_KERNEL; diff --git a/arch/arm64/kernel/perf_callchain.c b/arch/arm64/kernel/perf_callchain.c index 4a72c2727309..86d9f2013172 100644 --- a/arch/arm64/kernel/perf_callchain.c +++ b/arch/arm64/kernel/perf_callchain.c @@ -102,7 +102,9 @@ compat_user_backtrace(struct compat_frame_tail __user *tail, void perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); + + if (guest_cbs && guest_cbs->is_in_guest()) { /* We don't support guest os callchain now */ return; } @@ -147,9 +149,10 @@ static bool callchain_trace(void *data, unsigned long pc) void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); struct stackframe frame; - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { + if (guest_cbs && guest_cbs->is_in_guest()) { /* We don't support guest os callchain now */ return; } @@ -160,18 +163,21 @@ void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, unsigned long perf_instruction_pointer(struct pt_regs *regs) { - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) - return perf_guest_cbs->get_guest_ip(); + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); + + if (guest_cbs && guest_cbs->is_in_guest()) + return guest_cbs->get_guest_ip(); return instruction_pointer(regs); } unsigned long perf_misc_flags(struct pt_regs *regs) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); int misc = 0; - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { - if (perf_guest_cbs->is_user_mode()) + if (guest_cbs && guest_cbs->is_in_guest()) { + if (guest_cbs->is_user_mode()) misc |= PERF_RECORD_MISC_GUEST_USER; else misc |= PERF_RECORD_MISC_GUEST_KERNEL; diff --git a/arch/csky/kernel/perf_callchain.c b/arch/csky/kernel/perf_callchain.c index ab55e98ee8f6..35318a635a5f 100644 --- a/arch/csky/kernel/perf_callchain.c +++ b/arch/csky/kernel/perf_callchain.c @@ -86,10 +86,11 @@ static unsigned long user_backtrace(struct perf_callchain_entry_ctx *entry, void perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); unsigned long fp = 0; /* C-SKY does not support virtualization. */ - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) + if (guest_cbs && guest_cbs->is_in_guest()) return; fp = regs->regs[4]; @@ -110,10 +111,11 @@ void perf_callchain_user(struct perf_callchain_entry_ctx *entry, void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); struct stackframe fr; /* C-SKY does not support virtualization. */ - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { + if (guest_cbs && guest_cbs->is_in_guest()) { pr_warn("C-SKY does not support perf in guest mode!"); return; } diff --git a/arch/nds32/kernel/perf_event_cpu.c b/arch/nds32/kernel/perf_event_cpu.c index 0ce6f9f307e6..f38791960781 100644 --- a/arch/nds32/kernel/perf_event_cpu.c +++ b/arch/nds32/kernel/perf_event_cpu.c @@ -1363,6 +1363,7 @@ void perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); unsigned long fp = 0; unsigned long gp = 0; unsigned long lp = 0; @@ -1371,7 +1372,7 @@ perf_callchain_user(struct perf_callchain_entry_ctx *entry, leaf_fp = 0; - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { + if (guest_cbs && guest_cbs->is_in_guest()) { /* We don't support guest os callchain now */ return; } @@ -1479,9 +1480,10 @@ void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); struct stackframe fr; - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { + if (guest_cbs && guest_cbs->is_in_guest()) { /* We don't support guest os callchain now */ return; } @@ -1493,20 +1495,23 @@ perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, unsigned long perf_instruction_pointer(struct pt_regs *regs) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); + /* However, NDS32 does not support virtualization */ - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) - return perf_guest_cbs->get_guest_ip(); + if (guest_cbs && guest_cbs->is_in_guest()) + return guest_cbs->get_guest_ip(); return instruction_pointer(regs); } unsigned long perf_misc_flags(struct pt_regs *regs) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); int misc = 0; /* However, NDS32 does not support virtualization */ - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { - if (perf_guest_cbs->is_user_mode()) + if (guest_cbs && guest_cbs->is_in_guest()) { + if (guest_cbs->is_user_mode()) misc |= PERF_RECORD_MISC_GUEST_USER; else misc |= PERF_RECORD_MISC_GUEST_KERNEL; diff --git a/arch/riscv/kernel/perf_callchain.c b/arch/riscv/kernel/perf_callchain.c index 0bb1854dce83..8ecfc4c128bc 100644 --- a/arch/riscv/kernel/perf_callchain.c +++ b/arch/riscv/kernel/perf_callchain.c @@ -56,10 +56,11 @@ static unsigned long user_backtrace(struct perf_callchain_entry_ctx *entry, void perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); unsigned long fp = 0; /* RISC-V does not support perf in guest mode. */ - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) + if (guest_cbs && guest_cbs->is_in_guest()) return; fp = regs->s0; @@ -78,8 +79,10 @@ static bool fill_callchain(void *entry, unsigned long pc) void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); + /* RISC-V does not support perf in guest mode. */ - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { + if (guest_cbs && guest_cbs->is_in_guest()) { pr_warn("RISC-V does not support perf in guest mode!"); return; } diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index 1eb45139fcc6..ffb3e6c0d367 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -2761,10 +2761,11 @@ static bool perf_hw_regs(struct pt_regs *regs) void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); struct unwind_state state; unsigned long addr; - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { + if (guest_cbs && guest_cbs->is_in_guest()) { /* TODO: We don't support guest os callchain now */ return; } @@ -2864,10 +2865,11 @@ perf_callchain_user32(struct pt_regs *regs, struct perf_callchain_entry_ctx *ent void perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); struct stack_frame frame; const struct stack_frame __user *fp; - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { + if (guest_cbs && guest_cbs->is_in_guest()) { /* TODO: We don't support guest os callchain now */ return; } @@ -2944,18 +2946,21 @@ static unsigned long code_segment_base(struct pt_regs *regs) unsigned long perf_instruction_pointer(struct pt_regs *regs) { - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) - return perf_guest_cbs->get_guest_ip(); + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); + + if (guest_cbs && guest_cbs->is_in_guest()) + return guest_cbs->get_guest_ip(); return regs->ip + code_segment_base(regs); } unsigned long perf_misc_flags(struct pt_regs *regs) { + struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); int misc = 0; - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { - if (perf_guest_cbs->is_user_mode()) + if (guest_cbs && guest_cbs->is_in_guest()) { + if (guest_cbs->is_user_mode()) misc |= PERF_RECORD_MISC_GUEST_USER; else misc |= PERF_RECORD_MISC_GUEST_KERNEL; diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index fca7a6e2242f..9baa46185d94 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -2786,6 +2786,7 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status) { struct perf_sample_data data; struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events); + struct perf_guest_info_callbacks *guest_cbs; int bit; int handled = 0; u64 intel_ctrl = hybrid(cpuc->pmu, intel_ctrl); @@ -2852,9 +2853,11 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status) */ if (__test_and_clear_bit(GLOBAL_STATUS_TRACE_TOPAPMI_BIT, (unsigned long *)&status)) { handled++; - if (unlikely(perf_guest_cbs && perf_guest_cbs->is_in_guest() && - perf_guest_cbs->handle_intel_pt_intr)) - perf_guest_cbs->handle_intel_pt_intr(); + + guest_cbs = perf_get_guest_cbs(); + if (unlikely(guest_cbs && guest_cbs->is_in_guest() && + guest_cbs->handle_intel_pt_intr)) + guest_cbs->handle_intel_pt_intr(); else intel_pt_interrupt(); } diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 2d510ad750ed..6b0405e578c1 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1237,6 +1237,14 @@ extern void perf_event_bpf_event(struct bpf_prog *prog, u16 flags); extern struct perf_guest_info_callbacks *perf_guest_cbs; +static inline struct perf_guest_info_callbacks *perf_get_guest_cbs(void) +{ + /* Reg/unreg perf_guest_cbs waits for readers via synchronize_rcu(). */ + lockdep_assert_preemption_disabled(); + + /* Prevent reloading between a !NULL check and dereferences. */ + return READ_ONCE(perf_guest_cbs); +} extern int perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *callbacks); extern int perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *callbacks); diff --git a/kernel/events/core.c b/kernel/events/core.c index 464917096e73..2126f6327321 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -6491,14 +6491,19 @@ struct perf_guest_info_callbacks *perf_guest_cbs; int perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs) { - perf_guest_cbs = cbs; + if (WARN_ON_ONCE(perf_guest_cbs)) + return -EBUSY; + + WRITE_ONCE(perf_guest_cbs, cbs); + synchronize_rcu(); return 0; } EXPORT_SYMBOL_GPL(perf_register_guest_info_callbacks); int perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs) { - perf_guest_cbs = NULL; + WRITE_ONCE(perf_guest_cbs, NULL); + synchronize_rcu(); return 0; } EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks); From patchwork Sat Aug 28 00:35:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12463077 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 032D9C432BE for ; Sat, 28 Aug 2021 00:40:37 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BC94060ED5 for ; Sat, 28 Aug 2021 00:40:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org BC94060ED5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References :Mime-Version:Message-Id:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=HQ28Z9eBDuTahDDkmeQDzbuQoNc8V/ZX1TNLLZV/dq0=; b=MSQfQcaut+xHJ+ BGNggaKGBqfsTUsHqIc85Y+iOPxDVcgX8a8Huyei1ukFIfYYZzfMrWoU33LfYMOASMv5bk4vjg6gU OZRYmuOXaE6IyUrfV68igLUHyLqaZpKLxLoGbpA1xYZY9lb+hZ8mtwFt2l6BLG85HeDAa9HrLLbUV zJ8WGNaVz8lu9fmtL/D1z3OjJ+F7nRJQra2y6jW4HFALSYbebRhyz+m/3TkhhzUZYo7LJRf7eIGcC VpKIZwt99iine6YPj4xawG1Iy3M1K0AUS+XBfDuXs6kpvV8q+CqpQ7kEldeWz7Yy3YdZJqCfEc+rc xF0luqfa0yZLHMcJaMeQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmLD-00DRGj-Nx; Sat, 28 Aug 2021 00:36:59 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmKl-00DQxc-GR for linux-arm-kernel@lists.infradead.org; Sat, 28 Aug 2021 00:36:33 +0000 Received: by mail-yb1-xb49.google.com with SMTP id b9-20020a5b07890000b0290558245b7eabso651536ybq.10 for ; Fri, 27 Aug 2021 17:36:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=J1gV9tKuEUW347bCpCRKYVIA3sPFQNKnByLeTysAe1c=; b=QQC9kCQqRwL40ouGUsJ6vhRZON2TPobzHX8Wq8NP7Mf7/RKw30MhddUWUdztQM8ka7 Ewe++e/HjV2jGw30ZO9LaCG2WpwNsAvCft0FNwp9j2VxNuY0jXVJogAA4hui8IIZ1qr9 T7Ay+2a4aRkadNU1ca0JCC87XvjH4ea9fD4ReM/qOgy67MxneR4EHRBEt1lUd0VOX2+/ aMIald+U354j6b4DBU7OSxpd/vJmRGaMfQXYZFeXtKK7qkfMyGqETcknapO+6PjS0TvX f2FLBBpeqGMiBMtBIznPS2E0O9iUn3iZeMA1csLuIwLQaPaZSBmpT2biwVLtTLrwW9db 1vRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=J1gV9tKuEUW347bCpCRKYVIA3sPFQNKnByLeTysAe1c=; b=Zn4CpzeHBuqxC7rTt2Nq4mdCfsj5waFt8Lkg91ZQpmUEsJISBNM8Xm+Xo+zNYGwXOm 02i3I6a4U0mfIRwd8Ok6ykXScC9XHw/lmCswEzQomMSDavgq8N0SZrSmr9f57mjMAj5W pPY5kPzsSf5Isx7Dethi/mQAbEP536DAoI9Jk17GIoJFeHm1lKviEfRBkKxkc3chLdQp 5MWp0hzbeI2tqM4thl+ZH9bW6+FYUXXvETA11OrxPU+mXgccENvoSLPLIXyTp3PRPvTc LdmEx8EYHGY9Wvn/4UMjR2Xldu4cPGSrbf2WWZnYAGgU+WWqXMF4fow+s0R+XL6NGib2 jJig== X-Gm-Message-State: AOAM530FDmvYd31GLxFcCG6SqTxF/50YuDVBhrGGZmFHEnQobY/BVt1s 1/ASIONRiIGwG7GQP68HpsXbwSNrIAQ= X-Google-Smtp-Source: ABdhPJycwxh1le3BAg4292rl1ZqiFoDJidbv03xd08RKtKGj0W0JQHpqPcTgq5axIHXDILJ1EM67ZM2iLac= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:f66c:b851:7e79:7ed4]) (user=seanjc job=sendgmr) by 2002:a25:d307:: with SMTP id e7mr8478177ybf.119.1630110990084; Fri, 27 Aug 2021 17:36:30 -0700 (PDT) Date: Fri, 27 Aug 2021 17:35:47 -0700 In-Reply-To: <20210828003558.713983-1-seanjc@google.com> Message-Id: <20210828003558.713983-3-seanjc@google.com> Mime-Version: 1.0 References: <20210828003558.713983-1-seanjc@google.com> X-Mailer: git-send-email 2.33.0.259.gc128427fd7-goog Subject: [PATCH v2 02/13] KVM: x86: Register perf callbacks after calling vendor's hardware_setup() From: Sean Christopherson To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Will Deacon , Mark Rutland , Catalin Marinas , Marc Zyngier , Guo Ren , Nick Hu , Greentime Hu , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , Thomas Gleixner , Borislav Petkov , x86@kernel.org, Paolo Bonzini , Boris Ostrovsky , Juergen Gross Cc: Alexander Shishkin , Jiri Olsa , Namhyung Kim , James Morse , Alexandru Elisei , Suzuki K Poulose , "H. Peter Anvin" , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Stefano Stabellini , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Artem Kashkanov , Like Xu , Zhu Lingshan X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210827_173631_591649_0B38D4C9 X-CRM114-Status: GOOD ( 13.53 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Wait to register perf callbacks until after doing vendor hardaware setup. VMX's hardware_setup() configures Intel Processor Trace (PT) mode, and a future fix to register the Intel PT guest interrupt hook if and only if Intel PT is exposed to the guest will consume the configured PT mode. Delaying registration to hardware setup is effectively a nop as KVM's perf hooks all pivot on the per-CPU current_vcpu, which is non-NULL only when KVM is handling an IRQ/NMI in a VM-Exit path. I.e. current_vcpu will be NULL throughout both kvm_arch_init() and kvm_arch_hardware_setup(). Cc: Alexander Shishkin Cc: Artem Kashkanov Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 86539c1686fa..fb6015f97f9e 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8426,8 +8426,6 @@ int kvm_arch_init(void *opaque) kvm_timer_init(); - perf_register_guest_info_callbacks(&kvm_guest_cbs); - if (boot_cpu_has(X86_FEATURE_XSAVE)) { host_xcr0 = xgetbv(XCR_XFEATURE_ENABLED_MASK); supported_xcr0 = host_xcr0 & KVM_SUPPORTED_XCR0; @@ -8461,7 +8459,6 @@ void kvm_arch_exit(void) clear_hv_tscchange_cb(); #endif kvm_lapic_exit(); - perf_unregister_guest_info_callbacks(&kvm_guest_cbs); if (!boot_cpu_has(X86_FEATURE_CONSTANT_TSC)) cpufreq_unregister_notifier(&kvmclock_cpufreq_notifier_block, @@ -11064,6 +11061,8 @@ int kvm_arch_hardware_setup(void *opaque) memcpy(&kvm_x86_ops, ops->runtime_ops, sizeof(kvm_x86_ops)); kvm_ops_static_call_update(); + perf_register_guest_info_callbacks(&kvm_guest_cbs); + if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES)) supported_xss = 0; @@ -11091,6 +11090,8 @@ int kvm_arch_hardware_setup(void *opaque) void kvm_arch_hardware_unsetup(void) { + perf_unregister_guest_info_callbacks(&kvm_guest_cbs); + static_call(kvm_x86_hardware_unsetup)(); } From patchwork Sat Aug 28 00:35:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12463069 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A5FDC432BE for ; Sat, 28 Aug 2021 00:39:31 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CBEB760FD9 for ; Sat, 28 Aug 2021 00:39:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org CBEB760FD9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References :Mime-Version:Message-Id:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wwr3Z0f2arGWEAFpP5AHgqIwvkosrndtGM7i6AwFVkI=; b=iDU7mOVy99NoQe VLRijPMOUmq/5nTIB8lelZr3O7iPip5oNRYD6gvScCRZ5dbjrLNxPTzZJpvgW87/jZq2sb9b1mzug T2hn/dMJqdNK8NYGBrpHETSmF1kivEWV/JaVGD9yWkAy790KfGpAV20oyD04uHbwfNSKmALuBeAfN nsTcrDweZlLf/JJUZevbrHiLK+YxQDNpo514iK+FJws5AQ1mF3qsskKfpasxicvufa9a/H9APqLhX bwm6H/awRUKD1BnIdfs0UjzwYPpanlgTJvbaBa1Ymbx200GGv3SkC0YwTJHPmFuDMggDEGR5gWvXO lt6Gedf1AVd5wePSkqmg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmLo-00DRT3-Qw; Sat, 28 Aug 2021 00:37:37 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmKn-00DQzd-R3 for linux-arm-kernel@lists.infradead.org; Sat, 28 Aug 2021 00:36:37 +0000 Received: by mail-yb1-xb49.google.com with SMTP id u1-20020a25ab01000000b0059949024617so818211ybi.17 for ; Fri, 27 Aug 2021 17:36:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=oqmEir0ICv/3G92ai+heTZuDyr/2G3d8WRYAGw86atk=; b=PcPAOihT8OlOhhUR4O32+YWwPzAPmLXJ1HEV+moJpFC1Qy84KcVrj+ajHpTlQK+a3F iyxUIoBJmALw6qqcNEgkIBWBH6GlQPAe+u7xlbyeAychGYyySNpy/bvcaaDuenDkzBb1 Tjlhu6cHuXug1LfrtevGrfztVaOuMtHQ1BbUdWdKfT8tes9/0JNecwDm/BWfVrT79Qhh U/SjMl4oOYlt5v38Pnm8Mc6W1SnA5LO9EWqHnie5e5sXhbuMbSanMYmcJC2M9uljBuiN FTc3AiajdilK67F7Z2gaOx4+/6EEfK+b8GQmcjFFKC29NzjAngrdpD9yW51Odds/BoDr blsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=oqmEir0ICv/3G92ai+heTZuDyr/2G3d8WRYAGw86atk=; b=Z7xbz9cWkt6kDZkTdBLoo0a3QmhYwEqFLufRCBN03K0pV7twwTX4wI7NsKUJKCWoMc 2/6BnE8rUCLIUGKJ+ncT0bNZuQ9g64rwGKS5nvGmoCRb4DvY6ScDCBOlDRx98Ly1ur5D kb3QVHPLSyWadefHuCKabauDdNqpbajCdnAVuXJUB9QrpJT9SZ1EEJgxzJuIv1rofrBk nNq00zcIuenMdQJWZVwK7IeScJOf1Fxil3OZ61AcLEUuWdibaWPceqph1DF+rqS+fnXk tALfIAVHgBBbB39zHS2hW6z/8RIH6QPtJ3PDBEieaxzsvmvoeLZApZRgJI00vOT80G00 McrQ== X-Gm-Message-State: AOAM531q4ufxsAjX/sUm77Dfhv2ZxYMVhxnkzYCdDIpCDK2nnCuT/s2B nIAZZARKZ2QYZqrPQRWBHad9+j+qQ3o= X-Google-Smtp-Source: ABdhPJyTmAQoKJ2tjPvfJaPWBN34/RS1O5zwXSkZLsm9h521rPpR1j//umIRZ/UqerB16AfCO1AGIAPsHvE= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:f66c:b851:7e79:7ed4]) (user=seanjc job=sendgmr) by 2002:a25:4545:: with SMTP id s66mr8318615yba.191.1630110992229; Fri, 27 Aug 2021 17:36:32 -0700 (PDT) Date: Fri, 27 Aug 2021 17:35:48 -0700 In-Reply-To: <20210828003558.713983-1-seanjc@google.com> Message-Id: <20210828003558.713983-4-seanjc@google.com> Mime-Version: 1.0 References: <20210828003558.713983-1-seanjc@google.com> X-Mailer: git-send-email 2.33.0.259.gc128427fd7-goog Subject: [PATCH v2 03/13] KVM: x86: Register Processor Trace interrupt hook iff PT enabled in guest From: Sean Christopherson To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Will Deacon , Mark Rutland , Catalin Marinas , Marc Zyngier , Guo Ren , Nick Hu , Greentime Hu , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , Thomas Gleixner , Borislav Petkov , x86@kernel.org, Paolo Bonzini , Boris Ostrovsky , Juergen Gross Cc: Alexander Shishkin , Jiri Olsa , Namhyung Kim , James Morse , Alexandru Elisei , Suzuki K Poulose , "H. Peter Anvin" , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Stefano Stabellini , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Artem Kashkanov , Like Xu , Zhu Lingshan X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210827_173634_023567_D58482EA X-CRM114-Status: GOOD ( 16.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Override the Processor Trace (PT) interrupt handler for guest mode if and only if PT is configured for host+guest mode, i.e. is being used independently by both host and guest. If PT is configured for system mode, the host fully controls PT and must handle all events. Fixes: 8479e04e7d6b ("KVM: x86: Inject PMI for KVM guest") Cc: stable@vger.kernel.org Cc: Like Xu Reported-by: Alexander Shishkin Reported-by: Artem Kashkanov Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/vmx/vmx.c | 1 + arch/x86/kvm/x86.c | 5 ++++- 3 files changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 09b256db394a..1ea4943a73d7 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1494,6 +1494,7 @@ struct kvm_x86_init_ops { int (*disabled_by_bios)(void); int (*check_processor_compatibility)(void); int (*hardware_setup)(void); + bool (*intel_pt_intr_in_guest)(void); struct kvm_x86_ops *runtime_ops; }; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index fada1055f325..f19d72136f77 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7896,6 +7896,7 @@ static struct kvm_x86_init_ops vmx_init_ops __initdata = { .disabled_by_bios = vmx_disabled_by_bios, .check_processor_compatibility = vmx_check_processor_compat, .hardware_setup = hardware_setup, + .intel_pt_intr_in_guest = vmx_pt_mode_is_host_guest, .runtime_ops = &vmx_x86_ops, }; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index fb6015f97f9e..ffc6c2d73508 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8305,7 +8305,7 @@ static struct perf_guest_info_callbacks kvm_guest_cbs = { .is_in_guest = kvm_is_in_guest, .is_user_mode = kvm_is_user_mode, .get_guest_ip = kvm_get_guest_ip, - .handle_intel_pt_intr = kvm_handle_intel_pt_intr, + .handle_intel_pt_intr = NULL, }; #ifdef CONFIG_X86_64 @@ -11061,6 +11061,8 @@ int kvm_arch_hardware_setup(void *opaque) memcpy(&kvm_x86_ops, ops->runtime_ops, sizeof(kvm_x86_ops)); kvm_ops_static_call_update(); + if (ops->intel_pt_intr_in_guest && ops->intel_pt_intr_in_guest()) + kvm_guest_cbs.handle_intel_pt_intr = kvm_handle_intel_pt_intr; perf_register_guest_info_callbacks(&kvm_guest_cbs); if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES)) @@ -11091,6 +11093,7 @@ int kvm_arch_hardware_setup(void *opaque) void kvm_arch_hardware_unsetup(void) { perf_unregister_guest_info_callbacks(&kvm_guest_cbs); + kvm_guest_cbs.handle_intel_pt_intr = NULL; static_call(kvm_x86_hardware_unsetup)(); } From patchwork Sat Aug 28 00:35:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12463075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F38D6C432BE for ; Sat, 28 Aug 2021 00:40:32 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BFF9660C3E for ; Sat, 28 Aug 2021 00:40:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org BFF9660C3E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References :Mime-Version:Message-Id:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=L71rjZBkXdhFXrLDCwlRYUVWuR3x64ywECITsADEEFo=; b=Nr0jlh2w/WaE9M w/1H7tnasY6Ro9IL0kTAva8DvhPtVz3zOlBQadFBLFJii5ZIBodEYIXPO/vj8WWuOUsb+asqELjm9 WtjZZzxBwZLw/esIchzoM8mLwwC1cbT2iJwMXWXCzHoEX2UFaB5ocHQYJ/BxmtJUtdvg2300SkkU1 5lqWeweNuoKtPw5O6yRx71hCBMeilOKFEGNDwroA/b2VeLu8goiEFX2U87OieIcgML94lGb6eCIGj 1KBxDbIb630IIRxakReMYObM8BFPvnKoskGMIPXa2CVmwC/dExK57L6RMIdXBEMA/8KtLJbleTs4h y/+hZ3RM7WmkFuCmLJqw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmMN-00DRl1-M7; Sat, 28 Aug 2021 00:38:11 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmKq-00DR1W-5N for linux-arm-kernel@lists.infradead.org; Sat, 28 Aug 2021 00:36:39 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id z6-20020a257e06000000b0059bad6decfbso880718ybc.16 for ; Fri, 27 Aug 2021 17:36:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=l+qrTcHHbVOVfx3csFqTQJW+sIvVNtSUAKnqiNDseGU=; b=YeIUfDPPv26/qQ9C6zbTR90RkfDch9YFOKAlHKUjSEVddL4Hpoya9tjkJY3A1wbIBn QolRIg0F84AYiP8jYQF3PWlN+ULQv1tmI4e/BbUmbvQ35zPcx2ZQSqLKQ5AVao+1UhgO fkWFWktVzgenJwImxC38FiwFCxzMEKkS1qQUUdmj1KFZcMw9Lyq83HZeYJH/euP/cfqF 3aiAClPO6WWDoE2XklbezJvqEtsZgyUPgX+DMcdLAHn5l7sHl4CT18xkeTalHsiGr3MZ YjN79LXtHsUEDIVLT82BEQrYcZmX8jMYXoxZPFQJObu1cRhSKHjBMlVs35/os2e+0Sxl Ec7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=l+qrTcHHbVOVfx3csFqTQJW+sIvVNtSUAKnqiNDseGU=; b=JqH5w465qwPzU3t+ZlMdR8VEwUXimH64z93EAHwY5yRVWICGoMhaK9rfxxTjaXZWqQ s4EOJSdyB7HN6RFej4LqHSmoyD2bsG/+ZtiXLqZ9yo7cKtRWtSs+u5tLbStHooYsvDoF cvvV6WVi2oPjkNrmX1jkajyzdnh6JxVZ56wJkGGfM6+wpft3BZyznT0WtRPEwLPOH+Lc sOJMthPFdbWQTSBriQtt6JNNOtdJTNDVhwrj1++oXsccNdNb9O7La5W/sCgSSxVTZIk6 HSUWpvKeQV8/ul0fO2wlA+6kCYFpzaDttA6q+ueng1cOx2OPG1FbCROMhkKwlX1xTGPe BtBw== X-Gm-Message-State: AOAM533luV6ltQszLLsh85s1Ty90IBtVf8jvEjsYpzowt2+6W9M+mlO7 31zlQ50aYRNV0DvDWEoLFy2+u7dtum4= X-Google-Smtp-Source: ABdhPJx5idLVf7lCndwsU5Ce4jbb5QTi9k0wtLEJzPA1boh4/EA6RmmuRvKiVHeqlDtlM2jyABObo3ywihc= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:f66c:b851:7e79:7ed4]) (user=seanjc job=sendgmr) by 2002:a25:7cc3:: with SMTP id x186mr8484190ybc.227.1630110994660; Fri, 27 Aug 2021 17:36:34 -0700 (PDT) Date: Fri, 27 Aug 2021 17:35:49 -0700 In-Reply-To: <20210828003558.713983-1-seanjc@google.com> Message-Id: <20210828003558.713983-5-seanjc@google.com> Mime-Version: 1.0 References: <20210828003558.713983-1-seanjc@google.com> X-Mailer: git-send-email 2.33.0.259.gc128427fd7-goog Subject: [PATCH v2 04/13] perf: Stop pretending that perf can handle multiple guest callbacks From: Sean Christopherson To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Will Deacon , Mark Rutland , Catalin Marinas , Marc Zyngier , Guo Ren , Nick Hu , Greentime Hu , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , Thomas Gleixner , Borislav Petkov , x86@kernel.org, Paolo Bonzini , Boris Ostrovsky , Juergen Gross Cc: Alexander Shishkin , Jiri Olsa , Namhyung Kim , James Morse , Alexandru Elisei , Suzuki K Poulose , "H. Peter Anvin" , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Stefano Stabellini , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Artem Kashkanov , Like Xu , Zhu Lingshan X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210827_173636_242514_B25DEFCD X-CRM114-Status: GOOD ( 19.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Drop the 'int' return value from the perf (un)register callbacks helpers and stop pretending perf can support multiple callbacks. The 'int' returns are not future proofing anything as none of the callers take action on an error. It's also not obvious that there will ever be co-tenant hypervisors, and if there are, that allowing multiple callbacks to be registered is desirable or even correct. No functional change intended. Signed-off-by: Sean Christopherson --- arch/arm64/include/asm/kvm_host.h | 4 ++-- arch/arm64/kvm/perf.c | 8 ++++---- arch/x86/kvm/x86.c | 2 +- include/linux/perf_event.h | 11 +++++------ kernel/events/core.c | 14 +++----------- 5 files changed, 15 insertions(+), 24 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 41911585ae0c..ed940aec89e0 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -670,8 +670,8 @@ unsigned long kvm_mmio_read_buf(const void *buf, unsigned int len); int kvm_handle_mmio_return(struct kvm_vcpu *vcpu); int io_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa); -int kvm_perf_init(void); -int kvm_perf_teardown(void); +void kvm_perf_init(void); +void kvm_perf_teardown(void); long kvm_hypercall_pv_features(struct kvm_vcpu *vcpu); gpa_t kvm_init_stolen_time(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/perf.c b/arch/arm64/kvm/perf.c index 151c31fb9860..039fe59399a2 100644 --- a/arch/arm64/kvm/perf.c +++ b/arch/arm64/kvm/perf.c @@ -48,15 +48,15 @@ static struct perf_guest_info_callbacks kvm_guest_cbs = { .get_guest_ip = kvm_get_guest_ip, }; -int kvm_perf_init(void) +void kvm_perf_init(void) { if (kvm_pmu_probe_pmuver() != 0xf && !is_protected_kvm_enabled()) static_branch_enable(&kvm_arm_pmu_available); - return perf_register_guest_info_callbacks(&kvm_guest_cbs); + perf_register_guest_info_callbacks(&kvm_guest_cbs); } -int kvm_perf_teardown(void) +void kvm_perf_teardown(void) { - return perf_unregister_guest_info_callbacks(&kvm_guest_cbs); + perf_unregister_guest_info_callbacks(); } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index ffc6c2d73508..bae951344e28 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -11092,7 +11092,7 @@ int kvm_arch_hardware_setup(void *opaque) void kvm_arch_hardware_unsetup(void) { - perf_unregister_guest_info_callbacks(&kvm_guest_cbs); + perf_unregister_guest_info_callbacks(); kvm_guest_cbs.handle_intel_pt_intr = NULL; static_call(kvm_x86_hardware_unsetup)(); diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 6b0405e578c1..2b77e2154b61 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1245,8 +1245,8 @@ static inline struct perf_guest_info_callbacks *perf_get_guest_cbs(void) /* Prevent reloading between a !NULL check and dereferences. */ return READ_ONCE(perf_guest_cbs); } -extern int perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *callbacks); -extern int perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *callbacks); +extern void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *callbacks); +extern void perf_unregister_guest_info_callbacks(void); extern void perf_event_exec(void); extern void perf_event_comm(struct task_struct *tsk, bool exec); @@ -1489,10 +1489,9 @@ perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr) { } static inline void perf_bp_event(struct perf_event *event, void *data) { } -static inline int perf_register_guest_info_callbacks -(struct perf_guest_info_callbacks *callbacks) { return 0; } -static inline int perf_unregister_guest_info_callbacks -(struct perf_guest_info_callbacks *callbacks) { return 0; } +static inline void perf_register_guest_info_callbacks +(struct perf_guest_info_callbacks *callbacks) { } +static inline void perf_unregister_guest_info_callbacks(void) { } static inline void perf_event_mmap(struct vm_area_struct *vma) { } diff --git a/kernel/events/core.c b/kernel/events/core.c index 2126f6327321..1be95d9ace46 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -6482,29 +6482,21 @@ static void perf_pending_event(struct irq_work *entry) perf_swevent_put_recursion_context(rctx); } -/* - * We assume there is only KVM supporting the callbacks. - * Later on, we might change it to a list if there is - * another virtualization implementation supporting the callbacks. - */ struct perf_guest_info_callbacks *perf_guest_cbs; - -int perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs) +void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs) { if (WARN_ON_ONCE(perf_guest_cbs)) - return -EBUSY; + return; WRITE_ONCE(perf_guest_cbs, cbs); synchronize_rcu(); - return 0; } EXPORT_SYMBOL_GPL(perf_register_guest_info_callbacks); -int perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs) +void perf_unregister_guest_info_callbacks(void) { WRITE_ONCE(perf_guest_cbs, NULL); synchronize_rcu(); - return 0; } EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks); From patchwork Sat Aug 28 00:35:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12463081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26F2EC432BE for ; Sat, 28 Aug 2021 00:42:02 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E5C5F60ED5 for ; Sat, 28 Aug 2021 00:42:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E5C5F60ED5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References :Mime-Version:Message-Id:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vy4+M+5VHtzElIDKbKiJVeRWjfJzWs8e6vHfh74Y+tg=; b=NWNMyPBZtUqA9w WkDcM3lXbSRhfHRX8ZwUr+6ySGg9bTVnZdLIY2/FvoYTgjm/O0AsR7AJVKXX/0VHKgSQwjToC9bHB W2+BqhPkxS/BUCKiLYynFzK7JIkQqzn0/Wh3zbTx/uBzw5AEnFi0L5wRiJuRW3hquCt1gQvrtu/Qr gB0YHRICxnBka+S/Es8mAyAABskv439OEdNCYi1GtOwiZRaIJDcW4lpd8wz2ysvHTqTiUrZj82IpC uBI6xzuZwBfxwPg+LwZBmMVhMOgZiNbvl9RsH5eIS3hXqVq3/GFbF624gDwD3+ushuXPLzY13Pkno 1AF1k9RLOx/7m0r5L0TA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmOE-00DSgS-3v; Sat, 28 Aug 2021 00:40:07 +0000 Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmKt-00DR3o-Fh for linux-arm-kernel@lists.infradead.org; Sat, 28 Aug 2021 00:36:44 +0000 Received: by mail-qt1-x849.google.com with SMTP id k3-20020ac80203000000b0029f2c3c6b34so519744qtg.4 for ; Fri, 27 Aug 2021 17:36:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=KfNIEWB+/soEzHjyrFkspDQOOy4Avf/K1hvEC4YdhO0=; b=etOlWN2jXHZScPROjSWHQA1MNEG1/zyCCB37FYjhRMZAupHKwl05Vuz0C7AWyzfha4 d2CCq9F9mL5lN5ykDeErVjB/ZNBbEiFy8d2kGuqW1+ug/RYcdPXB4sbWTrSfjXO90R4z 73zXeM2wXaRrf99wFnbbFW707nrO0547RUqb8hl5FW1iu84mA9RxJ1PVbKMclW8cbNS+ WRreCLkLh5vjw0dtJr9As1V0HfQmGbO3x82w3wplaPA3dNmzXIGJZNO4zpNc6Qil7pKZ qM8HyMDmY8Eag7Hk9WOcaSNcFKLIK0Ql4Boc8KUl6REYnjOOtwXg4iC0XEbDCbpOh6EH HTWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=KfNIEWB+/soEzHjyrFkspDQOOy4Avf/K1hvEC4YdhO0=; b=G1eCH9VKANcOHKMauoDkc9+UxussKMGyAHBJZCJts5d0/Vk50ZufMfVWJg1TCfbkGh av6gPcSMP2D3xvb2rPuGCEgDvCivaaNrYsncAeegQXXGHuhRSVVybmtXkaxfqZJFog8d VLolqkZN2suEwOwnN7MdcM309i/Ovef75ahLUAldPiCpgJCkI3Wncj5IHjNrKNGdozXK tTKMGKlN8J3qd3AI82YQUFhHMTRoY+ZM9kHymotSpbW+w7c16mWmFySpl3SBouAuEgcr 9rB0Az8JEVyOLO8E6pRkniuJdNmIWJ/I3Gzptl3YmIUgVdBbXiiKrJ70XUejQPSkYCWJ GxcQ== X-Gm-Message-State: AOAM531rM9MpIwdFf9wxfCgrwQd4w+MgW9vRO9+nYV2hQFONEvTpc6R1 YW5ZXzmHBbjsPRjsU/tjaZR7ce0bFG8= X-Google-Smtp-Source: ABdhPJy5x+uxZDfRklRUjRTGMvNBMlTUvIpJQ5+ZJo6EJZI2mP41SXSUQuik9OAwPT5faDa6Pg/AO3ViM5g= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:f66c:b851:7e79:7ed4]) (user=seanjc job=sendgmr) by 2002:a05:6214:21cc:: with SMTP id d12mr12369768qvh.22.1630110997003; Fri, 27 Aug 2021 17:36:37 -0700 (PDT) Date: Fri, 27 Aug 2021 17:35:50 -0700 In-Reply-To: <20210828003558.713983-1-seanjc@google.com> Message-Id: <20210828003558.713983-6-seanjc@google.com> Mime-Version: 1.0 References: <20210828003558.713983-1-seanjc@google.com> X-Mailer: git-send-email 2.33.0.259.gc128427fd7-goog Subject: [PATCH v2 05/13] perf: Force architectures to opt-in to guest callbacks From: Sean Christopherson To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Will Deacon , Mark Rutland , Catalin Marinas , Marc Zyngier , Guo Ren , Nick Hu , Greentime Hu , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , Thomas Gleixner , Borislav Petkov , x86@kernel.org, Paolo Bonzini , Boris Ostrovsky , Juergen Gross Cc: Alexander Shishkin , Jiri Olsa , Namhyung Kim , James Morse , Alexandru Elisei , Suzuki K Poulose , "H. Peter Anvin" , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Stefano Stabellini , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Artem Kashkanov , Like Xu , Zhu Lingshan X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210827_173639_575118_704ECFF9 X-CRM114-Status: GOOD ( 20.61 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce HAVE_GUEST_PERF_EVENTS and require architectures to select it to allow registering guest callbacks in perf. Future patches will convert the callbacks to static_call. Rather than churn a bunch of arch code (that was presumably copy+pasted from x86), remove it wholesale as it's useless and at best wasting cycles. Wrap even the stubs with an #ifdef to avoid an arch sneaking in a bogus registration with CONFIG_PERF_EVENTS=n. Signed-off-by: Sean Christopherson --- arch/arm/kernel/perf_callchain.c | 33 ++++------------------------- arch/arm64/Kconfig | 1 + arch/csky/kernel/perf_callchain.c | 12 ----------- arch/nds32/kernel/perf_event_cpu.c | 34 ++++-------------------------- arch/riscv/kernel/perf_callchain.c | 13 ------------ arch/x86/Kconfig | 1 + include/linux/perf_event.h | 4 ++++ init/Kconfig | 3 +++ kernel/events/core.c | 2 ++ 9 files changed, 19 insertions(+), 84 deletions(-) diff --git a/arch/arm/kernel/perf_callchain.c b/arch/arm/kernel/perf_callchain.c index 1626dfc6f6ce..bc6b246ab55e 100644 --- a/arch/arm/kernel/perf_callchain.c +++ b/arch/arm/kernel/perf_callchain.c @@ -62,14 +62,8 @@ user_backtrace(struct frame_tail __user *tail, void perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); struct frame_tail __user *tail; - if (guest_cbs && guest_cbs->is_in_guest()) { - /* We don't support guest os callchain now */ - return; - } - perf_callchain_store(entry, regs->ARM_pc); if (!current->mm) @@ -99,44 +93,25 @@ callchain_trace(struct stackframe *fr, void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); struct stackframe fr; - if (guest_cbs && guest_cbs->is_in_guest()) { - /* We don't support guest os callchain now */ - return; - } - arm_get_current_stackframe(regs, &fr); walk_stackframe(&fr, callchain_trace, entry); } unsigned long perf_instruction_pointer(struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); - - if (guest_cbs && guest_cbs->is_in_guest()) - return guest_cbs->get_guest_ip(); - return instruction_pointer(regs); } unsigned long perf_misc_flags(struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); int misc = 0; - if (guest_cbs && guest_cbs->is_in_guest()) { - if (guest_cbs->is_user_mode()) - misc |= PERF_RECORD_MISC_GUEST_USER; - else - misc |= PERF_RECORD_MISC_GUEST_KERNEL; - } else { - if (user_mode(regs)) - misc |= PERF_RECORD_MISC_USER; - else - misc |= PERF_RECORD_MISC_KERNEL; - } + if (user_mode(regs)) + misc |= PERF_RECORD_MISC_USER; + else + misc |= PERF_RECORD_MISC_KERNEL; return misc; } diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index b5b13a932561..72a201a686c5 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -190,6 +190,7 @@ config ARM64 select HAVE_NMI select HAVE_PATA_PLATFORM select HAVE_PERF_EVENTS + select HAVE_GUEST_PERF_EVENTS select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP select HAVE_REGS_AND_STACK_ACCESS_API diff --git a/arch/csky/kernel/perf_callchain.c b/arch/csky/kernel/perf_callchain.c index 35318a635a5f..92057de08f4f 100644 --- a/arch/csky/kernel/perf_callchain.c +++ b/arch/csky/kernel/perf_callchain.c @@ -86,13 +86,8 @@ static unsigned long user_backtrace(struct perf_callchain_entry_ctx *entry, void perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); unsigned long fp = 0; - /* C-SKY does not support virtualization. */ - if (guest_cbs && guest_cbs->is_in_guest()) - return; - fp = regs->regs[4]; perf_callchain_store(entry, regs->pc); @@ -111,15 +106,8 @@ void perf_callchain_user(struct perf_callchain_entry_ctx *entry, void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); struct stackframe fr; - /* C-SKY does not support virtualization. */ - if (guest_cbs && guest_cbs->is_in_guest()) { - pr_warn("C-SKY does not support perf in guest mode!"); - return; - } - fr.fp = regs->regs[4]; fr.lr = regs->lr; walk_stackframe(&fr, entry); diff --git a/arch/nds32/kernel/perf_event_cpu.c b/arch/nds32/kernel/perf_event_cpu.c index f38791960781..a78a879e7ef1 100644 --- a/arch/nds32/kernel/perf_event_cpu.c +++ b/arch/nds32/kernel/perf_event_cpu.c @@ -1363,7 +1363,6 @@ void perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); unsigned long fp = 0; unsigned long gp = 0; unsigned long lp = 0; @@ -1372,11 +1371,6 @@ perf_callchain_user(struct perf_callchain_entry_ctx *entry, leaf_fp = 0; - if (guest_cbs && guest_cbs->is_in_guest()) { - /* We don't support guest os callchain now */ - return; - } - perf_callchain_store(entry, regs->ipc); fp = regs->fp; gp = regs->gp; @@ -1480,13 +1474,8 @@ void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); struct stackframe fr; - if (guest_cbs && guest_cbs->is_in_guest()) { - /* We don't support guest os callchain now */ - return; - } fr.fp = regs->fp; fr.lp = regs->lp; fr.sp = regs->sp; @@ -1495,32 +1484,17 @@ perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, unsigned long perf_instruction_pointer(struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); - - /* However, NDS32 does not support virtualization */ - if (guest_cbs && guest_cbs->is_in_guest()) - return guest_cbs->get_guest_ip(); - return instruction_pointer(regs); } unsigned long perf_misc_flags(struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); int misc = 0; - /* However, NDS32 does not support virtualization */ - if (guest_cbs && guest_cbs->is_in_guest()) { - if (guest_cbs->is_user_mode()) - misc |= PERF_RECORD_MISC_GUEST_USER; - else - misc |= PERF_RECORD_MISC_GUEST_KERNEL; - } else { - if (user_mode(regs)) - misc |= PERF_RECORD_MISC_USER; - else - misc |= PERF_RECORD_MISC_KERNEL; - } + if (user_mode(regs)) + misc |= PERF_RECORD_MISC_USER; + else + misc |= PERF_RECORD_MISC_KERNEL; return misc; } diff --git a/arch/riscv/kernel/perf_callchain.c b/arch/riscv/kernel/perf_callchain.c index 8ecfc4c128bc..1fc075b8f764 100644 --- a/arch/riscv/kernel/perf_callchain.c +++ b/arch/riscv/kernel/perf_callchain.c @@ -56,13 +56,8 @@ static unsigned long user_backtrace(struct perf_callchain_entry_ctx *entry, void perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); unsigned long fp = 0; - /* RISC-V does not support perf in guest mode. */ - if (guest_cbs && guest_cbs->is_in_guest()) - return; - fp = regs->s0; perf_callchain_store(entry, regs->epc); @@ -79,13 +74,5 @@ static bool fill_callchain(void *entry, unsigned long pc) void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); - - /* RISC-V does not support perf in guest mode. */ - if (guest_cbs && guest_cbs->is_in_guest()) { - pr_warn("RISC-V does not support perf in guest mode!"); - return; - } - walk_stackframe(NULL, regs, fill_callchain, entry); } diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 49270655e827..a4de4aa7a3df 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -229,6 +229,7 @@ config X86 select HAVE_PERF_EVENTS select HAVE_PERF_EVENTS_NMI select HAVE_HARDLOCKUP_DETECTOR_PERF if PERF_EVENTS && HAVE_PERF_EVENTS_NMI + select HAVE_GUEST_PERF_EVENTS select HAVE_PCI select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 2b77e2154b61..e75971f85cc8 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1236,6 +1236,7 @@ extern void perf_event_bpf_event(struct bpf_prog *prog, enum perf_bpf_event_type type, u16 flags); +#ifdef CONFIG_HAVE_GUEST_PERF_EVENTS extern struct perf_guest_info_callbacks *perf_guest_cbs; static inline struct perf_guest_info_callbacks *perf_get_guest_cbs(void) { @@ -1247,6 +1248,7 @@ static inline struct perf_guest_info_callbacks *perf_get_guest_cbs(void) } extern void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *callbacks); extern void perf_unregister_guest_info_callbacks(void); +#endif /* CONFIG_HAVE_GUEST_PERF_EVENTS */ extern void perf_event_exec(void); extern void perf_event_comm(struct task_struct *tsk, bool exec); @@ -1489,9 +1491,11 @@ perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr) { } static inline void perf_bp_event(struct perf_event *event, void *data) { } +#ifdef CONFIG_HAVE_GUEST_PERF_EVENTS static inline void perf_register_guest_info_callbacks (struct perf_guest_info_callbacks *callbacks) { } static inline void perf_unregister_guest_info_callbacks(void) { } +#endif static inline void perf_event_mmap(struct vm_area_struct *vma) { } diff --git a/init/Kconfig b/init/Kconfig index 55f9f7738ebb..9ef51ae53977 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1776,6 +1776,9 @@ config HAVE_PERF_EVENTS help See tools/perf/design.txt for details. +config HAVE_GUEST_PERF_EVENTS + bool + config PERF_USE_VMALLOC bool help diff --git a/kernel/events/core.c b/kernel/events/core.c index 1be95d9ace46..d7f606e06446 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -6482,6 +6482,7 @@ static void perf_pending_event(struct irq_work *entry) perf_swevent_put_recursion_context(rctx); } +#ifdef CONFIG_HAVE_GUEST_PERF_EVENTS struct perf_guest_info_callbacks *perf_guest_cbs; void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs) { @@ -6499,6 +6500,7 @@ void perf_unregister_guest_info_callbacks(void) synchronize_rcu(); } EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks); +#endif static void perf_output_sample_regs(struct perf_output_handle *handle, From patchwork Sat Aug 28 00:35:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12463079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF118C432BE for ; Sat, 28 Aug 2021 00:41:04 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8BBEE60EBD for ; Sat, 28 Aug 2021 00:41:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 8BBEE60EBD Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References :Mime-Version:Message-Id:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=l4HCtW/J+QKBA/5jttwhhgGh0JeAVVtFO6gyU81HMRk=; b=Byj6Fio0/x9EBT Sb/39fZByaWGGJNHtA0GQ5uU9MlHwtYkFUHCvcZdkah3NfykiERoiihQKw5rQMdB57BUtWOWvU54/ MjFz6qy0Eq9xr7pWpzRDN7pH0RXnMzslMH7zPssM90bx/qvf3dgql0NLfRdtTyZrDL3LOIW/LBKvb MybQYkMOqsvWElGxLBotBoBvh6TjQWlO+eQ/GBbVqpJYuuKHxHJ25rStST95AMb3nsQUcKZKBzIKh 0RJBuIxEKOblRoslKz8SsF9Hw1n1eNSZ7eMRQLS/Y7FO3+XQNSfu/CClTY9/j60tlVCDlwGylEQOA srKouTnCbq4BQlvOSMyA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmNA-00DS59-5H; Sat, 28 Aug 2021 00:39:02 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmKv-00DR4R-2P for linux-arm-kernel@lists.infradead.org; Sat, 28 Aug 2021 00:36:43 +0000 Received: by mail-yb1-xb49.google.com with SMTP id a62-20020a254d410000b0290592f360b0ccso8242205ybb.14 for ; Fri, 27 Aug 2021 17:36:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=0YXYNwfO2nP0d/8tcUlJOlRfQipYSfj6KR9im5qFSzU=; b=M/Xe0rMtGmxdkWl9mi5EOW9t7UlutI0HqYiemomWjM6+KLBUeVT3ObS3xgkFOcpBHA tjEreWTZww/GaibpuFp3TP6uJw/XXrbB7rgTViYSk8FfdflF+cKbmXPkirpzaUgFSwuF vLlNrQEp4R0vbB3TstwUqA/I7NNS4Ui2hk2YFYsQ9QF7i+W+jF4GnB9pW6MAQRukcU/q md3KD2Uem7PScXCJD2QVRXMKOLsNPxZapGXO+7H3E2dqptEoTOpWEpWO/03jkHSd2XRu P5sFFQ6xUG9CInfC2TWxYV5PcoFZlV4zv4dllrmT/FnIzl+/h7hBbAB8LVIwt89dViD6 wzQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=0YXYNwfO2nP0d/8tcUlJOlRfQipYSfj6KR9im5qFSzU=; b=o9WZEWJcETrmt8yPCFY5CeijhjMyP1yaVb+I7soW3wSNbRZKBgafaOMNIMP8TJExi4 N5/h5LGqj3R7UwQWmo8xhLWsi61R0KHmkYpeUlNdRKD6kRPyd7xSuRfj6qJ0JCADOiSd +YSCznFagLhh/pPs9GoiJeD24hZT1qh34OLUFY3GxCI7+Al19Ld5AUGlErJ5TLe6V9sC qvH/JFz7koAY7V5YE2nFoGxgWMIN2R06QZGxyYwzBQeI750dK994t6bi+nK3/IpwftiZ r1iN+Hr76NJJaKKCu5nn/0ykhlPV7SY37zLPPXNLgrAeTT59lQwOhKLC/DQBgwXbs6mv qiMw== X-Gm-Message-State: AOAM532ZKuAsHh9olWRDRF13kWP0VdmoliZAdfg1EZ4pK+N4PBfL6TWH v/75NxrtNVHN6Vv9r4jigbcNkuvN/qs= X-Google-Smtp-Source: ABdhPJx7+h99mRUNQ5W6YGQDjwrL7hApQN1LKnrNBAh3ncOOeOsVF2BhDlqAamjvA/RKoXSaOf42ZGz737c= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:f66c:b851:7e79:7ed4]) (user=seanjc job=sendgmr) by 2002:a25:6545:: with SMTP id z66mr9189208ybb.149.1630110999440; Fri, 27 Aug 2021 17:36:39 -0700 (PDT) Date: Fri, 27 Aug 2021 17:35:51 -0700 In-Reply-To: <20210828003558.713983-1-seanjc@google.com> Message-Id: <20210828003558.713983-7-seanjc@google.com> Mime-Version: 1.0 References: <20210828003558.713983-1-seanjc@google.com> X-Mailer: git-send-email 2.33.0.259.gc128427fd7-goog Subject: [PATCH v2 06/13] perf/core: Rework guest callbacks to prepare for static_call support From: Sean Christopherson To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Will Deacon , Mark Rutland , Catalin Marinas , Marc Zyngier , Guo Ren , Nick Hu , Greentime Hu , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , Thomas Gleixner , Borislav Petkov , x86@kernel.org, Paolo Bonzini , Boris Ostrovsky , Juergen Gross Cc: Alexander Shishkin , Jiri Olsa , Namhyung Kim , James Morse , Alexandru Elisei , Suzuki K Poulose , "H. Peter Anvin" , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Stefano Stabellini , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Artem Kashkanov , Like Xu , Zhu Lingshan X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210827_173641_204390_6278D4FD X-CRM114-Status: GOOD ( 24.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Like Xu To prepare for using static_calls to optimize perf's guest callbacks, replace ->is_in_guest and ->is_user_mode with a new multiplexed hook ->state, tweak ->handle_intel_pt_intr to play nice with being called when there is no active guest, and drop "guest" from ->is_in_guest. Return '0' from ->state and ->handle_intel_pt_intr to indicate "not in guest" so that DEFINE_STATIC_CALL_RET0 can be used to define the static calls, i.e. no callback == !guest. Suggested-by: Peter Zijlstra (Intel) Originally-by: Peter Zijlstra (Intel) Signed-off-by: Like Xu Signed-off-by: Zhu Lingshan [sean: extracted from static_call patch, fixed get_ip() bug, wrote changelog] Signed-off-by: Sean Christopherson --- arch/arm64/kernel/perf_callchain.c | 13 +++++----- arch/arm64/kvm/perf.c | 35 +++++++++++--------------- arch/x86/events/core.c | 13 +++++----- arch/x86/events/intel/core.c | 5 +--- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/pmu.c | 2 +- arch/x86/kvm/x86.c | 40 ++++++++++++++++-------------- arch/x86/xen/pmu.c | 32 ++++++++++-------------- include/linux/perf_event.h | 10 +++++--- kernel/events/core.c | 1 + 10 files changed, 74 insertions(+), 79 deletions(-) diff --git a/arch/arm64/kernel/perf_callchain.c b/arch/arm64/kernel/perf_callchain.c index 86d9f2013172..274dc3e11b6d 100644 --- a/arch/arm64/kernel/perf_callchain.c +++ b/arch/arm64/kernel/perf_callchain.c @@ -104,7 +104,7 @@ void perf_callchain_user(struct perf_callchain_entry_ctx *entry, { struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); - if (guest_cbs && guest_cbs->is_in_guest()) { + if (guest_cbs && guest_cbs->state()) { /* We don't support guest os callchain now */ return; } @@ -152,7 +152,7 @@ void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); struct stackframe frame; - if (guest_cbs && guest_cbs->is_in_guest()) { + if (guest_cbs && guest_cbs->state()) { /* We don't support guest os callchain now */ return; } @@ -165,8 +165,8 @@ unsigned long perf_instruction_pointer(struct pt_regs *regs) { struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); - if (guest_cbs && guest_cbs->is_in_guest()) - return guest_cbs->get_guest_ip(); + if (guest_cbs && guest_cbs->state()) + return guest_cbs->get_ip(); return instruction_pointer(regs); } @@ -174,10 +174,11 @@ unsigned long perf_instruction_pointer(struct pt_regs *regs) unsigned long perf_misc_flags(struct pt_regs *regs) { struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); + unsigned int guest_state = guest_cbs ? guest_cbs->state() : 0; int misc = 0; - if (guest_cbs && guest_cbs->is_in_guest()) { - if (guest_cbs->is_user_mode()) + if (guest_state) { + if (guest_state & PERF_GUEST_USER) misc |= PERF_RECORD_MISC_GUEST_USER; else misc |= PERF_RECORD_MISC_GUEST_KERNEL; diff --git a/arch/arm64/kvm/perf.c b/arch/arm64/kvm/perf.c index 039fe59399a2..893de1a51fea 100644 --- a/arch/arm64/kvm/perf.c +++ b/arch/arm64/kvm/perf.c @@ -13,39 +13,34 @@ DEFINE_STATIC_KEY_FALSE(kvm_arm_pmu_available); -static int kvm_is_in_guest(void) +static unsigned int kvm_guest_state(void) { - return kvm_get_running_vcpu() != NULL; -} + struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); + unsigned int state; -static int kvm_is_user_mode(void) -{ - struct kvm_vcpu *vcpu; - - vcpu = kvm_get_running_vcpu(); + if (!vcpu) + return 0; - if (vcpu) - return !vcpu_mode_priv(vcpu); + state = PERF_GUEST_ACTIVE; + if (!vcpu_mode_priv(vcpu)) + state |= PERF_GUEST_USER; - return 0; + return state; } static unsigned long kvm_get_guest_ip(void) { - struct kvm_vcpu *vcpu; + struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); - vcpu = kvm_get_running_vcpu(); + if (WARN_ON_ONCE(!vcpu)) + return 0; - if (vcpu) - return *vcpu_pc(vcpu); - - return 0; + return *vcpu_pc(vcpu); } static struct perf_guest_info_callbacks kvm_guest_cbs = { - .is_in_guest = kvm_is_in_guest, - .is_user_mode = kvm_is_user_mode, - .get_guest_ip = kvm_get_guest_ip, + .state = kvm_guest_state, + .get_ip = kvm_get_guest_ip, }; void kvm_perf_init(void) diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index ffb3e6c0d367..3a7630fdd340 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -2765,7 +2765,7 @@ perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *re struct unwind_state state; unsigned long addr; - if (guest_cbs && guest_cbs->is_in_guest()) { + if (guest_cbs && guest_cbs->state()) { /* TODO: We don't support guest os callchain now */ return; } @@ -2869,7 +2869,7 @@ perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs struct stack_frame frame; const struct stack_frame __user *fp; - if (guest_cbs && guest_cbs->is_in_guest()) { + if (guest_cbs && guest_cbs->state()) { /* TODO: We don't support guest os callchain now */ return; } @@ -2948,8 +2948,8 @@ unsigned long perf_instruction_pointer(struct pt_regs *regs) { struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); - if (guest_cbs && guest_cbs->is_in_guest()) - return guest_cbs->get_guest_ip(); + if (guest_cbs && guest_cbs->state()) + return guest_cbs->get_ip(); return regs->ip + code_segment_base(regs); } @@ -2957,10 +2957,11 @@ unsigned long perf_instruction_pointer(struct pt_regs *regs) unsigned long perf_misc_flags(struct pt_regs *regs) { struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); + unsigned int guest_state = guest_cbs ? guest_cbs->state() : 0; int misc = 0; - if (guest_cbs && guest_cbs->is_in_guest()) { - if (guest_cbs->is_user_mode()) + if (guest_state) { + if (guest_state & PERF_GUEST_USER) misc |= PERF_RECORD_MISC_GUEST_USER; else misc |= PERF_RECORD_MISC_GUEST_KERNEL; diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index 9baa46185d94..524ad1f747bd 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -2855,10 +2855,7 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status) handled++; guest_cbs = perf_get_guest_cbs(); - if (unlikely(guest_cbs && guest_cbs->is_in_guest() && - guest_cbs->handle_intel_pt_intr)) - guest_cbs->handle_intel_pt_intr(); - else + if (likely(!guest_cbs || !guest_cbs->handle_intel_pt_intr())) intel_pt_interrupt(); } diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 1ea4943a73d7..1080166fc0cf 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1874,7 +1874,7 @@ int kvm_skip_emulated_instruction(struct kvm_vcpu *vcpu); int kvm_complete_insn_gp(struct kvm_vcpu *vcpu, int err); void __kvm_request_immediate_exit(struct kvm_vcpu *vcpu); -int kvm_is_in_guest(void); +unsigned int kvm_guest_state(void); void __user *__x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size); diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 0772bad9165c..5b68d4188de0 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -87,7 +87,7 @@ static void kvm_perf_overflow_intr(struct perf_event *perf_event, * woken up. So we should wake it, but this is impossible from * NMI context. Do it from irq work instead. */ - if (!kvm_is_in_guest()) + if (!kvm_guest_state()) irq_work_queue(&pmc_to_pmu(pmc)->irq_work); else kvm_make_request(KVM_REQ_PMI, pmc->vcpu); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index bae951344e28..b2a4d085aa4f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8267,44 +8267,48 @@ static void kvm_timer_init(void) DEFINE_PER_CPU(struct kvm_vcpu *, current_vcpu); EXPORT_PER_CPU_SYMBOL_GPL(current_vcpu); -int kvm_is_in_guest(void) +unsigned int kvm_guest_state(void) { - return __this_cpu_read(current_vcpu) != NULL; -} + struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu); + unsigned int state; -static int kvm_is_user_mode(void) -{ - int user_mode = 3; + if (!vcpu) + return 0; - if (__this_cpu_read(current_vcpu)) - user_mode = static_call(kvm_x86_get_cpl)(__this_cpu_read(current_vcpu)); + state = PERF_GUEST_ACTIVE; + if (static_call(kvm_x86_get_cpl)(vcpu)) + state |= PERF_GUEST_USER; - return user_mode != 0; + return state; } -static unsigned long kvm_get_guest_ip(void) +static unsigned long kvm_guest_get_ip(void) { - unsigned long ip = 0; + struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu); - if (__this_cpu_read(current_vcpu)) - ip = kvm_rip_read(__this_cpu_read(current_vcpu)); + if (WARN_ON_ONCE(!vcpu)) + return 0; - return ip; + return kvm_rip_read(vcpu); } -static void kvm_handle_intel_pt_intr(void) +static unsigned int kvm_handle_intel_pt_intr(void) { struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu); + /* '0' on failure so that the !PT case can use a RET0 static call. */ + if (!vcpu) + return 0; + kvm_make_request(KVM_REQ_PMI, vcpu); __set_bit(MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT, (unsigned long *)&vcpu->arch.pmu.global_status); + return 1; } static struct perf_guest_info_callbacks kvm_guest_cbs = { - .is_in_guest = kvm_is_in_guest, - .is_user_mode = kvm_is_user_mode, - .get_guest_ip = kvm_get_guest_ip, + .state = kvm_guest_state, + .get_ip = kvm_guest_get_ip, .handle_intel_pt_intr = NULL, }; diff --git a/arch/x86/xen/pmu.c b/arch/x86/xen/pmu.c index e13b0b49fcdf..89dd6b1708b0 100644 --- a/arch/x86/xen/pmu.c +++ b/arch/x86/xen/pmu.c @@ -413,34 +413,29 @@ int pmu_apic_update(uint32_t val) } /* perf callbacks */ -static int xen_is_in_guest(void) +static unsigned int xen_guest_state(void) { const struct xen_pmu_data *xenpmu_data = get_xenpmu_data(); + unsigned int state = 0; if (!xenpmu_data) { pr_warn_once("%s: pmudata not initialized\n", __func__); - return 0; + return state; } if (!xen_initial_domain() || (xenpmu_data->domain_id >= DOMID_SELF)) - return 0; + return state; - return 1; -} + state |= PERF_GUEST_ACTIVE; -static int xen_is_user_mode(void) -{ - const struct xen_pmu_data *xenpmu_data = get_xenpmu_data(); - - if (!xenpmu_data) { - pr_warn_once("%s: pmudata not initialized\n", __func__); - return 0; + if (xenpmu_data->pmu.pmu_flags & PMU_SAMPLE_PV) { + if (xenpmu_data->pmu.pmu_flags & PMU_SAMPLE_USER) + state |= PERF_GUEST_USER; + } else if (xenpmu_data->pmu.r.regs.cpl & 3) { + state |= PERF_GUEST_USER; } - if (xenpmu_data->pmu.pmu_flags & PMU_SAMPLE_PV) - return (xenpmu_data->pmu.pmu_flags & PMU_SAMPLE_USER); - else - return !!(xenpmu_data->pmu.r.regs.cpl & 3); + return state; } static unsigned long xen_get_guest_ip(void) @@ -456,9 +451,8 @@ static unsigned long xen_get_guest_ip(void) } static struct perf_guest_info_callbacks xen_guest_cbs = { - .is_in_guest = xen_is_in_guest, - .is_user_mode = xen_is_user_mode, - .get_guest_ip = xen_get_guest_ip, + .state = xen_guest_state, + .get_ip = xen_get_guest_ip, }; /* Convert registers from Xen's format to Linux' */ diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index e75971f85cc8..a5d5893b80b0 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -26,11 +26,13 @@ # include #endif +#define PERF_GUEST_ACTIVE 0x01 +#define PERF_GUEST_USER 0x02 + struct perf_guest_info_callbacks { - int (*is_in_guest)(void); - int (*is_user_mode)(void); - unsigned long (*get_guest_ip)(void); - void (*handle_intel_pt_intr)(void); + unsigned int (*state)(void); + unsigned long (*get_ip)(void); + unsigned int (*handle_intel_pt_intr)(void); }; #ifdef CONFIG_HAVE_HW_BREAKPOINT diff --git a/kernel/events/core.c b/kernel/events/core.c index d7f606e06446..ec36e7aded89 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -6484,6 +6484,7 @@ static void perf_pending_event(struct irq_work *entry) #ifdef CONFIG_HAVE_GUEST_PERF_EVENTS struct perf_guest_info_callbacks *perf_guest_cbs; + void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs) { if (WARN_ON_ONCE(perf_guest_cbs)) From patchwork Sat Aug 28 00:35:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12463083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6AFCC432BE for ; Sat, 28 Aug 2021 00:42:58 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8572D60F14 for ; Sat, 28 Aug 2021 00:42:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 8572D60F14 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References :Mime-Version:Message-Id:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+xgj7zL8Jxsi8auBX3TFe17V835bjnEGpSnAXKfYLZY=; b=5BF3hJNUxdYDLu 6/JIgZHddRfEH+RqEqjB5MDWZDOfAWX0j+wjBtOnAA8BKX/JIHDQ3FjdPwZa2L4AAA3zUtuvOEp6O gdJOszE5KkPj4HAVtO9eMxwXk7tje40nTG3CyvzmgwYEH3aDodoaeoecq/tzL2aUrQG6Osjh3NBmK J/i1GMVR8te7EUaVwneQd+fmZ8WkVIMw5/mbjbVU2b9+AaCvTxEl/TvyAhLUydepQ9bP4fJxSVQ7x tEkGXLFv6TU81GxIobgsBZX45OzggSoHzFktJZYzoVJ4EvW8zYfx/qvgjlNzRaXywebPgT7tzXxD7 co0d/kyzPL7EzibB0FPA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmPB-00DT6b-UM; Sat, 28 Aug 2021 00:41:06 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmKx-00DR5l-2E for linux-arm-kernel@lists.infradead.org; Sat, 28 Aug 2021 00:36:47 +0000 Received: by mail-yb1-xb49.google.com with SMTP id q9-20020a252a09000000b0059b9b61f6ceso8281505ybq.19 for ; Fri, 27 Aug 2021 17:36:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=HKJZFfDLXh82oPfU5c20Gn43VIGGnboNJpddzA+BkHg=; b=rxzaCm/AN4VZGThWPRoQD+RNM2EHTsKr0Ts1TI5RsbpW5tPvu4XmXZqedCw9N2UdfD 4BaICFECH3j4loy5TpRObFTxBfrUdbBWvTHBjr6GJLVbonFbfguEMCgYX3LAy00BMX8W u4zq+GNNnFkS61eZe7VYyzsuyilBvIa/AmriXFxKDtOyV9SB/RH39B3Z4ZRtQLKCn0LG 75GCLxiyV2Wjkp5NjbGKf221E276nnqTHN+nKLi3sBSe656AbgGBtm5gJBcoRQ+b28Fh EFWEQESVDIOvAnq6i3fM/hFe7419BHD3Hwa7ggBFCXbKuNGTahW2VLC+3ni2ePvL+UNy zQvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=HKJZFfDLXh82oPfU5c20Gn43VIGGnboNJpddzA+BkHg=; b=L8uUrw5+UtL0/Y+yAnh98ty6wKQR4HR3L0A75Kw26LvuIu9uGl3Z1eHXm8HJ8zoxmU oRjNwicl6waO7ZgXeICzdPqiuouDapQDmzy4PLOn/5uZAWUtSbbmxdC+Lv4l1iXxzwOy yg4Ot3rqCXu+WVok8eq9ZVLrf+V3GE8MYNUjlx6fQ4pnSZ78kXw50mwm9DBewiaPQNS8 O9KzbdYAGC9oNZG77Bz13JM+1KzUL1vb3+N06GyFf5Lvaz74KQp+pChYn4DzAQ0qK0Li IqdIu6yvGrP9hB177mq0w3OaE2MUeDWklgWixfbZNYJzbvhonoHioMqFrOX8elE4kCQ8 W5Rw== X-Gm-Message-State: AOAM530oD+SJxzt/zA4jdpzkCHZOiv7cn/GoyMY5E6CCcKQ0+H9OnYEb gXxOPLflj3dXOWMWtZFe8YsDBgOmh8Y= X-Google-Smtp-Source: ABdhPJyKixaYStMukhYbHbK555CGQM6iYXLxxdSAEvoAp4FLIZA18987uVI34E4+zGb67g6x4Ia3ty7kWs8= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:f66c:b851:7e79:7ed4]) (user=seanjc job=sendgmr) by 2002:a25:9ac6:: with SMTP id t6mr8860851ybo.190.1630111001657; Fri, 27 Aug 2021 17:36:41 -0700 (PDT) Date: Fri, 27 Aug 2021 17:35:52 -0700 In-Reply-To: <20210828003558.713983-1-seanjc@google.com> Message-Id: <20210828003558.713983-8-seanjc@google.com> Mime-Version: 1.0 References: <20210828003558.713983-1-seanjc@google.com> X-Mailer: git-send-email 2.33.0.259.gc128427fd7-goog Subject: [PATCH v2 07/13] perf/core: Use static_call to optimize perf_guest_info_callbacks From: Sean Christopherson To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Will Deacon , Mark Rutland , Catalin Marinas , Marc Zyngier , Guo Ren , Nick Hu , Greentime Hu , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , Thomas Gleixner , Borislav Petkov , x86@kernel.org, Paolo Bonzini , Boris Ostrovsky , Juergen Gross Cc: Alexander Shishkin , Jiri Olsa , Namhyung Kim , James Morse , Alexandru Elisei , Suzuki K Poulose , "H. Peter Anvin" , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Stefano Stabellini , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Artem Kashkanov , Like Xu , Zhu Lingshan X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210827_173643_178382_AE5A063B X-CRM114-Status: GOOD ( 22.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Like Xu Use static_call to optimize perf's guest callbacks on arm64 and x86, which are now the only architectures that define the callbacks. Use DEFINE_STATIC_CALL_RET0 as the default/NULL for all guest callbacks, as the callback semantics are that a return value '0' means "not in guest". static_call obviously avoids the overhead of CONFIG_RETPOLINE=y, but is also advantageous versus other solutions, e.g. per-cpu callbacks, in that a per-cpu memory load is not needed to detect the !guest case. Suggested-by: Peter Zijlstra (Intel) Originally-by: Peter Zijlstra (Intel) Signed-off-by: Like Xu Signed-off-by: Zhu Lingshan [sean: split out patch, drop __weak, tweak updaters, rewrite changelog] Signed-off-by: Sean Christopherson --- arch/arm64/kernel/perf_callchain.c | 31 +++++++++++++++--------- arch/x86/events/core.c | 38 ++++++++++++++++++++++-------- arch/x86/events/intel/core.c | 7 +++--- include/linux/perf_event.h | 9 +------ kernel/events/core.c | 2 ++ 5 files changed, 54 insertions(+), 33 deletions(-) diff --git a/arch/arm64/kernel/perf_callchain.c b/arch/arm64/kernel/perf_callchain.c index 274dc3e11b6d..18cf6e608778 100644 --- a/arch/arm64/kernel/perf_callchain.c +++ b/arch/arm64/kernel/perf_callchain.c @@ -5,6 +5,7 @@ * Copyright (C) 2015 ARM Limited */ #include +#include #include #include @@ -99,12 +100,24 @@ compat_user_backtrace(struct compat_frame_tail __user *tail, } #endif /* CONFIG_COMPAT */ +DEFINE_STATIC_CALL_RET0(arm64_guest_state, *(perf_guest_cbs->state)); +DEFINE_STATIC_CALL_RET0(arm64_guest_get_ip, *(perf_guest_cbs->get_ip)); + +void arch_perf_update_guest_cbs(struct perf_guest_info_callbacks *guest_cbs) +{ + if (guest_cbs) { + static_call_update(arm64_guest_state, guest_cbs->state); + static_call_update(arm64_guest_get_ip, guest_cbs->get_ip); + } else { + static_call_update(arm64_guest_state, (void *)&__static_call_return0); + static_call_update(arm64_guest_get_ip, (void *)&__static_call_return0); + } +} + void perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); - - if (guest_cbs && guest_cbs->state()) { + if (static_call(arm64_guest_state)()) { /* We don't support guest os callchain now */ return; } @@ -149,10 +162,9 @@ static bool callchain_trace(void *data, unsigned long pc) void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); struct stackframe frame; - if (guest_cbs && guest_cbs->state()) { + if (static_call(arm64_guest_state)()) { /* We don't support guest os callchain now */ return; } @@ -163,18 +175,15 @@ void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, unsigned long perf_instruction_pointer(struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); - - if (guest_cbs && guest_cbs->state()) - return guest_cbs->get_ip(); + if (static_call(arm64_guest_state)()) + return static_call(arm64_guest_get_ip)(); return instruction_pointer(regs); } unsigned long perf_misc_flags(struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); - unsigned int guest_state = guest_cbs ? guest_cbs->state() : 0; + unsigned int guest_state = static_call(arm64_guest_state)(); int misc = 0; if (guest_state) { diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index 3a7630fdd340..508a677edd8c 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -90,6 +90,29 @@ DEFINE_STATIC_CALL_NULL(x86_pmu_pebs_aliases, *x86_pmu.pebs_aliases); */ DEFINE_STATIC_CALL_RET0(x86_pmu_guest_get_msrs, *x86_pmu.guest_get_msrs); +DEFINE_STATIC_CALL_RET0(x86_guest_state, *(perf_guest_cbs->state)); +DEFINE_STATIC_CALL_RET0(x86_guest_get_ip, *(perf_guest_cbs->get_ip)); +DEFINE_STATIC_CALL_RET0(x86_guest_handle_intel_pt_intr, *(perf_guest_cbs->handle_intel_pt_intr)); + +void arch_perf_update_guest_cbs(struct perf_guest_info_callbacks *guest_cbs) +{ + if (guest_cbs) { + static_call_update(x86_guest_state, guest_cbs->state); + static_call_update(x86_guest_get_ip, guest_cbs->get_ip); + } else { + static_call_update(x86_guest_state, (void *)&__static_call_return0); + static_call_update(x86_guest_get_ip, (void *)&__static_call_return0); + } + + /* Implementing ->handle_intel_pt_intr is optional. */ + if (guest_cbs && guest_cbs->handle_intel_pt_intr) + static_call_update(x86_guest_handle_intel_pt_intr, + guest_cbs->handle_intel_pt_intr); + else + static_call_update(x86_guest_handle_intel_pt_intr, + (void *)&__static_call_return0); +} + u64 __read_mostly hw_cache_event_ids [PERF_COUNT_HW_CACHE_MAX] [PERF_COUNT_HW_CACHE_OP_MAX] @@ -2761,11 +2784,10 @@ static bool perf_hw_regs(struct pt_regs *regs) void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); struct unwind_state state; unsigned long addr; - if (guest_cbs && guest_cbs->state()) { + if (static_call(x86_guest_state)()) { /* TODO: We don't support guest os callchain now */ return; } @@ -2865,11 +2887,10 @@ perf_callchain_user32(struct pt_regs *regs, struct perf_callchain_entry_ctx *ent void perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); struct stack_frame frame; const struct stack_frame __user *fp; - if (guest_cbs && guest_cbs->state()) { + if (static_call(x86_guest_state)()) { /* TODO: We don't support guest os callchain now */ return; } @@ -2946,18 +2967,15 @@ static unsigned long code_segment_base(struct pt_regs *regs) unsigned long perf_instruction_pointer(struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); - - if (guest_cbs && guest_cbs->state()) - return guest_cbs->get_ip(); + if (static_call(x86_guest_state)()) + return static_call(x86_guest_get_ip)(); return regs->ip + code_segment_base(regs); } unsigned long perf_misc_flags(struct pt_regs *regs) { - struct perf_guest_info_callbacks *guest_cbs = perf_get_guest_cbs(); - unsigned int guest_state = guest_cbs ? guest_cbs->state() : 0; + unsigned int guest_state = static_call(x86_guest_state)(); int misc = 0; if (guest_state) { diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index 524ad1f747bd..fb1bd7a0e1a6 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -2782,11 +2782,12 @@ static void intel_pmu_reset(void) local_irq_restore(flags); } +DECLARE_STATIC_CALL(x86_guest_handle_intel_pt_intr, *(perf_guest_cbs->handle_intel_pt_intr)); + static int handle_pmi_common(struct pt_regs *regs, u64 status) { struct perf_sample_data data; struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events); - struct perf_guest_info_callbacks *guest_cbs; int bit; int handled = 0; u64 intel_ctrl = hybrid(cpuc->pmu, intel_ctrl); @@ -2853,9 +2854,7 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status) */ if (__test_and_clear_bit(GLOBAL_STATUS_TRACE_TOPAPMI_BIT, (unsigned long *)&status)) { handled++; - - guest_cbs = perf_get_guest_cbs(); - if (likely(!guest_cbs || !guest_cbs->handle_intel_pt_intr())) + if (!static_call(x86_guest_handle_intel_pt_intr)()) intel_pt_interrupt(); } diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index a5d5893b80b0..3fa1014218f4 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1240,14 +1240,7 @@ extern void perf_event_bpf_event(struct bpf_prog *prog, #ifdef CONFIG_HAVE_GUEST_PERF_EVENTS extern struct perf_guest_info_callbacks *perf_guest_cbs; -static inline struct perf_guest_info_callbacks *perf_get_guest_cbs(void) -{ - /* Reg/unreg perf_guest_cbs waits for readers via synchronize_rcu(). */ - lockdep_assert_preemption_disabled(); - - /* Prevent reloading between a !NULL check and dereferences. */ - return READ_ONCE(perf_guest_cbs); -} +extern void arch_perf_update_guest_cbs(struct perf_guest_info_callbacks *guest_cbs); extern void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *callbacks); extern void perf_unregister_guest_info_callbacks(void); #endif /* CONFIG_HAVE_GUEST_PERF_EVENTS */ diff --git a/kernel/events/core.c b/kernel/events/core.c index ec36e7aded89..fb0fd670ab23 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -6491,6 +6491,7 @@ void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs) return; WRITE_ONCE(perf_guest_cbs, cbs); + arch_perf_update_guest_cbs(cbs); synchronize_rcu(); } EXPORT_SYMBOL_GPL(perf_register_guest_info_callbacks); @@ -6498,6 +6499,7 @@ EXPORT_SYMBOL_GPL(perf_register_guest_info_callbacks); void perf_unregister_guest_info_callbacks(void) { WRITE_ONCE(perf_guest_cbs, NULL); + arch_perf_update_guest_cbs(NULL); synchronize_rcu(); } EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks); From patchwork Sat Aug 28 00:35:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12463085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C8F4C432BE for ; Sat, 28 Aug 2021 00:43:39 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E388960F14 for ; Sat, 28 Aug 2021 00:43:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E388960F14 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References :Mime-Version:Message-Id:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wzKcJiYyNjNAeruBVoVwN7Y+EoWzThOXvGURZIX3gVY=; b=vDaPdEeuV07gFb 5LUvnM8XCEnNESnuW8wlYfL3W9VaU7U/ktdn1h9af6NvV7TJSXc+vB0oueD/enmYk5h1oqpWcQidf RbtJm6ZuHR58kGL420DY2lX7jxb6jgz2HmpSa4wgJA5k5TzQ0QBVSeEf0YKOnGiN8Ps/fz0Y7GGp0 Ssxixs5maXGdjyJpa1N8SsDFx4t0X0560pY1UByVHvYruUzDDgyhUczvYzwKTCVQvmj00QuAFE9xi SIzNFDv/BzN7eGBx4QjqY5qaUwUDcwp+hLPmI0yeTBF0Pf0YNSMdQU1VRktYTXw4ykEu7TgySTusw RGiDvdzevzWqpCVSlqmw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmPo-00DTQh-Db; Sat, 28 Aug 2021 00:41:44 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmKz-00DR77-6J for linux-arm-kernel@lists.infradead.org; Sat, 28 Aug 2021 00:36:49 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id c4-20020a0ce7c4000000b00370a5bb605eso2911616qvo.0 for ; Fri, 27 Aug 2021 17:36:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=DHvdKUJPhOfdwKYi3gDlAxqlqPxsEI3eu21bt3UUIpQ=; b=cRSRDiBD1t9urqOzflAsrMtB378EAPg5jZzAQBu3WBpF9lCRiO9i1nInagQ5WfJGFI mazzpjLYk+9+pL+F9otrGLLg/n08ZevlzMBPHEYOnN+VdIB4A7wxwtFH6ZpoJLJJI+SZ rxf/NY8q5ZYsMDzhr085D9FW1b3CZFNUgnmsBtUOczgqmJA3n25Cy7DHpE8+8o64Przh loBGBaIDfQqOoqdZDGmQF17Fg+dmpa+acKaF2TUeYuJThxmscBQMPn7PZ7/xDoZrL4eO EjriiepZFNfgucM3RIv8HD8YQIWksY61k2qaludpcxy2qAakOC66DdAdCPUomdcFD1HM mmjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=DHvdKUJPhOfdwKYi3gDlAxqlqPxsEI3eu21bt3UUIpQ=; b=uk+tCEpbTUuRC6POFRIDSzLX2oSvKGtY/mGAyyOorgyBNXHYCvkxBqwi94DGA3L3Aq qBN36fJs3hfPO3k0qlkREgyKGv7L0tZye/hg4xgtKME6p1XCzQh2J15T9PlZnu3Sv+A7 ZAVk2r2DVlDsoGN8XoyUrP+3Md89v5vnSzAtywyG8u+0UUiCGiasCsmS2zLQTtWgKjI6 k91P6rQFtQVBSkH5JKikkRktwyR3ncwA2yx24H68dT3B9N9+MZltsBlKiH4kS3d1ZldW f72dEkFV1aiFrVCMrq+ar6Eov7Vs6m9N8gwsJyc7DKX9ET46Qmb11IXf9uP9p8zOWKCR b6Tg== X-Gm-Message-State: AOAM531l1DyKaoPMJC5OfxkSPvMl+30y5g91m9LrHJ2r6u7EcciI7ber gQ3erX2TPH7xPGOKIAEvBDvgD4CTLEY= X-Google-Smtp-Source: ABdhPJwPL5V/RrJYzEwuqQz/lpqtoTngCSsfC6hM7seqpt+XgOfCSrLYZLMKAACl21hF27dX3suwlB0z88Y= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:f66c:b851:7e79:7ed4]) (user=seanjc job=sendgmr) by 2002:ad4:5346:: with SMTP id v6mr12225725qvs.40.1630111003639; Fri, 27 Aug 2021 17:36:43 -0700 (PDT) Date: Fri, 27 Aug 2021 17:35:53 -0700 In-Reply-To: <20210828003558.713983-1-seanjc@google.com> Message-Id: <20210828003558.713983-9-seanjc@google.com> Mime-Version: 1.0 References: <20210828003558.713983-1-seanjc@google.com> X-Mailer: git-send-email 2.33.0.259.gc128427fd7-goog Subject: [PATCH v2 08/13] KVM: x86: Drop current_vcpu for kvm_running_vcpu + kvm_arch_vcpu variable From: Sean Christopherson To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Will Deacon , Mark Rutland , Catalin Marinas , Marc Zyngier , Guo Ren , Nick Hu , Greentime Hu , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , Thomas Gleixner , Borislav Petkov , x86@kernel.org, Paolo Bonzini , Boris Ostrovsky , Juergen Gross Cc: Alexander Shishkin , Jiri Olsa , Namhyung Kim , James Morse , Alexandru Elisei , Suzuki K Poulose , "H. Peter Anvin" , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Stefano Stabellini , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Artem Kashkanov , Like Xu , Zhu Lingshan X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210827_173645_340544_C2EA2C75 X-CRM114-Status: GOOD ( 20.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Use the generic kvm_running_vcpu plus a new 'handling_intr_from_guest' variable in kvm_arch_vcpu instead of the semi-redundant current_vcpu. kvm_before/after_interrupt() must be called while the vCPU is loaded, (which protects against preemption), thus kvm_running_vcpu is guaranteed to be non-NULL when handling_intr_from_guest is non-zero. Switching to kvm_get_running_vcpu() will allows moving KVM's perf callbacks to generic code, and the new flag will be used in a future patch to more precisely identify the "NMI from guest" case. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 3 +-- arch/x86/kvm/pmu.c | 2 +- arch/x86/kvm/x86.c | 21 ++++++++++++--------- arch/x86/kvm/x86.h | 10 ++++++---- 4 files changed, 20 insertions(+), 16 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 1080166fc0cf..2d86a2dfc775 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -763,6 +763,7 @@ struct kvm_vcpu_arch { unsigned nmi_pending; /* NMI queued after currently running handler */ bool nmi_injected; /* Trying to inject an NMI this entry */ bool smi_pending; /* SMI queued after currently running handler */ + u8 handling_intr_from_guest; struct kvm_mtrr mtrr_state; u64 pat; @@ -1874,8 +1875,6 @@ int kvm_skip_emulated_instruction(struct kvm_vcpu *vcpu); int kvm_complete_insn_gp(struct kvm_vcpu *vcpu, int err); void __kvm_request_immediate_exit(struct kvm_vcpu *vcpu); -unsigned int kvm_guest_state(void); - void __user *__x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size); bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu); diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 5b68d4188de0..eef48258e50f 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -87,7 +87,7 @@ static void kvm_perf_overflow_intr(struct perf_event *perf_event, * woken up. So we should wake it, but this is impossible from * NMI context. Do it from irq work instead. */ - if (!kvm_guest_state()) + if (!kvm_handling_nmi_from_guest(pmc->vcpu)) irq_work_queue(&pmc_to_pmu(pmc)->irq_work); else kvm_make_request(KVM_REQ_PMI, pmc->vcpu); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index b2a4d085aa4f..6df300c55461 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8264,15 +8264,17 @@ static void kvm_timer_init(void) kvmclock_cpu_online, kvmclock_cpu_down_prep); } -DEFINE_PER_CPU(struct kvm_vcpu *, current_vcpu); -EXPORT_PER_CPU_SYMBOL_GPL(current_vcpu); +static inline bool kvm_pmi_in_guest(struct kvm_vcpu *vcpu) +{ + return vcpu && vcpu->arch.handling_intr_from_guest; +} -unsigned int kvm_guest_state(void) +static unsigned int kvm_guest_state(void) { - struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu); + struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); unsigned int state; - if (!vcpu) + if (!kvm_pmi_in_guest(vcpu)) return 0; state = PERF_GUEST_ACTIVE; @@ -8284,9 +8286,10 @@ unsigned int kvm_guest_state(void) static unsigned long kvm_guest_get_ip(void) { - struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu); + struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); - if (WARN_ON_ONCE(!vcpu)) + /* Retrieving the IP must be guarded by a call to kvm_guest_state(). */ + if (WARN_ON_ONCE(!kvm_pmi_in_guest(vcpu))) return 0; return kvm_rip_read(vcpu); @@ -8294,10 +8297,10 @@ static unsigned long kvm_guest_get_ip(void) static unsigned int kvm_handle_intel_pt_intr(void) { - struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu); + struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); /* '0' on failure so that the !PT case can use a RET0 static call. */ - if (!vcpu) + if (!kvm_pmi_in_guest(vcpu)) return 0; kvm_make_request(KVM_REQ_PMI, vcpu); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 7d66d63dc55a..a9c107e7c907 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -387,18 +387,20 @@ static inline bool kvm_cstate_in_guest(struct kvm *kvm) return kvm->arch.cstate_in_guest; } -DECLARE_PER_CPU(struct kvm_vcpu *, current_vcpu); - static inline void kvm_before_interrupt(struct kvm_vcpu *vcpu) { - __this_cpu_write(current_vcpu, vcpu); + WRITE_ONCE(vcpu->arch.handling_intr_from_guest, 1); } static inline void kvm_after_interrupt(struct kvm_vcpu *vcpu) { - __this_cpu_write(current_vcpu, NULL); + WRITE_ONCE(vcpu->arch.handling_intr_from_guest, 0); } +static inline bool kvm_handling_nmi_from_guest(struct kvm_vcpu *vcpu) +{ + return !!vcpu->arch.handling_intr_from_guest; +} static inline bool kvm_pat_valid(u64 data) { From patchwork Sat Aug 28 00:35:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12463137 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 438F3C432BE for ; Sat, 28 Aug 2021 00:44:25 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0C46060ED5 for ; Sat, 28 Aug 2021 00:44:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 0C46060ED5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References :Mime-Version:Message-Id:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=cxo/ZRkfPqqlRlJEzinqHv3BiSPKYhI+RtZ+yGjvWyw=; b=fSyt95MH1RIUef RpDRzxlJjskhYhJaW6v4oNB2W4jmTlxUzdZQZINLA9gE13oiPHifH+qVB1JLF2QORpHhBX3BA9R5n ANQ20NaUqfJ/actQ/ARv+y6tmPwbl4OaTiu4c741RyS99KfwCHZFoZXeou52uKBS3I3kLy/xyZl2f LVBVy6F0CIM7Zy207MsUQi9XrTkGCXID7z1dY4GxT4TImhNvWwbgZPmPlDURaO4SOTGymG3eJuBSQ LrfiF8RRAUBQaNHBKZ2spIAqDgXoY2yaU0kYl5PF157b4oU50TbFBjkCZimLJ5AjnF7y1hmjO1D7V O7hzaU/GM+s/XoEGqh+A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmQV-00DTgZ-RS; Sat, 28 Aug 2021 00:42:30 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmL1-00DR8U-Eb for linux-arm-kernel@lists.infradead.org; Sat, 28 Aug 2021 00:36:51 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id l23-20020a252517000000b0059bf0bed21fso1236126ybl.18 for ; Fri, 27 Aug 2021 17:36:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=ZqLtQecZGSWC4AA1MNUminuBnJfUTZ97werLGL77udw=; b=W7YW+zmYlqt7RZocpxt+Zt9pY0UpoY2KAKci8cZwj6B9FcylXpzGzQs460J2eCLeO7 teq3g48IMVj8VY2IbRoULnBfpkow62kQ4WR9lSbT6+S3/VfO1mprPaUeR/TGh9eLXRlv Lc3Rfpl3D1r7APNREe18xnEdMpPT4WiL/4dbHUEcReZ3q/7v2SIOWBkUSbCpfx5nigei 7+ogWXDBydp9r/EwqpMZCW2iShx49w75B8rV8caRVnqX3Ryr6RI6XSvcak0ct7nIrRDY DLc6aNoMH3Xu51qkT4HMU2d6imm3Ju+neyD/xf4o05nAqKv3vaT7w7RaHNmHBwGJNnFa GvhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=ZqLtQecZGSWC4AA1MNUminuBnJfUTZ97werLGL77udw=; b=EgXxQsp593b9mAODPASV/8Tg5ZGctah6EUD4SvXKyDc4V9Vp7zSyISD+V5TyKYTNkn TYObtT4kV1ErzE1t66MLdKrt8wuc5z0Hb4GRbsD9ofqmwq+fn6cm5SzWBsexHCAKVft4 Va6Ap2/gAyTo9fadS3SLxoI2ChxFFYNglJHapITqOsFJDwMBUge7VJ/4XDm/0538/CMq 1yN4Jyq/59mDxUnUqZw1ITMi+6wTR2zZHYwT2DN1jA/YoYUSaiMNbLLaMzMcST9cfbKx 2SNyh+kq7GJHG1MTiMHmMM0Z7I6QFD3ZAkeULd94jah81Wk80GPzH7P+kTfTQ3D/ZvSL yaTw== X-Gm-Message-State: AOAM530LnVKfkIiX9G8KXo810Hc4caxqHYZRQnhkiDbj3hcCigY0sbF1 RRm70aKth2r6n+KSoy1o1za7i4lx4Ok= X-Google-Smtp-Source: ABdhPJypKVd0cLFh8EuRLmmqpexQyXCav+rSvE30xuwBhSiRSkaWIzvoFPoqFTfkE5i16U7H5ekOzDpkr2E= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:f66c:b851:7e79:7ed4]) (user=seanjc job=sendgmr) by 2002:a25:f503:: with SMTP id a3mr8148726ybe.501.1630111005865; Fri, 27 Aug 2021 17:36:45 -0700 (PDT) Date: Fri, 27 Aug 2021 17:35:54 -0700 In-Reply-To: <20210828003558.713983-1-seanjc@google.com> Message-Id: <20210828003558.713983-10-seanjc@google.com> Mime-Version: 1.0 References: <20210828003558.713983-1-seanjc@google.com> X-Mailer: git-send-email 2.33.0.259.gc128427fd7-goog Subject: [PATCH v2 09/13] KVM: x86: More precisely identify NMI from guest when handling PMI From: Sean Christopherson To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Will Deacon , Mark Rutland , Catalin Marinas , Marc Zyngier , Guo Ren , Nick Hu , Greentime Hu , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , Thomas Gleixner , Borislav Petkov , x86@kernel.org, Paolo Bonzini , Boris Ostrovsky , Juergen Gross Cc: Alexander Shishkin , Jiri Olsa , Namhyung Kim , James Morse , Alexandru Elisei , Suzuki K Poulose , "H. Peter Anvin" , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Stefano Stabellini , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Artem Kashkanov , Like Xu , Zhu Lingshan X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210827_173647_608618_1DD1669D X-CRM114-Status: GOOD ( 18.04 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Differntiate between IRQ and NMI for KVM's PMC overflow callback, which was originally invoked in response to an NMI that arrived while the guest was running, but was inadvertantly changed to fire on IRQs as well when support for perf without PMU/NMI was added to KVM. In practice, this should be a nop as the PMC overflow callback shouldn't be reached, but it's a cheap and easy fix that also better documents the situation. Note, this also doesn't completely prevent false positives if perf somehow ends up calling into KVM, e.g. an NMI can arrive in host after KVM sets its flag. Fixes: dd60d217062f ("KVM: x86: Fix perf timer mode IP reporting") Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/vmx/vmx.c | 4 +++- arch/x86/kvm/x86.c | 2 +- arch/x86/kvm/x86.h | 13 ++++++++++--- 4 files changed, 15 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 1a70e11f0487..0a0c01744b63 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -3843,7 +3843,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) } if (unlikely(svm->vmcb->control.exit_code == SVM_EXIT_NMI)) - kvm_before_interrupt(vcpu); + kvm_before_interrupt(vcpu, KVM_HANDLING_NMI); kvm_load_host_xsave_state(vcpu); stgi(); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index f19d72136f77..61a4f5ff2acd 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6344,7 +6344,9 @@ void vmx_do_interrupt_nmi_irqoff(unsigned long entry); static void handle_interrupt_nmi_irqoff(struct kvm_vcpu *vcpu, unsigned long entry) { - kvm_before_interrupt(vcpu); + bool is_nmi = entry == (unsigned long)asm_exc_nmi_noist; + + kvm_before_interrupt(vcpu, is_nmi ? KVM_HANDLING_NMI : KVM_HANDLING_IRQ); vmx_do_interrupt_nmi_irqoff(entry); kvm_after_interrupt(vcpu); } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 6df300c55461..1427ac1fc1f2 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9676,7 +9676,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) * interrupts on processors that implement an interrupt shadow, the * stat.exits increment will do nicely. */ - kvm_before_interrupt(vcpu); + kvm_before_interrupt(vcpu, KVM_HANDLING_IRQ); local_irq_enable(); ++vcpu->stat.exits; local_irq_disable(); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index a9c107e7c907..9b26f9b09d2a 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -387,9 +387,16 @@ static inline bool kvm_cstate_in_guest(struct kvm *kvm) return kvm->arch.cstate_in_guest; } -static inline void kvm_before_interrupt(struct kvm_vcpu *vcpu) +enum kvm_intr_type { + /* Values are arbitrary, but must be non-zero. */ + KVM_HANDLING_IRQ = 1, + KVM_HANDLING_NMI, +}; + +static inline void kvm_before_interrupt(struct kvm_vcpu *vcpu, + enum kvm_intr_type intr) { - WRITE_ONCE(vcpu->arch.handling_intr_from_guest, 1); + WRITE_ONCE(vcpu->arch.handling_intr_from_guest, (u8)intr); } static inline void kvm_after_interrupt(struct kvm_vcpu *vcpu) @@ -399,7 +406,7 @@ static inline void kvm_after_interrupt(struct kvm_vcpu *vcpu) static inline bool kvm_handling_nmi_from_guest(struct kvm_vcpu *vcpu) { - return !!vcpu->arch.handling_intr_from_guest; + return vcpu->arch.handling_intr_from_guest == KVM_HANDLING_NMI; } static inline bool kvm_pat_valid(u64 data) From patchwork Sat Aug 28 00:35:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12463139 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0935C432BE for ; Sat, 28 Aug 2021 00:46:15 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 98DE460ED5 for ; Sat, 28 Aug 2021 00:46:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 98DE460ED5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References :Mime-Version:Message-Id:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=yZyyafy7PZxOOV4Lc0dgDKq6M9/Eyw+RnYzjiR4ynE8=; b=yNtUwGiGiSueio aswXUB/v4rMkkt/w3F8rbygQgASdroOqnWmnKF/sY4IyVHFEZ+6LXYrHBj2Hf9AdJHXoLH7t2UNTa rkTJd+8rJ7OJEUUIWzhauUETh+Stx5a30PqhLptCMblMfnGgQ5cOAAvOh8F/LiaF4aM+0kRy52BVq GhREIGIoKxZaerO9/y7EqaRp84JIw6PRnPG0nkzyj/J90EnKfexSn1j5XlspzP3hxa94fVC893TfO r59rypPDIji7QpN9+PAdx9ZEkGiJS89luZEZ3G8zTg0inajBumvyTgTh7T618WuabqKCzGS27ReRm RSxSa1Ji3vvFNFDscfuQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmRT-00DU4m-Sc; Sat, 28 Aug 2021 00:43:29 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmL3-00DR9q-IV for linux-arm-kernel@lists.infradead.org; Sat, 28 Aug 2021 00:36:53 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id e14-20020a056214162e00b00375ec21dd99so755606qvw.23 for ; Fri, 27 Aug 2021 17:36:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=kV+43gCo9lUv9XBpq5GszAcUqdFbD0VTdN9gkn90biY=; b=rxQFl/E0q7TG4eTtS8cvaKjpz+1uZzXKYUahAXBLz4zSPj8Lw+VLh8Mb1BBBtiQKNw BlNjA876gHwdxDucFChcGa5H2Zi316XYlCmTuKX+ROmAwLv/BIgb2shswMUBN0+4HLwS zQZ7ZkLSnOZFUPXFuFHGYUDr9nX4Oea1w/M1oQqjqJxqbK6ba2arrO/e5Qo8o9ZtAlj/ FOSv9aQsEN9L3IoU1DIQcVeEZ3bqRPKaoHTa52LbneQ89SWlRvFoztysKfBpWoaqqjBB nB92v+uOO+V41If/3hHsbxNnCSO1+LGRQHtXQib4wWZe8INcZp3C8P1oHhmk1iC8OAIJ Gmig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=kV+43gCo9lUv9XBpq5GszAcUqdFbD0VTdN9gkn90biY=; b=AhNVdsysLhsXpJs3la8YFaIE+ZBvyX19d76j4YWKXR/HrkY9pvmZNmEXShsPIrXvGX ++AisoBVM6ZhHkyZvCgENOPGxldfz8Hu7teCFl6Egwm+t3cOW9wiN8dqqoIsav9QyRHy w8eK4dTFEb9smfVz01YH09i5/3FxbCIC06HoVzluLKoHCNq5Ixxu14GVJt65UFMvW4aN hysdJemRl7zPtBOrKiTHPFyPJWEw3LmJP8WRuqRozTfVWJ4FDht+VvVFPWxvjBm7BxiM xePm70+5tq1uhulwrRtcKNPdGlO3SVmfFpB2QisyZjys34vF8aXTZLZpvRg+DQ1PhMS7 Ihqw== X-Gm-Message-State: AOAM533GOw56iEltwo7bH5mr5uD0OD5iljDmAIOtJjJ4HvDl5hjihblf YHSu87r5vhLgU+X+IZwOadmHEwaiRu4= X-Google-Smtp-Source: ABdhPJzs4DFqQHNeRKzV+tkWBighuxilPoPlwqBBJW7KkYH0Bkccb5PXEDnQkRBBiHYdU/kdTKgE4wfebtg= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:f66c:b851:7e79:7ed4]) (user=seanjc job=sendgmr) by 2002:a0c:f88f:: with SMTP id u15mr12630183qvn.38.1630111008043; Fri, 27 Aug 2021 17:36:48 -0700 (PDT) Date: Fri, 27 Aug 2021 17:35:55 -0700 In-Reply-To: <20210828003558.713983-1-seanjc@google.com> Message-Id: <20210828003558.713983-11-seanjc@google.com> Mime-Version: 1.0 References: <20210828003558.713983-1-seanjc@google.com> X-Mailer: git-send-email 2.33.0.259.gc128427fd7-goog Subject: [PATCH v2 10/13] KVM: Move x86's perf guest info callbacks to generic KVM From: Sean Christopherson To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Will Deacon , Mark Rutland , Catalin Marinas , Marc Zyngier , Guo Ren , Nick Hu , Greentime Hu , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , Thomas Gleixner , Borislav Petkov , x86@kernel.org, Paolo Bonzini , Boris Ostrovsky , Juergen Gross Cc: Alexander Shishkin , Jiri Olsa , Namhyung Kim , James Morse , Alexandru Elisei , Suzuki K Poulose , "H. Peter Anvin" , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Stefano Stabellini , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Artem Kashkanov , Like Xu , Zhu Lingshan X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210827_173649_682866_14AEE5EC X-CRM114-Status: GOOD ( 20.61 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move x86's perf guest callbacks into common KVM, as they are semantically identical to arm64's callbacks (the only other such KVM callbacks). arm64 will convert to the common versions in a future patch. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 4 +++ arch/x86/kvm/x86.c | 53 +++++++-------------------------- include/linux/kvm_host.h | 12 ++++++++ virt/kvm/kvm_main.c | 40 +++++++++++++++++++++++++ 4 files changed, 67 insertions(+), 42 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 2d86a2dfc775..a98c7907110c 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1543,6 +1543,10 @@ static inline int kvm_arch_flush_remote_tlb(struct kvm *kvm) return -ENOTSUPP; } +#define __KVM_WANT_PERF_CALLBACKS +#define kvm_arch_pmi_in_guest(vcpu) \ + ((vcpu) && (vcpu)->arch.handling_intr_from_guest) + int kvm_mmu_module_init(void); void kvm_mmu_module_exit(void); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 1427ac1fc1f2..1bea616402e6 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8264,43 +8264,12 @@ static void kvm_timer_init(void) kvmclock_cpu_online, kvmclock_cpu_down_prep); } -static inline bool kvm_pmi_in_guest(struct kvm_vcpu *vcpu) -{ - return vcpu && vcpu->arch.handling_intr_from_guest; -} - -static unsigned int kvm_guest_state(void) -{ - struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); - unsigned int state; - - if (!kvm_pmi_in_guest(vcpu)) - return 0; - - state = PERF_GUEST_ACTIVE; - if (static_call(kvm_x86_get_cpl)(vcpu)) - state |= PERF_GUEST_USER; - - return state; -} - -static unsigned long kvm_guest_get_ip(void) -{ - struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); - - /* Retrieving the IP must be guarded by a call to kvm_guest_state(). */ - if (WARN_ON_ONCE(!kvm_pmi_in_guest(vcpu))) - return 0; - - return kvm_rip_read(vcpu); -} - static unsigned int kvm_handle_intel_pt_intr(void) { struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); /* '0' on failure so that the !PT case can use a RET0 static call. */ - if (!kvm_pmi_in_guest(vcpu)) + if (!kvm_arch_pmi_in_guest(vcpu)) return 0; kvm_make_request(KVM_REQ_PMI, vcpu); @@ -8309,12 +8278,6 @@ static unsigned int kvm_handle_intel_pt_intr(void) return 1; } -static struct perf_guest_info_callbacks kvm_guest_cbs = { - .state = kvm_guest_state, - .get_ip = kvm_guest_get_ip, - .handle_intel_pt_intr = NULL, -}; - #ifdef CONFIG_X86_64 static void pvclock_gtod_update_fn(struct work_struct *work) { @@ -11068,9 +11031,11 @@ int kvm_arch_hardware_setup(void *opaque) memcpy(&kvm_x86_ops, ops->runtime_ops, sizeof(kvm_x86_ops)); kvm_ops_static_call_update(); + /* Temporary ugliness. */ if (ops->intel_pt_intr_in_guest && ops->intel_pt_intr_in_guest()) - kvm_guest_cbs.handle_intel_pt_intr = kvm_handle_intel_pt_intr; - perf_register_guest_info_callbacks(&kvm_guest_cbs); + kvm_register_perf_callbacks(kvm_handle_intel_pt_intr); + else + kvm_register_perf_callbacks(NULL); if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES)) supported_xss = 0; @@ -11099,8 +11064,7 @@ int kvm_arch_hardware_setup(void *opaque) void kvm_arch_hardware_unsetup(void) { - perf_unregister_guest_info_callbacks(); - kvm_guest_cbs.handle_intel_pt_intr = NULL; + kvm_unregister_perf_callbacks(); static_call(kvm_x86_hardware_unsetup)(); } @@ -11727,6 +11691,11 @@ bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu) return vcpu->arch.preempted_in_kernel; } +unsigned long kvm_arch_vcpu_get_ip(struct kvm_vcpu *vcpu) +{ + return kvm_rip_read(vcpu); +} + int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index e4d712e9f760..34d99034852f 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1163,6 +1163,18 @@ static inline bool kvm_arch_intc_initialized(struct kvm *kvm) } #endif +#ifdef __KVM_WANT_PERF_CALLBACKS + +void kvm_set_intel_pt_intr_handler(unsigned int (*handler)(void)); +unsigned long kvm_arch_vcpu_get_ip(struct kvm_vcpu *vcpu); + +void kvm_register_perf_callbacks(unsigned int (*pt_intr_handler)(void)); +static inline void kvm_unregister_perf_callbacks(void) +{ + perf_unregister_guest_info_callbacks(); +} +#endif + int kvm_arch_init_vm(struct kvm *kvm, unsigned long type); void kvm_arch_destroy_vm(struct kvm *kvm); void kvm_arch_sync_events(struct kvm *kvm); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 3e67c93ca403..2c61db39b501 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -5460,6 +5460,46 @@ struct kvm_vcpu * __percpu *kvm_get_running_vcpus(void) return &kvm_running_vcpu; } +#ifdef __KVM_WANT_PERF_CALLBACKS +static unsigned int kvm_guest_state(void) +{ + struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); + unsigned int state; + + if (!kvm_arch_pmi_in_guest(vcpu)) + return 0; + + state = PERF_GUEST_ACTIVE; + if (!kvm_arch_vcpu_in_kernel(vcpu)) + state |= PERF_GUEST_USER; + + return state; +} + +static unsigned long kvm_guest_get_ip(void) +{ + struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); + + /* Retrieving the IP must be guarded by a call to kvm_guest_state(). */ + if (WARN_ON_ONCE(!kvm_arch_pmi_in_guest(vcpu))) + return 0; + + return kvm_arch_vcpu_get_ip(vcpu); +} + +static struct perf_guest_info_callbacks kvm_guest_cbs = { + .state = kvm_guest_state, + .get_ip = kvm_guest_get_ip, + .handle_intel_pt_intr = NULL, +}; + +void kvm_register_perf_callbacks(unsigned int (*pt_intr_handler)(void)) +{ + kvm_guest_cbs.handle_intel_pt_intr = pt_intr_handler; + perf_register_guest_info_callbacks(&kvm_guest_cbs); +} +#endif + struct kvm_cpu_compat_check { void *opaque; int *ret; From patchwork Sat Aug 28 00:35:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12463141 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F643C432BE for ; Sat, 28 Aug 2021 00:46:48 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3CCAF60F14 for ; Sat, 28 Aug 2021 00:46:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3CCAF60F14 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References :Mime-Version:Message-Id:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=PtaoA9p9osJLWdYZFzVSd6Jx/0E9s+z6IGWxS/mnIy4=; b=tUluA5o93o5fTn jqB+2GEkCQWiAbxmbElbZzY4Q8mC1Vy48vzBLpWSMtWRNStXl1+qu45ScCCMHFUOKsACPDnKOYSqo +YoPIWP0dFZNueqgQx4Kr5m+c8kBs+SQNrr8/gQSCVsYi8K/ryizQwoMY2eH07EQEQVryw+D604YF vVeEumYSDRJeo5apSz9uM+n8hqVUpSZjgm8EC5zFnQ0rcQeYF0XsZ94pwTlsbPTySe/Nh7CVNj/uA H9MkflxVBEuD6F50/KBPFbBS96/O+pvLFyEdL+duh2yHKvKgXd4dLsYFPQOrYIuoRbHG4eLS34r8L qbuYEetFg1MXn488lW9A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmSf-00DUYL-Sc; Sat, 28 Aug 2021 00:44:43 +0000 Received: from mail-qv1-xf49.google.com ([2607:f8b0:4864:20::f49]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmL5-00DRBH-Nr for linux-arm-kernel@lists.infradead.org; Sat, 28 Aug 2021 00:36:55 +0000 Received: by mail-qv1-xf49.google.com with SMTP id q2-20020ad45ca2000000b00374fa0dbedfso4469951qvh.1 for ; Fri, 27 Aug 2021 17:36:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=12ZRJNbNqDVAbxhZSOCyDcPf9hFLX5MHnweni3QFi0c=; b=iLNSgtMMmWhPukAfNbXvTTIYc6dXiH564tVo4Ccok40pcj7Q1+P/ARsT7PV8AyBdmK HJj6MjluW1BB5E0dRBPWWlbq5RPbONE16r5eytzJ5dtT3Mx/PdOBt38hIWAuJRNbURm/ xNeeH/oYQ/ranGoB1l4Hm10ertIZkv2GfYm55Wd5igk0ilFh3pXhtUy2sEOwRtYDMAyh 6rcUqhAqdWJQ80KdyFCjp1TGV/qxmUBqN/OP8u5cNvZ4wuCxHf+yb2U7VWMwLJg0Tjpm 8+MmLRoRfyDYZSJ9jwmqHx7eyPBkiWNXYqNrFjX3lIda1O9H5NM6CV+Oc6WxIubml381 UDrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=12ZRJNbNqDVAbxhZSOCyDcPf9hFLX5MHnweni3QFi0c=; b=j5fm372VmbRJMFmplbgVVmqdpurr7WgJJkaai+ba//AiglA9bjZycqtmmaQaf2pofr zcFFAT7wNr+Jyc28W2jWVxPOE2xLVOvDKyy+xyQlcsVLOKMj+9bbO6a+xHfpFEg9w/Br 5mGwoORX/fqBrYHYMLirsXj8fMMz0DW4IPa1bLsXI4pGjVV1yDV4+SgHKhDes9rcMhNG u5UG/M1dqul/F+wy3eyFcYRVqpilaZSqZpjL3810+zCvtR2466tT/7A3yIPsZJQ7R8CR f+Vbf03e7DWHdVqzZ4CGiE9xzUDlTkq4VrzcRr1GS2eKibumdKzcEu6T2ux6DD4CL04q /p4w== X-Gm-Message-State: AOAM531YFOYrI10exkFojcm2M6H2UBcLv9O24SF61ivCfNi2HJPxbgou 9U6XXsfA+4oajGVdgGTqgIGze/BZDtc= X-Google-Smtp-Source: ABdhPJz2jvWRX6eHn8Br1Aba6NZj9m8KgLtjAjXtG8dC54Dd/WTNajNGxSMzGoZjDoekQIJTzJgi/4Z3X0I= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:f66c:b851:7e79:7ed4]) (user=seanjc job=sendgmr) by 2002:a0c:ef0d:: with SMTP id t13mr12401497qvr.21.1630111010073; Fri, 27 Aug 2021 17:36:50 -0700 (PDT) Date: Fri, 27 Aug 2021 17:35:56 -0700 In-Reply-To: <20210828003558.713983-1-seanjc@google.com> Message-Id: <20210828003558.713983-12-seanjc@google.com> Mime-Version: 1.0 References: <20210828003558.713983-1-seanjc@google.com> X-Mailer: git-send-email 2.33.0.259.gc128427fd7-goog Subject: [PATCH v2 11/13] KVM: x86: Move Intel Processor Trace interrupt handler to vmx.c From: Sean Christopherson To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Will Deacon , Mark Rutland , Catalin Marinas , Marc Zyngier , Guo Ren , Nick Hu , Greentime Hu , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , Thomas Gleixner , Borislav Petkov , x86@kernel.org, Paolo Bonzini , Boris Ostrovsky , Juergen Gross Cc: Alexander Shishkin , Jiri Olsa , Namhyung Kim , James Morse , Alexandru Elisei , Suzuki K Poulose , "H. Peter Anvin" , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Stefano Stabellini , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Artem Kashkanov , Like Xu , Zhu Lingshan X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210827_173651_910621_DD2AB2C4 X-CRM114-Status: GOOD ( 19.13 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that all state needed for VMX's PT interrupt handler is exposed to vmx.c (specifically the currently running vCPU), move the handler into vmx.c where it belongs. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/vmx/vmx.c | 22 +++++++++++++++++++++- arch/x86/kvm/x86.c | 20 +------------------- include/linux/kvm_host.h | 2 -- 4 files changed, 23 insertions(+), 23 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index a98c7907110c..7a3d1dcfef39 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1495,7 +1495,7 @@ struct kvm_x86_init_ops { int (*disabled_by_bios)(void); int (*check_processor_compatibility)(void); int (*hardware_setup)(void); - bool (*intel_pt_intr_in_guest)(void); + unsigned int (*handle_intel_pt_intr)(void); struct kvm_x86_ops *runtime_ops; }; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 61a4f5ff2acd..33f92febe3ce 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7687,6 +7687,20 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, }; +static unsigned int vmx_handle_intel_pt_intr(void) +{ + struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); + + /* '0' on failure so that the !PT case can use a RET0 static call. */ + if (!kvm_arch_pmi_in_guest(vcpu)) + return 0; + + kvm_make_request(KVM_REQ_PMI, vcpu); + __set_bit(MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT, + (unsigned long *)&vcpu->arch.pmu.global_status); + return 1; +} + static __init void vmx_setup_user_return_msrs(void) { @@ -7713,6 +7727,8 @@ static __init void vmx_setup_user_return_msrs(void) kvm_add_user_return_msr(vmx_uret_msrs_list[i]); } +static struct kvm_x86_init_ops vmx_init_ops __initdata; + static __init int hardware_setup(void) { unsigned long host_bndcfgs; @@ -7873,6 +7889,10 @@ static __init int hardware_setup(void) return -EINVAL; if (!enable_ept || !cpu_has_vmx_intel_pt()) pt_mode = PT_MODE_SYSTEM; + if (pt_mode == PT_MODE_HOST_GUEST) + vmx_init_ops.handle_intel_pt_intr = vmx_handle_intel_pt_intr; + else + vmx_init_ops.handle_intel_pt_intr = NULL; setup_default_sgx_lepubkeyhash(); @@ -7898,7 +7918,7 @@ static struct kvm_x86_init_ops vmx_init_ops __initdata = { .disabled_by_bios = vmx_disabled_by_bios, .check_processor_compatibility = vmx_check_processor_compat, .hardware_setup = hardware_setup, - .intel_pt_intr_in_guest = vmx_pt_mode_is_host_guest, + .handle_intel_pt_intr = NULL, .runtime_ops = &vmx_x86_ops, }; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 1bea616402e6..b79b2d29260d 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8264,20 +8264,6 @@ static void kvm_timer_init(void) kvmclock_cpu_online, kvmclock_cpu_down_prep); } -static unsigned int kvm_handle_intel_pt_intr(void) -{ - struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); - - /* '0' on failure so that the !PT case can use a RET0 static call. */ - if (!kvm_arch_pmi_in_guest(vcpu)) - return 0; - - kvm_make_request(KVM_REQ_PMI, vcpu); - __set_bit(MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT, - (unsigned long *)&vcpu->arch.pmu.global_status); - return 1; -} - #ifdef CONFIG_X86_64 static void pvclock_gtod_update_fn(struct work_struct *work) { @@ -11031,11 +11017,7 @@ int kvm_arch_hardware_setup(void *opaque) memcpy(&kvm_x86_ops, ops->runtime_ops, sizeof(kvm_x86_ops)); kvm_ops_static_call_update(); - /* Temporary ugliness. */ - if (ops->intel_pt_intr_in_guest && ops->intel_pt_intr_in_guest()) - kvm_register_perf_callbacks(kvm_handle_intel_pt_intr); - else - kvm_register_perf_callbacks(NULL); + kvm_register_perf_callbacks(ops->handle_intel_pt_intr); if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES)) supported_xss = 0; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 34d99034852f..b9235c3ac6af 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1164,8 +1164,6 @@ static inline bool kvm_arch_intc_initialized(struct kvm *kvm) #endif #ifdef __KVM_WANT_PERF_CALLBACKS - -void kvm_set_intel_pt_intr_handler(unsigned int (*handler)(void)); unsigned long kvm_arch_vcpu_get_ip(struct kvm_vcpu *vcpu); void kvm_register_perf_callbacks(unsigned int (*pt_intr_handler)(void)); From patchwork Sat Aug 28 00:35:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12463147 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BF65C432BE for ; Sat, 28 Aug 2021 00:49:54 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D187460E73 for ; Sat, 28 Aug 2021 00:49:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D187460E73 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References :Mime-Version:Message-Id:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=pWKV1C1LypIKHscb3TdqBmAy+eVfACNHQTgwIQ0rKzc=; b=MCtkptP8dXuRIa rWYa8VKmboEqvruvkstU+gGSQKYTxsdzZyGAHEi100xKj5r2jM7Zn1hb70DmAvQNi3oYis7S7RbA/ 61eyyOQhTuC1iUCwagrfrxSa6lts3nU7WrT68hrEfmxGjdRqM9WwovwVXLqNT+1/GnqB9aW4ZqPng und3qETVi/lk5MtOHd/2Ns7SjmYwIxeykeZ/Ua0aEZ1MCVlHXD/qfom27VZaxmXZnzrYAH5Oca6qJ TPMST0hFK2uporxZB9Yd+d+/xUye5JYFQtuvd22vJN5trRaPQZ7e7kt1g8Pw46KQK9fPzZ3WMO8V5 k70Y+5sETyN62Bv6Vuaw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmU2-00DV44-Lm; Sat, 28 Aug 2021 00:46:09 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmL7-00DRCr-On for linux-arm-kernel@lists.infradead.org; Sat, 28 Aug 2021 00:36:56 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id c63-20020a25e5420000b0290580b26e708aso8289573ybh.12 for ; Fri, 27 Aug 2021 17:36:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=8sgYBBoPv+2JwgnSGYhHNhVlG76cGIWAxgCtg/V/coQ=; b=H5N4ryA5hWTUAeyuBk5XizZ7N0trA7JIKj0csm/SnRrBSvZI4fvOfdylQj2HI9uGWb FlgA7RqivFC20QT3Fgdghq9BURCUg84MC8EYToh6OEPEn485kgBO2eBqSmuI+BgUif0L X0t7BSgnBR2SJjDLf8mzg4n0cr87GdYxedcIPWDQMs1CgXbr3dAKzNI+Ec51COkbfFee M/XLorZxISoZWBr6d43w4+OCYBUTM9KVOhPDBEHcmSrOb67+e2vyyj7m1qI8rapB1/sj sUKaaIlY0oFXPavLrApCVJnrUrYLouBq/sRrV1srDbUivxaMXuuYz/XSg705ezUGG1EI utYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=8sgYBBoPv+2JwgnSGYhHNhVlG76cGIWAxgCtg/V/coQ=; b=cqhGyRW1HPR+ApAeIUbV8szuq9Ir5fFl3If7yHXy+fAUkMUuYbm1L5lonB1Wu37HWf 8pAAAMGA4ie8wRlQgRfsDD/DsSFNGwLIt/4qYfaIZwhZIZcdktkFplPEQ7BAJf0m28yi kq7G/wRHc8b3ARwp3ijaftg2GJ9FHJE5JO+nfNG8q6y+hOawNizbjytWGTj06+kMM80h 5wop1n8+wejhYL2cyJNlvF+z3jeR/0DlSPJiZMo2km/Sw3zvM6mU7oRPK9UuTdMBISaC emgb9s5szDN7HVEOoVQHDU5XlX8EA/pQiTKAMvaaFtj29neg8hz3JgxRjSPkdg4WHudI Eh/w== X-Gm-Message-State: AOAM533oLV0ht29gT9Ro+seBFi8vQmf+VsX2fYAur1Zpo7B/K2bbjh16 5ExKtDGOnnO2EzYx2/oTnDWNpLlt0zw= X-Google-Smtp-Source: ABdhPJzymXWQza3vAsItF0sAPdCE8iLSMVzwPpM+LtdWMA9TXI8otz549S9ZetrsOJXPExeTt/x3hD0mXJY= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:f66c:b851:7e79:7ed4]) (user=seanjc job=sendgmr) by 2002:a25:bb48:: with SMTP id b8mr7895492ybk.275.1630111012286; Fri, 27 Aug 2021 17:36:52 -0700 (PDT) Date: Fri, 27 Aug 2021 17:35:57 -0700 In-Reply-To: <20210828003558.713983-1-seanjc@google.com> Message-Id: <20210828003558.713983-13-seanjc@google.com> Mime-Version: 1.0 References: <20210828003558.713983-1-seanjc@google.com> X-Mailer: git-send-email 2.33.0.259.gc128427fd7-goog Subject: [PATCH v2 12/13] KVM: arm64: Convert to the generic perf callbacks From: Sean Christopherson To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Will Deacon , Mark Rutland , Catalin Marinas , Marc Zyngier , Guo Ren , Nick Hu , Greentime Hu , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , Thomas Gleixner , Borislav Petkov , x86@kernel.org, Paolo Bonzini , Boris Ostrovsky , Juergen Gross Cc: Alexander Shishkin , Jiri Olsa , Namhyung Kim , James Morse , Alexandru Elisei , Suzuki K Poulose , "H. Peter Anvin" , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Stefano Stabellini , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Artem Kashkanov , Like Xu , Zhu Lingshan X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210827_173653_941591_020F4C48 X-CRM114-Status: GOOD ( 13.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Drop arm64's version of the callbacks in favor of the callbacks provided by generic KVM, which are semantically identical. Implement the "get ip" hook as needed. Signed-off-by: Sean Christopherson --- arch/arm64/include/asm/kvm_host.h | 12 +++++++++++ arch/arm64/kvm/arm.c | 5 +++++ arch/arm64/kvm/perf.c | 34 ++----------------------------- 3 files changed, 19 insertions(+), 32 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index ed940aec89e0..73dc402ded1f 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -673,6 +673,18 @@ int io_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa); void kvm_perf_init(void); void kvm_perf_teardown(void); +#ifdef CONFIG_PERF_EVENTS +#define __KVM_WANT_PERF_CALLBACKS +static inline bool kvm_arch_pmi_in_guest(struct kvm_vcpu *vcpu) +{ + /* Any callback while a vCPU is loaded is considered to be in guest. */ + return !!vcpu; +} +#else +static inline void kvm_register_perf_callbacks(void) {} +static inline void kvm_unregister_perf_callbacks(void) {} +#endif + long kvm_hypercall_pv_features(struct kvm_vcpu *vcpu); gpa_t kvm_init_stolen_time(struct kvm_vcpu *vcpu); void kvm_update_stolen_time(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index e9a2b8f27792..2b542fdc237e 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -500,6 +500,11 @@ bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu) return vcpu_mode_priv(vcpu); } +unsigned long kvm_arch_vcpu_get_ip(struct kvm_vcpu *vcpu) +{ + return *vcpu_pc(vcpu); +} + /* Just ensure a guest exit from a particular CPU */ static void exit_vm_noop(void *info) { diff --git a/arch/arm64/kvm/perf.c b/arch/arm64/kvm/perf.c index 893de1a51fea..0b902e0d5b5d 100644 --- a/arch/arm64/kvm/perf.c +++ b/arch/arm64/kvm/perf.c @@ -13,45 +13,15 @@ DEFINE_STATIC_KEY_FALSE(kvm_arm_pmu_available); -static unsigned int kvm_guest_state(void) -{ - struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); - unsigned int state; - - if (!vcpu) - return 0; - - state = PERF_GUEST_ACTIVE; - if (!vcpu_mode_priv(vcpu)) - state |= PERF_GUEST_USER; - - return state; -} - -static unsigned long kvm_get_guest_ip(void) -{ - struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); - - if (WARN_ON_ONCE(!vcpu)) - return 0; - - return *vcpu_pc(vcpu); -} - -static struct perf_guest_info_callbacks kvm_guest_cbs = { - .state = kvm_guest_state, - .get_ip = kvm_get_guest_ip, -}; - void kvm_perf_init(void) { if (kvm_pmu_probe_pmuver() != 0xf && !is_protected_kvm_enabled()) static_branch_enable(&kvm_arm_pmu_available); - perf_register_guest_info_callbacks(&kvm_guest_cbs); + kvm_register_perf_callbacks(NULL); } void kvm_perf_teardown(void) { - perf_unregister_guest_info_callbacks(); + kvm_unregister_perf_callbacks(); } From patchwork Sat Aug 28 00:35:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12463149 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6AFCC432BE for ; Sat, 28 Aug 2021 00:50:45 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AC99360F14 for ; Sat, 28 Aug 2021 00:50:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org AC99360F14 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References :Mime-Version:Message-Id:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=MOaHcqB0saTJckVxATzVGMaO8JdUx2RoZTM5MWuVdpI=; b=2uZ6BV4LVWV8DB /X5Cciq0C9TO8yEGfqGDleeNJWvBIxf5jzLWJK4j8qzl9zENA7wPWH2eRDbOo2oKdUcrBDg5a5Pah ueOxfJ+p+fY5iDigiEbSGDrQaMtzyhvdvfIOz9M+YL+pFbTONL9E6NnPXNZW6oBmS9OlFmpfSZi+K 2rO+mW+3w4QMUoAcffevhnOGtO4yPNpzPU6Sf7EJjkOZ5SV/ORw34TkfS08XWMZQJiOqKa/GDN0mR SD4i9epjsK0dc+O19srZH6pxEtT//xtX95M8TS37PBqZ7yUVYmaIle3BT6uAMjwiqHRFLm6jBs14+ n3dYf/l8uhVBLn2QrYpg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmVS-00DVWP-RK; Sat, 28 Aug 2021 00:47:38 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mJmLA-00DREQ-3p for linux-arm-kernel@lists.infradead.org; Sat, 28 Aug 2021 00:36:58 +0000 Received: by mail-yb1-xb49.google.com with SMTP id g192-20020a25dbc9000000b0059bd2958c8aso5465243ybf.5 for ; Fri, 27 Aug 2021 17:36:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=MvwJUHJ7nL7QAZpmwIc3YN3b2GBJgXnzulug8SK2Qjw=; b=Rd8GZlOM7483aZ/NulymMp3S9HO5bS1npFsrbHkWVgmtUT7AYmnlfjuL+oAn/+3Dri +5cR2ODgibWqpXhDO8BZ5XeMEmoa6kHMNeLnQOJzh9PWAYLTkPF2NgJK5Mby7Cvmotge 2PyjseoJ7Slvdtb6cq6mgARo7D+wvyYizSBuQjnshLNdziqxPWZnRNLzvpaZZ+ivbpGR 1gkx3rIpMiVsfiqaiEDc1i/t4zHv55U+FFPE/qx5o8z68VoLNkaUyOM/1xRhzVqRYHrY re+Q8PzB8BWoikOAEufy0epFWC9x2ug/ZDxYOVgylgofjF85zivnlpT4bnrEu+QzpLoF hYZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=MvwJUHJ7nL7QAZpmwIc3YN3b2GBJgXnzulug8SK2Qjw=; b=mrY+kxpqNtkW6Ik69EROoeJz6sCPP73S7nx4VJOkWqV6DVJ+K6yGDyU3A2Nn/7EkpK LiYHRkkt6BNRPE7Ve8CRkD2tJJYLbwEvyC8yeQCDoaKMHpo6y1OTKNNnAv0l+p/V1AAc 4Zbo6urar2el1Sh/8ACrSLzfEuQrWNEIcwsM1eoNv7uZyCzTZWHc8s4UolAsu1MGnc7K mqyuAR/tvF1ordPJgOtUsmbHiHxROABV82TpB159uxkyLQMFXsWYJLTjw9cE6KlBKXII aEXRTOSyZAzoT4IxYLy2eh6LObfoRrkWhE8rr9zwc281viTmokheWLY2Q4PC6eYcPvrF qWCg== X-Gm-Message-State: AOAM530IIjkl69dVfCv/ICkI/ndvoSQDDI2EsQw8sY8yVpTkhkSGlzvM +l6v5M+ereFWgou1ky1I923FLzC2jAU= X-Google-Smtp-Source: ABdhPJyAd60vBQXQHXvEDagnL5C8DqU2H00FG/h0rAqZ+ilMScmE0veBYXU9kzdGqI+AZkskYlaiz8r6ASk= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:f66c:b851:7e79:7ed4]) (user=seanjc job=sendgmr) by 2002:a25:9d83:: with SMTP id v3mr8323765ybp.97.1630111014478; Fri, 27 Aug 2021 17:36:54 -0700 (PDT) Date: Fri, 27 Aug 2021 17:35:58 -0700 In-Reply-To: <20210828003558.713983-1-seanjc@google.com> Message-Id: <20210828003558.713983-14-seanjc@google.com> Mime-Version: 1.0 References: <20210828003558.713983-1-seanjc@google.com> X-Mailer: git-send-email 2.33.0.259.gc128427fd7-goog Subject: [PATCH v2 13/13] KVM: arm64: Drop perf.c and fold its tiny bits of code into arm.c / pmu.c From: Sean Christopherson To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Will Deacon , Mark Rutland , Catalin Marinas , Marc Zyngier , Guo Ren , Nick Hu , Greentime Hu , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , Thomas Gleixner , Borislav Petkov , x86@kernel.org, Paolo Bonzini , Boris Ostrovsky , Juergen Gross Cc: Alexander Shishkin , Jiri Olsa , Namhyung Kim , James Morse , Alexandru Elisei , Suzuki K Poulose , "H. Peter Anvin" , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Stefano Stabellini , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Artem Kashkanov , Like Xu , Zhu Lingshan X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210827_173656_204604_9154F0F8 X-CRM114-Status: GOOD ( 20.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Call KVM's (un)register perf callbacks helpers directly from arm.c, and move the PMU bits into pmu.c and rename the related helper accordingly. Signed-off-by: Sean Christopherson --- arch/arm64/include/asm/kvm_host.h | 3 --- arch/arm64/kvm/Makefile | 2 +- arch/arm64/kvm/arm.c | 6 ++++-- arch/arm64/kvm/perf.c | 27 --------------------------- arch/arm64/kvm/pmu.c | 8 ++++++++ include/kvm/arm_pmu.h | 1 + 6 files changed, 14 insertions(+), 33 deletions(-) delete mode 100644 arch/arm64/kvm/perf.c diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 73dc402ded1f..d549b58120bc 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -670,9 +670,6 @@ unsigned long kvm_mmio_read_buf(const void *buf, unsigned int len); int kvm_handle_mmio_return(struct kvm_vcpu *vcpu); int io_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa); -void kvm_perf_init(void); -void kvm_perf_teardown(void); - #ifdef CONFIG_PERF_EVENTS #define __KVM_WANT_PERF_CALLBACKS static inline bool kvm_arch_pmi_in_guest(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile index 989bb5dad2c8..0bcc378b7961 100644 --- a/arch/arm64/kvm/Makefile +++ b/arch/arm64/kvm/Makefile @@ -12,7 +12,7 @@ obj-$(CONFIG_KVM) += hyp/ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \ $(KVM)/vfio.o $(KVM)/irqchip.o $(KVM)/binary_stats.o \ - arm.o mmu.o mmio.o psci.o perf.o hypercalls.o pvtime.o \ + arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \ inject_fault.o va_layout.o handle_exit.o \ guest.o debug.o reset.o sys_regs.o \ vgic-sys-reg-v3.o fpsimd.o pmu.o \ diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 2b542fdc237e..48f89d80f464 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1744,7 +1744,9 @@ static int init_subsystems(void) if (err) goto out; - kvm_perf_init(); + kvm_pmu_init(); + kvm_register_perf_callbacks(NULL); + kvm_sys_reg_table_init(); out: @@ -2160,7 +2162,7 @@ int kvm_arch_init(void *opaque) /* NOP: Compiling as a module not supported */ void kvm_arch_exit(void) { - kvm_perf_teardown(); + kvm_unregister_perf_callbacks(); } static int __init early_kvm_mode_cfg(char *arg) diff --git a/arch/arm64/kvm/perf.c b/arch/arm64/kvm/perf.c deleted file mode 100644 index 0b902e0d5b5d..000000000000 --- a/arch/arm64/kvm/perf.c +++ /dev/null @@ -1,27 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * Based on the x86 implementation. - * - * Copyright (C) 2012 ARM Ltd. - * Author: Marc Zyngier - */ - -#include -#include - -#include - -DEFINE_STATIC_KEY_FALSE(kvm_arm_pmu_available); - -void kvm_perf_init(void) -{ - if (kvm_pmu_probe_pmuver() != 0xf && !is_protected_kvm_enabled()) - static_branch_enable(&kvm_arm_pmu_available); - - kvm_register_perf_callbacks(NULL); -} - -void kvm_perf_teardown(void) -{ - kvm_unregister_perf_callbacks(); -} diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c index 03a6c1f4a09a..d98b57a17043 100644 --- a/arch/arm64/kvm/pmu.c +++ b/arch/arm64/kvm/pmu.c @@ -7,6 +7,14 @@ #include #include +DEFINE_STATIC_KEY_FALSE(kvm_arm_pmu_available); + +void kvm_pmu_init(void) +{ + if (kvm_pmu_probe_pmuver() != 0xf && !is_protected_kvm_enabled()) + static_branch_enable(&kvm_arm_pmu_available); +} + /* * Given the perf event attributes and system type, determine * if we are going to need to switch counters at guest entry/exit. diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h index 864b9997efb2..42270676498d 100644 --- a/include/kvm/arm_pmu.h +++ b/include/kvm/arm_pmu.h @@ -14,6 +14,7 @@ #define ARMV8_PMU_MAX_COUNTER_PAIRS ((ARMV8_PMU_MAX_COUNTERS + 1) >> 1) DECLARE_STATIC_KEY_FALSE(kvm_arm_pmu_available); +void kvm_pmu_init(void); static __always_inline bool kvm_arm_support_pmu_v3(void) {