From patchwork Fri Apr 12 01:50:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kyle Huey X-Patchwork-Id: 13626688 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-qk1-f169.google.com (mail-qk1-f169.google.com [209.85.222.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 120C5DF58 for ; Fri, 12 Apr 2024 01:50:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712886648; cv=none; b=mr4DSjoq2HLnCFs1CRYliaBhLcPzC0Zbay1GMUqwkNK1fdNgjhVR2O/S6GJECkq+X3b8/PGu4iMQaWnGrsVl4JTuUZ1oOFb8KB2TLxbl+3arTH3voVJVvB0+xZNmBgYzTIH5DX5wAoB3hC3BSGXex3iwZ1UVrcbRfZmVv+G0d/k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712886648; c=relaxed/simple; bh=lLcJXMGYAt18ckXzjBJt3RZcIlon0AJDakZC8+N8M0M=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=r5kY52Snju41gC8fmzL8a59Zwd3U0MGd3pawRavC3EOmRijSayx3xdhOLgLaj+SAbC0Op0KprhVq+2Nodj2vGLCzI3s/CWDbl1/WPBxDcDZxKKrJO90ifsqwfyereQBTDOB0nmUB8eg+8iXCU04Wd19MZcygSTFZqAZE/GdhaZU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com; spf=pass smtp.mailfrom=kylehuey.com; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b=FQ3BuQZb; arc=none smtp.client-ip=209.85.222.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b="FQ3BuQZb" Received: by mail-qk1-f169.google.com with SMTP id af79cd13be357-78ec78c4fceso56190785a.0 for ; Thu, 11 Apr 2024 18:50:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kylehuey.com; s=google; t=1712886646; x=1713491446; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2I8ioTIE3St0NCO304/67c5ZZcTg34djE+ltGa2DGgk=; b=FQ3BuQZb8LtOhmt890Bmeq2fPTjqyrHlVC00FaR2K+bSiNS5ffhWMIBCbKqWbCoF9a yTKEojGuNSBLHgeauasteqnUs7ZXlVqVebsjOuOToLwj2QxBsBLCTIrm1YgnQ4ZdptSY Kn0JB14JBHBz64zM4H39qMyt7yft6iJNslfW7EP1u006R86EdrKnRdvwl1GqckKj6Ie2 bmgYnPNGoRRPQnX0i07qpCzGA/iBz/eNkYm2z65qVVDMY6G7ohPS6eRhxyOMeMtXrw90 /p4V0rsU/WxnRWomdyk0/CjfkbhOjQ+wcY/Gtshn6HRziwKWDkEIHi6FpxzvGm7vATrB pbfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712886646; x=1713491446; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2I8ioTIE3St0NCO304/67c5ZZcTg34djE+ltGa2DGgk=; b=bcW8NlyH6M5MJRK8ht3GWnB30IlG3A4NWYL7qPmp/XOfWeL3APQGbui96qljxQ40kQ YkhVjhlPLTmldYjyn8GaUEY+w3Qqxd4k3BU09sfFb5sviclJz9qWeLh9CEb/z+/vPcJp ePJCl5N4RdsU5yxsbzjw+B4TYLKoDUykSBclKhRvH5mt0gVD8UwgUNk7omaPXRk2uPye bvJYenaivTk0jIULfl++szUWqjs/YoJr4o+puRYyF/HguUeVnzrXFGSfMThnkwAtPlqL zC5ggQpkbUwf9cpgZ1jp4jcMD/nWcnYeznbs6W23FL+97upwrqWu+Ej+Qhgj7x+FHVTW T6Cw== X-Forwarded-Encrypted: i=1; AJvYcCXBIGKo2kAmVV9oJHddDvXHeeZTH5RXbmiY7KU8lqX2nT7s22+ZfmKYjhxFY5HUPyGaL45UTc6i6NlClkH/d9wHhEyK X-Gm-Message-State: AOJu0YxTZCSPscbaRNV4M6whKhOreMj9YZphGj36KguA2nox3mRd6zSN I6BxBgEUMk3ww4MeqtAcmK2CFEj4nW0IIxoBAbreow/1y+n0rBz7sdI1WKbUUQ== X-Google-Smtp-Source: AGHT+IEpPEvWFpAFcQSe+VQV8imp3rgz2Yo/hr1w0rVz+GPtikTxqKZX7MSio59IOMcIrmeqvbBuxg== X-Received: by 2002:a05:620a:4589:b0:78e:b996:db93 with SMTP id bp9-20020a05620a458900b0078eb996db93mr3057028qkb.3.1712886646096; Thu, 11 Apr 2024 18:50:46 -0700 (PDT) Received: from ip-172-31-44-15.us-east-2.compute.internal (ec2-52-15-100-147.us-east-2.compute.amazonaws.com. [52.15.100.147]) by smtp.googlemail.com with ESMTPSA id f10-20020a05620a15aa00b0078d76c1178esm1756677qkk.119.2024.04.11.18.50.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Apr 2024 18:50:45 -0700 (PDT) From: Kyle Huey X-Google-Original-From: Kyle Huey To: Kyle Huey , linux-kernel@vger.kernel.org, Andrii Nakryiko , Jiri Olsa , Namhyung Kim , Marco Elver , Yonghong Song , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo Cc: Robert O'Callahan , bpf@vger.kernel.org, Ingo Molnar , Mark Rutland , Alexander Shishkin , Ian Rogers , Adrian Hunter , linux-perf-users@vger.kernel.org Subject: [PATCH v6 1/7] perf/bpf: Reorder bpf_overflow_handler() ahead of __perf_event_overflow() Date: Thu, 11 Apr 2024 18:50:13 -0700 Message-Id: <20240412015019.7060-2-khuey@kylehuey.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240412015019.7060-1-khuey@kylehuey.com> References: <20240412015019.7060-1-khuey@kylehuey.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This will allow __perf_event_overflow() to call bpf_overflow_handler(). Signed-off-by: Kyle Huey --- kernel/events/core.c | 183 ++++++++++++++++++++++--------------------- 1 file changed, 92 insertions(+), 91 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 724e6d7e128f..ee025125a681 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -9544,6 +9544,98 @@ static inline bool sample_is_allowed(struct perf_event *event, struct pt_regs *r return true; } +#ifdef CONFIG_BPF_SYSCALL +static void bpf_overflow_handler(struct perf_event *event, + struct perf_sample_data *data, + struct pt_regs *regs) +{ + struct bpf_perf_event_data_kern ctx = { + .data = data, + .event = event, + }; + struct bpf_prog *prog; + int ret = 0; + + ctx.regs = perf_arch_bpf_user_pt_regs(regs); + if (unlikely(__this_cpu_inc_return(bpf_prog_active) != 1)) + goto out; + rcu_read_lock(); + prog = READ_ONCE(event->prog); + if (prog) { + perf_prepare_sample(data, event, regs); + ret = bpf_prog_run(prog, &ctx); + } + rcu_read_unlock(); +out: + __this_cpu_dec(bpf_prog_active); + if (!ret) + return; + + event->orig_overflow_handler(event, data, regs); +} + +static int perf_event_set_bpf_handler(struct perf_event *event, + struct bpf_prog *prog, + u64 bpf_cookie) +{ + if (event->overflow_handler_context) + /* hw breakpoint or kernel counter */ + return -EINVAL; + + if (event->prog) + return -EEXIST; + + if (prog->type != BPF_PROG_TYPE_PERF_EVENT) + return -EINVAL; + + if (event->attr.precise_ip && + prog->call_get_stack && + (!(event->attr.sample_type & PERF_SAMPLE_CALLCHAIN) || + event->attr.exclude_callchain_kernel || + event->attr.exclude_callchain_user)) { + /* + * On perf_event with precise_ip, calling bpf_get_stack() + * may trigger unwinder warnings and occasional crashes. + * bpf_get_[stack|stackid] works around this issue by using + * callchain attached to perf_sample_data. If the + * perf_event does not full (kernel and user) callchain + * attached to perf_sample_data, do not allow attaching BPF + * program that calls bpf_get_[stack|stackid]. + */ + return -EPROTO; + } + + event->prog = prog; + event->bpf_cookie = bpf_cookie; + event->orig_overflow_handler = READ_ONCE(event->overflow_handler); + WRITE_ONCE(event->overflow_handler, bpf_overflow_handler); + return 0; +} + +static void perf_event_free_bpf_handler(struct perf_event *event) +{ + struct bpf_prog *prog = event->prog; + + if (!prog) + return; + + WRITE_ONCE(event->overflow_handler, event->orig_overflow_handler); + event->prog = NULL; + bpf_prog_put(prog); +} +#else +static int perf_event_set_bpf_handler(struct perf_event *event, + struct bpf_prog *prog, + u64 bpf_cookie) +{ + return -EOPNOTSUPP; +} + +static void perf_event_free_bpf_handler(struct perf_event *event) +{ +} +#endif + /* * Generic event overflow handling, sampling. */ @@ -10422,97 +10514,6 @@ static void perf_event_free_filter(struct perf_event *event) ftrace_profile_free_filter(event); } -#ifdef CONFIG_BPF_SYSCALL -static void bpf_overflow_handler(struct perf_event *event, - struct perf_sample_data *data, - struct pt_regs *regs) -{ - struct bpf_perf_event_data_kern ctx = { - .data = data, - .event = event, - }; - struct bpf_prog *prog; - int ret = 0; - - ctx.regs = perf_arch_bpf_user_pt_regs(regs); - if (unlikely(__this_cpu_inc_return(bpf_prog_active) != 1)) - goto out; - rcu_read_lock(); - prog = READ_ONCE(event->prog); - if (prog) { - perf_prepare_sample(data, event, regs); - ret = bpf_prog_run(prog, &ctx); - } - rcu_read_unlock(); -out: - __this_cpu_dec(bpf_prog_active); - if (!ret) - return; - - event->orig_overflow_handler(event, data, regs); -} - -static int perf_event_set_bpf_handler(struct perf_event *event, - struct bpf_prog *prog, - u64 bpf_cookie) -{ - if (event->overflow_handler_context) - /* hw breakpoint or kernel counter */ - return -EINVAL; - - if (event->prog) - return -EEXIST; - - if (prog->type != BPF_PROG_TYPE_PERF_EVENT) - return -EINVAL; - - if (event->attr.precise_ip && - prog->call_get_stack && - (!(event->attr.sample_type & PERF_SAMPLE_CALLCHAIN) || - event->attr.exclude_callchain_kernel || - event->attr.exclude_callchain_user)) { - /* - * On perf_event with precise_ip, calling bpf_get_stack() - * may trigger unwinder warnings and occasional crashes. - * bpf_get_[stack|stackid] works around this issue by using - * callchain attached to perf_sample_data. If the - * perf_event does not full (kernel and user) callchain - * attached to perf_sample_data, do not allow attaching BPF - * program that calls bpf_get_[stack|stackid]. - */ - return -EPROTO; - } - - event->prog = prog; - event->bpf_cookie = bpf_cookie; - event->orig_overflow_handler = READ_ONCE(event->overflow_handler); - WRITE_ONCE(event->overflow_handler, bpf_overflow_handler); - return 0; -} - -static void perf_event_free_bpf_handler(struct perf_event *event) -{ - struct bpf_prog *prog = event->prog; - - if (!prog) - return; - - WRITE_ONCE(event->overflow_handler, event->orig_overflow_handler); - event->prog = NULL; - bpf_prog_put(prog); -} -#else -static int perf_event_set_bpf_handler(struct perf_event *event, - struct bpf_prog *prog, - u64 bpf_cookie) -{ - return -EOPNOTSUPP; -} -static void perf_event_free_bpf_handler(struct perf_event *event) -{ -} -#endif - /* * returns true if the event is a tracepoint, or a kprobe/upprobe created * with perf_event_open() From patchwork Fri Apr 12 01:50:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kyle Huey X-Patchwork-Id: 13626689 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-qk1-f174.google.com (mail-qk1-f174.google.com [209.85.222.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F0C5101EE for ; Fri, 12 Apr 2024 01:50:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712886651; cv=none; b=nGhRg001WuQjxfAwhPXrggVZnfwhFAnfMv8J0ueQWZiJTBluqOQGeRMJIoDyKnHXHqgdV3FEeRWzn62iHVD6NjS00T7vv0fntiV8EIEZlNCfD5mS4/s1tb7PFaoLFsvP7YO86WkCGmCKPGdwVXnno3ZER07OtD3dSInckrzAWco= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712886651; c=relaxed/simple; bh=7YouWyZuxbSYzHxqPUnW+aY1UMlIJgGnhI+PHUoOFhQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=e1BQHX8LOF0d2tTr5KCYHTQJ16I8XtTElR2BzRAWdwPiJ08+kUKe2eS5kgWLHy4VM0hJLt2puD1Brsy+BRUd23tT21keSCQM+/hyML3mZxBFKslEPYSawUvcr+bS+6rMT04bj2UHpWX4MTOCX/EUzzRcmXdBdfYV81RVvCA5caA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com; spf=pass smtp.mailfrom=kylehuey.com; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b=aU0bVvpI; arc=none smtp.client-ip=209.85.222.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b="aU0bVvpI" Received: by mail-qk1-f174.google.com with SMTP id af79cd13be357-789f00aba19so21723885a.0 for ; Thu, 11 Apr 2024 18:50:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kylehuey.com; s=google; t=1712886649; x=1713491449; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IR+aRAW+xEUckMtcdjKW/EEXnX6Dbx8rKcxaN3dNqzg=; b=aU0bVvpItdjjGy0Uh4y25anjArKIOtlIy5bEQk0mhmXbgUtnAoxQU+EWL0JLXBUsGE PaFzOpaJzH8GVLLDtvVTKCjsLTFLU3B0AAlawSh4HGgxhag7TfwZH/R+MYRm7QUwbMAG w8FRBHIep8Bt+j5TOyytwhtljABL69vAvnkrXANE7rldNPtRV4s9KqL/BVLLdaS0EEhU S7NYM1+pr3opDH0IQanKJieTjzymm+1VP+TTtawBfPlvipoLVVtG8Lq+0a4+iHZQoxji bYan7dULV/jISLCvZvrhf5LY4IUpVNt3UDYdMtkiPY1qHjhOBcRJAb9+2KRRgreFRl9R P3hA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712886649; x=1713491449; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IR+aRAW+xEUckMtcdjKW/EEXnX6Dbx8rKcxaN3dNqzg=; b=YGlHyKwklV/PqOSOw2yKyQdGimzMETeVha13bbaNNt2alHRhtk/CYMo48Y/SedB2dE 5puCxH6A5TWk3FR5lq4ittzwp4Lx42P5kSFf9OXOLTr5jZCf9OC5i+xH0rVCc+6CKVVd tuoelsICShGtkoQFl/bO0pcrZ6Z6L2FTSf34kpiBK0TYBsBBiHno4DJkKmeujGK9eC23 EKM0l0pa1QIXAZYjDZEd8B0+w8qbjPT4n7/5UEdCJLV0zZKVCOGpyNRtsmfUhjfW23M3 g+zp3crPnu3oyinliUnckK0sJ1RdJdi/lTXJvPLnAQGf1H74T7V6gHgWr+PufuIoWs19 g/zQ== X-Forwarded-Encrypted: i=1; AJvYcCVV2XHOH/3a8iT8wMUyU8a8mBv6WtMVYeEImvhZJNw95uvzHWN6YNP4fSi5F1Gf/CmQXlSYNrHzwwU8ZrE1iGNTLyKk X-Gm-Message-State: AOJu0Yw4rkoVXnzOg/4PCk0ob/ajMyDn22Dwglwv+rZUV/oADuUw3gpt sMUeSYgxTeI5RVeqPyZSX2K8/sCF4HjZpMuMrTLxCcIfFjDXdclD8jzgsMCZTQ== X-Google-Smtp-Source: AGHT+IHD5DOmOr2nUz59Bfple5VcMmCJtdiKzwwn2KQ/kHy7BLf5evZIyHYa8ySlQOOQSWDiJjefKQ== X-Received: by 2002:a05:620a:1983:b0:78d:6339:b943 with SMTP id bm3-20020a05620a198300b0078d6339b943mr1698615qkb.20.1712886649123; Thu, 11 Apr 2024 18:50:49 -0700 (PDT) Received: from ip-172-31-44-15.us-east-2.compute.internal (ec2-52-15-100-147.us-east-2.compute.amazonaws.com. [52.15.100.147]) by smtp.googlemail.com with ESMTPSA id f10-20020a05620a15aa00b0078d76c1178esm1756677qkk.119.2024.04.11.18.50.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Apr 2024 18:50:48 -0700 (PDT) From: Kyle Huey X-Google-Original-From: Kyle Huey To: Kyle Huey , linux-kernel@vger.kernel.org, Andrii Nakryiko , Jiri Olsa , Namhyung Kim , Marco Elver , Yonghong Song , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo Cc: Robert O'Callahan , bpf@vger.kernel.org, Ingo Molnar , Mark Rutland , Alexander Shishkin , Ian Rogers , Adrian Hunter , linux-perf-users@vger.kernel.org Subject: [PATCH v6 2/7] perf/bpf: Create bpf_overflow_handler() stub for !CONFIG_BPF_SYSCALL Date: Thu, 11 Apr 2024 18:50:14 -0700 Message-Id: <20240412015019.7060-3-khuey@kylehuey.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240412015019.7060-1-khuey@kylehuey.com> References: <20240412015019.7060-1-khuey@kylehuey.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This will allow __perf_event_overflow() (which is independent of CONFIG_BPF_SYSCALL) to call bpf_overflow_handler(). Signed-off-by: Kyle Huey --- kernel/events/core.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/kernel/events/core.c b/kernel/events/core.c index ee025125a681..a7c2a739a27c 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -9624,6 +9624,12 @@ static void perf_event_free_bpf_handler(struct perf_event *event) bpf_prog_put(prog); } #else +static void bpf_overflow_handler(struct perf_event *event, + struct perf_sample_data *data, + struct pt_regs *regs) +{ +} + static int perf_event_set_bpf_handler(struct perf_event *event, struct bpf_prog *prog, u64 bpf_cookie) From patchwork Fri Apr 12 01:50:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kyle Huey X-Patchwork-Id: 13626690 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C7F7113FF5 for ; Fri, 12 Apr 2024 01:50:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712886655; cv=none; b=d2yMuJzV1cuteXzegQPYfMbnFlmZFaloGQ7h0x2iC7c5X5SpZ2bL0Aig1Z9LxYcU2MpCZG1iBd+KR3l/12jGYsOWgFvLKVkOryO4Hi8ZU61E7lCkSXMFDbw9OfTJ3VEtJDIw3S094DVPp9wdwysalHGQiMlHZMvpX9iPA/ogbHs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712886655; c=relaxed/simple; bh=mqvWJPjJTCHg1USd6XVsQJr18Mdq8CtI+8+xAlWcLkc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=dvYxWQ0zIROP8y8exCQF0OZOd9krnhhtbiGJ2eoItih80h4Gq27eL+DLqt8+nUekqcsEI4CHwBwUosrkocWI4OUNB5/K8tEGutEqyfsHDAE8idUkjY5+uU+YivXkPhoVI1QkqDCMe4pZBngnVXjqptKXeKe1YyOHrrbLK0RNxfE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com; spf=pass smtp.mailfrom=kylehuey.com; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b=ikZxtLYP; arc=none smtp.client-ip=209.85.222.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b="ikZxtLYP" Received: by mail-qk1-f181.google.com with SMTP id af79cd13be357-78ecd752a7cso2073185a.0 for ; Thu, 11 Apr 2024 18:50:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kylehuey.com; s=google; t=1712886652; x=1713491452; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aQsjAVDSJojOvZcYxeh0Pu2aG5e5UAQY3ALs9BYs2YU=; b=ikZxtLYPc50oIz/X4NEG7KZwqzRruUor+I3VOuDCfG1LQyrc5eylNrOuhZSfmSHmAc am3oTb/lwx3Jy6pqHyzqgcZfOwSkwSbeGQex0IwdoOT4Kt6dbEVbigxJTP7RF3khhV0I 17fNqUWNiycdyHmG0bKl0Keqnqw+5Tc7G+k6rsLoJdlBuT0zmfU0DjBrMYAAA5Lrpglz l/cDG6eWN7C2FIUjo5yOr1CznpuXnDd6ImjaEGKYf66XxFnVXfc2oV6BRvzJpCWDMobG Af7OOslT8s9085vEGyCOAduaWwbVuKC+1AGlV3iSK6iA5dt1gegimT+XMjesqDuym9+u MQ+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712886652; x=1713491452; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aQsjAVDSJojOvZcYxeh0Pu2aG5e5UAQY3ALs9BYs2YU=; b=JaU+8uHuqcD3yU4MyWnD4TJaVC2o0aMJFMP0capIozcJJNViTW8NwCyHIopPMNQllQ xH+4j+DbnhPfIxo0bS7KVUs4ZP7lHAyxPEAtMzKJPG2YUrz/0fSwHF/ytnlFwEKj8SmY kt/BAkI2yXZZtlliLNRPZ6GcGsnzC6zqPPxCnfvjMKnpK7qpDXlVIAiaqVbuUHzqOsHv lBtqFwk81H0sK5RefG5vXsb41kNWcft9MAZ8TdZaqpjdf7cDM5PkQis4rNwUV4BzLhLB fImPNzPLcDlhdjYQ/61YYpqa2FnTZquc6QUpFjvkDb+Ucd+yLmtr+1+kokvNKsEm0R0v DZKw== X-Forwarded-Encrypted: i=1; AJvYcCXn6qrXw069UeC75HfCf062oX57yJiZazk4MLT/0cQQH9fem2jDu9rb7IUY7x1gxeAWUb88XgaCg4HKYR7HRlRpsi/9 X-Gm-Message-State: AOJu0YymbTMDhK5LOT3E1F631eY1v1BMQqjdg51SxbJEsOwpyu6DZ1I8 2pw78N38evQaQnND/5GQ3Uiwh7p9LtjPy3IpUeN48OaReFGHxWXcQE/zfjkXJK6t0NCqiRo368Y toDzd X-Google-Smtp-Source: AGHT+IGKuvAU3YlymoVRT8hC2x9jWoKM60y5jVQlZXrkYD5Z3LF1GHcVUdYxkusrFaNEaXHiGScCmQ== X-Received: by 2002:a05:620a:15ac:b0:78d:3b3f:50cd with SMTP id f12-20020a05620a15ac00b0078d3b3f50cdmr1344767qkk.65.1712886651836; Thu, 11 Apr 2024 18:50:51 -0700 (PDT) Received: from ip-172-31-44-15.us-east-2.compute.internal (ec2-52-15-100-147.us-east-2.compute.amazonaws.com. [52.15.100.147]) by smtp.googlemail.com with ESMTPSA id f10-20020a05620a15aa00b0078d76c1178esm1756677qkk.119.2024.04.11.18.50.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Apr 2024 18:50:51 -0700 (PDT) From: Kyle Huey X-Google-Original-From: Kyle Huey To: Kyle Huey , linux-kernel@vger.kernel.org, Andrii Nakryiko , Jiri Olsa , Namhyung Kim , Marco Elver , Yonghong Song , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo Cc: Robert O'Callahan , bpf@vger.kernel.org, Ingo Molnar , Mark Rutland , Alexander Shishkin , Ian Rogers , Adrian Hunter , linux-perf-users@vger.kernel.org Subject: [PATCH v6 3/7] perf/bpf: Remove #ifdef CONFIG_BPF_SYSCALL from struct perf_event members Date: Thu, 11 Apr 2024 18:50:15 -0700 Message-Id: <20240412015019.7060-4-khuey@kylehuey.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240412015019.7060-1-khuey@kylehuey.com> References: <20240412015019.7060-1-khuey@kylehuey.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This will allow __perf_event_overflow() (which is independent of CONFIG_BPF_SYSCALL) to use struct perf_event's prog to decide whether to call bpf_overflow_handler(). Suggested-by: Ingo Molnar Signed-off-by: Kyle Huey --- include/linux/perf_event.h | 2 -- 1 file changed, 2 deletions(-) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index d2a15c0c6f8a..07cd4722dedb 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -809,11 +809,9 @@ struct perf_event { u64 (*clock)(void); perf_overflow_handler_t overflow_handler; void *overflow_handler_context; -#ifdef CONFIG_BPF_SYSCALL perf_overflow_handler_t orig_overflow_handler; struct bpf_prog *prog; u64 bpf_cookie; -#endif #ifdef CONFIG_EVENT_TRACING struct trace_event_call *tp_event; From patchwork Fri Apr 12 01:50:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kyle Huey X-Patchwork-Id: 13626691 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-qk1-f171.google.com (mail-qk1-f171.google.com [209.85.222.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B5B2716426 for ; Fri, 12 Apr 2024 01:50:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712886656; cv=none; b=Tmfy34fi3VkMWq3lgWti9AEBgVxRipqCZqGAsPWO8XBhD0Ge2YUfrD5V1RLcotqDDgbzKtGquMBGu4yvaGPpIz+0xGYUaugyG9M0TxJRwVCfu1bStaDUDOP25eY1lOt6T0yk5s2YPljxku2UYwkeoeTJwuwCTjrDfolkRyaiYgk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712886656; c=relaxed/simple; bh=j+JlZhELkbUyrxfK0CeiZLK3yBU6Q8rGCixqPhBA6EU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jWMXgoF0uw0QRlUX8vcPT8IRIfp4rnbDdP0JsafNg8DUIz2oBQ5h50sMTjRBlH+mvQFnRP/cSkGiJgYPEcSZWK1C1tM0j5D1GcTJvMmAbHX8dt7f9mYtcEL4Yom3e0BMr5fg3Otj2uNG6CSNsvXoADUbDVmwjrAyXsIdMG2VvP8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com; spf=pass smtp.mailfrom=kylehuey.com; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b=U1Flfrhd; arc=none smtp.client-ip=209.85.222.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b="U1Flfrhd" Received: by mail-qk1-f171.google.com with SMTP id af79cd13be357-78d723c0dc5so30841785a.2 for ; Thu, 11 Apr 2024 18:50:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kylehuey.com; s=google; t=1712886653; x=1713491453; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MEkPkaevs/85pKXIFzAxTV+0rXNtmtbXbn0hIUFujbg=; b=U1FlfrhdO9xd3qiKzuDF5u2WHSrUxvKxOWjhZBBNwr8YsurwdNBdKHxx/tiiDYwODe FklQ9x+0lbEybH3z8QFAsoKPOhc+cvO7EHOSzErbl7YbBGlZjGW3xKO0s7sxv2Vyl9Sj nZA7WUxCK4LKCVqGkOL9Kkh55LjRZch68BX8XLMSteGSC2FWHb+x7qvzQ2R3oKFtJOhs 4zkoaZwXlvof4qMBNKBvqgn0OqWcUQJm2WeLURh8jAy1Hq8pxGZwWuly3tfXc4Bqk2U7 +cbLOAMj+O4d/kcfs2rWk/peyd/zfgmNqRNHBupYKo1O99tLKS119oRb3ivEmcbJWq3Y m2Bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712886653; x=1713491453; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MEkPkaevs/85pKXIFzAxTV+0rXNtmtbXbn0hIUFujbg=; b=dY9f6B4gO0FDY5ftjCeqcHNLIL0IRRBbHLiFoSJXId6/LIg7oOAtohDE1cJHbaIVNb fctTpPnFdBUUVFYejLWpz6yMthQj47a6jfyRwKGzD6fJPL+MkdoehxXf+SO3PnpqrhyJ 6btdp7U6CXELxM+EnnR0j0NjCFBkbZ98i8r9EAQJiyxvq3rg733RNmqxo0owx6wTv2ET umR7G2XyjV4egFIsdN3UJX0fTEWWWkaMGh+N6RkdR8FId6cPGC3MRdsETeHyJrZOueWU BgE7HpgI4Ijgdi4DCjlQahnpJWb1VnUHIq0zKr/8AmtzgCgsrEft88+lAIWKSAYikNP1 ka+w== X-Forwarded-Encrypted: i=1; AJvYcCVWfQfYb0IulvCAhawj2JMcxBV7eiMtKp+lxyysuKYHT/QIcrXeNL00ItJ4kM7xAoelPdCEdlP55lcvZ4tDGBxQg1KZ X-Gm-Message-State: AOJu0YwRCUS5aaXLEIU+QPmnaDiM/g1USc1bs8ptLqEtdsX7K8zlkG5Z UjBZp8LpL2BqWtPG6xZLRCP7ZeJbLvZHULGEzHCBbC+e4geTA4LDC7+sfhY0DA== X-Google-Smtp-Source: AGHT+IEbFaxDuu1GXCiEZNx7U9M0URvr9l8ow3pyXrdyfGxSiceYpsCWsh1lyCxVKzYndLYworc6Jw== X-Received: by 2002:a05:620a:1a93:b0:78d:5714:6698 with SMTP id bl19-20020a05620a1a9300b0078d57146698mr1675936qkb.4.1712886653450; Thu, 11 Apr 2024 18:50:53 -0700 (PDT) Received: from ip-172-31-44-15.us-east-2.compute.internal (ec2-52-15-100-147.us-east-2.compute.amazonaws.com. [52.15.100.147]) by smtp.googlemail.com with ESMTPSA id f10-20020a05620a15aa00b0078d76c1178esm1756677qkk.119.2024.04.11.18.50.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Apr 2024 18:50:53 -0700 (PDT) From: Kyle Huey X-Google-Original-From: Kyle Huey To: Kyle Huey , linux-kernel@vger.kernel.org, Andrii Nakryiko , Jiri Olsa , Namhyung Kim , Marco Elver , Yonghong Song , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo Cc: Robert O'Callahan , bpf@vger.kernel.org, Song Liu , Andrii Nakryiko , Ingo Molnar , Mark Rutland , Alexander Shishkin , Ian Rogers , Adrian Hunter , linux-perf-users@vger.kernel.org Subject: [PATCH v6 4/7] perf/bpf: Call bpf handler directly, not through overflow machinery Date: Thu, 11 Apr 2024 18:50:16 -0700 Message-Id: <20240412015019.7060-5-khuey@kylehuey.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240412015019.7060-1-khuey@kylehuey.com> References: <20240412015019.7060-1-khuey@kylehuey.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To ultimately allow bpf programs attached to perf events to completely suppress all of the effects of a perf event overflow (rather than just the sample output, as they do today), call bpf_overflow_handler() from __perf_event_overflow() directly rather than modifying struct perf_event's overflow_handler. Return the bpf program's return value from bpf_overflow_handler() so that __perf_event_overflow() knows how to proceed. Remove the now unnecessary orig_overflow_handler from struct perf_event. This patch is solely a refactoring and results in no behavior change. Suggested-by: Namhyung Kim Signed-off-by: Kyle Huey Acked-by: Song Liu Acked-by: Jiri Olsa Acked-by: Andrii Nakryiko --- include/linux/perf_event.h | 6 +----- kernel/events/core.c | 27 +++++++++++---------------- 2 files changed, 12 insertions(+), 21 deletions(-) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 07cd4722dedb..65ad1294218f 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -809,7 +809,6 @@ struct perf_event { u64 (*clock)(void); perf_overflow_handler_t overflow_handler; void *overflow_handler_context; - perf_overflow_handler_t orig_overflow_handler; struct bpf_prog *prog; u64 bpf_cookie; @@ -1355,10 +1354,7 @@ __is_default_overflow_handler(perf_overflow_handler_t overflow_handler) #ifdef CONFIG_BPF_SYSCALL static inline bool uses_default_overflow_handler(struct perf_event *event) { - if (likely(is_default_overflow_handler(event))) - return true; - - return __is_default_overflow_handler(event->orig_overflow_handler); + return is_default_overflow_handler(event); } #else #define uses_default_overflow_handler(event) \ diff --git a/kernel/events/core.c b/kernel/events/core.c index a7c2a739a27c..fd601d509cea 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -9545,9 +9545,9 @@ static inline bool sample_is_allowed(struct perf_event *event, struct pt_regs *r } #ifdef CONFIG_BPF_SYSCALL -static void bpf_overflow_handler(struct perf_event *event, - struct perf_sample_data *data, - struct pt_regs *regs) +static int bpf_overflow_handler(struct perf_event *event, + struct perf_sample_data *data, + struct pt_regs *regs) { struct bpf_perf_event_data_kern ctx = { .data = data, @@ -9568,10 +9568,8 @@ static void bpf_overflow_handler(struct perf_event *event, rcu_read_unlock(); out: __this_cpu_dec(bpf_prog_active); - if (!ret) - return; - event->orig_overflow_handler(event, data, regs); + return ret; } static int perf_event_set_bpf_handler(struct perf_event *event, @@ -9607,8 +9605,6 @@ static int perf_event_set_bpf_handler(struct perf_event *event, event->prog = prog; event->bpf_cookie = bpf_cookie; - event->orig_overflow_handler = READ_ONCE(event->overflow_handler); - WRITE_ONCE(event->overflow_handler, bpf_overflow_handler); return 0; } @@ -9619,15 +9615,15 @@ static void perf_event_free_bpf_handler(struct perf_event *event) if (!prog) return; - WRITE_ONCE(event->overflow_handler, event->orig_overflow_handler); event->prog = NULL; bpf_prog_put(prog); } #else -static void bpf_overflow_handler(struct perf_event *event, - struct perf_sample_data *data, - struct pt_regs *regs) +static int bpf_overflow_handler(struct perf_event *event, + struct perf_sample_data *data, + struct pt_regs *regs) { + return 1; } static int perf_event_set_bpf_handler(struct perf_event *event, @@ -9711,7 +9707,8 @@ static int __perf_event_overflow(struct perf_event *event, irq_work_queue(&event->pending_irq); } - READ_ONCE(event->overflow_handler)(event, data, regs); + if (!(event->prog && !bpf_overflow_handler(event, data, regs))) + READ_ONCE(event->overflow_handler)(event, data, regs); if (*perf_event_fasync(event) && event->pending_kill) { event->pending_wakeup = 1; @@ -11978,13 +11975,11 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu, overflow_handler = parent_event->overflow_handler; context = parent_event->overflow_handler_context; #if defined(CONFIG_BPF_SYSCALL) && defined(CONFIG_EVENT_TRACING) - if (overflow_handler == bpf_overflow_handler) { + if (parent_event->prog) { struct bpf_prog *prog = parent_event->prog; bpf_prog_inc(prog); event->prog = prog; - event->orig_overflow_handler = - parent_event->orig_overflow_handler; } #endif } From patchwork Fri Apr 12 01:50:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kyle Huey X-Patchwork-Id: 13626692 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-qk1-f174.google.com (mail-qk1-f174.google.com [209.85.222.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6449417BAB for ; Fri, 12 Apr 2024 01:50:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712886657; cv=none; b=MCfDzz5NYj43i6y2JKpVjIYmGqU6tgGBr/FUU9ivgz/wfV+/EejULcUipvsVuBtAuX+KvZPLS605G5GhzjmVOadsvTyqBhQWnkUTTjKkxzM+oX69IF5j4E5oeYEnbaVLMBsVDGUXZqVObYeGcGg0u1BGbthfli5iQjnT/28sN80= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712886657; c=relaxed/simple; bh=NRmEZmEzOcudL2cLJi5SCwBCdAXkrrS/5QKylKnvPRI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NTwSrXOKS9eYh75YUd2cE4h8II47XHD2Hbde6u2kVC/zG3GXfiKzjDcskQnGMbVk419742sMXp9g/cvMrX+m4UiGN9nqu1sTIYZZbaVYsUy3nubQeegskjZUr+xTLCESxmGdcxj8mvUbgHQwShZVYkJ+MAcEaVFA3tm9nndcxUY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com; spf=pass smtp.mailfrom=kylehuey.com; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b=Xn8JkP7N; arc=none smtp.client-ip=209.85.222.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b="Xn8JkP7N" Received: by mail-qk1-f174.google.com with SMTP id af79cd13be357-78ec0d2e1e9so18583385a.3 for ; Thu, 11 Apr 2024 18:50:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kylehuey.com; s=google; t=1712886655; x=1713491455; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XmAdJT3ZUCPeWfD1wo8jRuZywHZwwpve3RZWOW9haf0=; b=Xn8JkP7NU04LMLIgfilXIt3KHG4T6B1Vtz1uXM2qEKoiTj8FPRP5gA63Z4JLuoTKPa BUCWI7e4/gN7/ilOtU16JGX26IzFpvJhhRBiifpadbzRHLLNNWEz77cpp/tY5gns7cFx 8f2/jiQ7kCpX4O9YskqqC9JsIQclJSMMziIFEKlzvOq4Dj33AT0moOJSK9wzjhpDcYoV 2CGc8CYJNxGvgbZZifH8HPnX0bJnIyQRxFvt6+mW+opRh3QdiRdXr6+3MlH8+Lc+0ALI OYfgCe3oNsJ1PLp5eqiTM2Ehbs4QEtt4dwhtjWozsOXZ/Ej0ZcrY8tkoNTYEJ9jpDfXF Cpkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712886655; x=1713491455; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XmAdJT3ZUCPeWfD1wo8jRuZywHZwwpve3RZWOW9haf0=; b=QsL0JwJON0Y4EI8618LGCMcVwqe0P2b9tc3OUa/Xqk08RGzIUurkc+oWTVbWb0YkHR TiyEiSjbgjh3SXrcqz/XQqyNPYAQR9oVDrtgYxV5LYbhbJ+q9sDpn46Pube/8renuAgi PHIjb9/IztofVT+apqvpvyptsU10WjGuECYjZYSKzEYesgf5tCpNTyhq5wg3z8Vkbk4o JsdObcigSEIPRJsfIv51MtPdmtCdhpmLq9vJvQzFnvjnqZ8FlX/XQAdalxRXmPLAuUr+ pEYVnt0ilmOddq/n9sOqSslc8M42IVv7aaOAFbSqm2d9sjDRCifGq9cmWmjMzub5mfxO rRQQ== X-Forwarded-Encrypted: i=1; AJvYcCWVEXXmdBsLU7ze8NmiZZfJMbK2H8aLp3vYMzb1pFfZxBlx8ifuyKb4EP4gy1EgbnaWa9ArQv7VjjnJk4S19s7t+J74 X-Gm-Message-State: AOJu0Yye7P+sQFeeBjR5ZJ3naUeDlrpr4d3BMq4V7i8mzNFBHlw67B5r C17yWZO+RaSj/t3kM/l5riWJYLofcW76ds2QaksHQK7hrzCNimiH/hWGQ6f3oQ== X-Google-Smtp-Source: AGHT+IH1CdfzeVz3pZajhCl9CVe8VCWrOdYFKQJn9dqM+zf6JXUpBa7+69QfvqWLEhWCHiBE2XF4sw== X-Received: by 2002:a05:620a:2211:b0:78d:68d6:c1e9 with SMTP id m17-20020a05620a221100b0078d68d6c1e9mr1314228qkh.47.1712886655388; Thu, 11 Apr 2024 18:50:55 -0700 (PDT) Received: from ip-172-31-44-15.us-east-2.compute.internal (ec2-52-15-100-147.us-east-2.compute.amazonaws.com. [52.15.100.147]) by smtp.googlemail.com with ESMTPSA id f10-20020a05620a15aa00b0078d76c1178esm1756677qkk.119.2024.04.11.18.50.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Apr 2024 18:50:55 -0700 (PDT) From: Kyle Huey X-Google-Original-From: Kyle Huey To: Kyle Huey , linux-kernel@vger.kernel.org, Andrii Nakryiko , Jiri Olsa , Namhyung Kim , Marco Elver , Yonghong Song , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo Cc: Robert O'Callahan , bpf@vger.kernel.org, Will Deacon , Song Liu , Andrii Nakryiko , Mark Rutland , Russell King , Catalin Marinas , Ingo Molnar , Alexander Shishkin , Ian Rogers , Adrian Hunter , linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org Subject: [PATCH v6 5/7] perf/bpf: Remove unneeded uses_default_overflow_handler() Date: Thu, 11 Apr 2024 18:50:17 -0700 Message-Id: <20240412015019.7060-6-khuey@kylehuey.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240412015019.7060-1-khuey@kylehuey.com> References: <20240412015019.7060-1-khuey@kylehuey.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Now that struct perf_event's orig_overflow_handler is gone, there's no need for the functions and macros to support looking past overflow_handler to orig_overflow_handler. This patch is solely a refactoring and results in no behavior change. Signed-off-by: Kyle Huey Acked-by: Will Deacon Acked-by: Song Liu Acked-by: Jiri Olsa Acked-by: Andrii Nakryiko --- arch/arm/kernel/hw_breakpoint.c | 8 ++++---- arch/arm64/kernel/hw_breakpoint.c | 4 ++-- include/linux/perf_event.h | 17 +++-------------- 3 files changed, 9 insertions(+), 20 deletions(-) diff --git a/arch/arm/kernel/hw_breakpoint.c b/arch/arm/kernel/hw_breakpoint.c index dc0fb7a81371..054e9199f30d 100644 --- a/arch/arm/kernel/hw_breakpoint.c +++ b/arch/arm/kernel/hw_breakpoint.c @@ -626,7 +626,7 @@ int hw_breakpoint_arch_parse(struct perf_event *bp, hw->address &= ~alignment_mask; hw->ctrl.len <<= offset; - if (uses_default_overflow_handler(bp)) { + if (is_default_overflow_handler(bp)) { /* * Mismatch breakpoints are required for single-stepping * breakpoints. @@ -798,7 +798,7 @@ static void watchpoint_handler(unsigned long addr, unsigned int fsr, * Otherwise, insert a temporary mismatch breakpoint so that * we can single-step over the watchpoint trigger. */ - if (!uses_default_overflow_handler(wp)) + if (!is_default_overflow_handler(wp)) continue; step: enable_single_step(wp, instruction_pointer(regs)); @@ -811,7 +811,7 @@ static void watchpoint_handler(unsigned long addr, unsigned int fsr, info->trigger = addr; pr_debug("watchpoint fired: address = 0x%x\n", info->trigger); perf_bp_event(wp, regs); - if (uses_default_overflow_handler(wp)) + if (is_default_overflow_handler(wp)) enable_single_step(wp, instruction_pointer(regs)); } @@ -886,7 +886,7 @@ static void breakpoint_handler(unsigned long unknown, struct pt_regs *regs) info->trigger = addr; pr_debug("breakpoint fired: address = 0x%x\n", addr); perf_bp_event(bp, regs); - if (uses_default_overflow_handler(bp)) + if (is_default_overflow_handler(bp)) enable_single_step(bp, addr); goto unlock; } diff --git a/arch/arm64/kernel/hw_breakpoint.c b/arch/arm64/kernel/hw_breakpoint.c index 2f5755192c2b..722ac45f9f7b 100644 --- a/arch/arm64/kernel/hw_breakpoint.c +++ b/arch/arm64/kernel/hw_breakpoint.c @@ -655,7 +655,7 @@ static int breakpoint_handler(unsigned long unused, unsigned long esr, perf_bp_event(bp, regs); /* Do we need to handle the stepping? */ - if (uses_default_overflow_handler(bp)) + if (is_default_overflow_handler(bp)) step = 1; unlock: rcu_read_unlock(); @@ -734,7 +734,7 @@ static u64 get_distance_from_watchpoint(unsigned long addr, u64 val, static int watchpoint_report(struct perf_event *wp, unsigned long addr, struct pt_regs *regs) { - int step = uses_default_overflow_handler(wp); + int step = is_default_overflow_handler(wp); struct arch_hw_breakpoint *info = counter_arch_bp(wp); info->trigger = addr; diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 65ad1294218f..13a2b05cc431 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1339,8 +1339,10 @@ extern int perf_event_output(struct perf_event *event, struct pt_regs *regs); static inline bool -__is_default_overflow_handler(perf_overflow_handler_t overflow_handler) +is_default_overflow_handler(struct perf_event *event) { + perf_overflow_handler_t overflow_handler = event->overflow_handler; + if (likely(overflow_handler == perf_event_output_forward)) return true; if (unlikely(overflow_handler == perf_event_output_backward)) @@ -1348,19 +1350,6 @@ __is_default_overflow_handler(perf_overflow_handler_t overflow_handler) return false; } -#define is_default_overflow_handler(event) \ - __is_default_overflow_handler((event)->overflow_handler) - -#ifdef CONFIG_BPF_SYSCALL -static inline bool uses_default_overflow_handler(struct perf_event *event) -{ - return is_default_overflow_handler(event); -} -#else -#define uses_default_overflow_handler(event) \ - is_default_overflow_handler(event) -#endif - extern void perf_event_header__init_id(struct perf_event_header *header, struct perf_sample_data *data, From patchwork Fri Apr 12 01:50:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kyle Huey X-Patchwork-Id: 13626693 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f182.google.com (mail-yw1-f182.google.com [209.85.128.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D38B4DDC3 for ; Fri, 12 Apr 2024 01:50:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712886659; cv=none; b=GNpOFDQ+lZc/+LAbAkDJUFEvO2IRP8qTcp9lzpN6WQQFrJPIK/r96lZOb5gmPoMYzmBk8thQlZV9VsQITfcfbqsJjP/t7mthXhAIZvYmCXfG7uF2bzwJgrrvKNkkOKbtsshsEb80OQEn22iIPM/C65gGQep7XbiktbDiS2qj9gI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712886659; c=relaxed/simple; bh=AdlygNz0RFZef1mcIWoB8it1jPVZwP5sUo9GIGfOMoo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=gmf9q9oqIB0XmYnVnbRlH0pElKfbnwk8nW2BHNmZJsfos5L7mJkiLikM9pKyohVl2kIL+a+5Hg4+SLxID9Wbr3EG79Cvz3QpnXomTvOZ2O17G9/JRPGJlS2fo8eqNI4O5JZduyJDp7s0KpOzmolIunEXrbcvdxTaxO5wPJgt3QU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com; spf=pass smtp.mailfrom=kylehuey.com; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b=CCl9p0GX; arc=none smtp.client-ip=209.85.128.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b="CCl9p0GX" Received: by mail-yw1-f182.google.com with SMTP id 00721157ae682-6181b9dc647so4605357b3.1 for ; Thu, 11 Apr 2024 18:50:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kylehuey.com; s=google; t=1712886657; x=1713491457; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6W3XOu1JzCFnI8d0V7+Yp4cwFPdA5VsCmQsQwSODsxU=; b=CCl9p0GXVhmMlklK1/QoFpwlvhnnwH9VJ6mihBgzv9ai+0wV+zdgyYF5DDcwfVvZ0A NpAeMCJ5pzQDBeMCHLV+0RSIyNcdBBLSlWajhj8zmISfuWrCP/6eJbLx0NlvF4o+Vu/5 QJvaBVvfdXhfmvj2FBV7mONS8Pr0sxnKTRr1zdEpruDXSTIGTG+FS9BRCePljkO1+shF JxeBCMBF+8/FBc5e9LoLcpFnwUanMDBr7bFwUvABXtPlt9gt/LzxWUbpy5dhr+GjWMcc KLp0vUJeh4MtKQWg8Yhfa7JeaUt4OmhD+gmfZW6AJsmST2OSMn6sIct5UVZEu+MuGXLl 9hYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712886657; x=1713491457; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6W3XOu1JzCFnI8d0V7+Yp4cwFPdA5VsCmQsQwSODsxU=; b=umRwob/bmRuwy7gRmJSe7wI22YoqNLJGH80m/If3M89VmxYL+M0O7mRF/g4WXjuFcO /HTR0Pe5v3oDm/aAyH3BOsa/urCjiIm0aRsNPthdoYvazvnnSTGPwcf/js8yy2eDpjaS r16prrdDoKGsi7KdA90UwhvN0YJ9Se7HZblazDxZ6Udwr6zj5ljPQb0zXSpfrQTx+TpL UPSKl7iyB9xwdxRsipnigFfgRX52SJbYPLMAimkpisaW+dq2cxP00QHroUZo+Z+kIGjy aR7wOWGReUjQQkYZShRWjhmDW8qVEOzmO/8jMOkaqUr32MjsT1FqbyE2T8x9ZseSwYE/ TwGA== X-Forwarded-Encrypted: i=1; AJvYcCVdTBd2QO0JMke5+x/D5aPrBcpiB/a+KCHmqRApRIhs137b/eQ6MULKK9QhNnxZHnEz5hzHLbiXgvNg0ZE7c1bAIBUb X-Gm-Message-State: AOJu0YwVBjF+NgDJN8CnP50HZyTuJ/6Lrau/ysAKMgMDHu1LJwAXf43F 77QwfFT51yLB7XY+lt9L9aR1WA4hH0BWn4g+xRjKTJGQX1yCG9rM5VIcyMt4Kw== X-Google-Smtp-Source: AGHT+IHjXR7BfIecARc6VI1pATfh8TR/3Pi2gT4lAF8cWX7oY3B82GGlWqZk5F7vE4hcCX37K6VYow== X-Received: by 2002:a81:af54:0:b0:611:5ff6:2608 with SMTP id x20-20020a81af54000000b006115ff62608mr1280098ywj.28.1712886656935; Thu, 11 Apr 2024 18:50:56 -0700 (PDT) Received: from ip-172-31-44-15.us-east-2.compute.internal (ec2-52-15-100-147.us-east-2.compute.amazonaws.com. [52.15.100.147]) by smtp.googlemail.com with ESMTPSA id f10-20020a05620a15aa00b0078d76c1178esm1756677qkk.119.2024.04.11.18.50.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Apr 2024 18:50:56 -0700 (PDT) From: Kyle Huey X-Google-Original-From: Kyle Huey To: Kyle Huey , linux-kernel@vger.kernel.org, Andrii Nakryiko , Jiri Olsa , Namhyung Kim , Marco Elver , Yonghong Song , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo Cc: Robert O'Callahan , bpf@vger.kernel.org, Song Liu , Andrii Nakryiko , Ingo Molnar , Mark Rutland , Alexander Shishkin , Ian Rogers , Adrian Hunter , linux-perf-users@vger.kernel.org Subject: [PATCH v6 6/7] perf/bpf: Allow a bpf program to suppress all sample side effects Date: Thu, 11 Apr 2024 18:50:18 -0700 Message-Id: <20240412015019.7060-7-khuey@kylehuey.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240412015019.7060-1-khuey@kylehuey.com> References: <20240412015019.7060-1-khuey@kylehuey.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Returning zero from a bpf program attached to a perf event already suppresses any data output. Return early from __perf_event_overflow() in this case so it will also suppress event_limit accounting, SIGTRAP generation, and F_ASYNC signalling. Signed-off-by: Kyle Huey Acked-by: Song Liu Acked-by: Jiri Olsa Acked-by: Namhyung Kim Acked-by: Andrii Nakryiko --- kernel/events/core.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index fd601d509cea..cd88d1e89eb8 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -9658,6 +9658,9 @@ static int __perf_event_overflow(struct perf_event *event, ret = __perf_event_account_interrupt(event, throttle); + if (event->prog && !bpf_overflow_handler(event, data, regs)) + return ret; + /* * XXX event_limit might not quite work as expected on inherited * events @@ -9707,8 +9710,7 @@ static int __perf_event_overflow(struct perf_event *event, irq_work_queue(&event->pending_irq); } - if (!(event->prog && !bpf_overflow_handler(event, data, regs))) - READ_ONCE(event->overflow_handler)(event, data, regs); + READ_ONCE(event->overflow_handler)(event, data, regs); if (*perf_event_fasync(event) && event->pending_kill) { event->pending_wakeup = 1; From patchwork Fri Apr 12 01:50:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kyle Huey X-Patchwork-Id: 13626694 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f171.google.com (mail-yw1-f171.google.com [209.85.128.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD12A24A03 for ; Fri, 12 Apr 2024 01:51:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712886667; cv=none; b=SphAsiuGiaSxkdw0bkE50V3s+X7xZvy5FAv++B/NYIUZ9QyhrQ+qAklV4sxnvIQ1pnU9ESAIPnySmIsgTWQeILsQM0Bbssfd17CTwOMWJUpVO9jxUGdAobeesEyztrV2zS+G9IrcyDDBvABNcFwzrWEjPfnZ54wA1xumuCCveac= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712886667; c=relaxed/simple; bh=EfTqzolUvIxbeV77szfL3e0e1IBqxiKk0wQkbdth9Jg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=E/pRaQ5zCni/ujW2Om4c8vIXjukF3tbxQAxVnC95zSBXeUWkMmv4KYhbQgQK0UV8ll/ExvWJg6CkJSAbs6hFwxw9d+aUJNWJmODaqm+jmB8UuBOFg81fDK1DAbfQzEJ8A6/t62FMKQtmjjBBedsmBf9BAIqlJJnWB1rS2/eFKM0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com; spf=pass smtp.mailfrom=kylehuey.com; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b=BBumYXRF; arc=none smtp.client-ip=209.85.128.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b="BBumYXRF" Received: by mail-yw1-f171.google.com with SMTP id 00721157ae682-61816fc256dso4189027b3.0 for ; Thu, 11 Apr 2024 18:51:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kylehuey.com; s=google; t=1712886665; x=1713491465; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PLsAtO8E8WEI0YPLnWc4zGD4gPrub60dCUyhat+mc0A=; b=BBumYXRFyY4oAk5wBat/ahkj0vo4ZnGDizw9oe7uiI4of7VQSdzqMqicfgz6Iqva3+ TWB3iYBSoKCwGzfyBadYhTJblNxUidbYGWt29yAxB5uG5AVFqqKSDB9JIUZvJIq3h5un AqcdhEOTYPdqqEB4RwLqMoOsWiVbiB2K6Ecn3dTiLeWaFd5ZlStzsB5463mCQOB7hSHP EamyeE7eoRo/SMtDLC1565o/z8czbepN5SJj7jsJXvH6TKSiN6tBk8KfyjROBuQ25zXo YliCS1PX3q8xNQUWIf+bDWLpFXKq81+s1iRIV0vtGBpRQ4V73mrheJV4nWzRPm6DOH2i aitQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712886665; x=1713491465; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PLsAtO8E8WEI0YPLnWc4zGD4gPrub60dCUyhat+mc0A=; b=MXM5GsxVpXYSkbHfEWmvtScm1Q3rzeaBVe2zlwnYAiBhsWSF06J8/x33UZwxItDRYK Mnol1Ny5fbV91Uu3DYVeanqleEF125BdSEMwjAo8BG2ZWC/7jw0bar1k1rXJOC8/ScdW RGlx7siNAnZSI0wzQ11UFeynspAf8u1RV30q/IaQiT17gbvi/Io1DUT/D/1XaXdfJoGA VHBVCVjgP/SOXxU4EQxrsyF9BCJFtxKopFmsVUsq79X9/hM+cQVZzDKbqe4eX8o7y/lW Kx1trRMV7G5QatjVkDlv0b3qfIXhadHJi/A1jpbsvx3s0fs1V+JJWOmQL12PXJnIGn96 WQEg== X-Forwarded-Encrypted: i=1; AJvYcCWEHsUXdQIHLpjHDcV52MWK79muc+vKKNQleFEHcyWDTKbcaLWpe9Aq1yGSIhrEnrK2wuWjbaFd10yS9GYDvqPo2Dy0 X-Gm-Message-State: AOJu0YwlR/mOOqfqjo+TjSn2TsR5o9LRjZDLmisKDldnZMr3INrNuaFD hXgP3rptpemz65x6GE8zeGOMlxgblatBt2SQeC5RqtqrS4BDJh7/rJ0A/2eK8g== X-Google-Smtp-Source: AGHT+IF6KWE2O+uOj1W7BUPtfwmixy3UO/O70mlonobqoOzGHLL5MXKtYfA1MJKuTWaSz4KGSxyRhQ== X-Received: by 2002:a81:4ec7:0:b0:615:21e7:6bf6 with SMTP id c190-20020a814ec7000000b0061521e76bf6mr1172665ywb.14.1712886664785; Thu, 11 Apr 2024 18:51:04 -0700 (PDT) Received: from ip-172-31-44-15.us-east-2.compute.internal (ec2-52-15-100-147.us-east-2.compute.amazonaws.com. [52.15.100.147]) by smtp.googlemail.com with ESMTPSA id f10-20020a05620a15aa00b0078d76c1178esm1756677qkk.119.2024.04.11.18.51.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Apr 2024 18:51:04 -0700 (PDT) From: Kyle Huey X-Google-Original-From: Kyle Huey To: Kyle Huey , linux-kernel@vger.kernel.org, Andrii Nakryiko , Jiri Olsa , Namhyung Kim , Marco Elver , Yonghong Song , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo Cc: Robert O'Callahan , bpf@vger.kernel.org, Song Liu , Andrii Nakryiko , Eduard Zingerman , Mykola Lysenko , Alexei Starovoitov , Daniel Borkmann , Martin KaFai Lau , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Shuah Khan , linux-kselftest@vger.kernel.org Subject: [PATCH v6 7/7] selftest/bpf: Test a perf bpf program that suppresses side effects. Date: Thu, 11 Apr 2024 18:50:19 -0700 Message-Id: <20240412015019.7060-8-khuey@kylehuey.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240412015019.7060-1-khuey@kylehuey.com> References: <20240412015019.7060-1-khuey@kylehuey.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net The test sets a hardware breakpoint and uses a bpf program to suppress the side effects of a perf event sample, including I/O availability signals, SIGTRAPs, and decrementing the event counter limit, if the ip matches the expected value. Then the function with the breakpoint is executed multiple times to test that all effects behave as expected. Signed-off-by: Kyle Huey Acked-by: Song Liu Acked-by: Jiri Olsa Acked-by: Andrii Nakryiko --- .../selftests/bpf/prog_tests/perf_skip.c | 137 ++++++++++++++++++ .../selftests/bpf/progs/test_perf_skip.c | 15 ++ 2 files changed, 152 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/perf_skip.c create mode 100644 tools/testing/selftests/bpf/progs/test_perf_skip.c diff --git a/tools/testing/selftests/bpf/prog_tests/perf_skip.c b/tools/testing/selftests/bpf/prog_tests/perf_skip.c new file mode 100644 index 000000000000..37d8618800e4 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/perf_skip.c @@ -0,0 +1,137 @@ +// SPDX-License-Identifier: GPL-2.0 +#define _GNU_SOURCE + +#include +#include "test_perf_skip.skel.h" +#include +#include +#include + +#ifndef TRAP_PERF +#define TRAP_PERF 6 +#endif + +int sigio_count, sigtrap_count; + +static void handle_sigio(int sig __always_unused) +{ + ++sigio_count; +} + +static void handle_sigtrap(int signum __always_unused, + siginfo_t *info, + void *ucontext __always_unused) +{ + ASSERT_EQ(info->si_code, TRAP_PERF, "si_code"); + ++sigtrap_count; +} + +static noinline int test_function(void) +{ + asm volatile (""); + return 0; +} + +void serial_test_perf_skip(void) +{ + struct sigaction action = {}; + struct sigaction previous_sigtrap; + sighandler_t previous_sigio = SIG_ERR; + struct test_perf_skip *skel = NULL; + struct perf_event_attr attr = {}; + int perf_fd = -1; + int err; + struct f_owner_ex owner; + struct bpf_link *prog_link = NULL; + + action.sa_flags = SA_SIGINFO | SA_NODEFER; + action.sa_sigaction = handle_sigtrap; + sigemptyset(&action.sa_mask); + if (!ASSERT_OK(sigaction(SIGTRAP, &action, &previous_sigtrap), "sigaction")) + return; + + previous_sigio = signal(SIGIO, handle_sigio); + if (!ASSERT_NEQ(previous_sigio, SIG_ERR, "signal")) + goto cleanup; + + skel = test_perf_skip__open_and_load(); + if (!ASSERT_OK_PTR(skel, "skel_load")) + goto cleanup; + + attr.type = PERF_TYPE_BREAKPOINT; + attr.size = sizeof(attr); + attr.bp_type = HW_BREAKPOINT_X; + attr.bp_addr = (uintptr_t)test_function; + attr.bp_len = sizeof(long); + attr.sample_period = 1; + attr.sample_type = PERF_SAMPLE_IP; + attr.pinned = 1; + attr.exclude_kernel = 1; + attr.exclude_hv = 1; + attr.precise_ip = 3; + attr.sigtrap = 1; + attr.remove_on_exec = 1; + + perf_fd = syscall(__NR_perf_event_open, &attr, 0, -1, -1, 0); + if (perf_fd < 0 && (errno == ENOENT || errno == EOPNOTSUPP)) { + printf("SKIP:no PERF_TYPE_BREAKPOINT/HW_BREAKPOINT_X\n"); + test__skip(); + goto cleanup; + } + if (!ASSERT_OK(perf_fd < 0, "perf_event_open")) + goto cleanup; + + /* Configure the perf event to signal on sample. */ + err = fcntl(perf_fd, F_SETFL, O_ASYNC); + if (!ASSERT_OK(err, "fcntl(F_SETFL, O_ASYNC)")) + goto cleanup; + + owner.type = F_OWNER_TID; + owner.pid = syscall(__NR_gettid); + err = fcntl(perf_fd, F_SETOWN_EX, &owner); + if (!ASSERT_OK(err, "fcntl(F_SETOWN_EX)")) + goto cleanup; + + /* Allow at most one sample. A sample rejected by bpf should + * not count against this. + */ + err = ioctl(perf_fd, PERF_EVENT_IOC_REFRESH, 1); + if (!ASSERT_OK(err, "ioctl(PERF_EVENT_IOC_REFRESH)")) + goto cleanup; + + prog_link = bpf_program__attach_perf_event(skel->progs.handler, perf_fd); + if (!ASSERT_OK_PTR(prog_link, "bpf_program__attach_perf_event")) + goto cleanup; + + /* Configure the bpf program to suppress the sample. */ + skel->bss->ip = (uintptr_t)test_function; + test_function(); + + ASSERT_EQ(sigio_count, 0, "sigio_count"); + ASSERT_EQ(sigtrap_count, 0, "sigtrap_count"); + + /* Configure the bpf program to allow the sample. */ + skel->bss->ip = 0; + test_function(); + + ASSERT_EQ(sigio_count, 1, "sigio_count"); + ASSERT_EQ(sigtrap_count, 1, "sigtrap_count"); + + /* Test that the sample above is the only one allowed (by perf, not + * by bpf) + */ + test_function(); + + ASSERT_EQ(sigio_count, 1, "sigio_count"); + ASSERT_EQ(sigtrap_count, 1, "sigtrap_count"); + +cleanup: + bpf_link__destroy(prog_link); + if (perf_fd >= 0) + close(perf_fd); + test_perf_skip__destroy(skel); + + if (previous_sigio != SIG_ERR) + signal(SIGIO, previous_sigio); + sigaction(SIGTRAP, &previous_sigtrap, NULL); +} diff --git a/tools/testing/selftests/bpf/progs/test_perf_skip.c b/tools/testing/selftests/bpf/progs/test_perf_skip.c new file mode 100644 index 000000000000..7eb8b6de7a57 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_perf_skip.c @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include + +uintptr_t ip; + +SEC("perf_event") +int handler(struct bpf_perf_event_data *data) +{ + /* Skip events that have the correct ip. */ + return ip != PT_REGS_IP(&data->regs); +} + +char _license[] SEC("license") = "GPL"; From patchwork Fri Apr 12 09:55:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ingo Molnar X-Patchwork-Id: 13627480 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79F4F535DC; Fri, 12 Apr 2024 09:55:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712915707; cv=none; b=eWsJC28D7UQWWfYHoEO5G5uqgRwKoSvB2ckpGj/sVQGWW/odaZgrzevUcuEJtsaXpq5AjIsmSxd7hdHAg7Yuen3o0He7VOYGfJQlXKkC/JVJeUgaEohVbN+wDkBgNKmaZY0wPYovuoTcPNS0pWVvW/WnJ1Jn+L8Z58B6wWx+ae4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712915707; c=relaxed/simple; bh=MxCP72ppniCUwq/Q6jnueZ8iwN0KI8bkGQ/p/fDaVws=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=MpGcBHQi5ktkW4nixlDJZweHMU4fAHWW0Scd/uxi5vrpM1+pnKURbhplyNGVJJnSKS5JokB5BLLPdr8EiuvvvvrB5CMufsJv3uYAthqS/YUCiOfAgn1fAYNWpEKSgL/J2EV8BmnTrJqIkSl2yaMC9twXF3w/VFlm7NqXDRKZM9c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=fwbru7gt; arc=none smtp.client-ip=209.85.208.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fwbru7gt" Received: by mail-ed1-f44.google.com with SMTP id 4fb4d7f45d1cf-56e37503115so576130a12.1; Fri, 12 Apr 2024 02:55:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712915703; x=1713520503; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:from:to:cc:subject:date:message-id :reply-to; bh=V6GO2vbzaTzYvoUcV4Qpl0H9TBLeVNlexEEn9hFraCw=; b=fwbru7gta6k+X/lIIpKhMDd/u6Zv8CZvGwN5LJNPAC06+hptYFIlEFC4L1h+J/UfdZ vhFI6+9KeYaqkzYOHw7wssdjab7soF/YQ7eDbKp4E8E5fGavwF3SQoJyS5GjFm3gWy1Q i3OhcPx80GE50AgVO9eN5KexPyJD9hF6CzISQ9mOuWA5QvzcWtAKGKYMs7WWY+H9T9Nm DZhHJjDzwByG9Mqnw/apngWRUTrHuaXLLjNHXOMTKJ0G0zHb0V7DYbZ6lshU1F37P2DC jVwnPkWna/rrVAkjqAiwD7o9KHjeu7jBbQd1fzPBMv2RK1RYNEDUkoUuomejz/H1od1i f5Jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712915703; x=1713520503; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=V6GO2vbzaTzYvoUcV4Qpl0H9TBLeVNlexEEn9hFraCw=; b=NPaVglt3ch5vBMzYIMgpoFr541v1+ZaT9oE0sitIbAMIoMdq2n0ewH+v4hmP0zWI5+ NaOXPAWbWfhgYagnLhNLbPGM9zupYR2oHPgMgkJVyBN/9ZqdUJ3tbCWYXbciugDKaSyG kWxK+81PQhOh0QDl/t6HBriOLpAsl0FUAWIWN0xWYCPT4FVwafOR4MwLZ7Qz/mfQmuRN YGeNeJXahHtr3H/vDMHSOxu/mfzRQNE31wMfjEO0zKkBss/mJBi+L4dXiJtqdUeu8pyM YNHoUqmXQWCNzlaLvv+WaLNGDjzmEToGOYY2oYyGSUyfG4KsAuA/mVGolRGQ5fM+gJul o41g== X-Forwarded-Encrypted: i=1; AJvYcCX5ia0WQ0t/rew5N7BaIoYMvwzEi4wN+9/HtKXzQ15X1TVYLIOp0WSYuDb7tmq5nfLVfQDy3qfFb9dEn/F+21WYKP+YN0gFVXaYvXfkGEGsy+eReRB9H21wMswOHN18YynX X-Gm-Message-State: AOJu0YzJ4ZlZhqte869J0lZL7j11Io2cJ4MzpG/E9DQREcGTdCM6GjyH bLtcKw2fek1HQ+PKGxMN/BQOpp/U/MJHMV6gN9v3Q5D5n9afGD4S X-Google-Smtp-Source: AGHT+IGnk62JGdMQa/B7hDMdZpD/yfiPhQr3f+cJw9cDK3woWaOPl9ouq7KgndTx/SGys6Fc5mFNZw== X-Received: by 2002:a50:aa9c:0:b0:56e:2ff3:bb89 with SMTP id q28-20020a50aa9c000000b0056e2ff3bb89mr1383824edc.28.1712915703327; Fri, 12 Apr 2024 02:55:03 -0700 (PDT) Received: from gmail.com (1F2EF1A5.nat.pool.telekom.hu. [31.46.241.165]) by smtp.gmail.com with ESMTPSA id 14-20020a0564021f4e00b0056e78dbfe13sm44670edz.32.2024.04.12.02.55.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Apr 2024 02:55:02 -0700 (PDT) Sender: Ingo Molnar Date: Fri, 12 Apr 2024 11:55:00 +0200 From: Ingo Molnar To: Kyle Huey Cc: Kyle Huey , linux-kernel@vger.kernel.org, Andrii Nakryiko , Jiri Olsa , Namhyung Kim , Marco Elver , Yonghong Song , Peter Zijlstra , Arnaldo Carvalho de Melo , Robert O'Callahan , bpf@vger.kernel.org Subject: [PATCH 8/7] perf/bpf: Change the !CONFIG_BPF_SYSCALL stubs to static inlines Message-ID: References: <20240412015019.7060-1-khuey@kylehuey.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240412015019.7060-1-khuey@kylehuey.com> Otherwise the compiler will be unhappy if they go unused, which they do on allnoconfigs. Cc: Kyle Huey Signed-off-by: Ingo Molnar --- kernel/events/core.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 2212670cbe9b..6708c1121b9f 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -9638,21 +9638,21 @@ static void perf_event_free_bpf_handler(struct perf_event *event) bpf_prog_put(prog); } #else -static int bpf_overflow_handler(struct perf_event *event, - struct perf_sample_data *data, - struct pt_regs *regs) +static inline int bpf_overflow_handler(struct perf_event *event, + struct perf_sample_data *data, + struct pt_regs *regs) { return 1; } -static int perf_event_set_bpf_handler(struct perf_event *event, - struct bpf_prog *prog, - u64 bpf_cookie) +static inline int perf_event_set_bpf_handler(struct perf_event *event, + struct bpf_prog *prog, + u64 bpf_cookie) { return -EOPNOTSUPP; } -static void perf_event_free_bpf_handler(struct perf_event *event) +static inline void perf_event_free_bpf_handler(struct perf_event *event) { } #endif