From patchwork Tue Sep 27 15:41:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Clark X-Patchwork-Id: 12990874 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 367F4C6FA83 for ; Tue, 27 Sep 2022 15:45:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232817AbiI0PpZ (ORCPT ); Tue, 27 Sep 2022 11:45:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50650 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232565AbiI0PpA (ORCPT ); Tue, 27 Sep 2022 11:45:00 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 3E6DC188BF9; Tue, 27 Sep 2022 08:41:21 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E12551756; Tue, 27 Sep 2022 08:41:26 -0700 (PDT) Received: from e121896.Emea.Arm.com (e121896.Emea.Arm.com [10.32.36.24]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 878E03F73B; Tue, 27 Sep 2022 08:41:17 -0700 (PDT) From: James Clark To: acme@kernel.org, namhyung@kernel.org Cc: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, will@kernel.org, James Clark , Peter Zijlstra , Ingo Molnar , Mark Rutland , Alexander Shishkin , Jiri Olsa , bpf@vger.kernel.org Subject: [RFC PATCH 1/4] perf test: Add ability to test exit code for attr tests Date: Tue, 27 Sep 2022 16:41:01 +0100 Message-Id: <20220927154104.869029-2-james.clark@arm.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20220927154104.869029-1-james.clark@arm.com> References: <20220927154104.869029-1-james.clark@arm.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Currently the return value is used to skip the test, but sometimes it can be useful to test if a certain command should return a certain exit code. Signed-off-by: James Clark --- tools/perf/tests/attr.py | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/tools/perf/tests/attr.py b/tools/perf/tests/attr.py index 6d9b5312e6c2..7e053fa5d6ae 100644 --- a/tools/perf/tests/attr.py +++ b/tools/perf/tests/attr.py @@ -129,7 +129,8 @@ class Event(dict): # - needs to specify: # 'command' - perf command name # 'args' - special command arguments -# 'ret' - expected command return value (0 by default) +# 'ret' - Skip test if Perf doesn't exit with this value (0 by default) +# 'test_ret'- If set to 'true', fail test instead of skipping for 'ret' argument # 'arch' - architecture specific test (optional) # comma separated list, ! at the beginning # negates it. @@ -155,6 +156,8 @@ class Test(object): except: self.ret = 0 + self.test_ret = parser.getboolean('config', 'test_ret', fallback=False) + try: self.arch = parser.get('config', 'arch') log.warning("test limitation '%s'" % self.arch) @@ -232,7 +235,10 @@ class Test(object): log.info(" '%s' ret '%s', expected '%s'" % (cmd, str(ret), str(self.ret))) if not data_equal(str(ret), str(self.ret)): - raise Unsup(self) + if self.test_ret: + raise Fail(self, "Perf exit code failure") + else: + raise Unsup(self) def compare(self, expect, result): match = {} From patchwork Tue Sep 27 15:41:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Clark X-Patchwork-Id: 12990872 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0A4AC6FA83 for ; Tue, 27 Sep 2022 15:45:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232477AbiI0PpW (ORCPT ); Tue, 27 Sep 2022 11:45:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232505AbiI0Po5 (ORCPT ); Tue, 27 Sep 2022 11:44:57 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C22043FA0D; Tue, 27 Sep 2022 08:41:24 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 971501EDB; Tue, 27 Sep 2022 08:41:30 -0700 (PDT) Received: from e121896.Emea.Arm.com (e121896.Emea.Arm.com [10.32.36.24]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 222693F73B; Tue, 27 Sep 2022 08:41:20 -0700 (PDT) From: James Clark To: acme@kernel.org, namhyung@kernel.org Cc: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, will@kernel.org, James Clark , Peter Zijlstra , Ingo Molnar , Mark Rutland , Alexander Shishkin , Jiri Olsa , bpf@vger.kernel.org Subject: [RFC PATCH 2/4] perf test: Add mechanism for skipping attr tests on auxiliary vector values Date: Tue, 27 Sep 2022 16:41:02 +0100 Message-Id: <20220927154104.869029-3-james.clark@arm.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20220927154104.869029-1-james.clark@arm.com> References: <20220927154104.869029-1-james.clark@arm.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC This can be used to skip tests or provide different test values on different platforms. For example to run a test only where Arm SVE is present add this to the config section: auxv = auxv["AT_HWCAP"] & 0x200000 == 0x200000 The value is a freeform Python expression that is evaled in the context of a map called "auxv" that contains the decoded auxiliary vector. Signed-off-by: James Clark --- tools/perf/tests/attr.py | 33 +++++++++++++++++++++++++++++++-- 1 file changed, 31 insertions(+), 2 deletions(-) diff --git a/tools/perf/tests/attr.py b/tools/perf/tests/attr.py index 7e053fa5d6ae..2d6124801da0 100644 --- a/tools/perf/tests/attr.py +++ b/tools/perf/tests/attr.py @@ -8,7 +8,9 @@ import glob import optparse import tempfile import logging +import re import shutil +import subprocess try: import configparser @@ -134,6 +136,8 @@ class Event(dict): # 'arch' - architecture specific test (optional) # comma separated list, ! at the beginning # negates it. +# 'auxv' - Truthy statement that is evaled in the scope of the auxv map. When false, +# the test is skipped. For example 'auxv["AT_HWCAP"] == 10'. (optional) # # [eventX:base] # - one or multiple instances in file @@ -164,6 +168,7 @@ class Test(object): except: self.arch = '' + self.auxv = parser.get('config', 'auxv', fallback=None) self.expect = {} self.result = {} log.debug(" loading expected events"); @@ -175,7 +180,28 @@ class Test(object): else: return True - def skip_test(self, myarch): + def skip_test_auxv(self): + def new_auxv(a, pattern): + items = list(filter(None, pattern.split(a))) + # AT_HWCAP is hex but doesn't have a prefix, so special case it + if items[0] == "AT_HWCAP": + value = int(items[-1], 16) + else: + try: + value = int(items[-1], 0) + except: + value = items[-1] + return (items[0], value) + + if not self.auxv: + return False + auxv = subprocess.check_output("LD_SHOW_AUXV=1 sleep 0", shell=True) \ + .decode(sys.stdout.encoding) + pattern = re.compile(r"[: ]+") + auxv = dict([new_auxv(a, pattern) for a in auxv.splitlines()]) + return not eval(self.auxv) + + def skip_test_arch(self, myarch): # If architecture not set always run test if self.arch == '': # log.warning("test for arch %s is ok" % myarch) @@ -225,9 +251,12 @@ class Test(object): def run_cmd(self, tempdir): junk1, junk2, junk3, junk4, myarch = (os.uname()) - if self.skip_test(myarch): + if self.skip_test_arch(myarch): raise Notest(self, myarch) + if self.skip_test_auxv(): + raise Notest(self, "auxv skip") + cmd = "PERF_TEST_ATTR=%s %s %s -o %s/perf.data %s" % (tempdir, self.perf, self.command, tempdir, self.args) ret = os.WEXITSTATUS(os.system(cmd)) From patchwork Tue Sep 27 15:41:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Clark X-Patchwork-Id: 12990875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF048C6FA82 for ; Tue, 27 Sep 2022 15:45:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232707AbiI0Ppo (ORCPT ); Tue, 27 Sep 2022 11:45:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229572AbiI0PpK (ORCPT ); Tue, 27 Sep 2022 11:45:10 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 87A611B85C3; Tue, 27 Sep 2022 08:41:28 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 20F9E1F02; Tue, 27 Sep 2022 08:41:34 -0700 (PDT) Received: from e121896.Emea.Arm.com (e121896.Emea.Arm.com [10.32.36.24]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B0BE53F73B; Tue, 27 Sep 2022 08:41:24 -0700 (PDT) From: James Clark To: acme@kernel.org, namhyung@kernel.org Cc: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, will@kernel.org, James Clark , Peter Zijlstra , Ingo Molnar , Mark Rutland , Alexander Shishkin , Jiri Olsa , bpf@vger.kernel.org Subject: [RFC PATCH 3/4] perf test: Add mechanism for skipping attr tests on kernel versions Date: Tue, 27 Sep 2022 16:41:03 +0100 Message-Id: <20220927154104.869029-4-james.clark@arm.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20220927154104.869029-1-james.clark@arm.com> References: <20220927154104.869029-1-james.clark@arm.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC The first two version numbers are used since that is where the ABI changes happen, so seems to be the most useful for now. 'Until' is exclusive and 'since' is inclusive so that the same version number can be used to mark a point where the change comes into effect. This allows keeping the tests in a state where new tests will also pass on older kernels if the existence of a new feature isn't explicitly broadcast by the kernel. For example extended user regs are currently discovered by trial and error calls to perf_event_open. Signed-off-by: James Clark --- tools/perf/tests/attr.py | 28 +++++++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/tools/perf/tests/attr.py b/tools/perf/tests/attr.py index 2d6124801da0..e890c261ad26 100644 --- a/tools/perf/tests/attr.py +++ b/tools/perf/tests/attr.py @@ -6,6 +6,7 @@ import os import sys import glob import optparse +import platform import tempfile import logging import re @@ -125,6 +126,11 @@ class Event(dict): if not data_equal(self[t], other[t]): log.warning("expected %s=%s, got %s" % (t, self[t], other[t])) +def parse_version(version): + if not version: + return None + return [int(v) for v in version.split(".")[0:2]] + # Test file description needs to have following sections: # [config] # - just single instance in file @@ -138,7 +144,9 @@ class Event(dict): # negates it. # 'auxv' - Truthy statement that is evaled in the scope of the auxv map. When false, # the test is skipped. For example 'auxv["AT_HWCAP"] == 10'. (optional) -# +# 'kernel_since' - Inclusive kernel version from which the test will start running. Only the +# first two values are supported, for example "6.1" (optional) +# 'kernel_until' - Exclusive kernel version from which the test will stop running. (optional) # [eventX:base] # - one or multiple instances in file # - expected values assignments @@ -169,6 +177,8 @@ class Test(object): self.arch = '' self.auxv = parser.get('config', 'auxv', fallback=None) + self.kernel_since = parse_version(parser.get('config', 'kernel_since', fallback=None)) + self.kernel_until = parse_version(parser.get('config', 'kernel_until', fallback=None)) self.expect = {} self.result = {} log.debug(" loading expected events"); @@ -180,6 +190,16 @@ class Test(object): else: return True + def skip_test_kernel_since(self): + if not self.kernel_since: + return False + return not self.kernel_since <= parse_version(platform.release()) + + def skip_test_kernel_until(self): + if not self.kernel_until: + return False + return not parse_version(platform.release()) < self.kernel_until + def skip_test_auxv(self): def new_auxv(a, pattern): items = list(filter(None, pattern.split(a))) @@ -257,6 +277,12 @@ class Test(object): if self.skip_test_auxv(): raise Notest(self, "auxv skip") + if self.skip_test_kernel_since(): + raise Notest(self, "old kernel skip") + + if self.skip_test_kernel_until(): + raise Notest(self, "new kernel skip") + cmd = "PERF_TEST_ATTR=%s %s %s -o %s/perf.data %s" % (tempdir, self.perf, self.command, tempdir, self.args) ret = os.WEXITSTATUS(os.system(cmd)) From patchwork Tue Sep 27 15:41:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Clark X-Patchwork-Id: 12990876 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C4D9C6FA8E for ; Tue, 27 Sep 2022 15:45:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232762AbiI0Ppr (ORCPT ); Tue, 27 Sep 2022 11:45:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46200 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232237AbiI0PpN (ORCPT ); Tue, 27 Sep 2022 11:45:13 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 9748B1B85F6; Tue, 27 Sep 2022 08:41:31 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 96B9E1756; Tue, 27 Sep 2022 08:41:37 -0700 (PDT) Received: from e121896.Emea.Arm.com (e121896.Emea.Arm.com [10.32.36.24]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6403F3F73B; Tue, 27 Sep 2022 08:41:28 -0700 (PDT) From: James Clark To: acme@kernel.org, namhyung@kernel.org Cc: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, will@kernel.org, James Clark , Peter Zijlstra , Ingo Molnar , Mark Rutland , Alexander Shishkin , Jiri Olsa , bpf@vger.kernel.org Subject: [RFC PATCH 4/4] perf test arm64: Add attr tests for new VG register Date: Tue, 27 Sep 2022 16:41:04 +0100 Message-Id: <20220927154104.869029-5-james.clark@arm.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20220927154104.869029-1-james.clark@arm.com> References: <20220927154104.869029-1-james.clark@arm.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Ensure that the availability of the VG register behaves as expected depending on the kernel version and SVE support. Signed-off-by: James Clark --- .../attr/test-record-user-regs-no-sve-aarch64 | 9 +++++++++ .../attr/test-record-user-regs-old-sve-aarch64 | 10 ++++++++++ .../tests/attr/test-record-user-regs-sve-aarch64 | 14 ++++++++++++++ 3 files changed, 33 insertions(+) create mode 100644 tools/perf/tests/attr/test-record-user-regs-no-sve-aarch64 create mode 100644 tools/perf/tests/attr/test-record-user-regs-old-sve-aarch64 create mode 100644 tools/perf/tests/attr/test-record-user-regs-sve-aarch64 diff --git a/tools/perf/tests/attr/test-record-user-regs-no-sve-aarch64 b/tools/perf/tests/attr/test-record-user-regs-no-sve-aarch64 new file mode 100644 index 000000000000..fbb065842880 --- /dev/null +++ b/tools/perf/tests/attr/test-record-user-regs-no-sve-aarch64 @@ -0,0 +1,9 @@ +# Test that asking for VG fails if the system doesn't support SVE. This +# applies both before and after the feature was added in 6.1 +[config] +command = record +args = --no-bpf-event --user-regs=vg kill >/dev/null 2>&1 +ret = 129 +test_ret = true +arch = aarch64 +auxv = auxv["AT_HWCAP"] & 0x200000 == 0 diff --git a/tools/perf/tests/attr/test-record-user-regs-old-sve-aarch64 b/tools/perf/tests/attr/test-record-user-regs-old-sve-aarch64 new file mode 100644 index 000000000000..15ebfc3418e3 --- /dev/null +++ b/tools/perf/tests/attr/test-record-user-regs-old-sve-aarch64 @@ -0,0 +1,10 @@ +# Test that asking for VG always fails on old kernels because it was +# added in 6.1. This applies to systems that either support or don't +# support SVE. +[config] +command = record +args = --no-bpf-event --user-regs=vg kill >/dev/null 2>&1 +ret = 129 +test_ret = true +arch = aarch64 +kernel_until = 6.1 diff --git a/tools/perf/tests/attr/test-record-user-regs-sve-aarch64 b/tools/perf/tests/attr/test-record-user-regs-sve-aarch64 new file mode 100644 index 000000000000..c598c803221d --- /dev/null +++ b/tools/perf/tests/attr/test-record-user-regs-sve-aarch64 @@ -0,0 +1,14 @@ +# Test that asking for VG works if the system has SVE and after the +# feature was added in 6.1 +[config] +command = record +args = --no-bpf-event --user-regs=vg kill >/dev/null 2>&1 +ret = 1 +test_ret = true +arch = aarch64 +auxv = auxv["AT_HWCAP"] & 0x200000 == 0x200000 +kernel_since = 6.1 + +[event:base-record] +sample_type=4359 +sample_regs_user=70368744177664