From patchwork Thu Jan 7 12:07:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12003741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16458C433E0 for ; Thu, 7 Jan 2021 12:08:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7208523355 for ; Thu, 7 Jan 2021 12:08:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7208523355 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8D88C8D0138; Thu, 7 Jan 2021 07:08:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 888658D011F; Thu, 7 Jan 2021 07:08:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 750418D0138; Thu, 7 Jan 2021 07:08:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0158.hostedemail.com [216.40.44.158]) by kanga.kvack.org (Postfix) with ESMTP id 60FFA8D011F for ; Thu, 7 Jan 2021 07:08:27 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1AA31180AD804 for ; Thu, 7 Jan 2021 12:08:27 +0000 (UTC) X-FDA: 77678856654.07.sleet93_2c0a6c7274ea Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin07.hostedemail.com (Postfix) with ESMTP id EFBAF1803FFC1 for ; Thu, 7 Jan 2021 12:08:26 +0000 (UTC) X-HE-Tag: sleet93_2c0a6c7274ea X-Filterd-Recvd-Size: 10351 Received: from smtp-fw-4101.amazon.com (smtp-fw-4101.amazon.com [72.21.198.25]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Thu, 7 Jan 2021 12:08:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1610021306; x=1641557306; h=from:to:cc:subject:date:message-id:mime-version; bh=Pq/1QFmMwIoQiLmPrNgwyVK1ezuCrlRevhqPdfRTny8=; b=k2KXrXlhmjiaI0mI1KzL+x9Yzz4BvUVr6oYy4eCYel6vU9C2NJApRwP8 p+7COiguJ0pDyHPx/8Qi839LxkdwIVFsDkjGrR/w+ZtLQQZBi7faIOT8z OMoWdPsaE2lNlq/BY5KClBLZrPTWWsf7K0ttHPWEhjChSNghSEiQmH+Km M=; X-IronPort-AV: E=Sophos;i="5.79,329,1602547200"; d="scan'208";a="73626685" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-1a-af6a10df.us-east-1.amazon.com) ([10.43.8.2]) by smtp-border-fw-out-4101.iad4.amazon.com with ESMTP; 07 Jan 2021 12:08:19 +0000 Received: from EX13D31EUA001.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan3.iad.amazon.com [10.40.163.38]) by email-inbound-relay-1a-af6a10df.us-east-1.amazon.com (Postfix) with ESMTPS id A8E4EA207D; Thu, 7 Jan 2021 12:08:07 +0000 (UTC) Received: from u3f2cd687b01c55.ant.amazon.com (10.43.160.66) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 7 Jan 2021 12:07:50 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC PATCH] tools/perf: Integrate DAMON in perf Date: Thu, 7 Jan 2021 13:07:29 +0100 Message-ID: <20210107120729.22328-1-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-Originating-IP: [10.43.160.66] X-ClientProxiedBy: EX13D23UWC003.ant.amazon.com (10.43.162.81) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park NOTE: This RFC has a dependancy on DAMON (Data Access MONitor) patchset[1], which is not merged in the mainline yet. The aim of this is to show how DAMON would be evolved once it is merged in and get some comments early. So, if you have some interest in this, please consider reviewing the DAMON patchset, either. [1] https://lore.kernel.org/linux-mm/20201215115448.25633-1-sjpark@amazon.com --- Though there is a simple debugfs interface for DAMON and even a dedicated user space tool, integrating the interface in perf will make best user experiences. For the reason, this commit adds perf scripts for DAMON. After this commit, users can record the data access monitoring results and read in human readable form with: $ sudo perf script record damon $ sudo perf script report damon or simply, $ sudo perf script damon Nevertheless, above commands do not start the monitoring by themselves, so the user should turn it on in background. To make it easy, this commit also adds a convenient version of 'perf script record damon', which executes a command, turns on DAMON for the process, and records DAMON trace events. For example, it can be used as: $ sudo ~/libexec/perf-core/scripts/python/bin/damon-record.sh $ sudo perf script report damon Currently, the report command supports only raw format. Signed-off-by: SeongJae Park --- tools/perf/scripts/python/bin/damon-record | 4 + tools/perf/scripts/python/bin/damon-record.sh | 163 ++++++++++++++++++ tools/perf/scripts/python/bin/damon-report | 4 + tools/perf/scripts/python/damon.py | 44 +++++ 4 files changed, 215 insertions(+) create mode 100644 tools/perf/scripts/python/bin/damon-record create mode 100644 tools/perf/scripts/python/bin/damon-record.sh create mode 100644 tools/perf/scripts/python/bin/damon-report create mode 100644 tools/perf/scripts/python/damon.py diff --git a/tools/perf/scripts/python/bin/damon-record b/tools/perf/scripts/python/bin/damon-record new file mode 100644 index 000000000000..6e21802c8c0e --- /dev/null +++ b/tools/perf/scripts/python/bin/damon-record @@ -0,0 +1,4 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +perf record -e damon:damon_aggregated $@ diff --git a/tools/perf/scripts/python/bin/damon-record.sh b/tools/perf/scripts/python/bin/damon-record.sh new file mode 100644 index 000000000000..3e3e27343ccc --- /dev/null +++ b/tools/perf/scripts/python/bin/damon-record.sh @@ -0,0 +1,163 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# +# This is more convenient version of 'perf script record damon '. +# While the command assumes DAMON will be turned on by the user, this do that +# instead. That is, this command starts the command, turn DAMON on for the +# process, and record the trace events. + +pr_usage() +{ + >&2 echo "Usage: $0 [OPTION]... " + >&2 echo + >&2 echo "OPTION" + >&2 echo " --sampling Sampling interval (us)" + >&2 echo " --aggregate Aggregate interval (us)" + >&2 echo " --update Regions update interval (us)" + >&2 echo " --min-reg Minimum number of regions" + >&2 echo " --max-reg Maximum number of regions" +} + +# Default values (intervals are in us) +sampling_interval=5000 +aggregate_interval=100000 +regions_update_interval=1000000 +min_nr_regions=10 +max_nr_regions=1000 +cmd="" + +debugfs_dir=$(mount | grep -e "type debugfs" | awk '{print $3}') +if [ -z "$debugfs_dir" ] +then + >&2 echo "debugfs not found" + exit 1 +fi + +damon_dir="$debugfs_dir/damon" +if [ ! -d "$damon_dir" ] +then + >&2 echo "damon dir not found" + exit 1 +fi + +if [ $# -lt 1 ] +then + pr_usage + exit 1 +fi + +while [ $# -ne 0 ] +do + case $1 in + "--sampling") + if [ $# -lt 2 ] + then + >&2 echo " not given" + pr_usage + exit 1 + fi + sampling_interval=$2 + shift 2 + continue + ;; + "--aggregate") + if [ $# -lt 2 ] + then + >&2 echo " not given" + pr_usage + exit 1 + fi + aggregate_interval=$2 + shift 2 + continue + ;; + "--update") + if [ $# -lt 2 ] + then + >&2 echo " not given" + pr_usage + exit 1 + fi + regions_update_interval=$2 + shift 2 + continue + ;; + "--min_reg") + if [ $# -lt 2 ] + then + >&2 echo " not given" + pr_usage + exit 1 + fi + min_nr_regions=$2 + shift 2 + continue + ;; + "--max_reg") + if [ $# -lt 2 ] + then + >&2 echo " not given" + pr_usage + exit 1 + fi + max_nr_regions=$2 + shift 2 + continue + ;; + *) + if [ $# -lt 1 ] + then + >&2 echo " not given" + pr_usage + exit 1 + fi + cmd="$*" + break + ;; + esac +done + +if [ -z "$cmd" ] +then + pr_usage + exit 1 +fi + +orig_attrs=$(cat "$damon_dir/attrs") +attrs="$sampling_interval $aggregate_interval $regions_update_interval" +attrs="$attrs $min_nr_regions $max_nr_regions" + +echo "$attrs" > "$damon_dir/attrs" + +$cmd & +cmd_pid=$! + +echo "$cmd_pid" > "$damon_dir/target_ids" +echo "on" > "$damon_dir/monitor_on" + +perf record -e damon:damon_aggregated & +perf_pid=$! + +sigint_trap() +{ + kill 2 "$cmd_pid" + kill 2 "$perf_pid" + echo "$orig_attrs" > "$damon_dir/attrs" + exit +} + +trap sigint_trap INT + +>&2 echo "Press Control+C to stop recording" + +while :; +do + on_off=$(cat "$damon_dir/monitor_on") + if [ "$on_off" = "off" ] + then + kill 2 $perf_pid + echo "$orig_attrs" > "$damon_dir/attrs" + break + fi + sleep 1 +done diff --git a/tools/perf/scripts/python/bin/damon-report b/tools/perf/scripts/python/bin/damon-report new file mode 100644 index 000000000000..89ece171959e --- /dev/null +++ b/tools/perf/scripts/python/bin/damon-report @@ -0,0 +1,4 @@ +#!/bin/bash +# description: data access monitoring + +perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/damon.py diff --git a/tools/perf/scripts/python/damon.py b/tools/perf/scripts/python/damon.py new file mode 100644 index 000000000000..b71a9bdf00e7 --- /dev/null +++ b/tools/perf/scripts/python/damon.py @@ -0,0 +1,44 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Author: SeongJae Park + +from __future__ import print_function + +import os +import sys + +sys.path.append(os.environ['PERF_EXEC_PATH'] + \ + '/scripts/python/Perf-Trace-Util/lib/Perf/Trace') + +from perf_trace_context import * +from Core import * + + +def trace_begin(): + pass + +def trace_end(): + pass + +start_time = None +nr_printed = 0 +def damon__damon_aggregated(event_name, context, common_cpu, + common_secs, common_nsecs, common_pid, common_comm, + common_callchain, target_id, nr_regions, start, end, + nr_accesses, perf_sample_dict): + global start_time + global nr_printed + time = common_secs * 1000000000 + common_nsecs + if not start_time: + start_time = time + print('start_time: %d' % start_time) + if nr_printed == 0: + print('rel time: %d' % (time - start_time)) + print('target_id: %d' % target_id) + print('nr_regions: %d' % nr_regions) + print('%x-%x (%d): %u' % (start, end, end - start, nr_accesses)) + + nr_printed += 1 + if nr_printed == nr_regions: + nr_printed = 0 + print()