From patchwork Thu Aug 27 09:33:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Song Bao Hua (Barry Song)" X-Patchwork-Id: 11740481 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4C2E0109B for ; Thu, 27 Aug 2020 09:37:11 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 237EE20738 for ; Thu, 27 Aug 2020 09:37:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="tIVu2/eG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 237EE20738 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=hisilicon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:Subject:To:From: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=mBrtw1NbVMr94CydOnTAI8miJ/THMpEOEJJI2/9eG+E=; b=tIVu2/eG+JNQZclR3inuaPtokY 4ay6FtdDSIPpA6srkMt+6aO8+m1hs1ZNneEDlbkPK04a4b2cpsT5bCUt9vx6+vQo7vFyAfxk9uMx1 NKS+f8ZwnZ4lM8WQI6azEHcfHOzM1U6yNJ9LolhrQxlTMsj0tuRums+EQkDNNh0GP+6cYr6d3brRk hkXgXczBkEdxF7hnY4yo6xSdaYa5UgUBJaagK7fhUzv+VfM67/OCamho+0EVLCgFRXke3wSkDHxHZ rjpr4p2DVkESmbl6XFG3rBguw4rBZnP51lAi/ZBIaJUHY9aaZeIWkZ+teYFlA6UySXuPBrFtPqENI yU2/vBTA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kBEL1-0007ib-7b; Thu, 27 Aug 2020 09:36:55 +0000 Received: from szxga04-in.huawei.com ([45.249.212.190] helo=huawei.com) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kBEKw-0007hE-Rb for linux-arm-kernel@lists.infradead.org; Thu, 27 Aug 2020 09:36:52 +0000 Received: from DGGEMS404-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id E97BBE7CB5867A12A7BA; Thu, 27 Aug 2020 17:36:45 +0800 (CST) Received: from SWX921481.china.huawei.com (10.126.200.71) by DGGEMS404-HUB.china.huawei.com (10.3.19.204) with Microsoft SMTP Server id 14.3.487.0; Thu, 27 Aug 2020 17:36:36 +0800 From: Barry Song To: , Subject: [PATCH] iommu/arm-smmu-v3: add tracepoints for cmdq_issue_cmdlist Date: Thu, 27 Aug 2020 21:33:51 +1200 Message-ID: <20200827093351.15244-1-song.bao.hua@hisilicon.com> X-Mailer: git-send-email 2.21.0.windows.1 MIME-Version: 1.0 X-Originating-IP: [10.126.200.71] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200827_053652_149773_854FA9D2 X-CRM114-Status: GOOD ( 17.90 ) X-Spam-Score: 0.7 (/) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (0.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [45.249.212.190 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.0 RCVD_IN_MSPIKE_H4 RBL: Very Good reputation (+4) [45.249.212.190 listed in wl.mailspike.net] 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 3.0 AC_FROM_MANY_DOTS Multiple periods in From user name X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Barry Song , robin.murphy@arm.com, joro@8bytes.org, will@kernel.org, linuxarm@huawei.com Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org cmdq_issue_cmdlist() is the hotspot that uses a lot of time. This patch adds tracepoints for it to help debug. Signed-off-by: Barry Song --- * can furthermore develop an eBPF program to benchmark using this trace cmdlistlat.c: #include BPF_HASH(start, u32); BPF_HISTOGRAM(dist); TRACEPOINT_PROBE(arm_smmu_v3, issue_cmdlist_entry) { u32 pid; u64 ts, *val; pid = bpf_get_current_pid_tgid(); ts = bpf_ktime_get_ns(); start.update(&pid, &ts); return 0; } TRACEPOINT_PROBE(arm_smmu_v3, issue_cmdlist_exit) { u32 pid; u64 *tsp, delta; pid = bpf_get_current_pid_tgid(); tsp = start.lookup(&pid); if (tsp != 0) { delta = bpf_ktime_get_ns() - *tsp; dist.increment(bpf_log2l(delta)); start.delete(&pid); } return 0; } cmdlistlat.py: #!/usr/bin/python3 # from __future__ import print_function from bcc import BPF from ctypes import c_ushort, c_int, c_ulonglong from time import sleep from sys import argv def usage(): print("USAGE: %s [interval [count]]" % argv[0]) exit() # arguments interval = 5 count = -1 if len(argv) > 1: try: interval = int(argv[1]) if interval == 0: raise if len(argv) > 2: count = int(argv[2]) except: # also catches -h, --help usage() # load BPF program b = BPF(src_file = "cmdlistlat.c") # header print("Tracing... Hit Ctrl-C to end.") # output loop = 0 do_exit = 0 while (1): if count > 0: loop += 1 if loop > count: exit() try: sleep(interval) except KeyboardInterrupt: pass; do_exit = 1 print() b["dist"].print_log2_hist("nsecs") b["dist"].clear() if do_exit: exit() drivers/iommu/arm/arm-smmu-v3/Makefile | 1 + .../iommu/arm/arm-smmu-v3/arm-smmu-v3-trace.h | 48 +++++++++++++++++++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 8 ++++ 3 files changed, 57 insertions(+) create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-trace.h diff --git a/drivers/iommu/arm/arm-smmu-v3/Makefile b/drivers/iommu/arm/arm-smmu-v3/Makefile index 569e24e9f162..dba1087f91f3 100644 --- a/drivers/iommu/arm/arm-smmu-v3/Makefile +++ b/drivers/iommu/arm/arm-smmu-v3/Makefile @@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 +ccflags-y += -I$(src) # needed for trace events obj-$(CONFIG_ARM_SMMU_V3) += arm-smmu-v3.o diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-trace.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-trace.h new file mode 100644 index 000000000000..29ab96706124 --- /dev/null +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-trace.h @@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2020 Hisilicon Limited. + */ + +#undef TRACE_SYSTEM +#define TRACE_SYSTEM arm_smmu_v3 + +#if !defined(_ARM_SMMU_V3_TRACE_H) || defined(TRACE_HEADER_MULTI_READ) +#define _ARM_SMMU_V3_TRACE_H + +#include + +struct device; + +DECLARE_EVENT_CLASS(issue_cmdlist_class, + TP_PROTO(struct device *dev, int n, bool sync), + TP_ARGS(dev, n, sync), + + TP_STRUCT__entry( + __string(device, dev_name(dev)) + __field(int, n) + __field(bool, sync) + ), + TP_fast_assign( + __assign_str(device, dev_name(dev)); + __entry->n = n; + __entry->sync = sync; + ), + TP_printk("%s cmd number=%d sync=%d", + __get_str(device), __entry->n, __entry->sync) +); + +#define DEFINE_ISSUE_CMDLIST_EVENT(name) \ +DEFINE_EVENT(issue_cmdlist_class, name, \ + TP_PROTO(struct device *dev, int n, bool sync), \ + TP_ARGS(dev, n, sync)) + +DEFINE_ISSUE_CMDLIST_EVENT(issue_cmdlist_entry); +DEFINE_ISSUE_CMDLIST_EVENT(issue_cmdlist_exit); + +#endif /* _ARM_SMMU_V3_TRACE_H */ + +#undef TRACE_INCLUDE_PATH +#undef TRACE_INCLUDE_FILE +#define TRACE_INCLUDE_PATH . +#define TRACE_INCLUDE_FILE arm-smmu-v3-trace +#include diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 7332251dd8cd..e2d7d5f1d234 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -33,6 +33,8 @@ #include +#include "arm-smmu-v3-trace.h" + /* MMIO registers */ #define ARM_SMMU_IDR0 0x0 #define IDR0_ST_LVL GENMASK(28, 27) @@ -1389,6 +1391,8 @@ static int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_device *smmu, }, head = llq; int ret = 0; + trace_issue_cmdlist_entry(smmu->dev, n, sync); + /* 1. Allocate some space in the queue */ local_irq_save(flags); llq.val = READ_ONCE(cmdq->q.llq.val); @@ -1493,6 +1497,7 @@ static int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_device *smmu, } local_irq_restore(flags); + trace_issue_cmdlist_exit(smmu->dev, n, sync); return ret; } @@ -4166,6 +4171,9 @@ static struct platform_driver arm_smmu_driver = { }; module_platform_driver(arm_smmu_driver); +#define CREATE_TRACE_POINTS +#include "arm-smmu-v3-trace.h" + MODULE_DESCRIPTION("IOMMU API for ARM architected SMMUv3 implementations"); MODULE_AUTHOR("Will Deacon "); MODULE_ALIAS("platform:arm-smmu-v3");