[0/8] rtla improvements

Message

Daniel Bristot de Oliveira May 11, 2023, 6:04 p.m. UTC

This is a series of improvements for rtla, mainly as a result of our
daily usage of the tool debugging problems at red hat.

The cgroup support and house keeping options are from our usage
of the tool debugging containers.

The auto-analysis overhead reduction is needed when we go to
large boxes - but it is really hand in practice, as it gives an idea
of the problem without having to look at the trace.

Finally, running hwnoise 100 % of CPU time might cause some systems
to slow down too much. Reduce its utilization to 75% by default to
avoid problems for people using it for the first time.

Daniel Bristot de Oliveira (8):
  rtla: Add -C cgroup support
  rtla: Add --house-keeping option
  rtla: Change monitored_cpus from char * to cpu_set_t
  rtla: Automatically move rtla to a house-keeping cpu
  rtla/timerlat: Give timerlat auto analysis its own instance
  rtla/timerlat_hist: Add auto-analysis support
  rtla: Start the tracers after creating all instances
  rtla/hwnoise: Reduce runtime to 75%

 Documentation/tools/rtla/common_options.rst   |   8 +
 .../tools/rtla/common_timerlat_aa.rst         |   7 -
 .../tools/rtla/rtla-timerlat-hist.rst         |   7 +-
 .../tools/rtla/rtla-timerlat-top.rst          |   7 +
 tools/tracing/rtla/src/osnoise_hist.c         |  90 +++++-
 tools/tracing/rtla/src/osnoise_top.c          |  83 +++++-
 tools/tracing/rtla/src/timerlat_aa.c          |  35 ++-
 tools/tracing/rtla/src/timerlat_aa.h          |   5 +-
 tools/tracing/rtla/src/timerlat_hist.c        | 139 +++++++--
 tools/tracing/rtla/src/timerlat_top.c         | 127 +++++++--
 tools/tracing/rtla/src/utils.c                | 266 ++++++++++++++++--
 tools/tracing/rtla/src/utils.h                |   5 +
 12 files changed, 684 insertions(+), 95 deletions(-)

Comments

Daniel Bristot de Oliveira May 11, 2023, 6:04 p.m. UTC | #1

Result: OK
(mbox) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'From: Daniel Bristot de Oliveira <[e-mail]>'
(body) Adding cc: Steven Rostedt <[e-mail]> from line 'Cc: Steven Rostedt <[e-mail]>'
(body) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'Cc: Daniel Bristot de Oliveira <[e-mail]>'
(body) Adding cc: Masami Hiramatsu <[e-mail]> from line 'Cc: Masami Hiramatsu <[e-mail]>'
(body) Adding cc: Jonathan Corbet <[e-mail]> from line 'Cc: Jonathan Corbet <[e-mail]>'
(body) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'Signed-off-by: Daniel Bristot de Oliveira <[e-mail]>'
Dry-OK. Log says:
Server: mail.kernel.org
MAIL FROM:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
From: Daniel Bristot de Oliveira <[e-mail]>
To: [e-mail],
	Steven Rostedt <[e-mail]>
Cc: [e-mail],
	[e-mail],
	Juri Lelli <[e-mail]>,
	William White <[e-mail]>,
	Daniel Bristot de Oliveira <[e-mail]>,
	Masami Hiramatsu <[e-mail]>,
	Jonathan Corbet <[e-mail]>
Subject: [PATCH 1/8] rtla: Add -C cgroup support
Date: Thu, 11 May 2023 20:02:53 +0200
Message-Id: <[e-mail]>
X-Mailer: git-send-email 2.38.1
In-Reply-To: <[e-mail]>
References: <[e-mail]>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Result: OK
(mbox) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'From: Daniel Bristot de Oliveira <[e-mail]>'
(body) Adding cc: Steven Rostedt <[e-mail]> from line 'Cc: Steven Rostedt <[e-mail]>'
(body) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'Cc: Daniel Bristot de Oliveira <[e-mail]>'
(body) Adding cc: Masami Hiramatsu <[e-mail]> from line 'Cc: Masami Hiramatsu <[e-mail]>'
(body) Adding cc: Jonathan Corbet <[e-mail]> from line 'Cc: Jonathan Corbet <[e-mail]>'
(body) Adding cc: Juri Lelli <[e-mail]> from line 'Suggested-by: Juri Lelli <[e-mail]>'
(body) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'Signed-off-by: Daniel Bristot de Oliveira <[e-mail]>'
Dry-OK. Log says:
Server: mail.kernel.org
MAIL FROM:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
From: Daniel Bristot de Oliveira <[e-mail]>
To: [e-mail],
	Steven Rostedt <[e-mail]>
Cc: [e-mail],
	[e-mail],
	Juri Lelli <[e-mail]>,
	William White <[e-mail]>,
	Daniel Bristot de Oliveira <[e-mail]>,
	Masami Hiramatsu <[e-mail]>,
	Jonathan Corbet <[e-mail]>
Subject: [PATCH 2/8] rtla: Add --house-keeping option
Date: Thu, 11 May 2023 20:02:54 +0200
Message-Id: <[e-mail]>
X-Mailer: git-send-email 2.38.1
In-Reply-To: <[e-mail]>
References: <[e-mail]>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Result: OK
(mbox) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'From: Daniel Bristot de Oliveira <[e-mail]>'
(body) Adding cc: Steven Rostedt <[e-mail]> from line 'Cc: Steven Rostedt <[e-mail]>'
(body) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'Cc: Daniel Bristot de Oliveira <[e-mail]>'
(body) Adding cc: Masami Hiramatsu <[e-mail]> from line 'Cc: Masami Hiramatsu <[e-mail]>'
(body) Adding cc: Jonathan Corbet <[e-mail]> from line 'Cc: Jonathan Corbet <[e-mail]>'
(body) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'Signed-off-by: Daniel Bristot de Oliveira <[e-mail]>'
Dry-OK. Log says:
Server: mail.kernel.org
MAIL FROM:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
From: Daniel Bristot de Oliveira <[e-mail]>
To: [e-mail],
	Steven Rostedt <[e-mail]>
Cc: [e-mail],
	[e-mail],
	Juri Lelli <[e-mail]>,
	William White <[e-mail]>,
	Daniel Bristot de Oliveira <[e-mail]>,
	Masami Hiramatsu <[e-mail]>,
	Jonathan Corbet <[e-mail]>
Subject: [PATCH 3/8] rtla: Change monitored_cpus from char * to cpu_set_t
Date: Thu, 11 May 2023 20:02:55 +0200
Message-Id: <[e-mail]>
X-Mailer: git-send-email 2.38.1
In-Reply-To: <[e-mail]>
References: <[e-mail]>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Result: OK
(mbox) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'From: Daniel Bristot de Oliveira <[e-mail]>'
(body) Adding cc: Steven Rostedt <[e-mail]> from line 'Cc: Steven Rostedt <[e-mail]>'
(body) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'Cc: Daniel Bristot de Oliveira <[e-mail]>'
(body) Adding cc: Masami Hiramatsu <[e-mail]> from line 'Cc: Masami Hiramatsu <[e-mail]>'
(body) Adding cc: Jonathan Corbet <[e-mail]> from line 'Cc: Jonathan Corbet <[e-mail]>'
(body) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'Signed-off-by: Daniel Bristot de Oliveira <[e-mail]>'
Dry-OK. Log says:
Server: mail.kernel.org
MAIL FROM:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
From: Daniel Bristot de Oliveira <[e-mail]>
To: [e-mail],
	Steven Rostedt <[e-mail]>
Cc: [e-mail],
	[e-mail],
	Juri Lelli <[e-mail]>,
	William White <[e-mail]>,
	Daniel Bristot de Oliveira <[e-mail]>,
	Masami Hiramatsu <[e-mail]>,
	Jonathan Corbet <[e-mail]>
Subject: [PATCH 4/8] rtla: Automatically move rtla to a house-keeping cpu
Date: Thu, 11 May 2023 20:02:56 +0200
Message-Id: <[e-mail]>
X-Mailer: git-send-email 2.38.1
In-Reply-To: <[e-mail]>
References: <[e-mail]>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Result: OK
(mbox) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'From: Daniel Bristot de Oliveira <[e-mail]>'
(body) Adding cc: Steven Rostedt <[e-mail]> from line 'Cc: Steven Rostedt <[e-mail]>'
(body) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'Cc: Daniel Bristot de Oliveira <[e-mail]>'
(body) Adding cc: Masami Hiramatsu <[e-mail]> from line 'Cc: Masami Hiramatsu <[e-mail]>'
(body) Adding cc: Jonathan Corbet <[e-mail]> from line 'Cc: Jonathan Corbet <[e-mail]>'
(body) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'Signed-off-by: Daniel Bristot de Oliveira <[e-mail]>'
Dry-OK. Log says:
Server: mail.kernel.org
MAIL FROM:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
From: Daniel Bristot de Oliveira <[e-mail]>
To: [e-mail],
	Steven Rostedt <[e-mail]>
Cc: [e-mail],
	[e-mail],
	Juri Lelli <[e-mail]>,
	William White <[e-mail]>,
	Daniel Bristot de Oliveira <[e-mail]>,
	Masami Hiramatsu <[e-mail]>,
	Jonathan Corbet <[e-mail]>
Subject: [PATCH 5/8] rtla/timerlat: Give timerlat auto-analysis its own instance
Date: Thu, 11 May 2023 20:02:57 +0200
Message-Id: <[e-mail]>
X-Mailer: git-send-email 2.38.1
In-Reply-To: <[e-mail]>
References: <[e-mail]>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Result: OK
(mbox) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'From: Daniel Bristot de Oliveira <[e-mail]>'
(body) Adding cc: Steven Rostedt <[e-mail]> from line 'Cc: Steven Rostedt <[e-mail]>'
(body) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'Cc: Daniel Bristot de Oliveira <[e-mail]>'
(body) Adding cc: Masami Hiramatsu <[e-mail]> from line 'Cc: Masami Hiramatsu <[e-mail]>'
(body) Adding cc: Jonathan Corbet <[e-mail]> from line 'Cc: Jonathan Corbet <[e-mail]>'
(body) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'Signed-off-by: Daniel Bristot de Oliveira <[e-mail]>'
Dry-OK. Log says:
Server: mail.kernel.org
MAIL FROM:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
From: Daniel Bristot de Oliveira <[e-mail]>
To: [e-mail],
	Steven Rostedt <[e-mail]>
Cc: [e-mail],
	[e-mail],
	Juri Lelli <[e-mail]>,
	William White <[e-mail]>,
	Daniel Bristot de Oliveira <[e-mail]>,
	Masami Hiramatsu <[e-mail]>,
	Jonathan Corbet <[e-mail]>
Subject: [PATCH 6/8] rtla/timerlat_hist: Add auto-analysis support
Date: Thu, 11 May 2023 20:02:58 +0200
Message-Id: <[e-mail]>
X-Mailer: git-send-email 2.38.1
In-Reply-To: <[e-mail]>
References: <[e-mail]>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Result: OK
(mbox) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'From: Daniel Bristot de Oliveira <[e-mail]>'
(body) Adding cc: Steven Rostedt <[e-mail]> from line 'Cc: Steven Rostedt <[e-mail]>'
(body) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'Cc: Daniel Bristot de Oliveira <[e-mail]>'
(body) Adding cc: Masami Hiramatsu <[e-mail]> from line 'Cc: Masami Hiramatsu <[e-mail]>'
(body) Adding cc: Jonathan Corbet <[e-mail]> from line 'Cc: Jonathan Corbet <[e-mail]>'
(body) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'Signed-off-by: Daniel Bristot de Oliveira <[e-mail]>'
Dry-OK. Log says:
Server: mail.kernel.org
MAIL FROM:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
From: Daniel Bristot de Oliveira <[e-mail]>
To: [e-mail],
	Steven Rostedt <[e-mail]>
Cc: [e-mail],
	[e-mail],
	Juri Lelli <[e-mail]>,
	William White <[e-mail]>,
	Daniel Bristot de Oliveira <[e-mail]>,
	Masami Hiramatsu <[e-mail]>,
	Jonathan Corbet <[e-mail]>
Subject: [PATCH 7/8] rtla: Start the tracers after creating all instances
Date: Thu, 11 May 2023 20:02:59 +0200
Message-Id: <[e-mail]>
X-Mailer: git-send-email 2.38.1
In-Reply-To: <[e-mail]>
References: <[e-mail]>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Result: OK
(mbox) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'From: Daniel Bristot de Oliveira <[e-mail]>'
(body) Adding cc: Steven Rostedt <[e-mail]> from line 'Cc: Steven Rostedt <[e-mail]>'
(body) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'Cc: Daniel Bristot de Oliveira <[e-mail]>'
(body) Adding cc: Masami Hiramatsu <[e-mail]> from line 'Cc: Masami Hiramatsu <[e-mail]>'
(body) Adding cc: Jonathan Corbet <[e-mail]> from line 'Cc: Jonathan Corbet <[e-mail]>'
(body) Adding cc: Daniel Bristot de Oliveira <[e-mail]> from line 'Signed-off-by: Daniel Bristot de Oliveira <[e-mail]>'
Dry-OK. Log says:
Server: mail.kernel.org
MAIL FROM:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
RCPT TO:<[e-mail]>
From: Daniel Bristot de Oliveira <[e-mail]>
To: [e-mail],
	Steven Rostedt <[e-mail]>
Cc: [e-mail],
	[e-mail],
	Juri Lelli <[e-mail]>,
	William White <[e-mail]>,
	Daniel Bristot de Oliveira <[e-mail]>,
	Masami Hiramatsu <[e-mail]>,
	Jonathan Corbet <[e-mail]>
Subject: [PATCH 8/8] rtla/hwnoise: Reduce runtime to 75%
Date: Thu, 11 May 2023 20:03:00 +0200
Message-Id: <[e-mail]>
X-Mailer: git-send-email 2.38.1
In-Reply-To: <[e-mail]>
References: <[e-mail]>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Result: OK

Daniel Bristot de Oliveira May 11, 2023, 6:08 p.m. UTC | #2

ok, today is one of those days... Ignore this email, it is the log --dry-run I tried before.

I need to automate this final checks :-)

-- Daniel