[v8,00/44] Start KernelShark v2 transformation

Message

Yordan Karadzhov Jan. 4, 2021, 5:46 p.m. UTC

In this patch-set we are starting the introduction of the changes
in the C API of libkshark. The goal of this transformation is to
turn KernelShark into a general purpose toolkit for analysis of
tracing data that is no longer strongly coupled to the Ftrace/trace-cmd
data format. At the end of the transformation KernelShark will be able
to merge and simultaneously visualize data from different tracers,
recorded in different data formats. It will also significantly extend
the possibilities for user customization.

This  patch-set transforms the way KernelShark processes the raw tracing
data. Later, in a consecutive patch-sets we will introduce the changes
in the visualization instruments and in the KernelShark GUI itself.

Changes in v8:
 * In particular patch 6 splits the installation of the project in two
   components. This can be used to create two packages. One for the GUI
   and one "devel" package installing the headers. This patch are inspired
   by the talk "Deep CMake For Library Authors" presented at CppCon 2019
   by Craig Scott.
   https://crascit.com/2019/10/16/cppcon-2019-deep-cmake-for-library-authors/

 * Patches 35-44 has been sent already, but as part of a separated
   patch-set.

Changes in v7:
 * Patches 1-5 are new, but these are only infrastructureal changes
   that have no functional effects on the API.
   - License info is added.
   - README is updated.
   - KernelShark now builds and links against the stand-alone versions
     of libtraceevent and libtracefs.

Changes in v6:
 * The patches from the "OpenGL wrapper" patch-set (already reviewed by
   Steven) have been added here (patches 21-24).

 * Patches 25-27 are new, but these are only infrastructureal changes
   that have no functional effects on the API.


Changes in v5:
 * Only minor modifications addressing issues pointed out by Steven in
   his reviews of v4.

Changes in v4:
 * The integer data format identifier is replaced with a string value
   that gets registered by the readout plugin.

 * Plus several small modifications addressing the issues pointed out
   by Steven.


Changes in v3.
  This version contains number of small modifications addressing the
  issues pointed out by Steven in his review. However, it also contains
  the following major changes:

   * The session context (kshark_context) holds a pointer to unspecified
     data processing interface (void *interface). This will allow to
     easily introduce new versions of this interface in the future, while
     keeping the interface that we have now for backward compatibility.
     The necessary type checking when using the interface made the helper
     functions wrapping the corresponding methods of the interface
     relatively complicated, so the implementations have been moved from
     the header to the source file.

   * A more sophisticated management of the Data stream descriptors that
     can handle the case when Data streams are constantly added and
     removed has been implemented. The design for this change was
     suggested by Steven in his review.


Changes in v2:
 * Tried to address review comments for "kernel-shark: Introduce libkshark-hash"

 * The huge patch "kernel-shark: Integrate the stream definitions with
the C API" in the previous version is now split into 8 smaller patches.
The last of these 8 new patches (kernel-shark: Complete the stream
integration) is still a bit entangled. I can try to further break this
patch in future versions of the patch-set, if this is needed.

Yordan Karadzhov (1):
  kernel-shark: Add "github Actions" workflow

Yordan Karadzhov (VMware) (43):
  kernel-shark: Adopt trace-cmd API changes
  kernel-shark: Use libtraceevent and libtracefs
  kernel-shark: Add license information
  kernel-shark: Change the CMake minimum version required
  kernel-shark: Change default libraries install location
  kernel-shark: Split the installation in two components
  kernel-shark: Update README
  kernel-shark: Define build target for JSONC
  kernel-shark: Use only signed types in kshark_entry
  kernel-shark: Add stream_id to kshark_entry
  kernel-shark: Introduce libkshark-hash
  kernel-shark: Introduce Data streams
  kernel-shark: Rename static methods in libkshark
  kernel-shark: Add basic methods for Data streams
  kernel-shark: Housekeeping before implementing stream interface
  kernel-shark: Add stream interface for trace-cmd data
  kernel-shark: Start introducing KernelShark 2.0
  kernel-shark: Start using data streams
  kernel-shark: Remove dead code
  kernel-shark: Redesign the plugin interface
  kernel-shark: Complete the stream integration
  kernel-shark: Provide merging of multiple data streams
  kernel-shark: Integrate the stream definitions with data model
  kernel-shark: Use only signed types for model defs
  kernel-shark: Add ksmodel_get_bin()
  kernel-shark: Protect ksmodel_set_in_range_bining()
  kernel-shark: Add methods for time calibration
  kernel-shark: Integrate streams with libkshark-configio
  kernel-shark: Add support for drawing text
  kernel-shark: Make GLUT optional dependency
  kernel-shark: Add ksplot_draw_polyline()
  kernel-shark: Optimize ksplot_draw_polygon()
  kernel-shark: Add basic infrastructure for testing
  kernel-shark: Integrate streams with KsPlotTools
  kernel-shark: Add getStreamColorTable()
  kernel-shark: Redefine the args of KsPlot::getColor()
  kernel-shark: Add TextBox class to KsPlot namespace
  kernel-shark: Consistent method naming in KsPlotTools
  kernel-shark: Make the label part of the graph
  kernel-shark: Remove the hard-coded Idle PID
  kernel-shark: Add class Polyline to KsPlot namespace
  kernel-shark: Add VirtBridge and VirtGap classes
  kernel-shark: Add double click interface to PlotObject

 .github/workflows/main.yml       |   63 +
 CMakeLists.txt                   |   55 +-
 Documentation/doxygen/dox_config |    1 +
 LICENSES/GPL-2.0                 |  359 +++
 LICENSES/LGPL-2.1                |  503 +++
 README                           |  129 +-
 build/FindJSONC.cmake            |   70 +-
 build/FindTraceCmd.cmake         |  138 +-
 build/FindTraceEvent.cmake       |   76 +
 build/FindTraceFS.cmake          |   76 +
 build/cmake_clean.sh             |    4 +
 build/cmake_uninstall.sh         |   23 +-
 build/deff.h.cmake               |   15 +-
 build/install_gui.sh             |    1 +
 build/install_libkshark-devel.sh |    1 +
 examples/CMakeLists.txt          |   34 +-
 examples/configio.c              |   20 +-
 examples/datafilter.c            |   67 +-
 examples/datahisto.c             |   38 +-
 examples/dataload.c              |   28 +-
 examples/multibufferload.c       |   53 +
 src/CMakeLists.txt               |   70 +-
 src/KsPlotTools.cpp              |  412 ++-
 src/KsPlotTools.hpp              |  199 +-
 src/libkshark-collection.c       |  121 +-
 src/libkshark-configio.c         | 1208 +++++--
 src/libkshark-hash.c             |  239 ++
 src/libkshark-model.c            |  154 +-
 src/libkshark-model.h            |   55 +-
 src/libkshark-plot.c             |  317 +-
 src/libkshark-plot.h             |   73 +
 src/libkshark-plugin.c           |  658 +++-
 src/libkshark-plugin.h           |  294 +-
 src/libkshark-tepdata.c          | 1914 ++++++++++++
 src/libkshark-tepdata.h          |  112 +
 src/libkshark.c                  | 2619 +++++++++-------
 src/libkshark.h                  |  704 ++++-
 src/plugins/CMakeLists.txt       |    3 +-
 src/plugins/sched_events.c       |    4 +-
 src/stb_truetype.h               | 5011 ++++++++++++++++++++++++++++++
 tests/CMakeLists.txt             |   14 +
 tests/libkshark-tests.cpp        |   48 +
 42 files changed, 13839 insertions(+), 2144 deletions(-)
 create mode 100644 .github/workflows/main.yml
 create mode 100644 LICENSES/GPL-2.0
 create mode 100644 LICENSES/LGPL-2.1
 create mode 100644 build/FindTraceEvent.cmake
 create mode 100644 build/FindTraceFS.cmake
 create mode 100755 build/install_gui.sh
 create mode 100755 build/install_libkshark-devel.sh
 create mode 100644 examples/multibufferload.c
 create mode 100644 src/libkshark-hash.c
 create mode 100644 src/libkshark-tepdata.c
 create mode 100644 src/libkshark-tepdata.h
 create mode 100644 src/stb_truetype.h
 create mode 100644 tests/CMakeLists.txt
 create mode 100644 tests/libkshark-tests.cpp

Comments

Steven Rostedt Jan. 5, 2021, 12:27 a.m. UTC | #1

On Mon,  4 Jan 2021 19:46:40 +0200
"Yordan Karadzhov (VMware)" <[e-mail]> wrote:

> In this patch-set we are starting the introduction of the changes
> in the C API of libkshark. The goal of this transformation is to
> turn KernelShark into a general purpose toolkit for analysis of
> tracing data that is no longer strongly coupled to the Ftrace/trace-cmd
> data format. At the end of the transformation KernelShark will be able
> to merge and simultaneously visualize data from different tracers,
> recorded in different data formats. It will also significantly extend
> the possibilities for user customization.
> 
> This  patch-set transforms the way KernelShark processes the raw tracing
> data. Later, in a consecutive patch-sets we will introduce the changes
> in the visualization instruments and in the KernelShark GUI itself.
> 

Hi Yordan,

I don't think you noticed that I applied your last series, with tweaks
(commenting in the change log what I tweaked).

Can you rebase this series against mainline?

  git://git.kernel.org/pub/scm/utils/trace-cmd/kernel-shark.git

Thanks!

-- Steve