From patchwork Wed Feb 26 16:31:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 11406761 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 833B914B4 for ; Wed, 26 Feb 2020 16:32:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6275624670 for ; Wed, 26 Feb 2020 16:32:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="I9zkG5DT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727434AbgBZQcG (ORCPT ); Wed, 26 Feb 2020 11:32:06 -0500 Received: from mail-lf1-f54.google.com ([209.85.167.54]:42036 "EHLO mail-lf1-f54.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727173AbgBZQcG (ORCPT ); Wed, 26 Feb 2020 11:32:06 -0500 Received: by mail-lf1-f54.google.com with SMTP id 83so2440914lfh.9 for ; Wed, 26 Feb 2020 08:32:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=T4zUXxck4HFOGvDPhR/G9ns6j0YeeWdcvt2M0bB8PGM=; b=I9zkG5DTeViIvtRwgWSLZYN/PXXScflYj7bmmY8b7Nig8Lha3mckA0dHH/a+8UcYi6 ew2ZgU2Gv6+q9dBksZmTtdUQa0nB/KGxJfyhZbcO5TpKmLix5DRpI83LUVJOJq6Y4XOQ D3PABR1bo7GyWdcjrSsOb88rlpci+jqu1wbz3/80bFIrtyEfPaeDtcnx4wdUuYJ7NdPf hqQXw3s2nGwbPGuio9sQFIhz+7d/XQf+JuEzbloi2jxMCbZO0Zsu2DIbVRz7H3pix0kt MoUADxl8gd2Njnnj9s6zb4rmJnQM6ziNCjcaAIy4C4QZN3IhFMeyWkrJl86GC/SpYNCy E3sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=T4zUXxck4HFOGvDPhR/G9ns6j0YeeWdcvt2M0bB8PGM=; b=GlTak0yxgk5uO15edidv1/sjsG1EvITZB25D4ptU+uYxSGZYCK4gKWp/7nrVOUHHoQ RscxMv5XA3N+VX3Us/6jatSNXUe3N6ZtC/yB9WZc+RTLo2swzKaVMdmoKR5jvNhdUceI putJn+TLZWGYAJ6YB9e7yi6O6LF9C5n4EpJeNWAPPjaCBZDMHIU6QUrnwdSvoOEgcBZ3 mM1Pekkak8KsjVhFrv0yxzVPiuMnLwUKoD3CW5PSNavHRd2b76Mozsq6isjx3I1U+P3v j3tnxUoRImunxRYkkD6fwKZLjR/+2gNYbJqJfXV1l7e5a1xSS//KwW2eoVP4gtq3iOlZ +J3w== X-Gm-Message-State: APjAAAXh5eNEZIL5CKizNJzVMxTxDnS05/E02qhBNIG/ZSUGGpJoDSvF AC0R9m4tJNSXASyFoNteEgqGQbGkNeA= X-Google-Smtp-Source: APXvYqxKyKEiEiTHLlVd5BHPZYFqOKzGJe1PZ/oDZACM7wq69pN+cwfNRGSIshGOYD78dT9+ARIfUw== X-Received: by 2002:ac2:44a5:: with SMTP id c5mr2771256lfm.4.1582734723238; Wed, 26 Feb 2020 08:32:03 -0800 (PST) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s2sm1264133lfd.67.2020.02.26.08.32.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Feb 2020 08:32:02 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 1/5] trace-cmd: Add new libtracefs API tracefs_instance_file_append() Date: Wed, 26 Feb 2020 18:31:55 +0200 Message-Id: <20200226163159.20232-2-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200226163159.20232-1-tz.stoyanov@gmail.com> References: <20200226163159.20232-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The existing tracefs_instance_file_write() API truncates the file before writing to it. The are use cases where the file must not be truncated. The new tracefs_instance_file_append() API appends to the end of the file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- include/tracefs/tracefs.h | 2 ++ lib/tracefs/tracefs-instance.c | 53 +++++++++++++++++++++++++--------- 2 files changed, 41 insertions(+), 14 deletions(-) diff --git a/include/tracefs/tracefs.h b/include/tracefs/tracefs.h index bc8bebcb..22f147a3 100644 --- a/include/tracefs/tracefs.h +++ b/include/tracefs/tracefs.h @@ -30,6 +30,8 @@ tracefs_instance_get_file(struct tracefs_instance *instance, const char *file); char *tracefs_instance_get_dir(struct tracefs_instance *instance); int tracefs_instance_file_write(struct tracefs_instance *instance, const char *file, const char *str); +int tracefs_instance_file_append(struct tracefs_instance *instance, + const char *file, const char *str); char *tracefs_instance_file_read(struct tracefs_instance *instance, char *file, int *psize); diff --git a/lib/tracefs/tracefs-instance.c b/lib/tracefs/tracefs-instance.c index 67123e7c..cc4bc1b4 100644 --- a/lib/tracefs/tracefs-instance.c +++ b/lib/tracefs/tracefs-instance.c @@ -176,12 +176,16 @@ char *tracefs_instance_get_name(struct tracefs_instance *instance) return NULL; } -static int write_file(const char *file, const char *str) +static int write_file(const char *file, const char *str, bool appned) { int ret; int fd; - fd = open(file, O_WRONLY | O_TRUNC); + if (appned) + fd = open(file, O_WRONLY | O_APPEND); + else + fd = open(file, O_WRONLY | O_TRUNC); + if (fd < 0) { warning("Failed to open '%s'", file); return -1; @@ -191,17 +195,8 @@ static int write_file(const char *file, const char *str) return ret; } - -/** - * tracefs_instance_file_write - Write in trace file of specific instance. - * @instance: ftrace instance, can be NULL for the top instance - * @file: name of the file - * @str: nul terminated string, that will be written in the file. - * - * Returns the number of written bytes, or -1 in case of an error - */ -int tracefs_instance_file_write(struct tracefs_instance *instance, - const char *file, const char *str) +static int instance_file_write(struct tracefs_instance *instance, + const char *file, const char *str, bool appned) { struct stat st; char *path; @@ -212,12 +207,42 @@ int tracefs_instance_file_write(struct tracefs_instance *instance, return -1; ret = stat(path, &st); if (ret == 0) - ret = write_file(path, str); + ret = write_file(path, str, appned); tracefs_put_tracing_file(path); return ret; } +/** + * tracefs_instance_file_write - Write in trace file of specific instance. + * @instance: ftrace instance, can be NULL for the top instance + * @file: name of the file + * @str: nul terminated string, that will be written in the file. + * + * Returns the number of written bytes, or -1 in case of an error. + * The content of the file is replaced with the new one. + * + */ +int tracefs_instance_file_write(struct tracefs_instance *instance, + const char *file, const char *str) +{ + return instance_file_write(instance, file, str, false); +} + +/** + * tracefs_instance_file_append - Append to a trace file of specific instance. + * @instance: ftrace instance, can be NULL for the top instance + * @file: name of the file + * @str: nul terminated string, that will be written in the file. + * + * Returns the number of written bytes, or -1 in case of an error. + */ +int tracefs_instance_file_append(struct tracefs_instance *instance, + const char *file, const char *str) +{ + return instance_file_write(instance, file, str, true); +} + /** * tracefs_instance_file_read - Read from a trace file of specific instance. * @instance: ftrace instance, can be NULL for the top instance From patchwork Wed Feb 26 16:31:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 11406763 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A0A6D930 for ; Wed, 26 Feb 2020 16:32:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7F24D206E6 for ; Wed, 26 Feb 2020 16:32:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="tBQt+9oF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727381AbgBZQcH (ORCPT ); Wed, 26 Feb 2020 11:32:07 -0500 Received: from mail-lj1-f194.google.com ([209.85.208.194]:40596 "EHLO mail-lj1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727168AbgBZQcH (ORCPT ); Wed, 26 Feb 2020 11:32:07 -0500 Received: by mail-lj1-f194.google.com with SMTP id n18so3804853ljo.7 for ; Wed, 26 Feb 2020 08:32:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hoA9+WGSbu8C2p4LQyj6mqpIZwKtFWUsPpsIzHeMVwk=; b=tBQt+9oFdg5KpbnOebh7AZjciabIvUZnxGY9dIneJYZgAfVvUF3aLbPbm2iakccho1 mruAxBEuD5juhtMqeZ+ve5UfJu2ZTBo3ChIN7WZlAYNfaJygON2JGvTblf2puwrMKjzn 1Op7J5Jg2kWUJ4YP2BwXYCTHz25vypxVJoNettJdHMS8sBlx9Dq7ZZ/O1irrkv/vh9Su gbQBnnu2byUbYWm6+mm2KTvPr83+syH4e2mkJr5fwBbuh9OvdFFbNmktIaqRlNDejzS0 uKtVKcecm1xcQ1QIEGVXk9oPEeIBKzYaF3SeU9QAYtwgXS3BcMDnstFY4u94F8rK+9Ra P1Fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hoA9+WGSbu8C2p4LQyj6mqpIZwKtFWUsPpsIzHeMVwk=; b=sgiEVdM3p1FwtDICB1orznFVWP2tTm8eR7p93tD8qvfy5bEXhucz0Z/Frf6VUuEYWV mfZkQX3S/cQ0axvqPAxY8nnKjUjQHQBAzZFeU4Dfrxq4UDo1EZUHaUGjFRzclWwhBJOd K6tJPw88KBZas0QYzt4xhxA/RvraVMH/JapYke/K0BEKVo7X861tvu/jNvj6re8zzMH2 xZFTmiV72qJtyoj3Czmr1+BwSB2eKO/5FHw42ryXwXoBWYElySuWDh7JxAclklwf7mDV qxAfZl/mcozcSW4lB5wsWz7D3eWDC3uC7kci3NCBbW93eOaS55Lke3knePGfNpiJzu7q RnRw== X-Gm-Message-State: APjAAAWJZavRv12riQKvDo4xfpeHNe2JtJ4G6l5kX0GXaAhoWgoACcEo MICiyK+HHSYyczTOOmvdey0= X-Google-Smtp-Source: ADFU+vvSDb6V5DWmZ2pskwy7PlqHuhtb/DiUvresgvpBeSf2jhHBYA4M/BP04YccKNykE1xeantkCw== X-Received: by 2002:a2e:3504:: with SMTP id z4mr3508332ljz.273.1582734724563; Wed, 26 Feb 2020 08:32:04 -0800 (PST) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s2sm1264133lfd.67.2020.02.26.08.32.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Feb 2020 08:32:03 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 2/5] trace-cmd: Unit test for tracefs_instance_file_append() API Date: Wed, 26 Feb 2020 18:31:56 +0200 Message-Id: <20200226163159.20232-3-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200226163159.20232-1-tz.stoyanov@gmail.com> References: <20200226163159.20232-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org A unit test for tracefs_instance_file_append() API is added. Signed-off-by: Tzvetomir Stoyanov (VMware) --- utest/tracefs-utest.c | 106 +++++++++++++++++++++++++++++++++--------- 1 file changed, 84 insertions(+), 22 deletions(-) diff --git a/utest/tracefs-utest.c b/utest/tracefs-utest.c index 3f57ecad..1dc15e1b 100644 --- a/utest/tracefs-utest.c +++ b/utest/tracefs-utest.c @@ -176,6 +176,86 @@ static void test_instance_file_read(struct tracefs_instance *inst, char *fname) #define ALL_TRACERS "available_tracers" #define CUR_TRACER "current_tracer" #define PER_CPU "per_cpu" +#define SYNTH_EVENTS "synthetic_events" +static void test_trace_file_read(struct tracefs_instance *instance) +{ + test_instance_file_read(NULL, ALL_TRACERS); + test_instance_file_read(instance, ALL_TRACERS); +} + +static void test_trace_file_write(struct tracefs_instance *instance) +{ + char *tracer; + char *file1; + char *file2; + int size; + int ret; + + file1 = tracefs_instance_file_read(instance, ALL_TRACERS, NULL); + CU_TEST(file1 != NULL); + tracer = strtok(file1, " "); + CU_TEST(tracer != NULL); + ret = tracefs_instance_file_write(instance, CUR_TRACER, tracer); + CU_TEST(ret == strlen(tracer)); + file2 = tracefs_instance_file_read(instance, CUR_TRACER, &size); + CU_TEST(file2 != NULL); + CU_TEST(size >= strlen(tracer)); + CU_TEST(strncmp(file2, tracer, strlen(tracer)) == 0); + free(file1); + free(file2); +} + +static bool check_file_line(struct tracefs_instance *instance, + char *fname, char *line, bool last) +{ + bool found = false; + char *buf = NULL; + char *l; + int n; + + buf = tracefs_instance_file_read(instance, fname, &n); + if (!buf) + return 0; + l = strtok(buf, "\n"); + while (l) { + if (strncmp(l, line, strlen(line)) == 0) { + if (last) { + if (strtok(NULL, "\n") == NULL) + found = true; + } else + found = true; + break; + } + l = strtok(NULL, "\n"); + } + free(buf); + return found; +} + +static void test_trace_file_append(struct tracefs_instance *instance) +{ + char *sevent1 = "first u64 start; u64 end; pid_t pid; u64 delta"; + char *sevent2 = "second u64 start; u64 end; pid_t pid; u64 delta"; + char buf[256]; + int ret; + + if (!tracefs_file_exist(NULL, SYNTH_EVENTS)) + return; + + CU_TEST(!check_file_line(NULL, SYNTH_EVENTS, sevent1, false)); + ret = tracefs_instance_file_append(NULL, SYNTH_EVENTS, sevent1); + CU_TEST(ret == strlen(sevent1)); + CU_TEST(check_file_line(NULL, SYNTH_EVENTS, sevent1, true)); + ret = tracefs_instance_file_append(NULL, SYNTH_EVENTS, sevent2); + CU_TEST(ret == strlen(sevent2)); + CU_TEST(check_file_line(NULL, SYNTH_EVENTS, sevent1, false)); + CU_TEST(check_file_line(NULL, SYNTH_EVENTS, sevent2, true)); + snprintf(buf, 256, "!%s", sevent1); + tracefs_instance_file_write(NULL, SYNTH_EVENTS, buf); + snprintf(buf, 256, "!%s", sevent2); + tracefs_instance_file_write(NULL, SYNTH_EVENTS, buf); +} + static void test_instance_file(void) { struct tracefs_instance *instance = NULL; @@ -186,11 +266,6 @@ static void test_instance_file(void) char *inst_dir; struct stat st; char *fname; - char *file1; - char *file2; - char *tracer; - int size; - int ret; tdir = tracefs_get_tracing_dir(); CU_TEST(tdir != NULL); @@ -232,26 +307,13 @@ static void test_instance_file(void) inst_file = tracefs_instance_get_file(instance, ALL_TRACERS); CU_TEST(inst_file != NULL); CU_TEST(strcmp(fname, inst_file) == 0); - - test_instance_file_read(NULL, ALL_TRACERS); - test_instance_file_read(instance, ALL_TRACERS); - - file1 = tracefs_instance_file_read(instance, ALL_TRACERS, NULL); - CU_TEST(file1 != NULL); - tracer = strtok(file1, " "); - CU_TEST(tracer != NULL); - ret = tracefs_instance_file_write(instance, CUR_TRACER, tracer); - CU_TEST(ret == strlen(tracer)); - file2 = tracefs_instance_file_read(instance, CUR_TRACER, &size); - CU_TEST(file2 != NULL); - CU_TEST(size >= strlen(tracer)); - CU_TEST(strncmp(file2, tracer, strlen(tracer)) == 0); - free(file1); - free(file2); - tracefs_put_tracing_file(inst_file); free(fname); + test_trace_file_read(instance); + test_trace_file_write(instance); + test_trace_file_append(instance); + CU_TEST(tracefs_file_exist(NULL, (char *)name) == false); CU_TEST(tracefs_dir_exist(NULL, (char *)name) == false); CU_TEST(tracefs_file_exist(instance, (char *)name) == false); From patchwork Wed Feb 26 16:31:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 11406765 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 735A4930 for ; Wed, 26 Feb 2020 16:32:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5339724680 for ; Wed, 26 Feb 2020 16:32:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AeIz+1yB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726905AbgBZQcI (ORCPT ); Wed, 26 Feb 2020 11:32:08 -0500 Received: from mail-lj1-f194.google.com ([209.85.208.194]:42173 "EHLO mail-lj1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727173AbgBZQcH (ORCPT ); Wed, 26 Feb 2020 11:32:07 -0500 Received: by mail-lj1-f194.google.com with SMTP id d10so3802525ljl.9 for ; Wed, 26 Feb 2020 08:32:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ytAs4N8eHHrCE5vlEUflW+GDRTfBuxWUc/so7AGAKZ0=; b=AeIz+1yB8F4MI/hr6SmTV7dcLgtKqj8QA/+ZmmrTJtgwqnBjIA88WyQvJhkA5i/Qjj P94gpc+OxxU+l6mAm3lZZoVApJqtMg0hsMiBtzjJxGi3x3IJ0tzL0X93GEGSDF2xRbUA eS/IKdXpwvXcDAsigvkMd1SmODkusn/3tlCkKqw27qVoJVehrf2LoYLnWhZ+DIMYA7PR SiU5lf5QjaZHA/4HUTkjeBphFbY1vaGtFp0QO4KfaN6tmKyfYJ7aGMc8NPwxbr87t8Nk FerIqItw3kS5ohQ+FWeCUUZ2eYzrbMgPaMt9KP6ECgZX8WK2GeX4Y6DSLZ/F5vPmiI8n yVfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ytAs4N8eHHrCE5vlEUflW+GDRTfBuxWUc/so7AGAKZ0=; b=XEuXLDNFaYiNTt+hKgOUXhUp1+aQwslhKQYEIvzP8TcpvGtQI9LIfkBlM9sTYbfvZX JSa84KXv5AVhS9em9uSxrTowMJd+kNObkIF0AVV8+RcoWLk51zXlYKHBsYDqvKX+B9fc yAB8+qlUrmiC6+flae8slFvjCqvaJJqClBcpyu3GdV+IK/Doj/HAI0yHUN1n3YDLNRIf 4dfLDKpwwozVuub9GC2aEGJ6W+3wArpGZsAzjg9rDpnQzgMFF5JrDPt8gnBWrYYGW5/Y WOVs8cluMAVhS1HEHX9/rQFRY3eQk70+Vn99KV8nmcjvDuhjHyh/prQDf4Wxa5sbvqeE nvYw== X-Gm-Message-State: APjAAAXx2vocNniKTkhmLigy+WYBXBsK81nns+2+JajrmO2Ly3M0tRRT 42W+MsMlwMR6b3sXE76aJWCND3XLkgM= X-Google-Smtp-Source: APXvYqy2myAgGbE66fycqoKrjoXOIROQqZ5SIrtdyd4MzfJEvjNGrlirGLdc3+YrwMHQjmPMQP24rQ== X-Received: by 2002:a05:651c:204f:: with SMTP id t15mr3357761ljo.240.1582734725884; Wed, 26 Feb 2020 08:32:05 -0800 (PST) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s2sm1264133lfd.67.2020.02.26.08.32.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Feb 2020 08:32:05 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 3/5] trace-cmd: Add new libtraceevent flag to suppress parsing warnings Date: Wed, 26 Feb 2020 18:31:57 +0200 Message-Id: <20200226163159.20232-4-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200226163159.20232-1-tz.stoyanov@gmail.com> References: <20200226163159.20232-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When the event's format strings are parsed by tep_parse_format() API and there is a parsing error, a warning is printed. There are use cases when these errors are not important and should be suppressed. A new flag is added to enum tep_flag { ... TEP_NO_PARSING_WARNINGS ... } for such use cases. It can be set / cleared / checked using the existing libtraceevent APIs: void tep_set_flag(struct tep_handle *tep, int flag); void tep_clear_flag(struct tep_handle *tep, enum tep_flag flag); bool tep_test_flag(struct tep_handle *tep, enum tep_flag flag); Signed-off-by: Tzvetomir Stoyanov (VMware) --- include/traceevent/event-parse.h | 2 ++ lib/traceevent/event-parse.c | 6 ++++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/include/traceevent/event-parse.h b/include/traceevent/event-parse.h index 52bafa54..49e3d287 100644 --- a/include/traceevent/event-parse.h +++ b/include/traceevent/event-parse.h @@ -321,6 +321,8 @@ enum tep_flag { TEP_NSEC_OUTPUT = 1, /* output in NSECS */ TEP_DISABLE_SYS_PLUGINS = 1 << 1, TEP_DISABLE_PLUGINS = 1 << 2, + TEP_NO_PARSING_WARNINGS = 1 << 3, /* Disable warnings while parsing + event's format strings */ }; #define TEP_ERRORS \ diff --git a/lib/traceevent/event-parse.c b/lib/traceevent/event-parse.c index 064c100d..61cd7ea1 100644 --- a/lib/traceevent/event-parse.c +++ b/lib/traceevent/event-parse.c @@ -6222,6 +6222,7 @@ enum tep_errno __tep_parse_format(struct tep_event **eventp, struct tep_handle *tep, const char *buf, unsigned long size, const char *sys) { + int show_warning_state = show_warning; struct tep_event *event; int ret; @@ -6274,11 +6275,12 @@ enum tep_errno __tep_parse_format(struct tep_event **eventp, * If the event has an override, don't print warnings if the event * print format fails to parse. */ - if (tep && find_event_handle(tep, event)) + if (tep && (tep_test_flag(tep, TEP_NO_PARSING_WARNINGS) || + find_event_handle(tep, event))) show_warning = 0; ret = event_read_print(event); - show_warning = 1; + show_warning = show_warning_state; if (ret < 0) { ret = TEP_ERRNO__READ_PRINT_FAILED; From patchwork Wed Feb 26 16:31:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 11406767 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 71C131580 for ; Wed, 26 Feb 2020 16:32:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5109424670 for ; Wed, 26 Feb 2020 16:32:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="pHmokIL8" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727854AbgBZQcL (ORCPT ); Wed, 26 Feb 2020 11:32:11 -0500 Received: from mail-lj1-f193.google.com ([209.85.208.193]:33433 "EHLO mail-lj1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727173AbgBZQcK (ORCPT ); Wed, 26 Feb 2020 11:32:10 -0500 Received: by mail-lj1-f193.google.com with SMTP id y6so3854967lji.0 for ; Wed, 26 Feb 2020 08:32:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bAPoEhczHGp2R5b1dAXEMIuuV6OQAATrpRCgG0atUHY=; b=pHmokIL8VzhTWCm8o59/0ivtXBZwNy72T+rLNp/TIKOf7DOOTu2fH18Z5L73QQxO19 LIIos9YGKiDYB+IJUxPdPhQvvkBK9/xwuWOQOfAjp9gsiaa56s37NuE3tmacp+Rk2b22 RUCXDZNULvoW4xXJw1EARqMhtrrXw8lNH498yYewR1lqoHTPtQrVApcgc2wtf99o/Ysy NvDCADKi+ucLo3aWrI8dJTtTs5qrVNP4M72XMX7HLBBGvaX73sKfGMh++3I6QC/VLumf HRDhlRrH3bm1zbagWJRhJtC4z4ooL44uaOCQB5TN1cyuh6tyacyQlaXkJ7hdllcH7Gte ARYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bAPoEhczHGp2R5b1dAXEMIuuV6OQAATrpRCgG0atUHY=; b=sBB6Fs2NMKNCXROn3EOBTiWrk4xjWoFGB2E0MH+I0A0N4Hu2VfGwFHaY9H7Z57b8A3 KoormfaqjM6qPG+78M4wMLtlJgBq5ysjLZLBBvctcaHH+SX6us86JpCvXOzASqLvU7O1 SvFBILgzccKWtZnqX/monnSwoiZVVZQQettamBtebXfjHgv+/YE8k8s1RS94qkJhDxrD Tl9VI8kgGxcsbIp+MnaoVLM5EPkL2NC91WvG3CTpPKlUz7XbRpFx6dm/XEhsszLIscae ztyEMrIEpn7uQZfQr/3pmp1IpSsZEfB61bBz+JYt98TF56fa/HRK8RFrXd8Tep0kEv7d hXOA== X-Gm-Message-State: APjAAAXLW9n4d+QTi365g4RwmBLqHWqiktCv02bsyfMeb+BAOBC2c4oT I+WrNTnjbjx1opw7ht1H+u8= X-Google-Smtp-Source: ADFU+vtryBq6y1RUpdAdpkcE9k4zKklMJPMxp/BdvIoHpijj2iwcf9jO/R+i6w80g5FtTuZtfFAyow== X-Received: by 2002:a05:651c:1a2:: with SMTP id c2mr3670608ljn.79.1582734727435; Wed, 26 Feb 2020 08:32:07 -0800 (PST) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s2sm1264133lfd.67.2020.02.26.08.32.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Feb 2020 08:32:06 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 4/5] trace-cmd: Suppress parsing warnings in tracefs_local_events() API Date: Wed, 26 Feb 2020 18:31:58 +0200 Message-Id: <20200226163159.20232-5-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200226163159.20232-1-tz.stoyanov@gmail.com> References: <20200226163159.20232-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The tracefs_local_events() API allocates new tep handler and initializes it with the events from the local system. However, at that moment no traceevent plugins are loaded. Part of events loading is parsing the event's format file, but successful parsing of print format string from this file depends of traceevent plugins. That's why a lot of parsing could be printed. To suppress the warnings, the TEP_NO_PARSING_WARNINGS flag is set on the newly allocated tep handler. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/tracefs/tracefs-events.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/tracefs/tracefs-events.c b/lib/tracefs/tracefs-events.c index 8e825f50..0875e665 100644 --- a/lib/tracefs/tracefs-events.c +++ b/lib/tracefs/tracefs-events.c @@ -590,7 +590,7 @@ struct tep_handle *tracefs_local_events_system(const char *tracing_dir, tep = tep_alloc(); if (!tep) return NULL; - + tep_set_flag(tep, TEP_NO_PARSING_WARNINGS); if (fill_local_events_system(tracing_dir, tep, sys_names, NULL)) { tep_free(tep); tep = NULL; From patchwork Wed Feb 26 16:31:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 11406769 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D3D0F14B4 for ; Wed, 26 Feb 2020 16:32:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7973824670 for ; Wed, 26 Feb 2020 16:32:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fiqft5ed" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727173AbgBZQcP (ORCPT ); Wed, 26 Feb 2020 11:32:15 -0500 Received: from mail-lf1-f67.google.com ([209.85.167.67]:43486 "EHLO mail-lf1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727168AbgBZQcO (ORCPT ); Wed, 26 Feb 2020 11:32:14 -0500 Received: by mail-lf1-f67.google.com with SMTP id s23so2441285lfs.10 for ; Wed, 26 Feb 2020 08:32:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FBvjs0PdLIWnOMASdOgKGb5Ya9lxKpGmHonJlnw2Sdw=; b=fiqft5edsf3aJiej4Nd1O+G9Sgxa+8mkiOzt2VYOflOZow+cqckWbWN6BQHgzaYwqD Dultyv1Ysvqv1qAHy3ffXUFByOG/iy928AQpHt9uSbzmnM8DKHtTuDAnYYdK5MvIWltA RJHKzZmksfZ2wUPXT1CqKtev85tHuD37bL3VkI0idEn2RceCQKvEi7KB6W4QQlESU0s3 Uo28dq7C3W32OoSd+sx8wC1BlCeJIkIgMtoYyVY41HigjgkTfK2A/6UP/rDcna+/cXKG PqtthtWTIZOhiU+qERrBLvf3+BGO+yTS6aYTGvtcwKqojaa7+Erbvx08lz2ksOqgYvmd oGhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FBvjs0PdLIWnOMASdOgKGb5Ya9lxKpGmHonJlnw2Sdw=; b=ISxEqrBoRt0pOSJ2bi4YUYIvGQsRfZhA8CniiqR1QL3K5hQEh8fDxkQzyg9AXGA9Lv XEb9rJ3o27s/6SpLJPFvnpuVt24355Ia8n0e+zt1idUQKiA87G9979FLSpKO6KAcJ4VC Mi2szbUHgZfex5L07Sh+rj1U6eBJ9dLdzimCmp6Zg3Fv2kvX7jJOAsCkB0aJuAjKr7zZ mI3alTSAoA/BRl5BU8MHNZJvsN/Ny9fX5+vFiHAmOQIap3uFnTLvTsAj/YYO/x6jS6Bg XxyEDlS1cth0YjhNuLxdxiNSx0vUL59WlHFAuf2EXqFUlUxbqlauU4KN4IZkaK2gJRxu Y0gQ== X-Gm-Message-State: APjAAAUjC3PkAc9kGHBHnewMmcwPeCRXTI4Evo52+VG+XacRALIxJRfw nRwOo7BESMOBZU/cpLMWiQg= X-Google-Smtp-Source: APXvYqyt31fh3PgXnwYg3WyZ0Y0P65ihk4GN5KJVC4k9pExjjEKiD4uw8J92E6F4kP8Vt0kuvVXTdA== X-Received: by 2002:a19:ca15:: with SMTP id a21mr2934372lfg.67.1582734728963; Wed, 26 Feb 2020 08:32:08 -0800 (PST) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s2sm1264133lfd.67.2020.02.26.08.32.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Feb 2020 08:32:08 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 5/5] trace-cmd: Add "--sql" option to trace-cmd start and record sub commands Date: Wed, 26 Feb 2020 18:31:59 +0200 Message-Id: <20200226163159.20232-6-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200226163159.20232-1-tz.stoyanov@gmail.com> References: <20200226163159.20232-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Configuration of complex ftrace histograms could be hard, using the existing ftrace interface. The "--sql" option is implemented to simplify this task. It uses a sql-like syntax to define the histograms and generates necessary ftrace syntetic events and triggers. Each ftrace event represents a sql tible, while the fields of this event are rows in the table. Additional option "-V" is added to trace-cmd start and report sub commands, which can be used to dump the auto generated ftrace synthetic events and triggers configuration. Signed-off-by: Tzvetomir Stoyanov (VMware) --- Documentation/trace-cmd-record.1.txt | 37 + Makefile | 1 + tracecmd/Makefile | 12 +- tracecmd/include/trace-local.h | 4 + tracecmd/sqlparser/Makefile | 52 + tracecmd/sqlparser/sqlhist-api.h | 29 + tracecmd/sqlparser/sqlhist-parse.c | 1526 ++++++++++++++++++++++++++ tracecmd/sqlparser/sqlhist.h | 34 + tracecmd/sqlparser/sqlhist.l | 59 + tracecmd/sqlparser/sqlhist.y | 175 +++ tracecmd/trace-record.c | 142 ++- tracecmd/trace-usage.c | 2 + 12 files changed, 2070 insertions(+), 3 deletions(-) create mode 100644 tracecmd/sqlparser/Makefile create mode 100644 tracecmd/sqlparser/sqlhist-api.h create mode 100644 tracecmd/sqlparser/sqlhist-parse.c create mode 100644 tracecmd/sqlparser/sqlhist.h create mode 100644 tracecmd/sqlparser/sqlhist.l create mode 100644 tracecmd/sqlparser/sqlhist.y diff --git a/Documentation/trace-cmd-record.1.txt b/Documentation/trace-cmd-record.1.txt index 0d75e43d..fd482e1c 100644 --- a/Documentation/trace-cmd-record.1.txt +++ b/Documentation/trace-cmd-record.1.txt @@ -343,6 +343,43 @@ OPTIONS executed will not be changed. This is useful if you want to monitor the output of the command being executed, but not see the output from trace-cmd. +*--sql*:: + Generate necessary ftrace syntetic events and triggers, based on the given + sql query. A simplified sql syntax can be used to describe ftrace histograms. + Each ftrace event represents a sql tible, while the fields of this event are + rows in the table. A sql aliases can be used to define synthetic events and + simplify the sql statement. + + Example 1: + --sql 'select pid as key1, common_pid from sched_waking' + Configures a histogram, where the key is the pid from sched/sched_waking + event and the value is the common_pid filed from the same event. + The auto generated ftrace config, written in sched/sched_waking/trigger file + is: + hist:keys=pid:values=common_pid + + Example 2: + --sql '(select start.common_timestamp as start_time, end.common_timestamp + as end_time, start.pid, (end_time - start_time) as delta + from sched_waking as start join sched_switch as end on + start.pid = end.next_pid) as first' + A more complex histogram example, using a synthetic event. The synthetic + event "first" is configured in synthetic_events file: + first u64 start_time u64 end_time pid_t pid u64 delta + After that, a histogram trigger is configured in sched/sched_waking/trigger + file: + hist:keys=pid:start_time=common_timestamp,__arg0__=pid + A second histogram trigger is configured in sched/sched_switch/trigger file: + hist:keys=next_pid:__arg1__=$start_time,__arg2__=$__arg0__, + delta=common_timestamp-$__arg1__: + onmatch(sched.sched_waking).trace(first,$__arg1__,common_timestamp,$__arg2__,$delta) + The "onmatch" histogram action is used to trigger the synthetic event "first" + with interested parameters, when there is a match between next_pid and pid + fields from both events. + +*-V*:: + Dump ftrace configuration, auto generated by the *--sql* option. + EXAMPLES -------- diff --git a/Makefile b/Makefile index a3facaa9..1928f1cf 100644 --- a/Makefile +++ b/Makefile @@ -224,6 +224,7 @@ INCLUDES += -I$(src)/lib/traceevent/include INCLUDES += -I$(src)/lib/trace-cmd/include INCLUDES += -I$(src)/lib/tracefs/include INCLUDES += -I$(src)/tracecmd/include +INCLUDES += -I$(src)/tracecmd/sqlparser INCLUDES += -I$(obj)/tracecmd/include include $(src)/features.mk diff --git a/tracecmd/Makefile b/tracecmd/Makefile index 0976341c..678dab31 100644 --- a/tracecmd/Makefile +++ b/tracecmd/Makefile @@ -5,6 +5,9 @@ EXTRAVERSION := $(TC_EXTRAVERSION) bdir:=$(obj)/tracecmd +sqldir:=$(bdir)/sqlparser +LIBSQLPARSE = $(sqldir)/libsqlparse.a + TC_VERSION := $(bdir)/include/tc_version.h TARGETS = $(bdir)/trace-cmd $(TC_VERSION) @@ -42,6 +45,8 @@ ALL_OBJS := $(TRACE_CMD_OBJS:%.o=$(bdir)/%.o) all_objs := $(sort $(ALL_OBJS)) all_deps := $(all_objs:$(bdir)/%.o=$(bdir)/.%.d) +CFLAGS += -I$(sqldir) + CONFIG_INCLUDES = CONFIG_LIBS = -lrt CONFIG_FLAGS = @@ -63,7 +68,8 @@ $(all_objs): | $(bdir) $(bdir)/trace-cmd: $(ALL_OBJS) $(Q)$(do_app_build) -$(bdir)/trace-cmd: $(LIBTRACECMD_STATIC) $(LIBTRACEEVENT_STATIC) $(LIBTRACEFS_STATIC) +$(bdir)/trace-cmd: $(LIBTRACECMD_STATIC) $(LIBTRACEEVENT_STATIC)\ + $(LIBTRACEFS_STATIC) $(LIBSQLPARSE) $(bdir)/%.o: %.c $(Q)$(call do_compile) @@ -81,8 +87,12 @@ ifneq ($(dep_includes),) include $(dep_includes) endif +$(LIBSQLPARSE): force + $(Q)$(MAKE) -C $(sqldir) $@ + clean: $(RM) $(bdir)/*.a $(bdir)/*.so $(bdir)/*.o $(bdir)/.*.d $(TARGETS) + $(Q)$(MAKE) -C $(sqldir) $@ force: .PHONY: clean diff --git a/tracecmd/include/trace-local.h b/tracecmd/include/trace-local.h index 7a0804e2..108b9a0a 100644 --- a/tracecmd/include/trace-local.h +++ b/tracecmd/include/trace-local.h @@ -11,6 +11,7 @@ #include "trace-cmd.h" #include "event-utils.h" +#include "sqlhist-api.h" #define TRACE_AGENT_DEFAULT_PORT 823 @@ -213,6 +214,9 @@ struct buffer_instance { char *max_graph_depth; + struct sqlhist_trigger *auto_triggers; + struct sqlhist_synth_event *synth_events; + int flags; int tracing_on_init_val; int tracing_on_fd; diff --git a/tracecmd/sqlparser/Makefile b/tracecmd/sqlparser/Makefile new file mode 100644 index 00000000..f648b183 --- /dev/null +++ b/tracecmd/sqlparser/Makefile @@ -0,0 +1,52 @@ + +include $(src)/scripts/utils.mk + +bdir:=$(obj)/tracecmd/sqlparser + +TARGET = $(bdir)/libsqlparse.a + +all: $(TARGET) + +FLEX ?= flex +BISON ?= bison + +OBJS = +OBJS += sqlhist-parse.o +OBJS += lex.yy.o +OBJS += sqlhist.tab.o + +OBJS := $(OBJS:%.o=$(bdir)/%.o) +DEPS := $(OBJS:$(bdir)/%.o=$(bdir)/.%.d) + +$(bdir): + @mkdir -p $(bdir) + +$(OBJS): | $(bdir) +$(DEPS): | $(bdir) + +$(TARGET): $(OBJS) + $(call do_build_static_lib) + +sqlhist-parse.c: lex.yy.c sqlhist.tab.c + +$(bdir)/%.o: %.c + $(Q)$(call do_compile) + +$(DEPS): $(bdir)/.%.d: %.c + $(Q)$(CC) -M $(CPPFLAGS) $(CFLAGS) $< > $@ + +$(OBJS): $(bdir)/%.o : $(bdir)/.%.d + + +lex.yy.c: sqlhist.l + $(Q)$(FLEX) $^ + +sqlhist.tab.c: sqlhist.y + $(Q)$(BISON) -d -o $@ $^ + +clean: + $(RM) $(bdir)/*.o $(bdir)/*.a $(bdir)/.*.d + $(RM) lex.yy.c *~ sqlhist.output sqlhist.tab.[ch] + +PHONY += force +force: \ No newline at end of file diff --git a/tracecmd/sqlparser/sqlhist-api.h b/tracecmd/sqlparser/sqlhist-api.h new file mode 100644 index 00000000..bb47990c --- /dev/null +++ b/tracecmd/sqlparser/sqlhist-api.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: LGPL-2.0 */ +/* + * Copyright (C) 2020 VMware Inc, Steven Rostedt + * + */ +#ifndef __SQLHIST_API_H +#define __SQLHIST_API_H + +struct sqlhist_trigger { + struct sqlhist_trigger *next; + char *system; + char *event; + char *config; +}; + +struct sqlhist_synth_event { + struct sqlhist_synth_event *next; + char *name; + char *config; +}; + +int sqlhist_parse(struct tep_handle *tep, char *buff, + size_t buff_size, struct sqlhist_trigger **triggers, + struct sqlhist_synth_event **synt_events); + +void sqlhist_trigger_free(struct sqlhist_trigger *list); +void sqlhist_synth_event_free(struct sqlhist_synth_event *list); + +#endif /* __SQLHIST_API_H */ diff --git a/tracecmd/sqlparser/sqlhist-parse.c b/tracecmd/sqlparser/sqlhist-parse.c new file mode 100644 index 00000000..0d284b33 --- /dev/null +++ b/tracecmd/sqlparser/sqlhist-parse.c @@ -0,0 +1,1526 @@ +// SPDX-License-Identifier: LGPL-2.0 +/* + * Copyright (C) 2020 VMware Inc, Steven Rostedt + * + */ +#include +#include +#include +#include +#include +#include +#include + +#include "trace-seq.h" +#include "trace-local.h" +#include "sqlhist.h" + +#define SQL_PARSE_DEBUG 0 + +/* + * Ideally, we will conevert: + * + * trace-cmd start \ + * --sql '(select start.common_timestamp as start_time, + * end.common_timestamp as end_time, start.pid, + * (start_time - end_time) as delta + * from sched_waking as start + * join sched_switch as end + * on start.pid = end.next_pid) as first' + * + * to this: + * + * # echo 'first u64 start_time u64 end_time pid_t pid u64 delta' >> synthetic_events + * # echo 'hist:keys=pid:start=common_timestamp' > + * events/sched/sched_waking/trigger + * # echo 'hist:keys=next_pid:start2=$start,delta=common_timestamp-$start:onmatch(sched.sched_waking).trace(first,$start2,common_timestamp,next_pid,$delta)' > + * events/sched/sched_switch/trigger + */ + +#define HASH_BITS 10 + +struct str_hash { + struct str_hash *next; + char *str; +}; + +static struct str_hash *str_hash[1 << HASH_BITS]; + +enum label_type { + LABEL_STRING, + LABEL_EXPR, +}; + +struct label_map { + struct label_map *next; + enum label_type type; + char *label; + void *value; +}; + +struct match_map { + struct match_map *next; + const char *A; + const char *B; +}; + +struct selection { + struct selection *next; + const char *name; + void *item; +}; + +enum expr_type { + EXPR_FIELD, + EXPR_PLUS, + EXPR_MINUS, + EXPR_MULT, + EXPR_DIVID, +}; + +struct sql_table; + +struct expression { + enum expr_type type; + void *A; + void *B; + const char *name; + struct sql_table *table; +}; + +struct table_map { + struct table_map *next; + char *name; + struct sql_table *table; +}; + +struct sql_table { + char *name; + struct sql_table *parent; + struct sql_table *child; + struct label_map *labels; + struct match_map *matches; + struct table_map *tables; + struct selection *selections; + struct selection **next_selection; + const char *from; + const char *to; +}; + +static struct sql_table *curr_table; +static struct sql_table *top_table; +static struct table_map *table_list; + +static int parse_error; +static struct tep_handle *tep_glob; + +static int print_type(struct trace_seq *s, struct expression *e); + +static int no_table(void) +{ + if (curr_table) + return 0; +#if SQL_PARSE_DEBUG + { + static int once; + + if (!once++) + printf("No table?\n"); + } +#endif + return 1; +} + +void table_start(void) +{ + struct sql_table *table; + + table = calloc(1, sizeof(*table)); + if (!table) { + parse_error++; + return; + } + + table->next_selection = &table->selections; + + table->parent = curr_table; + if (curr_table) + curr_table->child = table; + else + top_table = table; + + curr_table = table; +} + +void add_from(void *item) +{ + curr_table->from = show_expr(item); +} + +void add_to(void *item) +{ + curr_table->to = show_expr(item); +} + +static void add_table(const char *label) +{ + struct table_map *tmap; + + if (no_table()) + return; + + tmap = malloc(sizeof(*tmap)); + if (!tmap) { + parse_error++; + return; + } + + tmap->table = curr_table; + tmap->name = store_str(label); + + tmap->next = table_list; + table_list = tmap; +} + +static void clean_table(struct sql_table *table) +{ + struct selection *sdel; + struct label_map *ldel; + struct match_map *mdel; + + while (table->selections) { + sdel = table->selections; + table->selections = table->selections->next; + free(sdel); + } + + while (table->labels) { + ldel = table->labels; + table->labels = table->labels->next; + if (ldel->type == LABEL_EXPR) + free(ldel->value); + free(ldel); + } + + while (table->matches) { + mdel = table->matches; + table->matches = table->matches->next; + free(mdel); + } + free(table); +} + +static void clean_tables(void) +{ + struct table_map *del; + + while (table_list) { + del = table_list; + table_list = table_list->next; + clean_table(del->table); + free(del); + } + +} + +static struct sql_table *find_table(const char *name) +{ + struct table_map *tmap; + + for (tmap = table_list; tmap; tmap = tmap->next) + if (strcmp(tmap->name, name) == 0) + return tmap->table; + return NULL; +} + +void table_end(const char *name) +{ + static int anony_cnt; + char *tname; + + if (!name) + tname = store_printf("Anonymous%d", anony_cnt++); + else + tname = store_str(name); + + add_table(tname); + + curr_table->name = tname; + curr_table = curr_table->parent; + +} + +void from_table_end(const char *name) +{ + if (curr_table->parent) + curr_table->parent->from = store_str(name); + table_end(name); +} + +/* Just a histogram table */ +void simple_table_end(void) +{ +} + +static void insert_label(const char *label, void *val, enum label_type type) +{ + struct label_map *lmap; + struct sql_table *table = curr_table; + + if (!table) + table = top_table; + + if (!table) { + no_table(); + return; + } + + lmap = malloc(sizeof(*lmap)); + if (!lmap) { + parse_error++; + return; + } + lmap->label = store_str(label); + lmap->value = val; + lmap->type = type; + + lmap->next = table->labels; + table->labels = lmap; +} + +void add_label(const char *label, const char *val) +{ + insert_label(label, store_str(val), LABEL_STRING); +} + +void add_match(const char *A, const char *B) +{ + struct match_map *map; + + if (no_table()) + return; + + map = malloc(sizeof(*map)); + if (!map) { + parse_error++; + return; + } + map->A = store_str(A); + map->B = store_str(B); + + map->next = curr_table->matches; + curr_table->matches = map; +} + +void add_selection(void *item) +{ + struct selection *selection; + struct expression *e = item; + + if (no_table()) + return; + + selection = malloc(sizeof(*selection)); + if (!selection) { + parse_error++; + return; + } + selection->item = e; + selection->name = e->name; + selection->next = NULL; + *curr_table->next_selection = selection; + curr_table->next_selection = &selection->next; +} + +static char *expr_op_connect(void *A, void *B, char *op, + const char *(*show)(void *A)) +{ + struct expression *eA = A; + struct expression *eB = B; + char *a = NULL, *b = NULL; + char *str = NULL; + char *ret = NULL; + int r; + + if (eA->name) { + r = asprintf(&a, "%s AS %s", show(A), eA->name); + if (r < 0) + goto out; + } + + if (eB->name) { + r = asprintf(&b, "%s AS %s", show(B), eB->name); + if (r < 0) + goto out; + } + + r = asprintf(&str, "(%s %s %s)", + a ? a : show(A), op, b ? b : show(B)); + if (r < 0) + goto out; + + ret = store_str(str); +out: + free(a); + free(b); + free(str); + + if (!ret) + parse_error++; + + return ret; +} + +static const char *show_raw_expr(void *e); +static const char *resolve(struct sql_table *table, const char *label); + +static const char *expand(const char *str) +{ + char *exp = strdup(str); + const char *label; + const char *ret; + char *p; + + if (!exp) { + parse_error++; + return NULL; + } + + p = strstr(exp, "."); + if (p) { + *p = 0; + label = resolve(curr_table, exp); + ret = store_printf("%s.%s", label, p+1); + *p = '.'; + } else { + ret = resolve(curr_table, str); + } + free(exp); + return ret; +} + +static const char *__show_expr(struct expression *e, bool eval) +{ + const char *(*show)(void *e); + char *ret; + + if (eval) + show = show_raw_expr; + else + show = show_expr; + + switch (e->type) { + case EXPR_FIELD: + ret = e->A; + if (eval) + return expand(e->A); + break; + case EXPR_PLUS: + ret = expr_op_connect(e->A, e->B, "+", show); + break; + case EXPR_MINUS: + ret = expr_op_connect(e->A, e->B, "-", show); + break; + case EXPR_MULT: + ret = expr_op_connect(e->A, e->B, "*", show); + break; + case EXPR_DIVID: + ret = expr_op_connect(e->A, e->B, "/", show); + break; + } + return ret; +} + +static const char *show_raw_expr(void *e) +{ + return __show_expr(e, true); +} + +const char *show_expr(void *expr) +{ + struct expression *e = expr; + + if (e->name) + return e->name; + + return __show_expr(expr, false); +} + +static struct expression *create_expression(void *A, void *B, enum expr_type type) +{ + struct expression *e; + + e = calloc(sizeof(*e), 1); + if (!e) { + parse_error++; + return NULL; + } + e->A = A; + e->B = B; + e->type = type; + e->table = curr_table; + + return e; +} + +void *add_plus(void *A, void *B) +{ + return create_expression(A, B, EXPR_PLUS); +} + +void *add_minus(void *A, void *B) +{ + return create_expression(A, B, EXPR_MINUS); +} + +void *add_mult(void *A, void *B) +{ + return create_expression(A, B, EXPR_MULT); +} + +void *add_divid(void *A, void *B) +{ + return create_expression(A, B, EXPR_DIVID); +} + +void add_expr(const char *label, void *A) +{ + struct expression *e = A; + + insert_label(label, A, LABEL_EXPR); + e->name = store_str(label); +} + +void *add_field(const char *field, const char *label) +{ + struct expression *e; + + e = create_expression(store_str(field), NULL, EXPR_FIELD); + if (label) + add_expr(label, e); + + return e; +} + +static inline unsigned int quick_hash(const char *str) +{ + unsigned int val = 0; + int len = strlen(str); + + for (; len >= 4; str += 4, len -= 4) { + val += str[0]; + val += str[1] << 8; + val += str[2] << 16; + val += str[3] << 24; + } + for (; len > 0; str++, len--) + val += str[0] << (len * 8); + + val *= 2654435761; + + return val & ((1 << HASH_BITS) - 1); +} + + +static struct str_hash *find_string(const char *str) +{ + unsigned int key = quick_hash(str); + struct str_hash *hash; + + for (hash = str_hash[key]; hash; hash = hash->next) { + if (!strcmp(hash->str, str)) + return hash; + } + return NULL; +} + +/* + * If @str is found, then return the hash string. + * This lets store_str() know to free str. + */ +static char **add_hash(const char *str) +{ + struct str_hash *hash; + unsigned int key; + + hash = find_string(str); + if (hash) + return &hash->str; + + hash = malloc(sizeof(*hash)); + if (!hash) { + parse_error++; + return NULL; + } + key = quick_hash(str); + hash->next = str_hash[key]; + str_hash[key] = hash; + hash->str = NULL; + return &hash->str; +} + +static void clean_str_hash(void) +{ + int max = 1 << HASH_BITS; + struct str_hash *del; + int i; + + for (i = 0; i < max; i++) { + if (!str_hash[i]) + continue; + while (str_hash[i]) { + del = str_hash[i]; + str_hash[i] = del->next; + free(del->str); + free(del); + } + } +} + +char *store_str(const char *str) +{ + char **pstr = add_hash(str); + + if (!(*pstr)) + *pstr = strdup(str); + + return *pstr; +} + +char *store_printf(const char *fmt, ...) +{ + va_list ap; + char **pstr; + char *str; + int ret; + + va_start(ap, fmt); + ret = vasprintf(&str, fmt, ap); + va_end(ap); + + if (!ret) { + parse_error++; + return NULL; + } + + pstr = add_hash(str); + if (*pstr) + free(str); + else + *pstr = str; + + return *pstr; +} + +#if SQL_PARSE_DEBUG +static void dump_label_map(struct sql_table *table) +{ + struct label_map *lmap; + struct table_map *tmap; + + if (table->labels) + printf("%s Labels:\n", table->name); + for (lmap = table->labels; lmap; lmap = lmap->next) { + switch (lmap->type) { + case LABEL_STRING: + printf(" %s = %s\n", + lmap->label, (char *)lmap->value); + break; + case LABEL_EXPR: + printf(" %s = (%s)\n", lmap->label, + show_raw_expr(lmap->value)); + break; + } + } + if (table->tables) + printf("%s Tables:\n", table->name); + for (tmap = table->tables; tmap; tmap = tmap->next) + printf(" %s = Table %s\n", tmap->name, tmap->table->name); +} + +static void dump_match_map(struct sql_table *table) +{ + struct match_map *map; + + if (table->matches) + printf("%s Matches:\n", table->name); + for (map = table->matches; map; map = map->next) + printf(" %s = %s\n", map->A, map->B); + +} + +static void dump_selections(struct sql_table *table) +{ + struct selection *sel; + struct trace_seq s; + + trace_seq_init(&s); + if (table->selections) + trace_seq_printf(&s, "%s Selections:\n", table->name); + for (sel = table->selections; sel; sel = sel->next) { + trace_seq_printf(&s, " %s::\n\t", sel->name); + show_raw_expr(sel->item); + print_type(&s, sel->item); + trace_seq_printf(&s, "\n --\n"); + } + trace_seq_do_printf(&s); + trace_seq_destroy(&s); +} + +static void dump_table(struct sql_table *table) +{ + struct sql_table *save_curr = curr_table; + + if (!table) + return; + + printf("\nTable: %s\n\tFrom: %s\n", table->name, table->from); + + dump_table(find_table(table->from)); + + curr_table = table; + + dump_label_map(table); + dump_match_map(table); + dump_selections(table); + + curr_table = save_curr; + + printf("\n\tTo: %s\n", table->to); + dump_table(find_table(table->to)); +} + + +static void dump_tables(void) +{ + static int debug = 1; + + if (!debug) + return; + + dump_table(top_table); +} +#endif + +static const char *event_match(const char *event, const char *val, int len) +{ + if (strncmp(event, val, len) == 0 && val[len] == '.') + return val + len + 1; + + return NULL; +} + +static char *make_dynamic_arg(void) +{ + static int arg_cnt; + + return store_printf("__arg%d__", arg_cnt++); +} + +static struct tep_event *find_event(struct tep_handle *tep, const char *name) +{ + static struct tep_event stub_event = { + .system = "(system)", + }; + + if (tep) + return tep_find_event_by_name(tep, NULL, name); + + return &stub_event; +} + +static struct tep_format_field *find_field(struct tep_event *event, char *name) +{ + static struct tep_format_field stub_field = { + .type = "(unknown)", + }; + + if (tep_glob) + return tep_find_any_field(event, name); + + return &stub_field; +} + +static int print_type(struct trace_seq *s, struct expression *e) +{ + struct tep_format_field *field; + struct tep_event *event; + char *name = NULL; + char *tok; + int ret = -1; + + while (e && e->type != EXPR_FIELD) + e = e->A; + + if (!e) + return -1; + + name = strdup(show_raw_expr(e)); + if (!name) + return -1; + + tok = strtok(name, "."); + + event = find_event(tep_glob, tok); + if (!event) { + tok = strtok(NULL, "."); + if (!tok) + goto out; + event = find_event(tep_glob, tok); + } + + tok = strtok(NULL, "."); + if (!tok || !event) + goto out; + + if (strcmp(tok, "common_timestamp") == 0) { + trace_seq_printf(s, " u64 "); + } else { + field = find_field(event, tok); + if (field) + trace_seq_printf(s, " %s ", field->type); + else + goto out; + } + + ret = 0; + +out: + free(name); + return ret; +} + +static int print_synthetic_field(struct trace_seq *s, struct sql_table *table, + struct selection *selection) +{ + struct expression *e = selection->item; + const char *name; + const char *actual; + const char *field; + const char *to; + int len; + int ret; + + ret = print_type(s, e); + if (ret < 0) + return ret; + + name = selection->name; + if (!name) + name = e->name; + if (name) { + trace_seq_printf(s, "%s", name); + return 0; + } + + to = resolve(table, table->to); + len = strlen(to); + + actual = show_raw_expr(e); + field = event_match(to, actual, len); + if (field) { + trace_seq_printf(s, "%s", field); + return 0; + } + + selection->name = make_dynamic_arg(); + e->name = selection->name; + + field = strstr(actual, "."); + if (field) { + /* Need to check for common_timestamp */ + trace_seq_printf(s, "%s", field + 1); + } else { + trace_seq_printf(s, "%s", e->name); + } + + return 0; +} + +static struct sqlhist_synth_event * +find_synthetic_events(struct sqlhist_synth_event *synt_events, char *name) +{ + while (synt_events) { + if (strcmp(synt_events->name, name)) + return synt_events; + synt_events = synt_events->next; + } + + return NULL; +} + +static int make_synthetic_events(struct sql_table *table, + struct sqlhist_synth_event **synt_events) +{ + int ret; + struct trace_seq str; + struct selection *selection; + struct sqlhist_synth_event *synt_event = NULL; + struct sql_table *save_curr = curr_table; + + if (!table || !table->to) + return 0; + + ret = make_synthetic_events(find_table(table->from), synt_events); + if (ret < 0) + return ret; + + if (find_synthetic_events(*synt_events, table->name)) + goto error; + + trace_seq_init(&str); + synt_event = calloc(1, sizeof(struct sqlhist_synth_event)); + if (!synt_event) + goto error; + synt_event->name = strdup(table->name); + if (!synt_event->name) + goto error; + + curr_table = table; + trace_seq_printf(&str, "%s ", table->name); + for (selection = table->selections; selection; selection = selection->next) { + if (print_synthetic_field(&str, table, selection) < 0) + goto error; + } + + synt_event->config = strdup(str.buffer); + if (!synt_event->config) + goto error; + + curr_table = save_curr; + trace_seq_destroy(&str); + + if (*synt_events) + (*synt_events)->next = synt_event; + else + *synt_events = synt_event; + + return make_synthetic_events(find_table(table->to), synt_events); + +error: + trace_seq_destroy(&str); + if (synt_event) { + free(synt_event->name); + free(synt_event->config); + free(synt_event); + } + + return -1; +} + +static const char *resolve(struct sql_table *table, const char *label) +{ + struct sql_table *save_curr = curr_table; + struct label_map *lmap; + struct expression *e; + + curr_table = table; + + for (lmap = table->labels; lmap; lmap = lmap->next) + if (strcmp(lmap->label, label) == 0) + break; + + if (lmap) { + switch (lmap->type) { + case LABEL_STRING: + label = (char *)lmap->value; + break; + case LABEL_EXPR: + e = lmap->value; + label = show_raw_expr(e); + break; + } + } + + curr_table = save_curr; + + return label; +} + +static int print_key(struct trace_seq *s, struct sql_table *table, + const char *event, const char *key1, const char *key2) +{ + int len = strlen(event); + const char *field; + + field = event_match(event, key1, len); + if (field) + trace_seq_printf(s, "%s", field); + + field = event_match(event, key2, len); + if (field) + trace_seq_printf(s, "%s", field); + + return 0; +} + +static int print_keys(struct trace_seq *s, + struct sql_table *table, const char *event) +{ + int ret = 0; + struct selection *selection; + struct match_map *map; + struct expression *e; + char *f, *p; + int start = 0; + + if (event) { + f = strdup(event); + if (!f) + return -1; + p = strstr(f, "."); + if (p) + *p = '\0'; + + for (map = table->matches; map; map = map->next) { + if (start++) + trace_seq_printf(s, ","); + ret = print_key(s, table, f, + expand(map->A), expand(map->B)); + if (ret < 0) + break; + } + + free(f); + } else { + for (selection = table->selections; selection; selection = selection->next) { + e = selection->item; + if (!e->name || strncmp(e->name, "key", 3) != 0) + continue; + if (start++) + trace_seq_printf(s, ","); + trace_seq_printf(s, "%s", show_raw_expr(e)); + } + } + + return ret; +} + +enum value_type { + VALUE_TO, + VALUE_FROM, +}; + +static void print_val_delim(struct trace_seq *s, bool *start) +{ + if (*start) { + trace_seq_printf(s, ":"); + *start = false; + } else + trace_seq_printf(s, ","); +} + +struct var_list { + struct var_list *next; + const char *var; + const char *val; +}; + +static const char *find_var(struct var_list **vars, const char *val) +{ + struct var_list *v; + + for (v = *vars; v; v = v->next) { + if (strcmp(v->val, val) == 0) + return v->var; + } + + return NULL; +} + +static int add_var(struct var_list **vars, const char *var, const char *val) +{ + struct var_list *v; + + v = malloc(sizeof(*v)); + if (!v) + return -1; + + v->var = var; + v->val = val; + v->next = *vars; + *vars = v; + + return 0; +} + +static int print_to_expr(struct trace_seq *s, + struct sql_table *table, const char *event, + struct expression *e, struct var_list **vars) +{ + int ret = 0; + const char *actual; + const char *field; + const char *var; + int len = strlen(event); + + switch (e->type) { + case EXPR_FIELD: + actual = show_raw_expr(e); + field = event_match(event, actual, len); + if (field) { + trace_seq_printf(s, "%s", field); + break; + } + + field = strstr(actual, "."); + if (!field) { + trace_seq_printf(s, "%s", field); + break; + } + var = find_var(vars, actual); + if (!var) + return -1; + trace_seq_printf(s, "$%s", var); + break; + default: + ret = print_to_expr(s, table, event, e->A, vars); + if (ret < 0) + break; + switch (e->type) { + case EXPR_PLUS: + trace_seq_printf(s, "+"); + break; + case EXPR_MINUS: + trace_seq_printf(s, "-"); + break; + case EXPR_MULT: + trace_seq_printf(s, "*"); + break; + case EXPR_DIVID: + trace_seq_printf(s, "/"); + break; + default: + break; + } + ret = print_to_expr(s, table, event, e->B, vars); + } + + return ret; +} + +static int print_from_expr(struct trace_seq *s, struct sql_table *table, + const char *event, struct expression *e, + bool *start, struct var_list **vars) +{ + int ret = 0; + const char *actual; + const char *field; + int len = strlen(event); + + switch (e->type) { + case EXPR_FIELD: + actual = show_raw_expr(e); + field = event_match(event, actual, len); + if (field && !find_var(vars, actual)) { + print_val_delim(s, start); + if (!e->name) + e->name = make_dynamic_arg(); + trace_seq_printf(s, "%s=%s", e->name, field); + ret = add_var(vars, e->name, actual); + break; + } + break; + default: + ret = print_from_expr(s, table, event, e->A, start, vars); + if (ret < 0) + break; + ret = print_from_expr(s, table, event, e->B, start, vars); + } + return ret; +} + +static int print_value(struct trace_seq *s, struct sql_table *table, + const char *event, struct selection *selection, + enum value_type type, bool *start, + struct var_list **vars) +{ + int ret = 0; + struct expression *e = selection->item; + const char *name = selection->name; + int len = strlen(event); + const char *actual; + const char *field; + + switch (e->type) { + case EXPR_FIELD: + if (!selection->name || !e->name) + break; + actual = show_raw_expr(e); + field = event_match(event, actual, len); + if (field) { + if (type != VALUE_TO) { + print_val_delim(s, start); + trace_seq_printf(s, "%s=%s", e->name, field); + ret = add_var(vars, e->name, actual); + } + } else if (type == VALUE_TO) { + const char *arg; + /* + * The selection wants to show the from, + * We need to save the from field in a variable + */ + print_val_delim(s, start); + arg = make_dynamic_arg(); + trace_seq_printf(s, "%s=$%s", arg, e->name); + ret = add_var(vars, arg, actual); + if (ret < 0) + break; + /* Now use the this arg for the parameters */ + selection->name = arg; + } + break; + default: + if (type == VALUE_TO) { + print_val_delim(s, start); + trace_seq_printf(s, "%s=", name); + ret = print_to_expr(s, table, event, e, vars); + } else { + ret = print_from_expr(s, table, event, e, start, vars); + } + break; + } + + return ret; +} + +static int print_values(struct trace_seq *s, struct sql_table *table, + const char *event, enum value_type type, + struct var_list **vars) +{ + int ret = -1; + struct selection *selection; + struct expression *e; + char *f = NULL, *p; + bool start = true; + + if (event) { + f = strdup(event); + if (!f) + goto out; + p = strstr(f, "."); + if (p) + *p = '\0'; + + for (selection = table->selections; selection; selection = selection->next) { + ret = print_value(s, table, f, selection, type, &start, vars); + if (ret < 0) + goto out; + } + ret = 0; + } else { + for (selection = table->selections; selection; selection = selection->next) { + e = selection->item; + if (e->name && strncmp(e->name, "key", 3) == 0) + continue; + if (start) { + trace_seq_printf(s, ":values="); + start = false; + } else { + trace_seq_printf(s, ","); + } + trace_seq_printf(s, "%s", show_raw_expr(e)); + } + ret = 0; + } +out: + free(f); + return ret; +} + +static int print_trace_field(struct trace_seq *s, struct sql_table *table, + struct selection *selection) +{ + struct expression *e = selection->item; + const char *name; + const char *actual; + const char *field; + const char *to; + int len; + + to = resolve(table, table->to); + len = strlen(to); + + actual = show_raw_expr(e); + field = event_match(to, actual, len); + if (field) { + trace_seq_printf(s, ",%s", field); + return 0; + } + + name = selection->name; + if (!name) + name = e->name; + if (name) { + trace_seq_printf(s, ",$%s", name); + return 0; + } + + return -1; +} + +static int print_trace(struct trace_seq *s, struct sql_table *table) +{ + struct selection *selection; + int ret; + + trace_seq_printf(s, ".trace(%s", table->name); + + for (selection = table->selections; selection; selection = selection->next) { + ret = print_trace_field(s, table, selection); + if (ret < 0) + return ret; + } + trace_seq_printf(s, ")"); + return 0; +} + +static int print_system_event(const char *text, + char **sys_name, char **event_name) +{ + struct tep_event *event; + char *name = strdup(text); + char *tok; + + *sys_name = NULL; + *event_name = NULL; + + strtok(name, "."); + tok = strtok(NULL, "."); + if (tok) { + *sys_name = strdup(tok); + *event_name = strdup(name); + goto out; + } + + event = find_event(tep_glob, name); + if (!event) + goto out; + + *sys_name = strdup(event->system); + *event_name = strdup(name); + + out: + free(name); + if (!*sys_name || !*event_name) { + free(*sys_name); + free(*event_name); + *sys_name = NULL; + *event_name = NULL; + return -1; + } + return 0; +} + +static int make_histograms(struct sql_table *table, + struct sqlhist_trigger **triggers) +{ + int ret; + struct sqlhist_trigger *trigger = NULL; + struct trace_seq str; + struct sql_table *save_curr = curr_table; + struct var_list *vars = NULL; + const char *from = NULL; + const char *to; + char *sys_name = NULL, *event_name = NULL; + + if (!table) + return 0; + + /* Need to do children and younger siblings first */ + ret = make_histograms(find_table(table->from), triggers); + if (ret < 0) + return ret; + trace_seq_init(&str); + curr_table = table; + trigger = calloc(1, sizeof(struct sqlhist_trigger)); + if (!trigger) + goto error; + + if (table->to) + from = resolve(table, table->from); + + trace_seq_printf(&str, "hist:keys="); + ret = print_keys(&str, table, from); + if (ret < 0) + goto error; + ret = print_values(&str, table, from, VALUE_FROM, &vars); + if (ret < 0) + goto error; + + trigger->config = strdup(str.buffer); + if (!trigger->config) + goto error; + + if (!table->to) + from = resolve(table, table->from); + ret = print_system_event(from, &trigger->system, &trigger->event); + if (ret < 0) + goto error; + + if (*triggers) + (*triggers)->next = trigger; + else + *triggers = trigger; + + if (!table->to) + goto out; + + trigger = calloc(1, sizeof(struct sqlhist_trigger)); + if (!trigger) + goto error; + + trace_seq_reset(&str); + trace_seq_printf(&str, "hist:keys="); + to = resolve(table, table->to); + ret = print_keys(&str, table, to); + if (ret < 0) + goto error; + ret = print_values(&str, table, to, VALUE_TO, &vars); + if (ret < 0) + goto error; + trace_seq_printf(&str, ":onmatch("); + print_system_event(from, &sys_name, &event_name); + if (!sys_name || !event_name) + goto error; + trace_seq_printf(&str, "%s.%s", sys_name, event_name); + trace_seq_printf(&str, ")"); + + ret = print_trace(&str, table); + if (ret < 0) + goto error; + + print_system_event(to, &trigger->system, &trigger->event); + if (ret < 0) + goto error; + + trigger->config = strdup(str.buffer); + if (!trigger->config) + goto error; + + if (*triggers) + (*triggers)->next = trigger; + else + *triggers = trigger; + + while (vars) { + struct var_list *v = vars; + + vars = v->next; + free(v); + } + free(sys_name); + free(event_name); + out: + curr_table = save_curr; + trace_seq_destroy(&str); + return make_histograms(find_table(table->to), triggers); + +error: + free(sys_name); + free(event_name); + trace_seq_destroy(&str); + if (trigger) { + free(trigger->config); + free(trigger->event); + free(trigger->system); + free(trigger); + } + return -1; +} + +static char *buffer; +static size_t buffer_size; +static size_t buffer_idx; + +int my_yyinput(char *buf, int max) +{ + if (!buffer) + return read(0, buf, max); + + if (buffer_idx + max > buffer_size) + max = buffer_size - buffer_idx; + + if (max) + memcpy(buf, buffer + buffer_idx, max); + + buffer_idx += max; + + return max; +} + +void sqlhist_cleanup(void) +{ + clean_str_hash(); + clean_tables(); + parse_error = 0; + buffer_idx = 0; + curr_table = NULL; + top_table = NULL; +} + +void sqlhist_destroy(void) +{ + sqlhist_cleanup(); + tep_glob = NULL; +} + +void sqlhist_trigger_free(struct sqlhist_trigger *list) +{ + struct sqlhist_trigger *del, *trigger = list; + + while (trigger) { + del = trigger; + trigger = trigger->next; + free(del->config); + free(del->event); + free(del->system); + free(del); + } +} + +void sqlhist_synth_event_free(struct sqlhist_synth_event *list) +{ + struct sqlhist_synth_event *del, *event = list; + + while (event) { + del = event; + event = event->next; + free(del->config); + free(del->name); + free(del); + } +} + +int sqlhist_parse(struct tep_handle *tep, char *buff, + size_t buff_size, struct sqlhist_trigger **triggers, + struct sqlhist_synth_event **synt_events) +{ + int ret; + + sqlhist_cleanup(); + buffer = buff; + buffer_size = buff_size; + tep_glob = tep; + + ret = yyparse(); + if (ret < 0) + return ret; + +#if SQL_PARSE_DEBUG + dump_tables(); +#endif + + if (parse_error) + return -1; + + ret = make_synthetic_events(top_table, synt_events); + if (ret < 0) + return ret; + + ret = make_histograms(top_table, triggers); + if (ret < 0) + return ret; + + return ret; +} diff --git a/tracecmd/sqlparser/sqlhist.h b/tracecmd/sqlparser/sqlhist.h new file mode 100644 index 00000000..68ff196e --- /dev/null +++ b/tracecmd/sqlparser/sqlhist.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: LGPL-2.0 */ +/* + * Copyright (C) 2020 VMware Inc, Steven Rostedt + * + */ +#ifndef __SQLHIST_H +#define __SQLHIST_H + +#include +#include "sqlhist.tab.h" + +char *store_str(const char *str); +char *store_printf(const char *fmt, ...); +void add_label(const char *label, const char *val); +void add_match(const char *A, const char *B); +void table_start(void); +void table_end(const char *name); +void from_table_end(const char *name); +void simple_table_end(void); + +const char *show_expr(void *expr); +void *add_plus(void *A, void *B); +void *add_minus(void *A, void *B); +void *add_mult(void *A, void *B); +void *add_divid(void *A, void *B); +void *add_field(const char *field, const char *label); + +void add_expr(const char *name, void *expr); + +void add_selection(void *item); +void add_from(void *item); +void add_to(void *item); + +#endif /* __SQLHIST_H */ diff --git a/tracecmd/sqlparser/sqlhist.l b/tracecmd/sqlparser/sqlhist.l new file mode 100644 index 00000000..99dddcc6 --- /dev/null +++ b/tracecmd/sqlparser/sqlhist.l @@ -0,0 +1,59 @@ +%{ +/* SPDX-License-Identifier: LGPL-2.0 + * + * Copyright (C) 2020 VMware Inc, Steven Rostedt + * + */ + +#include +#include "sqlhist.h" + +extern int my_yyinput(char *buf, int max); + +#undef YY_INPUT +#define YY_INPUT(b, r, m) (r = my_yyinput(b, m)) + +#define YY_NO_INPUT +#define YY_NO_UNPUT + +static int line_no = 1; + +%} + +%option caseless + +%% + +select { return SELECT; } +as { return AS; } +from { return FROM; } +join { return JOIN; } +on { return ON; } + +\$[a-z][a-z0-9_]* { yylval.string = store_str(yytext); return VARIABLE; } + +[a-z0-9_\.]+ { yylval.string = store_str(yytext); return STRING; } + +[()\-\+\*/,=] { return yytext[0]; } + +[ \t] +\n { line_no++; } + +%% + +int yywrap(void) +{ + return 1; +} + +void yyerror(const char *fmt, ...) +{ + va_list ap; + + printf("\n"); + va_start(ap, fmt); + printf("ERROR: (%s) at line %d\n", yytext, line_no); + vprintf(fmt, ap); + printf("\n"); + va_end(ap); +} diff --git a/tracecmd/sqlparser/sqlhist.y b/tracecmd/sqlparser/sqlhist.y new file mode 100644 index 00000000..8a0004ae --- /dev/null +++ b/tracecmd/sqlparser/sqlhist.y @@ -0,0 +1,175 @@ +%{ +/* SPDX-License-Identifier: LGPL-2.0 + * + * Copyright (C) 2020 VMware Inc, Steven Rostedt + * + */ + +#include +#include +#include + +#include "sqlhist.h" + +extern int yylex(void); +extern void yyerror(char *fmt, ...); + +%} + +%union { + int s32; + char *string; + void *expr; +} + +%token AS SELECT FROM JOIN ON WHERE +%token STRING VARIABLE + +%left '+' '-' +%left '*' '/' + +%type name field label +%type selection_list table_exp selection_item +%type from_clause select_statement event_map +%type where_clause + +%type selection_expr item named_field join_clause + +%% + +start: + select_statement { table_end(NULL); } + | select_name + | simple_select { simple_table_end(); } + ; + +select_name : + '(' select_statement ')' label + { table_end($4); add_label($4, "SELECT"); } + ; + +label : AS name { $$ = store_printf("%s", $2); } + | name + ; + +select : SELECT { table_start(); } + ; + +simple_select : + select selection_list from_clause + | select selection_list from_clause where_clause + ; + +select_statement : + select selection_list table_exp + { + $$ = store_printf("SELECT %s %s", $2, $3); + } + ; + +selection_list : + selection_item + | selection_list ',' selection_item + { + $$ = store_printf("%s, %s", $1, $3); + } + ; + +selection_item : selection_expr { $$ = store_str(show_expr($1)); add_selection($1); } + ; + +selection_expr : + selection_expr '+' selection_expr + { + $$ = add_plus($1, $3); + } + | selection_expr '-' selection_expr + { + $$ = add_minus($1, $3); + } + | selection_expr '*' selection_expr + { + $$ = add_mult($1, $3); + } + | selection_expr '/' selection_expr + { + $$ = add_divid($1, $3); + } + | item + | '(' selection_expr ')' { $$ = $2; } + | '(' selection_expr ')' label + { + add_expr($4, $2); + $$ = $2; + } + ; + +item : + named_field + | field { $$ = add_field($1, NULL); } + ; + +field : + STRING + | VARIABLE + ; + +named_field : + field label { $$ = add_field($1, $2); } + ; + +name : + STRING + ; + +event_map : + from_clause join_clause on_clause { $$ = store_printf("%s TO %s", $1, show_expr($2)); } + ; + +where_clause : + WHERE item { $$ = store_printf(" WHERE %s", show_expr($2)); } + ; + +table_exp : + event_map + | event_map where_clause + ; + +from_clause : + FROM item + { + add_from($2); + $$ = store_printf("FROM %s", show_expr($2)); + } +/* + * Select from a from clause confuses the variable parsing. + * disable it for now. + + | FROM '(' select_statement ')' label + { + from_table_end($5); + $$ = store_printf("FROM (%s) AS %s", $3, $5); + } +*/ + ; + +join_clause : + JOIN item { add_to($2); $$ = $2; } + ; + +on_clause : + ON match_clause + ; + +match : + item '=' item { add_match(show_expr($1), show_expr($3)); } + ; + +match_clause : + match + | match_clause ',' match + ; + +%% + + diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 5c65a26f..8fd2d651 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -43,6 +43,7 @@ #include "version.h" #include "trace-local.h" #include "trace-msg.h" +#include "sqlhist-api.h" #define _STR(x) #x #define STR(x) _STR(x) @@ -214,6 +215,8 @@ enum trace_cmd { struct common_record_context { enum trace_cmd curr_cmd; struct buffer_instance *instance; + struct tep_handle *tep; + struct tep_plugin_list *tep_plugins; const char *output; char *date2ts; char *user; @@ -223,6 +226,7 @@ struct common_record_context { int total_disable; int disable; int events; + int auto_triggers; int global; int filtered; int date; @@ -2069,6 +2073,51 @@ static int clear_trigger(const char *file) return ret; } +static void clear_synthetic_events(void) +{ + struct sqlhist_synth_event *sevent; + struct buffer_instance *instance; + char *config; + + for_all_instances(instance) { + if (is_guest(instance) || !instance->synth_events) + continue; + for (sevent = instance->synth_events; sevent; sevent = sevent->next) { + if (asprintf(&config, "!%s", sevent->config) < 0) + continue; + tracefs_instance_file_append(NULL, + "synthetic_events", + config); + free(config); + } + sqlhist_synth_event_free(instance->synth_events); + instance->synth_events = NULL; + } +} + +static void clear_auto_triggers(void) +{ + struct buffer_instance *instance; + struct sqlhist_trigger *trigger; + char file[PATH_MAX]; + char *config; + + for_all_instances(instance) { + if (is_guest(instance) || !instance->auto_triggers) + continue; + for (trigger = instance->auto_triggers; trigger; trigger = trigger->next) { + if (asprintf(&config, "!%s", trigger->config) < 0) + continue; + snprintf(file, PATH_MAX, "events/%s/%s/trigger", + trigger->system, trigger->event); + tracefs_instance_file_append(instance->tracefs, file, config); + free(config); + } + sqlhist_trigger_free(instance->auto_triggers); + instance->auto_triggers = NULL; + } +} + static void clear_func_filter(const char *file) { char filter[BUFSIZ]; @@ -2568,6 +2617,52 @@ static void set_mask(struct buffer_instance *instance) instance->cpumask = NULL; } +static void set_synth_events(struct buffer_instance *instance) +{ + struct sqlhist_synth_event *sevent; + char *inst_name; + + if (is_guest(instance)) + return; + + inst_name = tracefs_instance_get_name(instance->tracefs); + for (sevent = instance->synth_events; sevent; sevent = sevent->next) { + pr_stat("Synthetic event (instance %s) %s:\n\t%s\n", + inst_name ? inst_name : "top", + sevent->name, sevent->config); + if (tracefs_instance_file_append(NULL, "synthetic_events", sevent->config) < 0) + die("Failed to set synthetic event %s", sevent->config); + } +} + +static void set_auto_triggers(struct buffer_instance *instance) +{ + struct sqlhist_trigger *trigger; + char file[PATH_MAX]; + char *inst_name; + + if (is_guest(instance)) + return; + + inst_name = tracefs_instance_get_name(instance->tracefs); + for (trigger = instance->auto_triggers; trigger; trigger = trigger->next) { + pr_stat("Trigger (instance %s) %s/%s:\n\t%s\n", + inst_name ? inst_name : "top", + trigger->system, trigger->event, trigger->config); + snprintf(file, PATH_MAX, "events/%s/%s/trigger", + trigger->system, trigger->event); + if (tracefs_instance_file_write(instance->tracefs, file, trigger->config) < 0) { + char *err_log = NULL; + + err_log = tracefs_instance_file_read(instance->tracefs, + "error_log", NULL); + die("Failed to set trigger %s:\n\t%s\nerror:\n\t%s", + file, trigger->config, + err_log ? err_log : "N/A"); + } + } +} + static void enable_events(struct buffer_instance *instance) { struct event_list *event; @@ -4816,7 +4911,7 @@ static void check_function_plugin(void) static int __check_doing_something(struct buffer_instance *instance) { return is_guest(instance) || (instance->flags & BUFFER_FL_PROFILE) || - instance->plugin || instance->events; + instance->plugin || instance->events || instance->auto_triggers; } static void check_doing_something(void) @@ -5230,6 +5325,7 @@ void init_top_instance(void) } enum { + OPT_sql = 242, OPT_user = 243, OPT_procmap = 244, OPT_quiet = 245, @@ -5425,6 +5521,23 @@ static void init_common_record_context(struct common_record_context *ctx, ctx->curr_cmd = curr_cmd; local_cpu_count = count_cpus(); init_top_instance(); + ctx->tep = tep_alloc(); + if (ctx->tep) { + ctx->tep_plugins = trace_load_plugins(ctx->tep); + tep_set_flag(ctx->tep, TEP_NO_PARSING_WARNINGS); + tracefs_fill_local_events(NULL, ctx->tep, NULL); + } +} + +static void free_common_record_context(struct common_record_context *ctx) +{ + if (ctx->tep) { + if (ctx->tep_plugins) + tep_unload_plugins(ctx->tep_plugins, ctx->tep); + ctx->tep_plugins = NULL; + tep_free(ctx->tep); + ctx->tep = NULL; + } } #define IS_EXTRACT(ctx) ((ctx)->curr_cmd == CMD_extract) @@ -5532,13 +5645,14 @@ static void parse_record_options(int argc, {"proc-map", no_argument, NULL, OPT_procmap}, {"user", required_argument, NULL, OPT_user}, {"module", required_argument, NULL, OPT_module}, + {"sql", required_argument, NULL, OPT_sql}, {NULL, 0, NULL, 0} }; if (IS_EXTRACT(ctx)) opts = "+haf:Fp:co:O:sr:g:l:n:P:N:tb:B:ksiT"; else - opts = "+hae:f:FA:p:cC:dDGo:O:s:r:vg:l:n:P:N:tb:R:B:ksSiTm:M:H:q"; + opts = "+hae:f:FA:p:cC:dDGo:O:s:r:vg:l:n:P:N:tb:R:B:ksSiTm:M:H:qV"; c = getopt_long (argc-1, argv+1, opts, long_options, &option_index); if (c == -1) break; @@ -5802,6 +5916,9 @@ static void parse_record_options(int argc, case 'i': ignore_event_not_found = 1; break; + case 'V': + show_status = 1; + break; case OPT_user: ctx->user = strdup(optarg); if (!ctx->user) @@ -5873,6 +5990,15 @@ static void parse_record_options(int argc, case 'q': quiet = true; break; + case OPT_sql: + ctx->auto_triggers = 1; + ret = sqlhist_parse(ctx->tep, optarg, + strlen(optarg) + 1, + &ctx->instance->auto_triggers, + &ctx->instance->synth_events); + if (ret < 0) + die("Failed to parse sql argument"); + break; default: usage(argv); } @@ -5950,6 +6076,8 @@ static void finalize_record_trace(struct common_record_context *ctx) update_reset_triggers(); if (clear_function_filters) clear_func_filters(); + clear_auto_triggers(); + clear_synthetic_events(); set_plugin("nop"); @@ -6024,6 +6152,11 @@ static void record_trace(int argc, char **argv, make_instances(); + if (ctx->auto_triggers) { + for_all_instances(instance) + set_synth_events(instance); + } + if (ctx->events) expand_event_list(); @@ -6051,6 +6184,10 @@ static void record_trace(int argc, char **argv, enable_events(instance); } + if (ctx->auto_triggers) { + for_all_instances(instance) + set_auto_triggers(instance); + } set_saved_cmdlines_size(ctx); set_buffer_size(); update_plugins(type); @@ -6198,6 +6335,7 @@ void trace_extract(int argc, char **argv) delete_thread_data(); destroy_stats(); finalize_record_trace(&ctx); + free_common_record_context(&ctx); exit(0); } diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index 502a4102..ba196eca 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -60,6 +60,8 @@ static struct usage_help usage_help[] = { " --no-filter include trace-cmd threads in the trace\n" " --proc-map save the traced processes address map into the trace.dat file\n" " --user execute the specified [command ...] as given user\n" + " --sql generate necessary ftrace syntetic events and triggers, based on the given sql query\n" + " -V dump ftrace configuration, auto generated by the --sql option" }, { "start",