From patchwork Fri Dec 10 10:59:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12669323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE3E6C433EF for ; Fri, 10 Dec 2021 10:59:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240114AbhLJLD1 (ORCPT ); Fri, 10 Dec 2021 06:03:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35606 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237285AbhLJLD0 (ORCPT ); Fri, 10 Dec 2021 06:03:26 -0500 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B731DC061746 for ; Fri, 10 Dec 2021 02:59:51 -0800 (PST) Received: by mail-ed1-x536.google.com with SMTP id e3so29073647edu.4 for ; Fri, 10 Dec 2021 02:59:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0JLOGryTX6Dx+lCa8BZt5/FliAFVkF7DjTVQAb57Za8=; b=TTNwR+NLXHuMZewLbYSq7/2rpynzJ1q0j2Kady/WrGcHxBQI6UYh+9ffa1llE++Z96 Q3cIQTe+/K/D6sn2NTz9p+K+itaMmrCBUltSzH3O0CQa+mguDPEQ0kX/HbQ40I7Mqt4/ Jy6MOlHA+g6bLJxcbjRYtdsik2JHlhBMZIDJLvsUfYUvAA6wvaWENDD5PO6w9CmFvPmQ XwoH89FB5ciLcnwpqqxvhFJXyFWiPe4izC4y4QPZq98WqVK+7uCrp3QRpQRhUdABlyOk gp1V3Nss66H7AiHCpZsxZrx9ey3tT5TYEQeTSDsJEu5IPxD9j1qxGk+7XRWbrwrh9cZ0 M8tg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0JLOGryTX6Dx+lCa8BZt5/FliAFVkF7DjTVQAb57Za8=; b=17bSOjVAWGrAmjTwHAtyiEDhJrlkfcZzQHNzKBSm6PFMCvXGS4sJ7fp7uCDr3n96Qp qLUvX/BVL7QOmIRd13eb4ZYshevxVXapZENCNVOPdhzOpj4Gl4E1BSUru9oxcLT7LATh qwOjmkBu/k8tOqxk9ap+q5WYZXQLtOXaKNme33dD833V0SLc2rK0JfUyhoomGVI+sMvY 7VQsYDC73VMvJILTI9NbK817NEd1okMiCNcQMtmh9wx7ck1v4TStTUOSBl8LCuid7Ycc 2qLsit79Nr0tygVc9t/73BOS9zwOkUiDZ1Qv8AORUvaLhEegqZbQX0s+vysqZySeoeJt aiwg== X-Gm-Message-State: AOAM531huPMpcHnLkQMPFceVD1nhfnGG/5SrKDJbRvfZoaNyIE/cXeTO aL39jN1Xxy+gONv8RbwYqP05ceRGRu0= X-Google-Smtp-Source: ABdhPJyHL1L9PCmQgc25Y87GHdOUYA8BcY7hMaqqLv4ehs02GHS/TicQIVmPh8GK0Nfrk81m8nxkZA== X-Received: by 2002:a17:907:629b:: with SMTP id nd27mr22941678ejc.24.1639133990304; Fri, 10 Dec 2021 02:59:50 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id j17sm1320379edj.0.2021.12.10.02.59.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 02:59:49 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 12/20] trace-cmd library: Extend the input handler with trace data decompression context Date: Fri, 10 Dec 2021 12:59:30 +0200 Message-Id: <20211210105938.98250-13-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211210105938.98250-1-tz.stoyanov@gmail.com> References: <20211210105938.98250-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The CPU trace data is compressed in chunks, as chunk's size is multiple trace pages. The input handler is extended with the necessary structures, to control the data decompression. There are two approaches for data decompression, both are supported and can be used in different use cases: - in-memory decompression, page by page. - using a temporary file Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 59 ++++++++++++++++++++++++++++++------- 1 file changed, 48 insertions(+), 11 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 78271fdf..dbfd881b 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -54,6 +54,24 @@ struct page { #endif }; +struct zchunk_cache { + struct list_head list; + struct tracecmd_compress_chunk *chunk; + void *map; + int ref; +}; + +struct cpu_zdata { + /* uncompressed cpu data */ + int fd; + char file[26]; /* strlen(COMPR_TEMP_FILE) */ + unsigned int count; + unsigned int last_chunk; + struct list_head cache; + struct tracecmd_compress_chunk *chunks; +}; + +#define COMPR_TEMP_FILE "/tmp/trace_cpu_dataXXXXXX" struct cpu_data { /* the first two never change */ unsigned long long file_offset; @@ -72,6 +90,7 @@ struct cpu_data { int page_cnt; int cpu; int pipe_fd; + struct cpu_zdata compress; }; struct cpu_file_data { @@ -151,6 +170,8 @@ struct tracecmd_input { bool use_trace_clock; bool read_page; bool use_pipe; + bool read_zpage; /* uncompress pages in memory, do not use tmp files */ + bool cpu_compressed; int file_version; unsigned int cpustats_size; struct cpu_data *cpu_data; @@ -3304,6 +3325,7 @@ static int init_cpu_data(struct tracecmd_input *handle) endian = KBUFFER_ENDIAN_LITTLE; for (cpu = 0; cpu < handle->cpus; cpu++) { + handle->cpu_data[cpu].compress.fd = -1; handle->cpu_data[cpu].kbuf = kbuffer_alloc(long_size, endian); if (!handle->cpu_data[cpu].kbuf) goto out_free; @@ -4085,6 +4107,7 @@ static inline void free_buffer(struct input_buffer_instance *buf) */ void tracecmd_close(struct tracecmd_input *handle) { + struct zchunk_cache *cache; struct file_section *del_sec; int cpu; int i; @@ -4104,17 +4127,31 @@ void tracecmd_close(struct tracecmd_input *handle) /* The tracecmd_peek_data may have cached a record */ free_next(handle, cpu); free_page(handle, cpu); - if (handle->cpu_data && handle->cpu_data[cpu].kbuf) { - kbuffer_free(handle->cpu_data[cpu].kbuf); - if (handle->cpu_data[cpu].page_map) - free_page_map(handle->cpu_data[cpu].page_map); - - if (handle->cpu_data[cpu].page_cnt) - tracecmd_warning("%d pages still allocated on cpu %d%s", - handle->cpu_data[cpu].page_cnt, cpu, - show_records(handle->cpu_data[cpu].pages, - handle->cpu_data[cpu].nr_pages)); - free(handle->cpu_data[cpu].pages); + if (handle->cpu_data) { + if (handle->cpu_data[cpu].kbuf) { + kbuffer_free(handle->cpu_data[cpu].kbuf); + if (handle->cpu_data[cpu].page_map) + free_page_map(handle->cpu_data[cpu].page_map); + + if (handle->cpu_data[cpu].page_cnt) + tracecmd_warning("%d pages still allocated on cpu %d%s", + handle->cpu_data[cpu].page_cnt, cpu, + show_records(handle->cpu_data[cpu].pages, + handle->cpu_data[cpu].nr_pages)); + free(handle->cpu_data[cpu].pages); + } + if (handle->cpu_data[cpu].compress.fd >= 0) { + close(handle->cpu_data[cpu].compress.fd); + unlink(handle->cpu_data[cpu].compress.file); + } + while (!list_empty(&handle->cpu_data[cpu].compress.cache)) { + cache = container_of(handle->cpu_data[cpu].compress.cache.next, + struct zchunk_cache, list); + list_del(&cache->list); + free(cache->map); + free(cache); + } + free(handle->cpu_data[cpu].compress.chunks); } }