From patchwork Thu Nov 11 15:11:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615105 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE3BEC433FE for ; Thu, 11 Nov 2021 15:12:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C60ED60E75 for ; Thu, 11 Nov 2021 15:12:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233959AbhKKPPA (ORCPT ); Thu, 11 Nov 2021 10:15:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234001AbhKKPO7 (ORCPT ); Thu, 11 Nov 2021 10:14:59 -0500 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7977AC061766 for ; Thu, 11 Nov 2021 07:12:10 -0800 (PST) Received: by mail-ed1-x52e.google.com with SMTP id j21so25221237edt.11 for ; Thu, 11 Nov 2021 07:12:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=68lYZsA3ue8BHX/KQfu7Y5XJCSNyGtkBGxThRpuUnB8=; b=FeXZuBxpvpfsEne0aw72KJ+lCm4j+RBk7lWkVd82z32sFSSjTkYmLLeoeN6qAQI0q/ EvuUupFBBQUUK6UQdiebKvBeuNJO1eBtPQt7dAJI9WVu/LHnuXqlhAbFS2E02+T2j9wA UlwjZ/Ph7rXxbhHOAbJN40CnynqYbULWycgYkEiSJ4i4Nw0ty/9sDu1K9UUIaZTSqsPb MXwGqfnQWRS/8l34aznFX4gwbQMi804G1rcY4BuPS82M5O0V8D2Jtpfp5sAXdz7M/ffb 3apc1ERtBplikc5qaSIOhBcBx1TpCiUv6GUzMh3hULI01ceq0MvhN3Hgd5JVKvsJLLsU hnFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=68lYZsA3ue8BHX/KQfu7Y5XJCSNyGtkBGxThRpuUnB8=; b=OZ/VzCq0aWQmG4Z/yAErUXvDZ52mg5PgOb0rkL4+q7rexKe5EApC6+EnmU0bRYXAS7 j+5V/hc9dt7jLg4oRMO5qGnNUrsZdIKY4vtrW0/cCX13XTBmCtEfpJSig1W4xGP5t+c6 v+SFnjHTviSPjbIZOPzU7K1MmPtgFejTL3i0ruQE2xIQAbvCTrmRgbRyvfIfznj8VOAb dsLCn6tsdLMddPwfM17sPZkiwQOiNm6unXV0tVhYlTEwL8xdoBKnKHv7xRvpdRYYgVSi gf40mcc4bzyQqKK6nnh5Im5LzlQDuI4TwlelKMvkNvanH+fy8I6CREz8knV0yps5iLJI i/Cg== X-Gm-Message-State: AOAM5322R0GymeIKPJDFrxwm1oyVupBeXVrgcVry96QGciRdHje05QxO wW8IBK4YraIS97UL6/mik7SqUGxg/mWftg== X-Google-Smtp-Source: ABdhPJww32hUvxjIS3CVQxwEsN51lBab8Hm9YnFb+C9c9UXgA5aYWRCd/H3EalqNIWmvztBVWvrZFg== X-Received: by 2002:a17:907:980e:: with SMTP id ji14mr10070688ejc.346.1636643529037; Thu, 11 Nov 2021 07:12:09 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:08 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 12/20] trace-cmd library: Extend the input handler with trace data decompression context Date: Thu, 11 Nov 2021 17:11:45 +0200 Message-Id: <20211111151153.86855-13-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The CPU trace data is compressed in chunks, as chunk's size is multiple trace pages. The input handler is extended with the necessary structures, to control the data decompression. There are two approaches for data decompression, both are supported and can be used in different use cases: - in-memory decompression, page by page. - using a temporary file Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 59 ++++++++++++++++++++++++++++++------- 1 file changed, 48 insertions(+), 11 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 2badc341..f46feac5 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -54,6 +54,24 @@ struct page { #endif }; +struct zchunk_cache { + struct list_head list; + struct tracecmd_compress_chunk *chunk; + void *map; + int ref; +}; + +struct cpu_zdata { + /* uncompressed cpu data */ + int fd; + char file[26]; /* strlen(COMPR_TEMP_FILE) */ + unsigned int count; + unsigned int last_chunk; + struct list_head cache; + struct tracecmd_compress_chunk *chunks; +}; + +#define COMPR_TEMP_FILE "/tmp/trace_cpu_dataXXXXXX" struct cpu_data { /* the first two never change */ unsigned long long file_offset; @@ -72,6 +90,7 @@ struct cpu_data { int page_cnt; int cpu; int pipe_fd; + struct cpu_zdata compress; }; struct cpu_file_data { @@ -151,6 +170,8 @@ struct tracecmd_input { bool use_trace_clock; bool read_page; bool use_pipe; + bool read_zpage; /* uncompress pages in memory, do not use tmp files */ + bool cpu_compressed; int file_version; unsigned int cpustats_size; struct cpu_data *cpu_data; @@ -3294,6 +3315,7 @@ static int init_cpu_data(struct tracecmd_input *handle) endian = KBUFFER_ENDIAN_LITTLE; for (cpu = 0; cpu < handle->cpus; cpu++) { + handle->cpu_data[cpu].compress.fd = -1; handle->cpu_data[cpu].kbuf = kbuffer_alloc(long_size, endian); if (!handle->cpu_data[cpu].kbuf) goto out_free; @@ -4030,6 +4052,7 @@ static inline void free_buffer(struct input_buffer_instance *buf) */ void tracecmd_close(struct tracecmd_input *handle) { + struct zchunk_cache *cache; struct file_section *del_sec; int cpu; int i; @@ -4049,17 +4072,31 @@ void tracecmd_close(struct tracecmd_input *handle) /* The tracecmd_peek_data may have cached a record */ free_next(handle, cpu); free_page(handle, cpu); - if (handle->cpu_data && handle->cpu_data[cpu].kbuf) { - kbuffer_free(handle->cpu_data[cpu].kbuf); - if (handle->cpu_data[cpu].page_map) - free_page_map(handle->cpu_data[cpu].page_map); - - if (handle->cpu_data[cpu].page_cnt) - tracecmd_warning("%d pages still allocated on cpu %d%s", - handle->cpu_data[cpu].page_cnt, cpu, - show_records(handle->cpu_data[cpu].pages, - handle->cpu_data[cpu].nr_pages)); - free(handle->cpu_data[cpu].pages); + if (handle->cpu_data) { + if (handle->cpu_data[cpu].kbuf) { + kbuffer_free(handle->cpu_data[cpu].kbuf); + if (handle->cpu_data[cpu].page_map) + free_page_map(handle->cpu_data[cpu].page_map); + + if (handle->cpu_data[cpu].page_cnt) + tracecmd_warning("%d pages still allocated on cpu %d%s", + handle->cpu_data[cpu].page_cnt, cpu, + show_records(handle->cpu_data[cpu].pages, + handle->cpu_data[cpu].nr_pages)); + free(handle->cpu_data[cpu].pages); + } + if (handle->cpu_data[cpu].compress.fd >= 0) { + close(handle->cpu_data[cpu].compress.fd); + unlink(handle->cpu_data[cpu].compress.file); + } + while (!list_empty(&handle->cpu_data[cpu].compress.cache)) { + cache = container_of(handle->cpu_data[cpu].compress.cache.next, + struct zchunk_cache, list); + list_del(&cache->list); + free(cache->map); + free(cache); + } + free(handle->cpu_data[cpu].compress.chunks); } }