From patchwork Wed Aug 23 00:26:50 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timofey Titovets X-Patchwork-Id: 9916415 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 3ED89603F9 for ; Wed, 23 Aug 2017 00:27:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 30F28287E3 for ; Wed, 23 Aug 2017 00:27:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 25DEF2892C; Wed, 23 Aug 2017 00:27:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BB634287E3 for ; Wed, 23 Aug 2017 00:27:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752916AbdHWA1T (ORCPT ); Tue, 22 Aug 2017 20:27:19 -0400 Received: from mail-wm0-f68.google.com ([74.125.82.68]:34787 "EHLO mail-wm0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752856AbdHWA1O (ORCPT ); Tue, 22 Aug 2017 20:27:14 -0400 Received: by mail-wm0-f68.google.com with SMTP id r187so711329wma.1 for ; Tue, 22 Aug 2017 17:27:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=peXnNnlIwBWDKMgkolvWoKj55HIezMPYme5H5+jm3jM=; b=QFviDhLyXIDUIfW+fRoOxob0oWHpqFDLfJtinn59s34RhCfXDon1mVPWu3ub0/8EaP dFRe9k1tqCG8dcM7gcHQ61ajjovVB17UfLAnJeqBKXM6gM8z7SOgPJWMeQGNis4plfGt wq94UuV6Z+5HXUIZasjwA6mr2TDHmbkej4oXwuB/CbFMaif9Zp0IgWccxda3IkWPzpiU M1BhVq8rJA9vvAWO45D7F2I9VYE5J6PuimZsNH7xPbnoLqfdXFutyjR1Q0ySPKExxTq9 S3bMA/E8etgifKZjCZUALNMqew0IwVsX++GTcVvqFtTF/6rsOrbOETXXnubR8lBKnAqK JIPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=peXnNnlIwBWDKMgkolvWoKj55HIezMPYme5H5+jm3jM=; b=oPbI/w1vJd1+m6xwy/s625n0g5DFbODg6faT1hprlOkUombCO9QUgJwwqDo21LMSnV QxFtyMfMBR2LuwaZMlVuuKmFFEsO0OaARpG5sVckP8D593kTAqNnkAW181uqLwtWNIS1 MxT+IhbocET0O3xvBsg//HW4nhzJ4XkQjCovM7KRgp5m/ThZDDh2gOyScrj2nS3eDJBS ZdJ5JrCU7vKphooYbs+AeXxb7mfbquJkIujQKqXbho5nyXp7siPfmbLAqA6ay4sQD45X 2K7IDgjEOEAsescuA5wU8UQIs0ELYD8czfb9YGwSsdkX0CQR6cL2tqcgvD9j4yugqUJP s65w== X-Gm-Message-State: AHYfb5hDUuoZm2LiDhJDj/7UbOCqByiXZE/DaxS7IgYQl+yKyQ/vIt+z jIs8FVResGCyMEnd X-Received: by 10.28.214.134 with SMTP id n128mr763465wmg.92.1503448032791; Tue, 22 Aug 2017 17:27:12 -0700 (PDT) Received: from titovetst-l.itransition.corp (nat3-minsk-pool-46-53-180-190.telecom.by. [46.53.180.190]) by smtp.gmail.com with ESMTPSA id k45sm136926wre.1.2017.08.22.17.27.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 22 Aug 2017 17:27:12 -0700 (PDT) From: Timofey Titovets To: linux-btrfs@vger.kernel.org Cc: Timofey Titovets Subject: [PATCH v5 6/6] Btrfs: heuristic add byte core set calculation Date: Wed, 23 Aug 2017 03:26:50 +0300 Message-Id: <20170823002650.3133-7-nefelim4ag@gmail.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20170823002650.3133-1-nefelim4ag@gmail.com> References: <20170823002650.3133-1-nefelim4ag@gmail.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Calculate byte core set for data sample: Sort bucket's numbers in decreasing order Count how many numbers use 90% of sample If core set are low (<=25%), data are easily compressible If core set high (>=80%), data are not compressible Signed-off-by: Timofey Titovets --- fs/btrfs/heuristic.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 50 insertions(+), 1 deletion(-) -- 2.14.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/fs/btrfs/heuristic.c b/fs/btrfs/heuristic.c index 953428fde305..14128f77d5ae 100644 --- a/fs/btrfs/heuristic.c +++ b/fs/btrfs/heuristic.c @@ -18,6 +18,7 @@ #include #include #include +#include #include "compression.h" #define READ_SIZE 16 @@ -32,6 +33,8 @@ #define MAX_INPUT_PAGES ((BTRFS_MAX_UNCOMPRESSED >> PAGE_SHIFT)+1) #define MAX_SAMPLE_SIZE (MAX_INPUT_PAGES*PAGE_SIZE*READ_SIZE/ITER_SHIFT) #define BYTE_SET_THRESHOLD 64 +#define BYTE_CORE_SET_LOW BYTE_SET_THRESHOLD +#define BYTE_CORE_SET_HIGH 200 // ~80% struct bucket_item { u32 count; @@ -74,6 +77,45 @@ static struct list_head *heuristic_alloc_workspace(void) return ERR_PTR(-ENOMEM); } +/* For bucket sorting */ +static inline int bucket_compare(const void *lv, const void *rv) +{ + struct bucket_item *l = (struct bucket_item *)(lv); + struct bucket_item *r = (struct bucket_item *)(rv); + + return r->count - l->count; +} + +/* + * Byte Core set size + * How many bytes use 90% of sample + */ +static int byte_core_set_size(struct workspace *workspace) +{ + int a = 0; + u32 coreset_sum = 0; + struct bucket_item *bucket = workspace->bucket; + u32 core_set_threshold = workspace->sample_size*90/100; + + /* Sort in reverse order */ + sort(bucket, BUCKET_SIZE, sizeof(*bucket), + &bucket_compare, NULL); + + for (; a < BYTE_CORE_SET_LOW; a++) + coreset_sum += bucket[a].count; + + if (coreset_sum > core_set_threshold) + return a; + + for (; a < BYTE_CORE_SET_HIGH && bucket[a].count > 0; a++) { + coreset_sum += bucket[a].count; + if (coreset_sum > core_set_threshold) + break; + } + + return a; +} + static int byte_set_size(const struct workspace *workspace) { int a = 0; @@ -161,7 +203,14 @@ static int heuristic(struct list_head *ws, struct inode *inode, if (a > BYTE_SET_THRESHOLD) return 2; - return 1; + a = byte_core_set_size(workspace); + if (a <= BYTE_CORE_SET_LOW) + return 3; + + if (a >= BYTE_CORE_SET_HIGH) + return 0; + + return 4; } const struct btrfs_compress_op btrfs_heuristic = {