From patchwork Sun Jan 16 08:48:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhang X-Patchwork-Id: 12714514 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66C63C433EF for ; Sun, 16 Jan 2022 08:49:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234433AbiAPItE (ORCPT ); Sun, 16 Jan 2022 03:49:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41466 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230421AbiAPItD (ORCPT ); Sun, 16 Jan 2022 03:49:03 -0500 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 57577C061574 for ; Sun, 16 Jan 2022 00:49:03 -0800 (PST) Received: by mail-pj1-x102b.google.com with SMTP id w12-20020a17090a528c00b001b276aa3aabso2833751pjh.0 for ; Sun, 16 Jan 2022 00:49:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id; bh=e3mvjCRCsR0Swov/K+9n7idBmM6wdvyYWBZQcPD4B9k=; b=UJkrpWWuhFUvUmb9AtBSaVo9Wcf93GPXVxVgiLhTf4sSw45d5dCGLc8jBA0PWwGq7N hKSmgfJ9FC1b7Otq2sZAubgV+DnCNli4nqNzq788X1muCOc7oOyxglIdfWie5WVUytIv AFPWC6iVmdAbGqdJmVJf+soHR3KTrb03OOIhMjwvKdyWxLRNxGe+MCZvkn62QSCwfOYr ffhSEb4CQQ5vPfTBY6RCIOWQe20R6tyI/gzQ3et/yQJtqH26mvtBDTLnaCTCRs6nhQnY W+YtsXQ3kEy87WC4+uPuRkZbScWaVeB9+88Vi2+wBGug5rOKpF00W+jpQlhD7WUV9YeI HoUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=e3mvjCRCsR0Swov/K+9n7idBmM6wdvyYWBZQcPD4B9k=; b=HL4X6hPYr6u77rJ/panz4eThHm4sjgOlbQM9cOwwXlPPfPMKKpJH+vKRhc1sdnmnO1 XLnwWYB7lwjID2k+QZJhjnRyOO3zV5xgFXA3uZ3zRyS3KPQSBRu1gLgjcKdiP3EHVrTq F28wpXsOJ3jH1TLMbs/NzO7j4mSOndbTwNa2p7UZSxvoLCVFsqQ2ujiF1ZU4GgAMXboo jEkO2lyd4BQy3EJS7r4UaDIxGWKDRH7TME+v9N5WzshFiGFsUYc6u0aXRIGSRSpfgpOo vPZfmH/5VRnSmDMp5fjGGV1ssGQsPjhyY+rm95Am9kTnoWhbov7xMwRyhIXSmlXb6EJy 09Rg== X-Gm-Message-State: AOAM53316WpJt1AZrEtjAk3WRah18rp1Z7qVqDRPziqzWUqnzByt7ZWq 93lXJmhZ0dAOi/QaSR5oPFX1UJ6W0D4ltLGF X-Google-Smtp-Source: ABdhPJy1zmdQcuKffEZJoL30cAF2lAxyMhvA/8twj9VihuIPK0q1We40Cv2ZFUO1D3Gq+G4pwTldnQ== X-Received: by 2002:a17:90b:384a:: with SMTP id nl10mr5114187pjb.233.1642322942337; Sun, 16 Jan 2022 00:49:02 -0800 (PST) Received: from zllke.localdomain ([113.99.5.116]) by smtp.gmail.com with ESMTPSA id ng18sm16964968pjb.36.2022.01.16.00.49.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 16 Jan 2022 00:49:01 -0800 (PST) From: Li Zhang To: linux-btrfs@vger.kernel.org Cc: zhanglikernel@gmail.com Subject: [PATCH 1/5] btrfs: Introduce memory-only flag BTRFS_INODE_HEURISTIC_NOCOMPRESS. Date: Sun, 16 Jan 2022 16:48:54 +0800 Message-Id: <1642322934-1804-1-git-send-email-zhanglikernel@gmail.com> X-Mailer: git-send-email 1.8.3.1 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Introduce the memory flag BTRFS_INODE_HEURISTIC_NOCOMPRESS which is kept in memory only. This flag should be unset every time the filesystem writes to disk. 1. If the compressed file is larger than the original size, BTRFS_INODE_HEURISTIC_NOCOMPRESS will be set. 2. If the btrfs file system wants to write the btrfs_inode flag to the disk or journal tree, the btrfs_inode flag should be and with ~BTRFS_INODE_ONLY_MEM_FLAG_MASK to filter the memory-only flag. 3. In the inode_need_compress function, if btrfs_info is set to BTRFS_INODE_HEURISTIC_NOCOMPRESS, it returns 0, which means that the file is not compressed as much as possible. Signed-off-by: Li Zhang --- fs/btrfs/ctree.h | 13 ++++++++++++- fs/btrfs/inode.c | 13 +++++++++---- fs/btrfs/tree-log.c | 8 ++++++-- 3 files changed, 27 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index b4a9b1c..754ac0f 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -1507,6 +1507,7 @@ enum { #define BTRFS_INODE_NOATIME (1U << 9) #define BTRFS_INODE_DIRSYNC (1U << 10) #define BTRFS_INODE_COMPRESS (1U << 11) +#define BTRFS_INODE_HEURISTIC_NOCOMPRESS (1U << 12) #define BTRFS_INODE_ROOT_ITEM_INIT (1U << 31) @@ -1523,7 +1524,17 @@ enum { BTRFS_INODE_NOATIME | \ BTRFS_INODE_DIRSYNC | \ BTRFS_INODE_COMPRESS | \ - BTRFS_INODE_ROOT_ITEM_INIT) + BTRFS_INODE_ROOT_ITEM_INIT | \ + BTRFS_INODE_HEURISTIC_NOCOMPRESS ) + +/* + * Inode only memory flags + */ + +#define BTRFS_INODE_ONLY_MEM_FLAG_MASK \ + (BTRFS_INODE_HEURISTIC_NOCOMPRESS) + + #define BTRFS_INODE_RO_VERITY (1U << 0) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 3b2403b..3fe485b 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -550,7 +550,8 @@ static inline int inode_need_compress(struct btrfs_inode *inode, u64 start, if (inode->defrag_compress) return 1; /* bad compression ratios */ - if (inode->flags & BTRFS_INODE_NOCOMPRESS) + if (inode->flags & BTRFS_INODE_NOCOMPRESS || + inode->flags & BTRFS_INODE_HEURISTIC_NOCOMPRESS) return 0; if (btrfs_test_opt(fs_info, COMPRESS) || inode->flags & BTRFS_INODE_COMPRESS || @@ -838,7 +839,7 @@ static noinline int compress_file_range(struct async_chunk *async_chunk) /* flag the file so we don't compress in the future */ if (!btrfs_test_opt(fs_info, FORCE_COMPRESS) && !(BTRFS_I(inode)->prop_compress)) { - BTRFS_I(inode)->flags |= BTRFS_INODE_NOCOMPRESS; + BTRFS_I(inode)->flags |= BTRFS_INODE_HEURISTIC_NOCOMPRESS; } } cleanup_and_bail_uncompressed: @@ -3970,8 +3971,12 @@ static void fill_inode_item(struct btrfs_trans_handle *trans, btrfs_set_token_inode_sequence(&token, item, inode_peek_iversion(inode)); btrfs_set_token_inode_transid(&token, item, trans->transid); btrfs_set_token_inode_rdev(&token, item, inode->i_rdev); - flags = btrfs_inode_combine_flags(BTRFS_I(inode)->flags, - BTRFS_I(inode)->ro_flags); + /* + * get rid of memory only flag + */ + flags = btrfs_inode_combine_flags( + BTRFS_I(inode)->flags & ~BTRFS_INODE_ONLY_MEM_FLAG_MASK, + BTRFS_I(inode)->ro_flags); btrfs_set_token_inode_flags(&token, item, flags); btrfs_set_token_inode_block_group(&token, item, 0); } diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c index c1ddbe8..2d4e5fc 100644 --- a/fs/btrfs/tree-log.c +++ b/fs/btrfs/tree-log.c @@ -4165,8 +4165,12 @@ static void fill_inode_item(struct btrfs_trans_handle *trans, btrfs_set_token_inode_sequence(&token, item, inode_peek_iversion(inode)); btrfs_set_token_inode_transid(&token, item, trans->transid); btrfs_set_token_inode_rdev(&token, item, inode->i_rdev); - flags = btrfs_inode_combine_flags(BTRFS_I(inode)->flags, - BTRFS_I(inode)->ro_flags); + /* + * clear memory only flag + */ + flags = btrfs_inode_combine_flags( + BTRFS_I(inode)->flags & ~BTRFS_INODE_ONLY_MEM_FLAG_MASK, + BTRFS_I(inode)->ro_flags); btrfs_set_token_inode_flags(&token, item, flags); btrfs_set_token_inode_block_group(&token, item, 0); } From patchwork Sun Jan 16 08:49:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhang X-Patchwork-Id: 12714515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D46B2C433F5 for ; Sun, 16 Jan 2022 08:49:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234486AbiAPIt0 (ORCPT ); Sun, 16 Jan 2022 03:49:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230421AbiAPItY (ORCPT ); Sun, 16 Jan 2022 03:49:24 -0500 Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D3D33C061574 for ; Sun, 16 Jan 2022 00:49:23 -0800 (PST) Received: by mail-pg1-x52c.google.com with SMTP id s1so6860736pga.5 for ; Sun, 16 Jan 2022 00:49:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id; bh=kj2rL3+EXNXNSigOfpfTRHKmQGvuAazq1uuleqK9hc4=; b=nYSiOhwq60nZpL7ya6n4prIhAgKdcxyJF3106vvVpkiez3NfgUcUluKJtQ+f7ZInYh DdwPgTXO2DA1Ne5KbID11Hgl6UtMInzk7tXI35b0ybVlf3ESHZ+4A5hR1/J1HWVcV6O5 KUey/5JjRvHK0OWfrzPUVowVucrP+MYqc2Y0pPV/ljCqrRBSBl16RqTjTf14TiMvbJoT cP6bEPJxM/gHYiGmpmwc7B8dNYzX0THUMTZ79IZ4afWGdr59F+hRus719Qq6HnGUg1K0 5DQMSKyRXwelQrM70BXbeUUw7p572NtdQbVsMWk8ftOS6bRhjX/9qHx1ULohvG8H9oTb IMfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=kj2rL3+EXNXNSigOfpfTRHKmQGvuAazq1uuleqK9hc4=; b=K3NmQP0pe8E6kear4TJZxRQYIuxKZTj8mygyEGLJz049DeKE6BwH1iN1bWK2adZGQE NAGpF4j6v0t5j1NBMGVrz3NrMe6cbwJ/bbp8U5bomMxOXt2I1ZPPrDSerpb8FHWR3YH/ zquOpln+yfT+zdeNe0XVEoSToVeT/g0UlP+zAmOMIjH5YxDdfcJvjNCtAmIj+m+IVfSR 5w3cWRGrlJNrQD7A5n56dInVEw2vXJykwluBnvRrP8BJ//iE50eKiAsGFMwWKXaEE4R7 tq4djwWdTHZeGkT9vutjYYq2mxsWTkZ62GxUfUGFN2mpp6BLINX3G+hWZNo7ZE5D2p2L fwPA== X-Gm-Message-State: AOAM530xoFR3ySPooHG3M1uyHm3LOdiKL5Msta2aBqzcavk2yGsPx9gs 21lIIS6hE5kqscR9jPCsVNjgXmLbDNIZr6Jc X-Google-Smtp-Source: ABdhPJwG1nScW0U1poVvLdybTIHPHszntwGbsoU1qcmlEwcNALEKyX28YJcJ6kBi3kIwO1X5Z9z8bA== X-Received: by 2002:a05:6a00:1905:b0:4a8:2f86:3f18 with SMTP id y5-20020a056a00190500b004a82f863f18mr16178208pfi.52.1642322963036; Sun, 16 Jan 2022 00:49:23 -0800 (PST) Received: from zllke.localdomain ([113.99.5.116]) by smtp.gmail.com with ESMTPSA id 10sm10549626pjc.6.2022.01.16.00.49.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 16 Jan 2022 00:49:22 -0800 (PST) From: Li Zhang To: linux-btrfs@vger.kernel.org Cc: zhanglikernel@gmail.com Subject: [PATCH 2/5] btrfs: Introduce helper functions for compression. Date: Sun, 16 Jan 2022 16:49:13 +0800 Message-Id: <1642322953-1856-1-git-send-email-zhanglikernel@gmail.com> X-Mailer: git-send-email 1.8.3.1 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org 1. Introduce the compression type and compression level combination function. Use the lowest 4 bits of unsigned int to record the compression type and 5 - 8 bits to record the compression level. 2. Introduce functions to parse compression description strings into filesystem flags and vice versa. Signed-off-by: Li Zhang --- fs/btrfs/compression.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++ fs/btrfs/compression.h | 14 ++++++++++++++ 2 files changed, 65 insertions(+) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index 71e5b2e..019ed2d 100644 --- a/fs/btrfs/compression.c +++ b/fs/btrfs/compression.c @@ -49,6 +49,38 @@ const char* btrfs_compress_type2str(enum btrfs_compression_type type) return NULL; } +/* + * convert str to combination of compression type and level + * ret: combination of compression type and compress level + * return type and level separately if caller specify type and level pointer + */ +unsigned int btrfs_compress_str2type_level(const char *str, unsigned int *type, unsigned int *level){ + int i; + unsigned int result_type, result_level; + size_t len = strlen(str); + result_type = BTRFS_COMPRESS_NONE; + result_level = 0; + for(i = 1; i < ARRAY_SIZE(btrfs_compress_types); i ++){ + size_t comp_len = strlen(btrfs_compress_types[i]); + if(len < comp_len){ + continue; + } + if(!strncmp(btrfs_compress_types[i], str, comp_len)){ + result_type = BTRFS_COMPRESS_NONE + i; + result_level = btrfs_compress_str2level(result_type, str + comp_len); + break; + } + } + if(type != NULL){ + *type = result_type; + } + if(level != NULL){ + *level = result_level; + } + return btrfs_compress_combine_type_level(result_type, result_level); +} + + bool btrfs_compress_is_valid_type(const char *str, size_t len) { int i; @@ -1888,3 +1920,22 @@ unsigned int btrfs_compress_str2level(unsigned int type, const char *str) return level; } + + +/* + * convert combination of compression type and level to meaningful string + * the caller should specify buffer and buffer size + */ +char *btrfs_compress_type_level2str(unsigned type, unsigned level, char *out, size_t out_len){ + switch (type) { + case BTRFS_COMPRESS_ZLIB: + case BTRFS_COMPRESS_LZO: + case BTRFS_COMPRESS_ZSTD: + case BTRFS_COMPRESS_NONE: + snprintf(out, out_len, "%s:%d", btrfs_compress_types[type], btrfs_compress_set_level(type, level)); + return out; + default: + break; + } + return NULL; +} diff --git a/fs/btrfs/compression.h b/fs/btrfs/compression.h index 56eef08..b76473c 100644 --- a/fs/btrfs/compression.h +++ b/fs/btrfs/compression.h @@ -76,6 +76,16 @@ static inline unsigned int btrfs_compress_level(unsigned int type_level) return ((type_level & 0xF0) >> 4); } +/* + * helper function combine compression type and level + * combine compression type and level in one byte + * the lower 4bit represent compression type + * the upper 4bit represent compression level + */ +static inline unsigned int btrfs_compress_combine_type_level(unsigned int type, unsigned level){ + return (level << 4) | type; +} + void __init btrfs_init_compress(void); void __cold btrfs_exit_compress(void); @@ -101,6 +111,10 @@ blk_status_t btrfs_submit_compressed_read(struct inode *inode, struct bio *bio, unsigned int btrfs_compress_str2level(unsigned int type, const char *str); +unsigned int btrfs_compress_str2type_level(const char *str, unsigned int * type, unsigned int *level); + +char *btrfs_compress_type_level2str(unsigned type, unsigned level, char *out, size_t out_len); + enum btrfs_compression_type { BTRFS_COMPRESS_NONE = 0, BTRFS_COMPRESS_ZLIB = 1, From patchwork Sun Jan 16 08:50:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhang X-Patchwork-Id: 12714516 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 920ACC433F5 for ; Sun, 16 Jan 2022 08:50:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234513AbiAPIuS (ORCPT ); Sun, 16 Jan 2022 03:50:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41736 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230421AbiAPIuR (ORCPT ); Sun, 16 Jan 2022 03:50:17 -0500 Received: from mail-pf1-x432.google.com (mail-pf1-x432.google.com [IPv6:2607:f8b0:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C86F1C061574 for ; Sun, 16 Jan 2022 00:50:17 -0800 (PST) Received: by mail-pf1-x432.google.com with SMTP id a5so6920873pfo.5 for ; Sun, 16 Jan 2022 00:50:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id; bh=zN+XDftrgAUHFMiUX9svS8GFo19XMARuamUVJNyydnc=; b=WA40ANAqHR25SxTE3F5rnerQBxi3aHLvquzS3PswkUgH8Au991IJ0ANg5AOc0IWztZ kkIzYsrgRmisaY/Ne9Ud4ZOt8Zgj/yoATefxBlJnSahegswQkb6rPS4PIPnNhmfpBBHu kHo2btJRiXhcUmuRdmXBsbV5mtt9MtZEkV4Bm1kLxr5KawDwW92UHvewn2AHro1995U9 NSkPb42Up1MvEd6GXNsRkkGywqpxh33XaFBSYK+3BGQyj9zD+8RP3+rd+y+jt+C6Y0WM NsYurx46O+DzY4+bLil+AcThUrl2YMRVAPMTvFpsrpsqBFiDmhjqYsveEU4z+y13w4pu PGpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=zN+XDftrgAUHFMiUX9svS8GFo19XMARuamUVJNyydnc=; b=05mxAoI8E7AJcPdbRqujiV52gwnmJA9Cd0BumiRhS9vTYgAYPUl6hIp9oWu+WKxe6G I7ZulYIya15fYDbJDP+WoC3SPtbBKHr2z7RI+JrStlUffuoXD2UH+drYgv3WzL6grUDL Tjfosqa9ZLoZkpgbGT0sg6icT8oedNDbfYHsb7I1t/eWIMXirmC7xwE2fUEtCoXEth6q dktaM05wk3X7skyUVEAzUb4lPGR6BY9SHedm5ozA0kdTtFC7Hq9782alXmdIPQDsZwrA R2ivjYNRY01fsC4a6w/48sQ68QnpTsFOJugKfhnLT9O/gZpfpXVv/kA2Z39txsn8PJBo 5e6Q== X-Gm-Message-State: AOAM530kuvO0iwoj3v/JwrMj0Fk1b9fYMU/zJfI1/SLYRJ9sl8WtfS7B P4pYghgkbaoDQmZb+WfvQ2hC9TWsFFZau6sX X-Google-Smtp-Source: ABdhPJy0HdpZnuLJrmB2CiizMTqGNZxDKTTsOiUnQJazYf1PTTaF8FpKXB5FMmHfd9spXXT1saEZUw== X-Received: by 2002:a05:6a00:1a94:b0:4c3:a8f8:1e46 with SMTP id e20-20020a056a001a9400b004c3a8f81e46mr3487020pfv.2.1642323016979; Sun, 16 Jan 2022 00:50:16 -0800 (PST) Received: from zllke.localdomain ([113.99.5.116]) by smtp.gmail.com with ESMTPSA id l2sm9849028pfe.189.2022.01.16.00.50.15 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 16 Jan 2022 00:50:16 -0800 (PST) From: Li Zhang To: linux-btrfs@vger.kernel.org Cc: zhanglikernel@gmail.com Subject: [PATCH 3/5] btrfs: Convert compression description strings into system flags. Date: Sun, 16 Jan 2022 16:50:09 +0800 Message-Id: <1642323009-1953-1-git-send-email-zhanglikernel@gmail.com> X-Mailer: git-send-email 1.8.3.1 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org 1. Calls the function in compress.c to Clean up the redundant code that parses the compression description string into system flags. 2. Convert the compressed description string to a combination of compression type and compression level. Signed-off-by: Li Zhang Reported-by: kernel test robot --- fs/btrfs/props.c | 52 +++++++++++++++++++++++---------- fs/btrfs/super.c | 89 ++++++++++++++++++++++++-------------------------------- 2 files changed, 74 insertions(+), 67 deletions(-) diff --git a/fs/btrfs/props.c b/fs/btrfs/props.c index 1a6d2d5..f07be37 100644 --- a/fs/btrfs/props.c +++ b/fs/btrfs/props.c @@ -272,13 +272,16 @@ static int prop_compression_apply(struct inode *inode, const char *value, { struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb); int type; + char* clean_value; + int ret; /* Reset to defaults */ if (len == 0) { BTRFS_I(inode)->flags &= ~BTRFS_INODE_COMPRESS; BTRFS_I(inode)->flags &= ~BTRFS_INODE_NOCOMPRESS; BTRFS_I(inode)->prop_compress = BTRFS_COMPRESS_NONE; - return 0; + ret = 0; + goto out; } /* Set NOCOMPRESS flag */ @@ -287,27 +290,44 @@ static int prop_compression_apply(struct inode *inode, const char *value, BTRFS_I(inode)->flags |= BTRFS_INODE_NOCOMPRESS; BTRFS_I(inode)->flags &= ~BTRFS_INODE_COMPRESS; BTRFS_I(inode)->prop_compress = BTRFS_COMPRESS_NONE; - - return 0; + ret = 0; + goto out; } - if (!strncmp("lzo", value, 3)) { - type = BTRFS_COMPRESS_LZO; - btrfs_set_fs_incompat(fs_info, COMPRESS_LZO); - } else if (!strncmp("zlib", value, 4)) { - type = BTRFS_COMPRESS_ZLIB; - } else if (!strncmp("zstd", value, 4)) { - type = BTRFS_COMPRESS_ZSTD; - btrfs_set_fs_incompat(fs_info, COMPRESS_ZSTD); - } else { - return -EINVAL; - } + /* + * The value contains gibberish at the end, we should clean it up. + */ + clean_value = kzalloc(len + 1, GFP_NOFS); + if(!clean_value){ + ret = -ENOMEM; + goto out; + } + strncpy(clean_value, value, len); + + if(!btrfs_compress_is_valid_type(value, len)){ + ret = -EINVAL; + goto free_clean_value; + } + type = btrfs_compress_str2type_level(clean_value, NULL, NULL); + switch(btrfs_compress_type(type)){ + case BTRFS_COMPRESS_LZO: + btrfs_set_fs_incompat(fs_info, COMPRESS_LZO); + break; + case BTRFS_COMPRESS_ZSTD: + btrfs_set_fs_incompat(fs_info, COMPRESS_ZSTD); + break; + default: + break; + } BTRFS_I(inode)->flags &= ~BTRFS_INODE_NOCOMPRESS; BTRFS_I(inode)->flags |= BTRFS_INODE_COMPRESS; BTRFS_I(inode)->prop_compress = type; - - return 0; + ret = 0; +free_clean_value: + kfree(clean_value); +out: + return ret; } static const char *prop_compression_extract(struct inode *inode) diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 0ec09fe..fea7e2d 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -574,6 +574,7 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options, int saved_compress_level; bool saved_compress_force; int no_compress = 0; + unsigned compress_type_flag, compress_level_flag; if (btrfs_fs_compat_ro(info, FREE_SPACE_TREE)) btrfs_set_opt(info->mount_opt, FREE_SPACE_TREE); @@ -660,62 +661,48 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options, saved_compress_force = btrfs_test_opt(info, FORCE_COMPRESS); saved_compress_level = info->compress_level; - if (token == Opt_compress || - token == Opt_compress_force || - strncmp(args[0].from, "zlib", 4) == 0) { - compress_type = "zlib"; - - info->compress_type = BTRFS_COMPRESS_ZLIB; - info->compress_level = BTRFS_ZLIB_DEFAULT_LEVEL; - /* - * args[0] contains uninitialized data since - * for these tokens we don't expect any - * parameter. - */ - if (token != Opt_compress && - token != Opt_compress_force) - info->compress_level = - btrfs_compress_str2level( - BTRFS_COMPRESS_ZLIB, - args[0].from + 4); - btrfs_set_opt(info->mount_opt, COMPRESS); - btrfs_clear_opt(info->mount_opt, NODATACOW); - btrfs_clear_opt(info->mount_opt, NODATASUM); - no_compress = 0; - } else if (strncmp(args[0].from, "lzo", 3) == 0) { - compress_type = "lzo"; - info->compress_type = BTRFS_COMPRESS_LZO; + if( token == Opt_compress || token == Opt_compress_force ){ + /* + * defaule compression type + */ + compress_type = "zlib"; + } else { + compress_type = args[0].from; + } + if(!strncmp(compress_type, "no", 2)){ + /* + * Handle no compression specially + */ + btrfs_clear_opt(info->mount_opt, COMPRESS); + btrfs_clear_opt(info->mount_opt, FORCE_COMPRESS); + compress_force = false; + info->compress_type = 0; info->compress_level = 0; + no_compress++; + } else { + if(!btrfs_compress_is_valid_type( + compress_type, strlen(compress_type))){ + ret = -EINVAL; + goto out; + } + btrfs_compress_str2type_level(compress_type, + &compress_type_flag, &compress_level_flag); + switch(compress_type_flag){ + case BTRFS_COMPRESS_ZSTD: + btrfs_set_fs_incompat(info, COMPRESS_ZSTD); + break; + case BTRFS_COMPRESS_LZO: + btrfs_set_fs_incompat(info, COMPRESS_LZO); + break; + } btrfs_set_opt(info->mount_opt, COMPRESS); btrfs_clear_opt(info->mount_opt, NODATACOW); btrfs_clear_opt(info->mount_opt, NODATASUM); - btrfs_set_fs_incompat(info, COMPRESS_LZO); - no_compress = 0; - } else if (strncmp(args[0].from, "zstd", 4) == 0) { - compress_type = "zstd"; - info->compress_type = BTRFS_COMPRESS_ZSTD; - info->compress_level = - btrfs_compress_str2level( - BTRFS_COMPRESS_ZSTD, - args[0].from + 4); - btrfs_set_opt(info->mount_opt, COMPRESS); - btrfs_clear_opt(info->mount_opt, NODATACOW); - btrfs_clear_opt(info->mount_opt, NODATASUM); - btrfs_set_fs_incompat(info, COMPRESS_ZSTD); no_compress = 0; - } else if (strncmp(args[0].from, "no", 2) == 0) { - compress_type = "no"; - info->compress_level = 0; - info->compress_type = 0; - btrfs_clear_opt(info->mount_opt, COMPRESS); - btrfs_clear_opt(info->mount_opt, FORCE_COMPRESS); - compress_force = false; - no_compress++; - } else { - ret = -EINVAL; - goto out; - } - + info->compress_type = compress_type_flag; + info->compress_level = compress_level_flag; + } + if (compress_force) { btrfs_set_opt(info->mount_opt, FORCE_COMPRESS); } else { From patchwork Sun Jan 16 08:50:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhang X-Patchwork-Id: 12714517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42E38C433F5 for ; Sun, 16 Jan 2022 08:51:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234528AbiAPIvM (ORCPT ); Sun, 16 Jan 2022 03:51:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41934 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230421AbiAPIvM (ORCPT ); Sun, 16 Jan 2022 03:51:12 -0500 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC08FC061574 for ; Sun, 16 Jan 2022 00:51:11 -0800 (PST) Received: by mail-pj1-x102f.google.com with SMTP id y16-20020a17090a6c9000b001b13ffaa625so26516196pjj.2 for ; Sun, 16 Jan 2022 00:51:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id; bh=Aj8k9+/OKTeZFkRxHYPidLp3FJBXLDv6NXCNj/xJqMw=; b=c1B7NegVS6xaFrxmZ07aCC/AzCZunkf/NG2gi+ngiciFV/U53SK4W1GJmsIJcx369e MWRJPRHZcx+ukAjWtfO9uVgsFCBwf2pmbjhTeVg1CjBWJ0+ybA/aY4Va4Dxq7eCe8uG1 OfWdAssZ0dHzkTwgVcMmQHlmRMSr0sJf0hNWzOOC2WXgoaN3pD5qmjoTMQNM46qW5Kqg 7lqkKv+NIvcXUjAt0aU5U94Sh+RXBSk42vG8Rtk1bERLPtEn2/UXMke2N+zqkCAhuepB 5SQbLwUeQjOIQ7tTZDsNRVL+gAiw//5iOW/GY/uFGyPjfAXGpXx1/shQP56rzdADxv+t wDRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=Aj8k9+/OKTeZFkRxHYPidLp3FJBXLDv6NXCNj/xJqMw=; b=Qu2pnmrAQPfQ+XYHLEuPmAUQ5hy7wER5lYtAZj2wd+DSLcfp1N9KQ/zXD2Ue0N8k4m yATWjD67b8Re2awg/0kF0RPDHeBmtbU4Ymdu46icBQC9TxC/zJvaeF5H4b1MffnDc2/o OHIXEV7uZL7nA14xY7DEbHtHkohE8ElBpsET1FTL7IPIcOcJ6gohrFEUYCHNHfaMbKcc GQ2jil+wHg5vODXDOAotpmAAVxlVwdmnTvLs0WURn4abNyuoj9rh1ieg52LwzCskHurL Czsgtl6SBsK28+uD0pr19HsIaxxIfkODzR4VKxeWxwmzz6gHOkhy+8bDHljkHlx7rDOB W4hg== X-Gm-Message-State: AOAM5327M6Lb4SGZdBbQkN1kJ8wIw/dAovMFPclrPxrypUQVHBiUBM5v g5VJ4mVw8Tik/WhK+MC4XCjnq8zUhMTEc4yf X-Google-Smtp-Source: ABdhPJwCascWQNh94M4afHF++/YsWkA/FAQh75rbtCC/Ld8c3wfRoHU93uhQVMZ0cs3QgImju1cvhQ== X-Received: by 2002:a17:903:191:b0:14a:5a6d:e86e with SMTP id z17-20020a170903019100b0014a5a6de86emr16876847plg.111.1642323071018; Sun, 16 Jan 2022 00:51:11 -0800 (PST) Received: from zllke.localdomain ([113.99.5.116]) by smtp.gmail.com with ESMTPSA id c21sm667787pgw.41.2022.01.16.00.51.09 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 16 Jan 2022 00:51:10 -0800 (PST) From: Li Zhang To: linux-btrfs@vger.kernel.org Cc: zhanglikernel@gmail.com Subject: [PATCH 4/5] btrfs: Synchronize compression flag BTRFS_INODE_NOCOMPRESS with xattr. Date: Sun, 16 Jan 2022 16:50:57 +0800 Message-Id: <1642323057-2044-1-git-send-email-zhanglikernel@gmail.com> X-Mailer: git-send-email 1.8.3.1 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org 1. If the btrfs_inode flag is set to BTRFS_INODE_NOCOMPRESS, insert axttr btrfs.compression=none. Signed-off-by: Li Zhang --- fs/btrfs/ioctl.c | 36 ++++++++++++++++++++++++++---------- 1 file changed, 26 insertions(+), 10 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index a5bd692..7036463 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -223,7 +223,8 @@ int btrfs_fileattr_set(struct user_namespace *mnt_userns, struct btrfs_trans_handle *trans; unsigned int fsflags, old_fsflags; int ret; - const char *comp = NULL; + const char *comp; + char comp_xattr_str[512]; u32 binode_flags; if (btrfs_root_readonly(root)) @@ -304,13 +305,19 @@ int btrfs_fileattr_set(struct user_namespace *mnt_userns, } /* - * The COMPRESS flag can only be changed by users, while the NOCOMPRESS - * flag may be changed automatically if compression code won't make - * things smaller. + * The COMPRESS and NOCOMPRESS flag can only be changed by users, while the + * BTRFS_INODE_HEURISTIC_NOCOMPRESS flag may be changed automatically if + * compression code won't make things smaller. */ + memset(comp_xattr_str, 0, sizeof(comp_xattr_str)); if (fsflags & FS_NOCOMP_FL) { binode_flags &= ~BTRFS_INODE_COMPRESS; binode_flags |= BTRFS_INODE_NOCOMPRESS; + /* + * set xattr btrfs.compression to none if user set the NOCOMPRESS flag + */ + strncpy(comp_xattr_str, "none", 5); + comp = comp_xattr_str; } else if (fsflags & FS_COMPR_FL) { if (IS_SWAPFILE(inode)) @@ -318,10 +325,19 @@ int btrfs_fileattr_set(struct user_namespace *mnt_userns, binode_flags |= BTRFS_INODE_COMPRESS; binode_flags &= ~BTRFS_INODE_NOCOMPRESS; - - comp = btrfs_compress_type2str(fs_info->compress_type); - if (!comp || comp[0] == 0) - comp = btrfs_compress_type2str(BTRFS_COMPRESS_ZLIB); + /* + * set xattr btrfs.compression to filesystem specify compression type:level + */ + comp = btrfs_compress_type_level2str(fs_info->compress_type, + fs_info->compress_level, comp_xattr_str, sizeof(comp_xattr_str)); + /* + * filesystem doesn't specify compression method, use default compression type + */ + if(!comp || comp[0] == ':') { + btrfs_compress_type_level2str(BTRFS_COMPRESS_ZLIB, + BTRFS_ZLIB_DEFAULT_LEVEL, comp_xattr_str, + sizeof(comp_xattr_str)); + } } else { binode_flags &= ~(BTRFS_INODE_COMPRESS | BTRFS_INODE_NOCOMPRESS); } @@ -335,8 +351,8 @@ int btrfs_fileattr_set(struct user_namespace *mnt_userns, return PTR_ERR(trans); if (comp) { - ret = btrfs_set_prop(trans, inode, "btrfs.compression", comp, - strlen(comp), 0); + ret = btrfs_set_prop(trans, inode, "btrfs.compression", comp_xattr_str, + strlen(comp_xattr_str), 0); if (ret) { btrfs_abort_transaction(trans, ret); goto out_end_trans; From patchwork Sun Jan 16 08:51:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhang X-Patchwork-Id: 12714518 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 120F3C433F5 for ; Sun, 16 Jan 2022 08:51:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234536AbiAPIve (ORCPT ); Sun, 16 Jan 2022 03:51:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230421AbiAPIvd (ORCPT ); Sun, 16 Jan 2022 03:51:33 -0500 Received: from mail-pf1-x432.google.com (mail-pf1-x432.google.com [IPv6:2607:f8b0:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C5FCCC061574 for ; Sun, 16 Jan 2022 00:51:33 -0800 (PST) Received: by mail-pf1-x432.google.com with SMTP id f144so6163246pfa.6 for ; Sun, 16 Jan 2022 00:51:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id; bh=ZA5kyCBNMxZ16dMYBW2Z9FsXO+BvPxsy2eLlwspomlw=; b=hVugHvCfyveGVHi8HOHOh+QvQtZY2TcKWc2shZCmdBpRh4pma9kArS2Xr6njNjfOpY nRBG8OEBYOVctol4vHx2lbDg83yyd5KBrBzEV3Rxsi1snt5DWuE2LkoRTXvBIY/rbgTy jdPdD2IJtw0rODYwyqMLP/6HRFNyItk6gu82kg/2LqzwWSfXNDbuU89FuYYblOMk8d21 SwnXlQdszCB+5pF/r3RoK/ioCAcvxrf6TEiJUu/Q0wUoKFsugTFylHP+GMktuBOs6gP8 TSXT7+5LaXQj798qA9vIsQX+iyTCx+aPNOdt/iUby8FEWuLg/f6724E821iJ4tTJWAVd Ml4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=ZA5kyCBNMxZ16dMYBW2Z9FsXO+BvPxsy2eLlwspomlw=; b=f7tlUBJ2ZrWW2e5PtkfgDvXTWNDEd7vqk7txMmLQkbh8Lo+bjrTHAa2bkovrL4i8zV SAYihdLRdMEZ9aL0iGAV5JU78YwXPMlhNR6mczmGlQ2wYhBa36VEzRRjxC4e8216CBEu 1AejTl1f80Hdr31v9wlne8IF5vXysd3N8EA8AT6/opdLe8ta8AF0nEnu6IavPbF6S+xk M2i2IQHLC7b2iZSdZvGWKiS/z7ZLfRjxkNihbSUB9k82XeRqRJjrK3N6Bh8gi6NxkR3o XoQtHQYJxCKkNecGdNdbvku3xRLcvLwjmsSPbVaJbMF3lXuG27fc6mPch24nNlgEOzZE atUQ== X-Gm-Message-State: AOAM532JR0P1HSJxht20v9AtOYlbhOQ5qC628lpla+CXHoPtmEMVO/M1 DcN7hv1Bd7wBos9jaeitqetppcbS/00SKQCz X-Google-Smtp-Source: ABdhPJxvYUw78fcUpJ7j7hX18s4xN27yJhd+MySr4AkjSbHSwVpM+qz6p6WbQh+et2t4xMve8dcIng== X-Received: by 2002:a63:9e01:: with SMTP id s1mr14513392pgd.45.1642323093004; Sun, 16 Jan 2022 00:51:33 -0800 (PST) Received: from zllke.localdomain ([113.99.5.116]) by smtp.gmail.com with ESMTPSA id h3sm6484560pfg.18.2022.01.16.00.51.31 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 16 Jan 2022 00:51:32 -0800 (PST) From: Li Zhang To: linux-btrfs@vger.kernel.org Cc: zhanglikernel@gmail.com Subject: [PATCH 5/5] btrfs: Compresses files using a combination of compression type and compression level. Date: Sun, 16 Jan 2022 16:51:23 +0800 Message-Id: <1642323083-2088-1-git-send-email-zhanglikernel@gmail.com> X-Mailer: git-send-email 1.8.3.1 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org 1. If the file is being defragmented, combine defrag_compress and fs_info compression_level (defrag_compress will be extended in the future). 2. Extract the compression type btrfs_inode using and write to extent, because decompression does not need to know the compression level. Signed-off-by: Li Zhang --- fs/btrfs/inode.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 3fe485b..fb44899 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -602,7 +602,8 @@ static noinline int compress_file_range(struct async_chunk *async_chunk) unsigned long total_in = 0; int i; int will_compress; - int compress_type = fs_info->compress_type; + int compress_type = btrfs_compress_combine_type_level( + fs_info->compress_type, fs_info -> compress_level); int compressed_extents = 0; int redirty = 0; @@ -683,7 +684,8 @@ static noinline int compress_file_range(struct async_chunk *async_chunk) } if (BTRFS_I(inode)->defrag_compress) - compress_type = BTRFS_I(inode)->defrag_compress; + compress_type = btrfs_compress_combine_type_level( + BTRFS_I(inode)->defrag_compress, fs_info->compress_level); else if (BTRFS_I(inode)->prop_compress) compress_type = BTRFS_I(inode)->prop_compress; @@ -706,7 +708,7 @@ static noinline int compress_file_range(struct async_chunk *async_chunk) /* Compression level is applied here and only here */ ret = btrfs_compress_pages( - compress_type | (fs_info->compress_level << 4), + compress_type, inode->i_mapping, start, pages, &nr_pages, @@ -743,7 +745,7 @@ static noinline int compress_file_range(struct async_chunk *async_chunk) /* try making a compressed inline extent */ ret = cow_file_range_inline(BTRFS_I(inode), start, end, total_compressed, - compress_type, pages); + btrfs_compress_type(compress_type), pages); } if (ret <= 0) { unsigned long clear_flags = EXTENT_DELALLOC | @@ -808,10 +810,12 @@ static noinline int compress_file_range(struct async_chunk *async_chunk) * The async work queues will take care of doing actual * allocation on disk for these compressed pages, and * will submit them to the elevator. + * It only need to record compression type, because decompress don't need + * to know compression level */ add_async_extent(async_chunk, start, total_in, total_compressed, pages, nr_pages, - compress_type); + btrfs_compress_type(compress_type)); if (start + total_in < end) { start += total_in;