[PATCHv4,0/9] zram: Support multiple compression streams

Message

Sergey Senozhatsky Oct. 18, 2022, 4:55 a.m. UTC

Hello,

	This series adds support for multiple (per-CPU)
compression streams (at point only 2). The main idea is that
different compression algorithms have different characteristics
and zram may benefit when it uses a combination of algorithms:
a default algorithm that is faster but have lower compression
rate and a secondary algorithm that can use higher compression
rate at a price of slower compression/decompression.

	There are several use-case for this functionality:

- huge pages re-compression: zstd or deflate can successfully
compress huge pages (~50% of huge pages on my synthetic ChromeOS
tests), IOW pages that lzo was not able to compress.

- idle pages re-compression: idle/cold pages sit in the memory
and we may reduce zsmalloc memory usage if we recompress those
idle pages.

	User-space has a number of ways to control the behavior
and impact of zram recompression: what type of pages should be
recompressed, size watermarks, etc. Please refer to documentation
patch.

v4:
-- added IS_ERR_VALUE patch (Andrew)
-- documented SIZE units (bytes) (Andrew)
-- re-phrased writeback BIO error comment (Andrew)
-- return zs_malloc() error code from zram_recompress()
-- do not lose zram_recompress() error in recompress_store()
-- corrected a typo
-- fixed previous rebase errors
-- rebased the series

v3:
-- conditionally reschedule during recompression loop so that
   we don't stall RCU grace periods
-- fixed a false-positive WARN_ON

v2:
-- rebased
-- mark completely incompressible pages (neither default nor secondary
   algorithm can compress them) with a new flag so that we don't attempt
   to recompress them all the time

Sergey Senozhatsky (9):
  zram: Preparation for multi-zcomp support
  zram: Add recompression algorithm sysfs knob
  zram: Factor out WB and non-WB zram read functions
  zram: Introduce recompress sysfs knob
  documentation: Add recompression documentation
  zram: Add recompression algorithm choice to Kconfig
  zram: Add recompress flag to read_block_state()
  zram: Clarify writeback_store() comment
  zram: Use IS_ERR_VALUE() to check for zs_malloc() errors

 Documentation/admin-guide/blockdev/zram.rst |  64 ++-
 drivers/block/zram/Kconfig                  |  55 +++
 drivers/block/zram/zcomp.c                  |   6 +-
 drivers/block/zram/zcomp.h                  |   2 +-
 drivers/block/zram/zram_drv.c               | 458 +++++++++++++++++---
 drivers/block/zram/zram_drv.h               |  16 +-
 6 files changed, 526 insertions(+), 75 deletions(-)

Comments

Minchan Kim Nov. 2, 2022, 8:07 p.m. UTC | #1

On Tue, Oct 18, 2022 at 01:55:24PM +0900, Sergey Senozhatsky wrote:
> Hello,
> 
> 	This series adds support for multiple (per-CPU)
> compression streams (at point only 2). The main idea is that
> different compression algorithms have different characteristics
> and zram may benefit when it uses a combination of algorithms:
> a default algorithm that is faster but have lower compression
> rate and a secondary algorithm that can use higher compression
> rate at a price of slower compression/decompression.
> 
> 	There are several use-case for this functionality:
> 
> - huge pages re-compression: zstd or deflate can successfully
> compress huge pages (~50% of huge pages on my synthetic ChromeOS
> tests), IOW pages that lzo was not able to compress.
> 
> - idle pages re-compression: idle/cold pages sit in the memory
> and we may reduce zsmalloc memory usage if we recompress those
> idle pages.
> 
> 	User-space has a number of ways to control the behavior
> and impact of zram recompression: what type of pages should be
> recompressed, size watermarks, etc. Please refer to documentation
> patch.

Hi Sergey,

First of all, I am really sorry to attend the party too late.

I absolutely agree the feature is really useful and even I am
thinking to support multiple comression trials on the fly for
future. So I'd like to introduce the feature more general shape
to be extended later so review will go.

Thanks!

Sergey Senozhatsky Nov. 3, 2022, 3:36 a.m. UTC | #2

Hi Minchan,

On (22/11/02 13:07), Minchan Kim wrote:
[..]
> Hi Sergey,
> 
> First of all, I am really sorry to attend the party too late.
> 
> I absolutely agree the feature is really useful and even I am
> thinking to support multiple comression trials on the fly for
> future. So I'd like to introduce the feature more general shape
> to be extended later so review will go.

On the fly recompression (from the same context) was what I had as
a first version (which was internal and was never published), and
we didn't like it at all. It's too limited, has zero flexibility,
zero extensibility and has too high of a price tag attached to it
(in terms of CPU cycles, power and time).

So we moved it to user-space (deferred context) and this unlocked
numerous possibilities: recompress only when we really need to and,
more importantly, can afford it; wire in numerous metrics: battery,
CPU load etc. And many more.