From patchwork Fri Feb 14 04:50:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5461AC02198 for ; Fri, 14 Feb 2025 04:52:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D2AA56B0088; Thu, 13 Feb 2025 23:52:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CB52B6B0085; Thu, 13 Feb 2025 23:52:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ADE29280001; Thu, 13 Feb 2025 23:52:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 868A06B0083 for ; Thu, 13 Feb 2025 23:52:28 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 34179161CFB for ; Fri, 14 Feb 2025 04:52:28 +0000 (UTC) X-FDA: 83117329176.18.E26F16F Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf04.hostedemail.com (Postfix) with ESMTP id 5933C40006 for ; Fri, 14 Feb 2025 04:52:26 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=f5NIvtQG; spf=pass (imf04.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.52 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508746; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8Ip299Sr+pOGhTseTfOOpc4VtCVbO+2qU2TEqvTRYlA=; b=wOpEcYgeJeAOOyw9E5uELp9tNoybHuewNoY2Z7NWQIlxrRlWECigM+1P1hHBfHtE6nkFFA itg8KFoPIsDkYPzgvlnrKmQp+anmPtOYnFJed7Dq3anrzBqgNB0LO9pmdeUXK6TRX0rRDu BL2sXoLSBQu2V+wizEwX/oCbVA5TlmY= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=f5NIvtQG; spf=pass (imf04.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.52 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508746; a=rsa-sha256; cv=none; b=E2sB6pc/2BIMgUEItWYnpNYweyQ7ufUOpLxyDTbBpZaD6pfBJAeNB6caKnfYtkYoIo97V2 lv9zG6o20p6NaOhlBXyAkV2/X4HtFNmdyEHd1ZkjjJPvLf1+qXUz3KJ0GZ7993EwfqcM2y vaLigts3J7xpn4Xu3Pr9SDj0xNcxyo0= Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-2f44353649aso2467399a91.0 for ; Thu, 13 Feb 2025 20:52:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508745; x=1740113545; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8Ip299Sr+pOGhTseTfOOpc4VtCVbO+2qU2TEqvTRYlA=; b=f5NIvtQGf8rXGJPC3/PTrsuPim+iGKiqUSPRdzT4Rwqlt0nIXN+6mMFk/A3qA0l3J9 xPDU1OIpqLR9CPBZ6BYVYjUg0ouqfJ8CnoVoS3C6w5Y4vPltj9FB6P8FRPsEdgtiXhqT DniXRg/OCeq7nvCHAivPgpSjLHXBizirC7WA0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508745; x=1740113545; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8Ip299Sr+pOGhTseTfOOpc4VtCVbO+2qU2TEqvTRYlA=; b=UGS++HWSCtOWeyGx78hNtdi1UqscqPN7hHf3XUK3kBMPKivVj4ao6+T/U9bhp8HvJ/ +zpSAnab4w7e5Kr2bYr6PT0Mowu/aUCEJBvdwTxixhXke75mYBy3j5Xu5Eujf9qLi+9y lWrkiw/5t7ymTeKR+mCd9g7dugLnwduqfC7XHAmYyFWeo4kIT8nwuFP+HCTOJmVWwkYs l78nivsERwI590vKMv8Oq64HSS9nLby45jicSA6NdjKNmBd2fPkoCnK8gLPCOQfrSBrg uE2pFsFPjTkl++VM2xzStjRuvzdBXDOEP/FLz27NhKYn560dwiBEfwKuqsiPx1E8ophg oATg== X-Forwarded-Encrypted: i=1; AJvYcCV/ne/kUj0RClvQ5Z+p2oSJGK1VKtyZ2PiWh3/qzQdomU4PjUKi1iDF3xIS84cnHmC+n90/k7lUNw==@kvack.org X-Gm-Message-State: AOJu0Ywpvb5adgcT4QtVDcRv00rACIIKW6sebXDIa6c1qiFn8saF5UUp +Gzf3vzZw3QGINPAj7iP7a8k+E5HLtUwH0jdxxklDIg/1uqtwZV5a0A/pCpHhw== X-Gm-Gg: ASbGncud5H9z1sV1L17P1v9uPVteGI5bZHOkKH4Xvb/UdyJZXR2z9TTOL0uz9QuoSBb AH0abyf3jH607ymWNxfOtfzK4BdHlEZ6nV48gVgQE/gz3PhakupBm2tG6AXzJyj00xSbLGb/ZWV YVdt46+EHx+yugHVNcXUB93X4ibo0R3EQGjnZYiM+z/aniTQEIrMMnH38UhiiuOIaSEFIlgHii6 QZuYe1JWlbc9XYpGYhU6PYgcOawMR6Cyay8Mz3qDL6ZzTLDtX3JXejRo0SGzsVb2tX6OSThnLzw w4O5xJvDgMRbQfoOEg== X-Google-Smtp-Source: AGHT+IElhSM58kSia8Ygok9EIOPYpgVDSk1Jw5b34RsIuUpVrp592z+ExnE3ByHbPmlZ4fx+OfAh+g== X-Received: by 2002:a05:6a00:3989:b0:729:a31:892d with SMTP id d2e1a72fcca58-7322c591baemr18875311b3a.8.1739508745187; Thu, 13 Feb 2025 20:52:25 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-7324f64a39asm588836b3a.69.2025.02.13.20.52.22 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:52:24 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 01/17] zram: sleepable entry locking Date: Fri, 14 Feb 2025 13:50:13 +0900 Message-ID: <20250214045208.1388854-2-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 5933C40006 X-Stat-Signature: 1uc5tar1b4id9gkqk6z7y661pr13u4ka X-HE-Tag: 1739508746-123828 X-HE-Meta: U2FsdGVkX1/4z5I2Uv5TmFmBRUzVkPyjwL1dLfd3SNbUgDIC31+pHQ+RUGcqdCscyfE/fOnBJe+OjayuTAAjQBv6mzjW4su2J33cxA+4Po38jHk77i8dUNrpPTpULceW6qWvanhxF6v6TrUwl5QXuOPLPi8t5ne1qnmJ6LwZ1Liztfd2QZZUbPyFdVWARBVcQo4WTn7oSIh+lSNy89Dx5M5L5ZYB2eN9mq0kNNnril3/G7RTBTaQEgndzTAw77sj1Bff/K4gzQgdUm6BqzE6MpWcr17Hfk/i27n0oFe4hHc51R1XNeVsYLmhZHminPIazq3H8vfolf1fH8fKUTdfpO0tpnkQg4WMUwhKuuvtx6NYAV1Ub0GN/4bA/0SBnhJKWhUkRpI60i+orjWvYWMfy8P2mg5aAAoh64y8sxkwxnxMEt/wtZSR2czS/qg8Nmla7zdT6UD4mE+J/Xe1ykma/RRChVb7cLoRX3gfDxyCugz3mwlSrVpKQ1lNUgcqTzwoYNfGNg0OybIzV871ZtAqX2/qV0y1j87MzssXnfe5+j2Ofshdx0oCArFBtEHmkJ+OrJOH8EIjdJ/MlILL1/2SWdd52+DDOrp/l58wW2oCX43OdcwQmtfLSaX9aMVsR9zY9E/FKHpnNIBxafHFX3h7SpwK/ZOO+72OVj9DX2Tmy7GM2IvLmpMu7k5ZMA+tUlqjyXUJMjx+qg45He7U8sJrxdYeLSgcsEXvnx3wrgRqjuQWG8hi4uEmHfwcqehXhsoxfH3IhKcDrKi0qM0cIN/QN4KxZINsKob3h20ltUm1Lqtv91j+CvkywW7/DHmqBXvEbd+gWud3eC86FAvHTMVo/3ONZux2UtVsdO7zLPtitb3pr96ZZa3pePGuZDR0bnQMV41ZqO2/UNwTBjVykqUyxL7RVpm6woejiUIq0MhzkDXLkVtrL67J64rsda3mqemioDjaPNlWWe+TMfSKM/f PapWGUxa og+aiHa+iSA/5qX/Y4QvEt8VksRR77yFpWyn5J50Wn0pd8Ic6j4XnCIxu0RyW8kksOD/QnIvBAUZDv8Trh/BS1yjsD1RuGzXyqVOFfxO1CzPfDshOyWKTRcKlTV4Ozw/82Bxvoxc+oJmKQfNst2F+JXU08lTzvMnEBrGiv7LDkHn40G6xh90NrKjWR+RCkTp3i9cOqN+x1/9mC2hfDwYdozmsFJ4hhkwv9eONiJefTumL2cr43gpVg+ZdLyUluyr5SHUiXLJEZ0rot6pyZOTLvJ6WB7rNqyAAOJR0KgZSxVyYX7tapEL8V3UlJe+6OeclIifyICpwVz9BTRaRIkisfIEQ78SRV9EknOwE9LvZVyxt4rMrXGiTFaObyeQTg+tcTByu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Concurrent modifications of meta table entries is now handled by per-entry spin-lock. This has a number of shortcomings. First, this imposes atomic requirements on compression backends. zram can call both zcomp_compress() and zcomp_decompress() under entry spin-lock, which implies that we can use only compression algorithms that don't schedule/sleep/wait during compression and decompression. This, for instance, makes it impossible to use some of the ASYNC compression algorithms (H/W compression, etc.) implementations. Second, this can potentially trigger watchdogs. For example, entry re-compression with secondary algorithms is performed under entry spin-lock. Given that we chain secondary compression algorithms and that some of them can be configured for best compression ratio (and worst compression speed) zram can stay under spin-lock for quite some time. Having a per-entry mutex (or, for instance, a rw-semaphore) significantly increases sizeof() of each entry and hence the meta table. Therefore entry locking returns back to bit locking, as before, however, this time also preempt-rt friendly, because if waits-on-bit instead of spinning-on-bit. Lock owners are also now permitted to schedule, which is a first step on the path of making zram non-atomic. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 105 ++++++++++++++++++++++++++++++---- drivers/block/zram/zram_drv.h | 20 +++++-- 2 files changed, 108 insertions(+), 17 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 9f5020b077c5..65e16117f2db 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -58,19 +58,99 @@ static void zram_free_page(struct zram *zram, size_t index); static int zram_read_from_zspool(struct zram *zram, struct page *page, u32 index); -static int zram_slot_trylock(struct zram *zram, u32 index) +static void zram_slot_lock_init(struct zram *zram, u32 index) { - return spin_trylock(&zram->table[index].lock); +#ifdef CONFIG_DEBUG_LOCK_ALLOC + lockdep_init_map(&zram->table[index].dep_map, + "zram->table[index].lock", + &zram->lock_class, 0); +#endif +} + +#ifdef CONFIG_DEBUG_LOCK_ALLOC +static inline bool __slot_trylock(struct zram *zram, u32 index) +{ + struct lockdep_map *dep_map = &zram->table[index].dep_map; + unsigned long *lock = &zram->table[index].flags; + + if (!test_and_set_bit_lock(ZRAM_ENTRY_LOCK, lock)) { + mutex_acquire(dep_map, 0, 1, _RET_IP_); + lock_acquired(dep_map, _RET_IP_); + return true; + } + + lock_contended(dep_map, _RET_IP_); + return false; +} + +static inline void __slot_lock(struct zram *zram, u32 index) +{ + struct lockdep_map *dep_map = &zram->table[index].dep_map; + unsigned long *lock = &zram->table[index].flags; + + mutex_acquire(dep_map, 0, 0, _RET_IP_); + wait_on_bit_lock(lock, ZRAM_ENTRY_LOCK, TASK_UNINTERRUPTIBLE); + lock_acquired(dep_map, _RET_IP_); +} + +static inline void __slot_unlock(struct zram *zram, u32 index) +{ + struct lockdep_map *dep_map = &zram->table[index].dep_map; + unsigned long *lock = &zram->table[index].flags; + + mutex_release(dep_map, _RET_IP_); + clear_and_wake_up_bit(ZRAM_ENTRY_LOCK, lock); +} +#else +static inline bool __slot_trylock(struct zram *zram, u32 index) +{ + unsigned long *lock = &zram->table[index].flags; + + if (!test_and_set_bit_lock(ZRAM_ENTRY_LOCK, lock)) + return true; + return false; +} + +static inline void __slot_lock(struct zram *zram, u32 index) +{ + unsigned long *lock = &zram->table[index].flags; + + wait_on_bit_lock(lock, ZRAM_ENTRY_LOCK, TASK_UNINTERRUPTIBLE); +} + +static inline void __slot_unlock(struct zram *zram, u32 index) +{ + unsigned long *lock = &zram->table[index].flags; + + clear_and_wake_up_bit(ZRAM_ENTRY_LOCK, lock); +} +#endif /* CONFIG_DEBUG_LOCK_ALLOC */ + +/* + * entry locking rules: + * + * 1) Lock is exclusive + * + * 2) lock() function can sleep waiting for the lock + * + * 3) Lock owner can sleep + * + * 4) Use TRY lock variant when in atomic context + * - must check return value and handle locking failers + */ +static __must_check bool zram_slot_trylock(struct zram *zram, u32 index) +{ + return __slot_trylock(zram, index); } static void zram_slot_lock(struct zram *zram, u32 index) { - spin_lock(&zram->table[index].lock); + return __slot_lock(zram, index); } static void zram_slot_unlock(struct zram *zram, u32 index) { - spin_unlock(&zram->table[index].lock); + return __slot_unlock(zram, index); } static inline bool init_done(struct zram *zram) @@ -93,7 +173,6 @@ static void zram_set_handle(struct zram *zram, u32 index, unsigned long handle) zram->table[index].handle = handle; } -/* flag operations require table entry bit_spin_lock() being held */ static bool zram_test_flag(struct zram *zram, u32 index, enum zram_pageflags flag) { @@ -1473,15 +1552,11 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize) huge_class_size = zs_huge_class_size(zram->mem_pool); for (index = 0; index < num_pages; index++) - spin_lock_init(&zram->table[index].lock); + zram_slot_lock_init(zram, index); + return true; } -/* - * To protect concurrent access to the same index entry, - * caller should hold this table index entry's bit_spinlock to - * indicate this index entry is accessing. - */ static void zram_free_page(struct zram *zram, size_t index) { unsigned long handle; @@ -2625,6 +2700,10 @@ static int zram_add(void) if (ret) goto out_cleanup_disk; +#ifdef CONFIG_DEBUG_LOCK_ALLOC + lockdep_register_key(&zram->lock_class); +#endif + zram_debugfs_register(zram); pr_info("Added device: %s\n", zram->disk->disk_name); return device_id; @@ -2681,6 +2760,10 @@ static int zram_remove(struct zram *zram) */ zram_reset_device(zram); +#ifdef CONFIG_DEBUG_LOCK_ALLOC + lockdep_unregister_key(&zram->lock_class); +#endif + put_disk(zram->disk); kfree(zram); return 0; diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index db78d7c01b9a..794c9234e627 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -28,7 +28,6 @@ #define ZRAM_SECTOR_PER_LOGICAL_BLOCK \ (1 << (ZRAM_LOGICAL_BLOCK_SHIFT - SECTOR_SHIFT)) - /* * ZRAM is mainly used for memory efficiency so we want to keep memory * footprint small and thus squeeze size and zram pageflags into a flags @@ -46,6 +45,7 @@ /* Flags for zram pages (table[page_no].flags) */ enum zram_pageflags { ZRAM_SAME = ZRAM_FLAG_SHIFT, /* Page consists the same element */ + ZRAM_ENTRY_LOCK, /* entry access lock bit */ ZRAM_WB, /* page is stored on backing_device */ ZRAM_PP_SLOT, /* Selected for post-processing */ ZRAM_HUGE, /* Incompressible page */ @@ -58,13 +58,18 @@ enum zram_pageflags { __NR_ZRAM_PAGEFLAGS, }; -/*-- Data structures */ - -/* Allocated for each disk page */ +/* + * Allocated for each disk page. We use bit-lock (ZRAM_ENTRY_LOCK bit + * of flags) to save memory. There can be plenty of entries and standard + * locking primitives (e.g. mutex) will significantly increase sizeof() + * of each entry and hence of the meta table. + */ struct zram_table_entry { unsigned long handle; - unsigned int flags; - spinlock_t lock; + unsigned long flags; +#ifdef CONFIG_DEBUG_LOCK_ALLOC + struct lockdep_map dep_map; +#endif #ifdef CONFIG_ZRAM_TRACK_ENTRY_ACTIME ktime_t ac_time; #endif @@ -137,5 +142,8 @@ struct zram { struct dentry *debugfs_dir; #endif atomic_t pp_in_progress; +#ifdef CONFIG_DEBUG_LOCK_ALLOC + struct lock_class_key lock_class; +#endif }; #endif From patchwork Fri Feb 14 04:50:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974464 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F44DC021A4 for ; Fri, 14 Feb 2025 04:52:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 16B0B6B0085; Thu, 13 Feb 2025 23:52:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0F4276B0089; Thu, 13 Feb 2025 23:52:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6173280001; Thu, 13 Feb 2025 23:52:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C5D7D6B0085 for ; Thu, 13 Feb 2025 23:52:33 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 3EC1D1C93E8 for ; Fri, 14 Feb 2025 04:52:33 +0000 (UTC) X-FDA: 83117329386.18.F0D447F Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by imf30.hostedemail.com (Postfix) with ESMTP id 68A298000B for ; Fri, 14 Feb 2025 04:52:31 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=lhYjrthL; spf=pass (imf30.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.181 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508751; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=F9dA9jZ1k0KVleOPZIXSXpGbAAlfZaWcl2QUySckvQY=; b=P0VTch20/XP3ZbhUSxqMxcSHr9j/gEzmj63PZfgbwZgwF9rJC9lF7FiNXkpxbYwuEx2k66 jQkcVZ0JinxooTyKbvtAQ9ga8ElH24Jen6Ie/bufmIkaS17QM/JNDVkxzlXQ3zCtY46dsU KY+jdT7gGc+vgEPTz4Ge28cEX4ZyVFU= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=lhYjrthL; spf=pass (imf30.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.181 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508751; a=rsa-sha256; cv=none; b=N/8BFfa/bndWA6dvDuj2CTT+uXPFg9cP4wzq3EY4s4tcPtIM6yuRDFt6jC875uy+hSBy/F eYPwaDjGP9BkCeD0B+ZyiHVh2HnRiEa6DL5M+tJ6j9Z0ngXmM+gxs1QipsQvpXn2tUqZkc +Wp2QdShFr4qwPo3CPLXTofEdwXq+F0= Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-220d398bea9so22388075ad.3 for ; Thu, 13 Feb 2025 20:52:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508750; x=1740113550; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=F9dA9jZ1k0KVleOPZIXSXpGbAAlfZaWcl2QUySckvQY=; b=lhYjrthLt/vxssY/KKlxqmhBMjfItV47A02v7rFCK6psbBoHDquHi7bux/o6pWzHJ/ Px7sbAcuV3JInFQ1Tyw2lqqP9fFu3Hu6mHRHSX5f0owQU1yfzGFH1nbHlFlfwQ498uoC uDIE94qOYjTz8h6DDc1vMsvsOXKG2K8D0Ym9M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508750; x=1740113550; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=F9dA9jZ1k0KVleOPZIXSXpGbAAlfZaWcl2QUySckvQY=; b=U29fhppXLPiVQFXzxEM0FCYAlcnpdiWcP8EhdwnZzlMOFLdKZeF9ybgL4PbdTRLgf6 cvTFGgwQrE3gaYYcnqAR0NVWVcmfFBYDSZhU3ePNCV68MblT4ejATUiFJpd7mW679FwI 26xL1qMcYcM3XEdh0dJMA6/FzDZUVfx5a8F4FJb037x2oA/0FQoJDXgBq4L95dbxRcg0 /KVoNtmi+8CkVmmQGdj9h8EFln53ECawLf7kdlk/N40dxsE8F2+WH0umHcrTqHT9kT1m ixsyDxnpkzAYzu6hQXzogyff2YU9z1GZsFwk9YYLkMAbdESxXsDgeJymv3btBLeVI/Az UTcA== X-Forwarded-Encrypted: i=1; AJvYcCXW/bT6Ld1A/KGjAABRfWsH4Jcr/YxHSrMjQ30Vv3eg0DR7YBtKx9LLowp9PvyIgTDybfBKB4Mf8g==@kvack.org X-Gm-Message-State: AOJu0YwH7Kbvt/qSSaxmvVA/z5xvW2TGsNeZxCtoDNDN4zZfpNWlZDtC aGr1bqN+YVHikLk9OjQ1oepNjywraN333CmHKfJ9gOYiGrLqV65rxpHiCAufHg== X-Gm-Gg: ASbGncvZ7bsu2Qs3YYvg3/ZAH3Bx8uOaYDj4FYvj7sk3J7qh3moihtzhAAfjSu8HP3H 0MM3NSLjHQAK4z0UQax9BUyUpyiJwnDvT7szhflxoURCDO1a2JJToWTxJ0UZuhftKV/DLlFwZ2X c/tejmYn8E9zirboiUPtFJkHCYMxzE9kBXrHltqUvBkUIsP7S7h75zwwqA49h5NPOglmxxALan/ j84rH2TB5Zmx/YuwCS4LRLR+8RUdk3Fky5pP81owscWIDsCYOmdiNFnasTxJnfCchMGub/PPRIR 6jSoXUefhr8YxLdPlA== X-Google-Smtp-Source: AGHT+IHeCccqw7BKz3Nps1nE3WUkuQd67prY/a7knQl57N4aYVrwMvzOwtITAs71quYBmeF0jfumUw== X-Received: by 2002:a05:6a21:898a:b0:1ee:5d05:a197 with SMTP id adf61e73a8af0-1ee5d05a2b1mr13268542637.35.1739508750392; Thu, 13 Feb 2025 20:52:30 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-73242761c13sm2252705b3a.136.2025.02.13.20.52.27 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:52:30 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 02/17] zram: permit preemption with active compression stream Date: Fri, 14 Feb 2025 13:50:14 +0900 Message-ID: <20250214045208.1388854-3-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 68A298000B X-Stat-Signature: mwde4jzw91mjcxngk7ugibrjamdnn8rj X-HE-Tag: 1739508751-631304 X-HE-Meta: U2FsdGVkX1/1YqLSvxRPJSgNZVWT3D2tzztg+hBOGUqjVlmj2taEMdhwtHNkW3nr/a7b9qSKFGAIkIPSMX77Ta07eKjyZuOfqSavXX2iXqgrPoAypiU0YiAhJZ1Q8HRTjKB1DfHvNHbEgck1q/eDEaa0AW7eXMFISVvYR0AtHXq4u+MAx+dspBiweGn0+zm6bRmOfz+/yVj8TQJXMQKo3CppxXpx6AeIXbwtSCo0OuFeXDkvFn62AWYdkq7fvwwJhO/n2y/P6AblPudHN3WX7qkX/K0oLYTsPJ+N2Dg992ythwyrsJoRynWCeOOKZ/jE8/Fiqhx1eW1rAY/hEs1kwZ1bV2ypWR6CWB8lMvCohsUavX54yZiLIEYIXX2xH/PqNaKwOJWefJPJYsgNanUnCCshQZ0jfl/2yUeoH7bvq0Sci9D2H9iy7oDEbO+bo1+5ZzpTrrP8SR8gjTpDfU0NRBgrd9WQdUuMfL5T1p2u3/q4I9nidfj2rMi8QN0KAqG8BZNZ+1H0uIry6/a/5/nCDlzvaEsP7b4lEpk0mfppZ0saISrTxCONMB9d6tZEVh04T/kVmgHxXU9rVVUySsC26N+2Eq+joR/ANXQqrtm+pREBsnFaiXK75rHYl/mI7yvKgAN9RGjqeQBImvWr/38pmtsvbat0NwPHT7GzzfqYdeyXwS2Y3jkHrW1dlkkhSXvuzGLsM4p7131WcDDW7/pxJpcBBqUtXWlOS02QUtHYYZRwFK4RYlQoI8RS4wm8Etp6yEJ0jBl/RoDTpTfjeOyw3NnCKU1EZd5BngHXCOZzg0FoAEhDF8fQ/bK9RDgoqFZdom3morzL9nn+2g9H5dCvsNvrlD0rJjKmFiC85FFJNxsHz3t7HK0q8NsQd/0W+dKbw/NN2KKoguBTofg/hmpDBXSzyzkH4No9T2eer7Sr4MmHXM7WS/UdTZp1r0LaeKDuYjbb9nl5VrWpa933eDZ /DN/xlX6 nV83a/7gcHkPEE7LyZ97PiQVDw+bvj+pHKgGZmGFDI/ZOr3695ruRNucIS3rH9Phmmndhz6ne8NM3flzldHg4VlyjYGdmbVJJrzO47RftuA7IcxCiYeA7xHIPcBIBxJU/L9TgrpcoQsNOkQo4UmYsbNHKCDGSGhxCHt9nPg8FbLgcGbm8ZeYRP2vDC2q7P3xYz9DAyF7s/oS4N5jhVlXrsduxF96NXfsvuHlsG3PLr90GGiI1mv0BikdB8nlfmbCaygnRdO2T63EPB/tRYnpVteL2WyENum8gANMB9T0oEpBWL6GUvHjMiZY88w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, per-CPU stream access is done from a non-preemptible (atomic) section, which imposes the same atomicity requirements on compression backends as entry spin-lock, and makes it impossible to use algorithms that can schedule/wait/sleep during compression and decompression. Switch to preemptible per-CPU model, similar to the one used in zswap. Instead of a per-CPU local lock, each stream carries a mutex which is locked throughout entire time zram uses it for compression or decompression, so that cpu-dead event waits for zram to stop using a particular per-CPU stream and release it. Suggested-by: Yosry Ahmed Signed-off-by: Sergey Senozhatsky Reviewed-by: Yosry Ahmed --- drivers/block/zram/zcomp.c | 41 +++++++++++++++++++++++++---------- drivers/block/zram/zcomp.h | 6 ++--- drivers/block/zram/zram_drv.c | 20 ++++++++--------- 3 files changed, 42 insertions(+), 25 deletions(-) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index bb514403e305..53e4c37441be 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -6,7 +6,7 @@ #include #include #include -#include +#include #include #include @@ -109,13 +109,29 @@ ssize_t zcomp_available_show(const char *comp, char *buf) struct zcomp_strm *zcomp_stream_get(struct zcomp *comp) { - local_lock(&comp->stream->lock); - return this_cpu_ptr(comp->stream); + for (;;) { + struct zcomp_strm *zstrm = raw_cpu_ptr(comp->stream); + + /* + * Inspired by zswap + * + * stream is returned with ->mutex locked which prevents + * cpu_dead() from releasing this stream under us, however + * there is still a race window between raw_cpu_ptr() and + * mutex_lock(), during which we could have been migrated + * from a CPU that has already destroyed its stream. If + * so then unlock and re-try on the current CPU. + */ + mutex_lock(&zstrm->lock); + if (likely(zstrm->buffer)) + return zstrm; + mutex_unlock(&zstrm->lock); + } } -void zcomp_stream_put(struct zcomp *comp) +void zcomp_stream_put(struct zcomp_strm *zstrm) { - local_unlock(&comp->stream->lock); + mutex_unlock(&zstrm->lock); } int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm, @@ -151,12 +167,9 @@ int zcomp_decompress(struct zcomp *comp, struct zcomp_strm *zstrm, int zcomp_cpu_up_prepare(unsigned int cpu, struct hlist_node *node) { struct zcomp *comp = hlist_entry(node, struct zcomp, node); - struct zcomp_strm *zstrm; + struct zcomp_strm *zstrm = per_cpu_ptr(comp->stream, cpu); int ret; - zstrm = per_cpu_ptr(comp->stream, cpu); - local_lock_init(&zstrm->lock); - ret = zcomp_strm_init(comp, zstrm); if (ret) pr_err("Can't allocate a compression stream\n"); @@ -166,16 +179,17 @@ int zcomp_cpu_up_prepare(unsigned int cpu, struct hlist_node *node) int zcomp_cpu_dead(unsigned int cpu, struct hlist_node *node) { struct zcomp *comp = hlist_entry(node, struct zcomp, node); - struct zcomp_strm *zstrm; + struct zcomp_strm *zstrm = per_cpu_ptr(comp->stream, cpu); - zstrm = per_cpu_ptr(comp->stream, cpu); + mutex_lock(&zstrm->lock); zcomp_strm_free(comp, zstrm); + mutex_unlock(&zstrm->lock); return 0; } static int zcomp_init(struct zcomp *comp, struct zcomp_params *params) { - int ret; + int ret, cpu; comp->stream = alloc_percpu(struct zcomp_strm); if (!comp->stream) @@ -186,6 +200,9 @@ static int zcomp_init(struct zcomp *comp, struct zcomp_params *params) if (ret) goto cleanup; + for_each_possible_cpu(cpu) + mutex_init(&per_cpu_ptr(comp->stream, cpu)->lock); + ret = cpuhp_state_add_instance(CPUHP_ZCOMP_PREPARE, &comp->node); if (ret < 0) goto cleanup; diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h index ad5762813842..23b8236b9090 100644 --- a/drivers/block/zram/zcomp.h +++ b/drivers/block/zram/zcomp.h @@ -3,7 +3,7 @@ #ifndef _ZCOMP_H_ #define _ZCOMP_H_ -#include +#include #define ZCOMP_PARAM_NO_LEVEL INT_MIN @@ -31,7 +31,7 @@ struct zcomp_ctx { }; struct zcomp_strm { - local_lock_t lock; + struct mutex lock; /* compression buffer */ void *buffer; struct zcomp_ctx ctx; @@ -77,7 +77,7 @@ struct zcomp *zcomp_create(const char *alg, struct zcomp_params *params); void zcomp_destroy(struct zcomp *comp); struct zcomp_strm *zcomp_stream_get(struct zcomp *comp); -void zcomp_stream_put(struct zcomp *comp); +void zcomp_stream_put(struct zcomp_strm *zstrm); int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm, const void *src, unsigned int *dst_len); diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 65e16117f2db..ca439f3b1b9a 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1650,7 +1650,7 @@ static int read_compressed_page(struct zram *zram, struct page *page, u32 index) ret = zcomp_decompress(zram->comps[prio], zstrm, src, size, dst); kunmap_local(dst); zs_unmap_object(zram->mem_pool, handle); - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zstrm); return ret; } @@ -1811,14 +1811,14 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) kunmap_local(mem); if (unlikely(ret)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zstrm); pr_err("Compression failed! err=%d\n", ret); zs_free(zram->mem_pool, handle); return ret; } if (comp_len >= huge_class_size) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zstrm); return write_incompressible_page(zram, page, index); } @@ -1842,7 +1842,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zstrm); atomic64_inc(&zram->stats.writestall); handle = zs_malloc(zram->mem_pool, comp_len, GFP_NOIO | __GFP_HIGHMEM | @@ -1854,7 +1854,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) } if (!zram_can_store_page(zram)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zstrm); zs_free(zram->mem_pool, handle); return -ENOMEM; } @@ -1862,7 +1862,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); memcpy(dst, zstrm->buffer, comp_len); - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zstrm); zs_unmap_object(zram->mem_pool, handle); zram_slot_lock(zram, index); @@ -2021,7 +2021,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, kunmap_local(src); if (ret) { - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zstrm); return ret; } @@ -2031,7 +2031,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, /* Continue until we make progress */ if (class_index_new >= class_index_old || (threshold && comp_len_new >= threshold)) { - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zstrm); continue; } @@ -2089,13 +2089,13 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zstrm); return PTR_ERR((void *)handle_new); } dst = zs_map_object(zram->mem_pool, handle_new, ZS_MM_WO); memcpy(dst, zstrm->buffer, comp_len_new); - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zstrm); zs_unmap_object(zram->mem_pool, handle_new); From patchwork Fri Feb 14 04:50:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974465 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93B35C021A4 for ; Fri, 14 Feb 2025 04:52:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 22D5D6B008A; Thu, 13 Feb 2025 23:52:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1B887280001; Thu, 13 Feb 2025 23:52:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0311D6B0092; Thu, 13 Feb 2025 23:52:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D75986B008A for ; Thu, 13 Feb 2025 23:52:38 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id A1B0E1C9969 for ; Fri, 14 Feb 2025 04:52:38 +0000 (UTC) X-FDA: 83117329596.11.6BD21B2 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf03.hostedemail.com (Postfix) with ESMTP id BB5A820003 for ; Fri, 14 Feb 2025 04:52:36 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=JAb2FaKJ; spf=pass (imf03.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.47 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508756; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HFfEwtj8s2kkg4bqWgjDWMl6rQxMO17ZCkJqcn1sbGY=; b=0G5aIvSnjf4Hk+ChwvP9C0Xe0SNDNg8N+xLqDdH+AY71ZVRBgS8BHJKZxgSUAv5HYEXvUT 4XV7F+TRn/VW9MspQmR5+o35mFfbxR2nPLEyXwJu2UCpx6NLcYStC/GouuAKPK9pHG8dsG WVt/FFGolH4CdSmSXM2tTzphhdKpwmQ= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=JAb2FaKJ; spf=pass (imf03.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.47 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508756; a=rsa-sha256; cv=none; b=g5z7wuMGBwxJvEcCmIYQaiENYF2gVMdBGOXx8VdW3fDJQTvCDkSBoEmm4pHvNS3Lqk4Id5 DGCjxTQUI7BCNU8pRnkyd9y3VzNyi7A/hugnB/pItGhsDvG56Vb7W3zhBw048cXX/fJu86 OIs5jdF3oio80KdYbkh4a7SO8Qs9pzo= Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-2fc32756139so89144a91.1 for ; Thu, 13 Feb 2025 20:52:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508756; x=1740113556; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HFfEwtj8s2kkg4bqWgjDWMl6rQxMO17ZCkJqcn1sbGY=; b=JAb2FaKJysntOND44gq1p/UeRA6x5KPLZEj90ysaBY8vQ8ywjat1NxcW+CBu/V0tlY IvbSss7fIPy2MF+hXulHBIWUHd65ubSXhaKDEQRJ9WzQ8Rv4c8064RXWZA2xIUWR/RaP TKbFFkSX+UsS+QfN7SWO3DxYYYdpCDgwTaw5Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508756; x=1740113556; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HFfEwtj8s2kkg4bqWgjDWMl6rQxMO17ZCkJqcn1sbGY=; b=uauFvN2fIxAyPVb4hcaU4dNr0wPvJ3b2UgeGsQ2RQJoxO6e5BxVDizCCi4wCqWRHzx bf2uMozz+lXf+6zCFF7cze5CCqqWFB6ACNF9kCrHSJbemSko8R15jl+kfYefyit5IW/Y CY0aV2TdhJWWqRy5wyootqMm0WvLXOHqo/j0tEw4Y36EituxrGTvbWUWO/51Hj/6D0fK JzYKqQ1GqmqNcfC4BQYcXe9t1912O9s3m/0HkEPrYUWzKIApx2iz3/zQBhqj8R5LyYKo 3rU9I6IPuIYpdPJIqGVWu/QrqdBPilFBbozcYPvIKCZqcqpQ6OEhdO2AqCamagSU4JRK tiDw== X-Forwarded-Encrypted: i=1; AJvYcCUHBGqTWKVtP5Aw7EZtj7ilcwr0z8YWDjXe4VERm3lX+yv6CA0GgV3aNW7SSnJWrAqG766vQJ8/NQ==@kvack.org X-Gm-Message-State: AOJu0YwmKRkFuTGFlb7P4fIeNQI8wdF8ORtbx7O53AWZrv49D3fb09af 55PL5xiWgtGfrO+g8fA3QhW55E4s/chk06O7o2xkNEweauDSFrOVItY7paduhQ== X-Gm-Gg: ASbGnctzWMNxX0Zog4fdiNjb9wNkJdghML7/xXYVdoyxFR5skkpXE7t2OFjprKeVSl8 H5Avd5wfojFy/T8vOJqludwMTWPRaandytbJkTja9IoOghno/NToyCUOkpxOKW3VoN/euQSvjcq oPFYoH2jafV1/6ZTTDw+fAKZrTspv5CBdkIXrsLePVbxjWwMZ/Kn2CFM5PINWvUMSBc9RPc8yht b9lSg59Sk1W6dWK1HGGwK4koEZoaZ28IQmwL8daZJAa/zOddS2VPmUhvZ1CHeUuYmMBXwswRk2d npl4wYWkLQd1yLwFbw== X-Google-Smtp-Source: AGHT+IHXO0KDJFtD3EzFKW9dTfi8p4jF/4dEQXWLK1VPzLY1TsXZB+Jthq5QTwRY8WV4GLgW0+AEXQ== X-Received: by 2002:a05:6a00:b95:b0:725:df1a:288 with SMTP id d2e1a72fcca58-7322c41b9damr17804053b3a.24.1739508755619; Thu, 13 Feb 2025 20:52:35 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-73242761676sm2272540b3a.142.2025.02.13.20.52.33 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:52:35 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 03/17] zram: remove unused crypto include Date: Fri, 14 Feb 2025 13:50:15 +0900 Message-ID: <20250214045208.1388854-4-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: BB5A820003 X-Stat-Signature: rrzwxm4bfjda6e86tp6r5d8ke8nkck98 X-HE-Tag: 1739508756-308494 X-HE-Meta: U2FsdGVkX1/3CWzPQfaG3r87Ju4wOtYFgzWID9DaLmWNaMy26He5llD3GO8ghcR6bQRONkDkFzVFNsFAMK+DbvL74OU9J+S+ftHVm6XOx5xynX1AjQLjxBEgYcbaPi1pC0K4xia5Dszke++PzJX65GfhfJJPHlnu7dcHL2+UsjD96nBx+KDBGHcxJjrhwMIX6O3X1GgiRasr3SRpj7zLCGmCALqEaeYTKq3nnIvGSnpmHK+SrYLyaHqDEZSPvsMd4zh0DTJf/Jgm5ZXPOWRIIeL2pKXF7lyK6afiIEcd50GHxPGLG0Nkpfqj5DU6Vbbp2FSs0sScPwmZDA7RVCq6DK4lFrBGsd2IjoO7SZ4BhV3YV1G8RiKXoyG8IU4mUCzlnZODm2++B2nRtvOuXZTfhQlT89KTYE5IvlCr29vh7+sC2YEgr/aHAnjS2pZ67UnQo1zkBiH3VA7amWk1biu7+uRNcVClsslnJ5wxMoZiMExQot8cwe3K3H0bdaMCAzItqTernMszO2ao9Poyobg2VsYKA0Gg7TCX1oNbpOSOE4fXrhciK2hKdenrgozs3p060Q703T8ppPoX+efaf4sVqCmdCGe6AkI9/cAsItMxWsgM55dd6uY58DW168W10iHECvWgN9geECA8EDAUgPS2UTahpm4Ty1xikJNXXqVOGgfi/1uML+xo+HxHvcCuflEBYTls1HhQ8uBEo4tKfsZ76lnYf4PvQCx4/q9KsckafADeG6oKPfEWZ/N61Gtbk3KyxrZj13fSTJG37ecOBTiBOJYm8zJtBI5hj4GKa6h7y2u2TW1dXQp2wKrCoxJXCa8fEfiRCia0h/119Z6Bt9JsvL+GPhadyK9Ba6n8xuT7j6lLkgTxfAV7oS4Dk+/DnA7PYI2f8H/783QXIhjvy15VkxYJ5z+0TqHYX4t49zDVN/zlT+5Smd1kNlU7225bVJD71cIBMHHnj34NFLeqAsl KOrVLOgG FYm890Gqy1U76lsK0HC2/WOZU1hwRawqff9I5OpLhBYNF5KGy09lDzihZsaEGg3Mc8cPwgwXRav47pugU7M9yWpi2+ovgWDvMHBVkHtTL8NbfWUUtuusUwzwhbqCHs74Z1TscpG/sd01ius3p7nu4VgvK3PWYFftKx4UFB2GEv3/X2udptNY7oNUEcbdsM4xF5/BVRnA3BrOc7DdbxLQAOXwqigiM7mTXdM/Z1aVRqGS9C/88AfyT9OAHeozh0pL2MK0QAamzPXz8zRTUHTHVkd0JPUFWQzY7HcaVkq+XQc4p7/HVwwvdVFwJggEfFOR2SgkZtKGTCGkqArtXNz6Rjq7H2Fx/kxYcs2Jz X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We stopped using crypto API (for the time being), so remove its include and replace CRYPTO_MAX_ALG_NAME with a local define. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zcomp.c | 1 - drivers/block/zram/zram_drv.c | 4 +++- drivers/block/zram/zram_drv.h | 1 - 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index 53e4c37441be..cfdde2e0748a 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -7,7 +7,6 @@ #include #include #include -#include #include #include "zcomp.h" diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index ca439f3b1b9a..0c8821b2b2d9 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -44,6 +44,8 @@ static DEFINE_MUTEX(zram_index_mutex); static int zram_major; static const char *default_compressor = CONFIG_ZRAM_DEF_COMP; +#define ZRAM_MAX_ALGO_NAME_SZ 128 + /* Module params (documentation at end) */ static unsigned int num_devices = 1; /* @@ -1191,7 +1193,7 @@ static int __comp_algorithm_store(struct zram *zram, u32 prio, const char *buf) size_t sz; sz = strlen(buf); - if (sz >= CRYPTO_MAX_ALG_NAME) + if (sz >= ZRAM_MAX_ALGO_NAME_SZ) return -E2BIG; compressor = kstrdup(buf, GFP_KERNEL); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 794c9234e627..2c380ea9a816 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -17,7 +17,6 @@ #include #include -#include #include "zcomp.h" From patchwork Fri Feb 14 04:50:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974466 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3ECE2C021A4 for ; Fri, 14 Feb 2025 04:52:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 848726B0092; Thu, 13 Feb 2025 23:52:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7D1BB6B0093; Thu, 13 Feb 2025 23:52:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 64C8B6B0095; Thu, 13 Feb 2025 23:52:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 416776B0092 for ; Thu, 13 Feb 2025 23:52:45 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A3B0844030 for ; Fri, 14 Feb 2025 04:52:44 +0000 (UTC) X-FDA: 83117329848.24.F0FB8F3 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by imf10.hostedemail.com (Postfix) with ESMTP id C1B1DC0002 for ; Fri, 14 Feb 2025 04:52:42 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=iWqNTlpa; spf=pass (imf10.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.181 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508762; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=N0yPBZnADO0X5Vq2D3cbKq9jy8e5O0FW3A6DJipxArU=; b=orH3R8FehJUzx9p/AQnPTRpG0Ko6euiWZBs2fNPivtGTpPjpTPU638RgcNziFswMwlrgdt 7gl//lhQm+ZIlzUKrifCrCN1Xj89qRpJ5f0+rG4pCkBYay2mzNoke0+05oT8CvZ3P8LU49 +flSq9a1Vz8JALs/omeWsyuI61HPJHA= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=iWqNTlpa; spf=pass (imf10.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.181 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508762; a=rsa-sha256; cv=none; b=sSt7bneeCqlfvs67ABrMTqpTnMqBm/34pN65K1zm2iTx28pQO6Dc4eq43tZU7Dl5Gm3f7n P86kyIOijXkSx1tarZre4OKXzTxUUz1sq1GuKL1zfpi81LRhlkMXQViKakEZxbvJBq84no 0sG6Yy49tX1JMFeyDMEcFKYrPe9qvcY= Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-21f44e7eae4so30059335ad.2 for ; Thu, 13 Feb 2025 20:52:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508762; x=1740113562; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=N0yPBZnADO0X5Vq2D3cbKq9jy8e5O0FW3A6DJipxArU=; b=iWqNTlpa0OF1NGpQsrKYCZpfSGy7hhJAcZUcTb+IsLE4WPd5hiItcp0Jdr71xmmMLn MxvyiBl3x7uQcm9tOoXdvIyyy6YDucQDvRJ2xXbHnFrTtkJjaNbYLq59QJSIUIC8brx9 l2CCrSHTFcYpOiEJvgSCr11/lXJjG0AtnI8Ow= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508762; x=1740113562; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=N0yPBZnADO0X5Vq2D3cbKq9jy8e5O0FW3A6DJipxArU=; b=XErH2z0PtK1AjufxALockRHrqjonMnGbMQLWpKcKMKT+l4PXdCfhug3l7zqeQUtp5i LIwmy3pUEdPh+10BGPlWoOTLOXGtKb9MV/5qU1EmdEdKOAHM84y61rD6cLErasx3BLfH z1uxnFzoQ4qmJTJ86tdJEB0ctUayMmBuqGlbpHftDb67rgMO5+cCwIwatbkj8DYYVG/B 61okLs6BRoajbMU5Be1jxu755J/3Y/r3BZsZ08Ghigth4/WkL3Ap7EvUKrx6aV+0yGMI nTnry/OnvvEpgbWn+MPTFg4Olx/HtPYcOXzJNi+1F2qPOR6DH6icGmqcI91xePsGRdTY AJuw== X-Forwarded-Encrypted: i=1; AJvYcCU7llmOnbClLzFbw61uWK60tALwnNdYHa3xTqPzNcYl74iwGsR9LKGdSNDkfpldeA91qpio4NU5Cw==@kvack.org X-Gm-Message-State: AOJu0YwzvyTaTupHe1cVrtu4qJr82fqUAoBAVaqhoddP1VfO9t0RavAi JwrFl3EwHsU0Sxu5g5UJVZULqNIginMvXcbSN0aBjuy9w5L5ZkB5FHMJp6MIGA== X-Gm-Gg: ASbGncsohIb8wn/4pkc5Y9ZpbrrodxatwgEWGe8fvpnd7Oi4RH5GAq3X2CmX085jn27 JvJ+iXOtl9UVwuzONwUy+Xq9dE3kLhoGwPYfJM/BGYxUU5gLHL0LzAqNX1BH6JBBUHoUUSOTezE eI3W3CfGK/om/NupgvHJCcngTvLBgFKRagrGtM+JdRoZmZ1BCFFEKp+v2RCXCIPcqte6/UDHwem kARUpNhxvYzkOog4s3y7gJwmiehLfAoSw98xfdFnhM0YxxtWWz7P9gduGCbJK09kfVjiOXr1Fsy ZfiQP/AMblc7dmDzNQ== X-Google-Smtp-Source: AGHT+IE0+HWw32m7Iz+CmsIdF7+EuP5uBVLXDu0uGMgAdMpaEB0vwPlOHOUHVWWjUpawD6FafZNm6g== X-Received: by 2002:a17:903:1ca:b0:220:d257:cdbd with SMTP id d9443c01a7336-220d257ce68mr96793445ad.48.1739508761612; Thu, 13 Feb 2025 20:52:41 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-220d545d474sm20667535ad.116.2025.02.13.20.52.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:52:41 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 04/17] zram: remove max_comp_streams device attr Date: Fri, 14 Feb 2025 13:50:16 +0900 Message-ID: <20250214045208.1388854-5-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: C1B1DC0002 X-Stat-Signature: a7orgx5hu374hsp7aoxctk5whftr3oe8 X-HE-Tag: 1739508762-921293 X-HE-Meta: U2FsdGVkX19vQ2mZGiOUWaznCESNUsNlebr4PZZGVY4JQUpNuNDZ+SbxWAjxQsB7/a9e+tAcNvSm88Y5tiPU6Jfypo/jEHl73mWjvmd5FMeWK/1peTQOPbpLf+4r6r7NOLF2LZrTyisVoTjdY9cNTxwNwzMWJG4n4Gtg+v71wEiY/OVyy4ekWCVIIuDPaE/tG5wiBs42SI1Y/+syakFni0Q9NPrHlCDGPUrPzMb90w5Vk8dH/1LvimK6/PaxQjL0aTQMHIvtpxo6kMKmLyVtCV+teT3qfcj57OjU7PyIE8ZEjvvnBYlo4JFAGKyeoXKbXilvreggtsTiVNsVWPTMZw58Q3KkVXrm/pxaz9ZojNg3ksmM0ppCPQrMS/quZCQinNYs73K/xipay0ANI5zJMWbECvvmr+y046haW3/47G6OceL0kx05nZVxYkKjnBooDSmR+l0a69tYs0tbPvM1aIixWn+pYZVSXijZL/iiYnkT1WXEqHvjplMswn2a8s2IYWRt6un5fYiOWNX2AcXRo9uJa2xxBZHM35Jvd73zkwbrmzpH6lteu9Y2hIHJbiYciuSnD4ykluxjnsnr76h/fiTxmUjI6tOgcc5+leM9LmGgfqkVVvWYQ6FSafC+ZyOVJGR/Evm4piW1df3wSh3hO6vPMHxHbJcwo0mO1AblC9wACCNeW8fbHvP+wbmgryacITfXUQNeslWZxMrXTQ96+SR1SmQpYQn4468EyE3DHq2F6jFwmqahsbWXo8D8lDfGAo8rUfOM3vGxP/rqn5pGPuVdoDBssmz52j4+thrqUp8Lvq8sHMXWn+upmj8Z4UAvEV+UXm/O72jnam7eHOxFtCLjhCI4mgWWvwkArG3WD/2ae3RdnfcTvq0YICcezbJ6uoih1hNS5gtmTIJlKXlcIVMMDqrhwEUpdgnvRPP98Rj63JqKq8Qza9WDyN1SEXTRLMgmuUkm2EzP5suhkJV HaecgPg5 BjKZvsNMtnw3GApboAdvb7vkI7u7Konioaq4Z5NiWI1OKqL8E/+BsynD15wnKkeYKetP5Jkv+UZ+q854qnAdqLsBGw/bDOFT6xIppFKWbRKFvL/8nqrM7RBJz+oY3mUhRrsj/btJRDJxcqgM8JTPoK2WtIDL7KwCThfnBY1/bLkNJ4k/PfIQSM9rQai8Y58wqMUpAFqMfpmWavprqhtw7xwnzC9MBiABBM9qOZt2my4wG3ZPWJSj75jRO8yREVKHSpur13ETQPzpuWH95YsE53UHlex+WRBLz3huB2T/2S6T0rUKPxB5P14dLCRQWNB1kh38PG0LXFm9IVwFndNvWZmHJ+0tssWyZkv9YVVOD3VIR99laNLfjjS5of75YWYLgVj+n X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: max_comp_streams device attribute has been defunct since May 2016 when zram switched to per-CPU compression streams, remove it. Signed-off-by: Sergey Senozhatsky --- Documentation/ABI/testing/sysfs-block-zram | 8 ----- Documentation/admin-guide/blockdev/zram.rst | 36 ++++++--------------- drivers/block/zram/zram_drv.c | 23 ------------- 3 files changed, 10 insertions(+), 57 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-block-zram b/Documentation/ABI/testing/sysfs-block-zram index 1ef69e0271f9..36c57de0a10a 100644 --- a/Documentation/ABI/testing/sysfs-block-zram +++ b/Documentation/ABI/testing/sysfs-block-zram @@ -22,14 +22,6 @@ Description: device. The reset operation frees all the memory associated with this device. -What: /sys/block/zram/max_comp_streams -Date: February 2014 -Contact: Sergey Senozhatsky -Description: - The max_comp_streams file is read-write and specifies the - number of backend's zcomp_strm compression streams (number of - concurrent compress operations). - What: /sys/block/zram/comp_algorithm Date: February 2014 Contact: Sergey Senozhatsky diff --git a/Documentation/admin-guide/blockdev/zram.rst b/Documentation/admin-guide/blockdev/zram.rst index 1576fb93f06c..9bdb30901a93 100644 --- a/Documentation/admin-guide/blockdev/zram.rst +++ b/Documentation/admin-guide/blockdev/zram.rst @@ -54,7 +54,7 @@ The list of possible return codes: If you use 'echo', the returned value is set by the 'echo' utility, and, in general case, something like:: - echo 3 > /sys/block/zram0/max_comp_streams + echo foo > /sys/block/zram0/comp_algorithm if [ $? -ne 0 ]; then handle_error fi @@ -73,21 +73,7 @@ This creates 4 devices: /dev/zram{0,1,2,3} num_devices parameter is optional and tells zram how many devices should be pre-created. Default: 1. -2) Set max number of compression streams -======================================== - -Regardless of the value passed to this attribute, ZRAM will always -allocate multiple compression streams - one per online CPU - thus -allowing several concurrent compression operations. The number of -allocated compression streams goes down when some of the CPUs -become offline. There is no single-compression-stream mode anymore, -unless you are running a UP system or have only 1 CPU online. - -To find out how many streams are currently available:: - - cat /sys/block/zram0/max_comp_streams - -3) Select compression algorithm +2) Select compression algorithm =============================== Using comp_algorithm device attribute one can see available and @@ -107,7 +93,7 @@ Examples:: For the time being, the `comp_algorithm` content shows only compression algorithms that are supported by zram. -4) Set compression algorithm parameters: Optional +3) Set compression algorithm parameters: Optional ================================================= Compression algorithms may support specific parameters which can be @@ -138,7 +124,7 @@ better the compression ratio, it even can take negatives values for some algorithms), for other algorithms `level` is acceleration level (the higher the value the lower the compression ratio). -5) Set Disksize +4) Set Disksize =============== Set disk size by writing the value to sysfs node 'disksize'. @@ -158,7 +144,7 @@ There is little point creating a zram of greater than twice the size of memory since we expect a 2:1 compression ratio. Note that zram uses about 0.1% of the size of the disk when not in use so a huge zram is wasteful. -6) Set memory limit: Optional +5) Set memory limit: Optional ============================= Set memory limit by writing the value to sysfs node 'mem_limit'. @@ -177,7 +163,7 @@ Examples:: # To disable memory limit echo 0 > /sys/block/zram0/mem_limit -7) Activate +6) Activate =========== :: @@ -188,7 +174,7 @@ Examples:: mkfs.ext4 /dev/zram1 mount /dev/zram1 /tmp -8) Add/remove zram devices +7) Add/remove zram devices ========================== zram provides a control interface, which enables dynamic (on-demand) device @@ -208,7 +194,7 @@ execute:: echo X > /sys/class/zram-control/hot_remove -9) Stats +8) Stats ======== Per-device statistics are exported as various nodes under /sys/block/zram/ @@ -228,8 +214,6 @@ mem_limit WO specifies the maximum amount of memory ZRAM can writeback_limit WO specifies the maximum amount of write IO zram can write out to backing device as 4KB unit writeback_limit_enable RW show and set writeback_limit feature -max_comp_streams RW the number of possible concurrent compress - operations comp_algorithm RW show and change the compression algorithm algorithm_params WO setup compression algorithm parameters compact WO trigger memory compaction @@ -310,7 +294,7 @@ a single line of text and contains the following stats separated by whitespace: Unit: 4K bytes ============== ============================================================= -10) Deactivate +9) Deactivate ============== :: @@ -318,7 +302,7 @@ a single line of text and contains the following stats separated by whitespace: swapoff /dev/zram0 umount /dev/zram1 -11) Reset +10) Reset ========= Write any positive value to 'reset' sysfs node:: diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 0c8821b2b2d9..cc4afa01b281 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1146,27 +1146,6 @@ static void zram_debugfs_register(struct zram *zram) {}; static void zram_debugfs_unregister(struct zram *zram) {}; #endif -/* - * We switched to per-cpu streams and this attr is not needed anymore. - * However, we will keep it around for some time, because: - * a) we may revert per-cpu streams in the future - * b) it's visible to user space and we need to follow our 2 years - * retirement rule; but we already have a number of 'soon to be - * altered' attrs, so max_comp_streams need to wait for the next - * layoff cycle. - */ -static ssize_t max_comp_streams_show(struct device *dev, - struct device_attribute *attr, char *buf) -{ - return scnprintf(buf, PAGE_SIZE, "%d\n", num_online_cpus()); -} - -static ssize_t max_comp_streams_store(struct device *dev, - struct device_attribute *attr, const char *buf, size_t len) -{ - return len; -} - static void comp_algorithm_set(struct zram *zram, u32 prio, const char *alg) { /* Do not free statically defined compression algorithms */ @@ -2583,7 +2562,6 @@ static DEVICE_ATTR_WO(reset); static DEVICE_ATTR_WO(mem_limit); static DEVICE_ATTR_WO(mem_used_max); static DEVICE_ATTR_WO(idle); -static DEVICE_ATTR_RW(max_comp_streams); static DEVICE_ATTR_RW(comp_algorithm); #ifdef CONFIG_ZRAM_WRITEBACK static DEVICE_ATTR_RW(backing_dev); @@ -2605,7 +2583,6 @@ static struct attribute *zram_disk_attrs[] = { &dev_attr_mem_limit.attr, &dev_attr_mem_used_max.attr, &dev_attr_idle.attr, - &dev_attr_max_comp_streams.attr, &dev_attr_comp_algorithm.attr, #ifdef CONFIG_ZRAM_WRITEBACK &dev_attr_backing_dev.attr, From patchwork Fri Feb 14 04:50:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974467 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DE0FC021A4 for ; Fri, 14 Feb 2025 04:52:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2CA6E6B0095; Thu, 13 Feb 2025 23:52:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 253976B0096; Thu, 13 Feb 2025 23:52:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0F4236B0098; Thu, 13 Feb 2025 23:52:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id DD8DA6B0095 for ; Thu, 13 Feb 2025 23:52:49 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 98C57B1902 for ; Fri, 14 Feb 2025 04:52:49 +0000 (UTC) X-FDA: 83117330058.08.9C20591 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf05.hostedemail.com (Postfix) with ESMTP id C2869100005 for ; Fri, 14 Feb 2025 04:52:47 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=W18fnxnj; spf=pass (imf05.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.169 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508767; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3bmpTeEVGC+rQQykpv7em5nfwUamOzB3Vnh8lKCRtbQ=; b=kSX7tZcrED9RGgPYiK7tHXM+f6K2pOa4BAPkIn0DrjAK3DGDdZPgKmMtyvrAPEvA2mY6nH vWxktdfgDi74zk3xzfLygDXhntfCFhDtiF40qNXbvCosU+1uZpJPOOc1Ia8bhOidyKGlZs gsTfFKoz/nK1nXnQ6A3CESuoeSVuERw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508767; a=rsa-sha256; cv=none; b=nZiUT6Aj3FZ+uWtkTmQ8kqETN/avZFe+Rh06yMyHCTwBmzPWM0kDx/URZD1tye/9O0t87P pha3e5Am9Ig4b8pSqtPeikwvlmcAQIhvI91TJm4IeJ3ZYCfRl4MKCiTB4fK95ke43Q2hlx +PNFZW3i1wppbnYdPzPi8Guyj8cYKYo= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=W18fnxnj; spf=pass (imf05.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.169 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-21f49bd087cso24735665ad.0 for ; Thu, 13 Feb 2025 20:52:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508767; x=1740113567; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3bmpTeEVGC+rQQykpv7em5nfwUamOzB3Vnh8lKCRtbQ=; b=W18fnxnjp3hc//CHkp1Ae8ypEimIkHWwkQebTZ4uCHRtXKiGZkfvM9AhZGvUWHlxZE DMpts54/742n9cUJp4P/01aUkazLbhWW4OKL7Y1wlYaRDYHsfmPaRPva2xStjJfqSeBX 4iaRXrXpR/jUBdLQQDfXSrjVz7hb8Pndrnc7k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508767; x=1740113567; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3bmpTeEVGC+rQQykpv7em5nfwUamOzB3Vnh8lKCRtbQ=; b=N7BgsmN0+Q7VWejq0/hMvz8qPNcRMk8a5dvhlQuWkrU1EpbMCyw7jcpCdL1sMmu/k4 ZzHFyyfjXL6H0QV277g+4sxndW++3WLKCCi347MwIWFAdHnDgXxIlGn6dTaA1iUdjwTl axA0TntyGarxhFt/FZsmKcRkkXZavaUReQoLr03xd/uPmO4Q0LAtlGYIeMFh+w8xJjYF XisvAukdO5cLu3crePUiGarDlq04lYVOtRmAijkcR66xbzCbTAUUVNWm7RlikdnqJnto aeqIi+fzWoYKBGlDT8XidI8GUnbKM5AE6zGpYSWDxCl9NlzYoSoeZeQ9BW/+BHxHb3ef iy1w== X-Forwarded-Encrypted: i=1; AJvYcCXzz7/M5RAZVg1mUBKIXJk4bU8xkjuUPgN6E72MWL5TwGJq2FzQvgF9yWid5EHsrZtPjwweI1RlKg==@kvack.org X-Gm-Message-State: AOJu0YzgucuGAV/+eztNXBVzPrm4ibRmEz7Oke8yn4ocMewi18dJwn4o 12hwqSq2P5P2kuK6ynHatz+Evb1meYsl2eQA2mr88bTyHSQOYegp5yL4NuoL2Q== X-Gm-Gg: ASbGncvggDil8fT4tc1W9xWH699tnxXU98psNKWca0iH565ARzd1H2RlxiurbhQLSg2 ANth5PdiUuFhwaPUWxTBCwlyPq4W3j3NTtM8fUP8My3Y0RxLSeu6UIOuqSx2Z1x7NAz9FoND5R6 6CzYIf+a2hbaceJVcqLuSDNSIuIqXDH+GDCyn4YhmyY9PbF3/QokeVqNt0uNomqlCQAKYgbeAYW /+6AZoS9aiMb2Yacv5RyRl35HMdEj7Fs56NxQZ3dqUipk3qdeksJ+c0nX120RPZlv0WzSpYfafi g4vZOpQfwU559JFRZg== X-Google-Smtp-Source: AGHT+IGEF7Dn0tVy7rM1XTaKFMcH+A1xI++T8yz9kJ3lVrfR5w/GZH5hDQABoV3SVr2TB7FEUspOGQ== X-Received: by 2002:a17:903:228b:b0:21f:6546:9af0 with SMTP id d9443c01a7336-220d216b686mr106732985ad.44.1739508766756; Thu, 13 Feb 2025 20:52:46 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-220d558fe3asm20666155ad.234.2025.02.13.20.52.44 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:52:46 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 05/17] zram: remove two-staged handle allocation Date: Fri, 14 Feb 2025 13:50:17 +0900 Message-ID: <20250214045208.1388854-6-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: C2869100005 X-Rspamd-Server: rspam07 X-Stat-Signature: 74ptkycu185mghmtdzsc56rp84takyh5 X-HE-Tag: 1739508767-177654 X-HE-Meta: U2FsdGVkX186KJqiWy4Q7ZURIVFv6a71QSiPbXfXKQn7PkcqR7e5K0BfiXOAK+9AOLDUMYxHDXjSvFxFGHn1raU0JHOcj/MT1tTHoccqF/sbq/XzyyrC+mIrP/GCrE3wEZQcqfZl5dVFPfZ7oXy9IVCUWEfc5/h13sWizjQVNUf7QSFt0uBv/QXtDmSB+qcK683FObwuEYTcRvDQTAcrYsRUz9uCgxHTJTJwrfnZV1Y21E8Su3kZt6w9s5KBBCiMgBAPNPr5YFHLypGGK4RFwgDk2pJ6yfhEKYKzpp2PG9isQImWFXWUoJZ4Lgxwt8GTSPkuT4FCmyZYuqWitM/Gj9P1bkHA/ovuf9Pzhblav4imdDSjUNM7TjlGLGrp6WIZZidyQ6j5CrICubXRsd3X/HvmQ08xuZVirp7R9L/sVpy8YQa1MrJAHg6By/fs92yHGu7de9Sdcye4CFndP0ugLt7jfV6y2M3fTjG3H0VKPeUWD8pHxMum0v49Xsq2zyNBBNggeDL5Zl5VXX/159STmWARmxDaBAi8nYsQb24TJ6E83fylp27FYJS2J2qG4e5pRAok4MIG7T5o2odvL7wW8XJu46W2nes8Pa3M4RZ43fBER46drBtmiyPW8KrJZfxK0JkJAE55ZXuzvh5lDC6X4t1OBgjBUWrVraCCfDZvekeN0KV2d8uY6TUMcgWudVb+s1qRtZE71mVQYtwiD6r6x39+Kmu93RUNHdUfw6FIIXzA2NxBniObT41/qKbz1rj38a93BB8UFhDilny8hiYRWHwRz87GQWJMbtUHLSZxgjkdx8mm2K2COOwjzMUziEMupPob6wTOQwP+OkWx5+20qb3hQq89glrVy9MacIJ5iUEP2wq11bLsEJqD3DIWFrsAimdBHyUg1I6aXSs2i+haLzVrH8MLCMDTajiRk8LFOTURGrFNOOiustHnqMtEhSjZEkjGuotB8betrZ6CyjR cHRQlqta jcHVB6RDBtPOgSllKAnjCnQ69Cq+B/6ajQnH+VN+onohdiRPlnd/G99Ab6qnpbgRd/0L3eEplOEbF73WEwZB6jZssvcs+aknbySgJiMLAV2yHXv5o/pmUXfrGa0d4DhsQTiWxKSb+OPe0oWXsPtzA+eu1oulHeOyWZChOAEerkuURx/H2qsuDFzdVZgLNhTaUCjvWwKg87auayhej7A08lHsd4veH5g8Va8Lymjl8NaT5OdXmc8bkWgOVAAgT9eLYvKO4nPE62S89MTeuvgXBSwOiDebthFj3TzEn2GyX9bSgStZmFPcAUyj4aA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Previously zram write() was atomic which required us to pass __GFP_KSWAPD_RECLAIM to zsmalloc handle allocation on a fast path and attempt a slow path allocation (with recompression) when the fast path failed. Since it's not atomic anymore we can permit direct reclaim during allocation, and remove fast allocation path and, also, drop the recompression path (which should reduce CPU/battery usage). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 38 ++++++----------------------------- 1 file changed, 6 insertions(+), 32 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index cc4afa01b281..b6bb52c49990 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1766,11 +1766,11 @@ static int write_incompressible_page(struct zram *zram, struct page *page, static int zram_write_page(struct zram *zram, struct page *page, u32 index) { int ret = 0; - unsigned long handle = -ENOMEM; - unsigned int comp_len = 0; + unsigned long handle; + unsigned int comp_len; void *dst, *mem; struct zcomp_strm *zstrm; - unsigned long element = 0; + unsigned long element; bool same_filled; /* First, free memory allocated to this slot (if any) */ @@ -1784,7 +1784,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (same_filled) return write_same_filled_page(zram, element, index); -compress_again: zstrm = zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); mem = kmap_local_page(page); ret = zcomp_compress(zram->comps[ZRAM_PRIMARY_COMP], zstrm, @@ -1794,7 +1793,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (unlikely(ret)) { zcomp_stream_put(zstrm); pr_err("Compression failed! err=%d\n", ret); - zs_free(zram->mem_pool, handle); return ret; } @@ -1803,35 +1801,11 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return write_incompressible_page(zram, page, index); } - /* - * handle allocation has 2 paths: - * a) fast path is executed with preemption disabled (for - * per-cpu streams) and has __GFP_DIRECT_RECLAIM bit clear, - * since we can't sleep; - * b) slow path enables preemption and attempts to allocate - * the page with __GFP_DIRECT_RECLAIM bit set. we have to - * put per-cpu compression stream and, thus, to re-do - * the compression once handle is allocated. - * - * if we have a 'non-null' handle here then we are coming - * from the slow path and handle has already been allocated. - */ - if (IS_ERR_VALUE(handle)) - handle = zs_malloc(zram->mem_pool, comp_len, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + handle = zs_malloc(zram->mem_pool, comp_len, + GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) { zcomp_stream_put(zstrm); - atomic64_inc(&zram->stats.writestall); - handle = zs_malloc(zram->mem_pool, comp_len, - GFP_NOIO | __GFP_HIGHMEM | - __GFP_MOVABLE); - if (IS_ERR_VALUE(handle)) - return PTR_ERR((void *)handle); - - goto compress_again; + return PTR_ERR((void *)handle); } if (!zram_can_store_page(zram)) { From patchwork Fri Feb 14 04:50:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974468 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03AD0C021A4 for ; Fri, 14 Feb 2025 04:52:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8E5336B0098; Thu, 13 Feb 2025 23:52:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 86ED46B0099; Thu, 13 Feb 2025 23:52:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 70F7B6B009A; Thu, 13 Feb 2025 23:52:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4D6136B0098 for ; Thu, 13 Feb 2025 23:52:55 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 0C4784BFC6 for ; Fri, 14 Feb 2025 04:52:55 +0000 (UTC) X-FDA: 83117330310.21.DF45362 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf10.hostedemail.com (Postfix) with ESMTP id 24F38C000C for ; Fri, 14 Feb 2025 04:52:52 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=lu0iMZWp; spf=pass (imf10.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.176 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508773; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6lbJ5wxYoLG970llEmrArbpgT7YZVPssHOdZszPf+hc=; b=YDBOainOXOHOyA6r2savtnlGAPOngnVHZCQQTI7L8RW2t+asmt1CXUlcMjl70/IMwbVz5o GJOWzTOlfQ4HAtTge5aYIULYtTj2LcreZI3bCOWZKFTW8jitQBBcpOBFj4RMN0k5ZJFFsn FSUfwAgRdmarQ83XkKbJ+nqvj0Vh6MQ= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=lu0iMZWp; spf=pass (imf10.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.176 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508773; a=rsa-sha256; cv=none; b=hH2buzPpZ79eLUz5PIMZJaCUw55ess4rhZGH5UV3T8GLkHipOMnPu3ykDpMVkbrvcnZDLh AOvd5cxbDKOQ1PGOmuWn+l+SN3e65qZiji2UEdhn0dbjQaTFnINmXpnHmtVrXCE9GQtUUh kxiTfJtQEObjG7BQZG/vrVXjGUXNoug= Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-21f2339dcfdso25104735ad.1 for ; Thu, 13 Feb 2025 20:52:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508772; x=1740113572; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6lbJ5wxYoLG970llEmrArbpgT7YZVPssHOdZszPf+hc=; b=lu0iMZWppk4I/J20vbucjEWkHRq/OLoPpHuKYHQ5KIfuYq/cwctqzHcUuJcaAHLfKE x6J0f6E/C5meE4CNO/IRJSn4aB2MHl333rW8oxNxkDBF9UPnJcZzHMSqax76NZmGbBwA 8xuqS90lAzxirU8Xi+FTsDb1ED+ImUi1jjs+I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508772; x=1740113572; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6lbJ5wxYoLG970llEmrArbpgT7YZVPssHOdZszPf+hc=; b=tpdaGuHPrnnHYHt5IoSWFDFOGvDb+hGR11z7Z9xxjnxbKGZcVVXfl/Oyscpjh1k9Ox hvkbNAqCq36Dt/AHl5fdtzJxz4/2zyCNTvT884ELoUD53HiSOrBl8Zt/D/sNllgJ98O+ U/gFK/WaoXZ3bSj6YCDi4Eq/t80cgHy2LtdCoF4nty+dWHI+Dan40HireVvsPdY2quBe pjVg9y/D5SYWpPEayKhRuEjMwiFiazIXthKnrV5nbF+mXxxHlAFzNWBJIL8+lu2tmg2n Kx1GFf+Xdb9ysitTqZuNYI/RPcR+17lPXBR+vzLT9ne1mnt6f7nLS3sJAMv+kq4jZ0EN GWFw== X-Forwarded-Encrypted: i=1; AJvYcCU3IpXretZhWLXszV+bTjQR0XyBPvUPiyYnsN7nkfk8r2dwJ5x03ExXvjL8WgpU2nrQwEjFd139bg==@kvack.org X-Gm-Message-State: AOJu0YyfQ+mDR5qKoD8VyPrNV86uAsmA0fGgo89ZW4bz33YmA0j6AvgW blDmLWidcXDXlR7miLI5vDBfGpx1Ca8dOdP3xTrDGgsZsRvEOnnga5fnTiMWjQ== X-Gm-Gg: ASbGnctvCAK3lfwQqofAr4y5w1Yr2jDM53kTp1ay/12SFafZLA6RXxYOD3b/asu9cGo N3D3pGmmMFYai+OJw5ZSx/2/gn9N9md81ZmJq3hFxD/JVxi8K7UPkqQufsI+9p+TN4zHFETa9NN EzN3zT4oCEXspaKOftBvLPrPmFTuNI8By3LMO/lfj3GMFBq1SfwM8vXcQtXgyGVxbEeIsvACcBG IEg+hS6JtlTiTA+FQX+qlcxvBGJzmlButu+OgHHEz3MqZ3zbCZdAtInN7viIna1ilTCEPJB45UG 7/R9kXjjxw63IJg4XQ== X-Google-Smtp-Source: AGHT+IF8aIJx2X7OJz6+UwgJWzttQh3HN4r8qEKkoAECXCtprMdGPtLnD2KHE/zkfmDa28LIY15LzQ== X-Received: by 2002:a17:902:ef4c:b0:215:8847:4377 with SMTP id d9443c01a7336-220d37151e7mr82187875ad.15.1739508772047; Thu, 13 Feb 2025 20:52:52 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-220d5348f34sm21017955ad.10.2025.02.13.20.52.49 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:52:51 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 06/17] zram: remove writestall zram_stats member Date: Fri, 14 Feb 2025 13:50:18 +0900 Message-ID: <20250214045208.1388854-7-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: ondydte5pm6kmnifnt66imkrmrzzgfo5 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 24F38C000C X-HE-Tag: 1739508772-67194 X-HE-Meta: U2FsdGVkX1/xbZ896IuI1of2Orp5wmDUhQkU39KyicLxYOTDtCorkt3Ossdjd6WmALAZMvVS2kKxbmgjJCVgflPCKioZQtaY7a1ebKA7JT6pURVy1jbrguMbPlxpSdMbbp9YBujoHqDECUozkFOoJ2lRIGXIreVYWouNhoLh+IKWszNPAWxsZ4Wdthd+9xiUadfFFCwAYaRyLvNQjR2Z7xfS+4/ATH7YDQMyyTHJC3K5aCSVLFT0AA+QnLuzxmYqYvFscZOxjTqic6FHKRiQbOOnRVwJDAYfd1mzVDSvJUC/TY9jIEoMHY0BcezxfIAEiV7N2X/7Dxz4y+QyCoak2509m8WDWgUb1Mt7HXw1m4uoevsbgG9CV0cB7D+C8YO7lEe5L5IPE9BKIO1sOxz1tKZxnlHEQ+rp/TAjUitDJQWNHcTp48HjIa+5tcu3tg1Y509tHDXl7qqdEsDP3oyZatG0iB0FyDJKcvWjUZTcyUbq0gg7Ms+gx8S0hR0hrhMHCurKm8zBC8uZxCLq+0dZi5Nk7obSE4ewq4+JnHTGqEwg1jEVr4obn5bMqy3f5WkI0FZWJF7wxU9j1i6wd3Ta7L8BxM3EUN3l2uQlVp3NcYOJPjo1kgE4oCpX89gPzWPkSXkcu4yYB33M1eSji2A8cVLVFxjgjl49cT4ij6Uo04fB8JtgwtfPBGh9+DAjSOchF6Eh2xUpeKRkeCjkXE1yPD/vG8FJTu9AcK5YzL341070DQd/QbgXLHqlA0l6LkmmbHtEhQUmXC3cjv+rslmTlE2tOA7AaIDGoeNI6Cd9zIyW+vd12aG6nFnMtK7rioajwJLIOWXNk2GSaJelPSnxhUHxyoWCVuDjGkS2FDyjZWnKsjTXuS1X63eng8XnlQ4UKEAikvliGlvZES8+KvY4O+pDImm8GoRY1/OYgnMVJ1TZUTTr1HRO8gV3mmQOClxltcVh1eNCsUBPZ83QtyH c9ZUcnSA CilggJYbWRaSEn9EOuj3wNZZ2jDVk8fQiWw6yvXB7gsea9r6h+kaf2As54SKw2SXo2gg8mghIJzB6D8fyMOhkECLXURvnpo1TkurH+9Qc0ssJJ8dG09/sybwxdmUcFbn5IoR+6qMn89bSebNG0pl3DFs/K1x7PVWLdgqGXcwRFcFbQjeFXTSI/5ZPXP0/gxFZKBKJWihp8HYSUi3P6kL5hoxkxDtpUMI72NEtqPx6cxE5i547snuVuDLdIm1UtfyMcavlgkbfk+wbGENVeqNzE+xQS+YO9VeCBD6AGbg6OwxeeajXTTI9vaRqACTWBV5//CXLV46GPdep+GoNu0vc50jtgP25zErZTva8BuJE43uew08= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There is no zsmalloc handle allocation slow path now and writestall is not possible any longer. Remove it from zram_stats. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 3 +-- drivers/block/zram/zram_drv.h | 1 - 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index b6bb52c49990..c28e1b97bf92 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1480,9 +1480,8 @@ static ssize_t debug_stat_show(struct device *dev, down_read(&zram->init_lock); ret = scnprintf(buf, PAGE_SIZE, - "version: %d\n%8llu %8llu\n", + "version: %d\n0 %8llu\n", version, - (u64)atomic64_read(&zram->stats.writestall), (u64)atomic64_read(&zram->stats.miss_free)); up_read(&zram->init_lock); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 2c380ea9a816..59c75154524f 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -84,7 +84,6 @@ struct zram_stats { atomic64_t huge_pages_since; /* no. of huge pages since zram set up */ atomic64_t pages_stored; /* no. of pages currently stored */ atomic_long_t max_used_pages; /* no. of maximum pages stored */ - atomic64_t writestall; /* no. of write slow paths */ atomic64_t miss_free; /* no. of missed free */ #ifdef CONFIG_ZRAM_WRITEBACK atomic64_t bd_count; /* no. of pages in backing device */ From patchwork Fri Feb 14 04:50:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974469 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1871DC021A4 for ; Fri, 14 Feb 2025 04:53:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9CEC86B009A; Thu, 13 Feb 2025 23:53:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 958996B009B; Thu, 13 Feb 2025 23:53:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D1EE280001; Thu, 13 Feb 2025 23:53:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5DEDE6B009A for ; Thu, 13 Feb 2025 23:53:00 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 11594161D25 for ; Fri, 14 Feb 2025 04:53:00 +0000 (UTC) X-FDA: 83117330520.13.CD62034 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf23.hostedemail.com (Postfix) with ESMTP id 31850140005 for ; Fri, 14 Feb 2025 04:52:58 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=j6UEszKl; spf=pass (imf23.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508778; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gJYj+/a+/IqgLXusoRYSgVepNrwhE22l9CBy17XcMXs=; b=2QrsmyfW237wqqYD5NqwzvqqPc7W5YdHQMyxscU56v2fhy9oWcRshWVNn2mwC3A4rvvjuL eEwy8OYi4KJ6XPX8dwfLz2aMxrwlUI6V7VFK6S3B0ztPnZgPs67u6E1ciS6nifXWfVl5fl h2jkFK/GAQfMu1d/qtFTtQ3PszQEH1U= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=j6UEszKl; spf=pass (imf23.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508778; a=rsa-sha256; cv=none; b=c5CVnHAHilSUURx/aqyHmdd85RLQe+S1dXRu6RnVehEv6ibEExMyBIWQBKDdcAKPRncNs6 r87TYCPMBIA+Ru3gQbJSp20nvPw6BXdo9fcae9ULeKQSBH39MNdGnk3qeyUgKfFQGaDMYI v+5uig9OpHrawMyGb3OS7dfnTgUuy+g= Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-21f3c119fe6so39151105ad.0 for ; Thu, 13 Feb 2025 20:52:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508777; x=1740113577; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gJYj+/a+/IqgLXusoRYSgVepNrwhE22l9CBy17XcMXs=; b=j6UEszKldFKs6stUpAgruOhaitJGz/Iut9o6GupOyFEwE466maH6/YGis05zPwCQ0H DHcB4WEYcMvoJK+uGbScBMa9FIeNTCmxpCYO/LdCmWqGpVt5UUPgbbk+qslnYKl8huig LDIB1lDr1vik3wKvPRDWNBDOhQ1dybIN2XeB0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508777; x=1740113577; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gJYj+/a+/IqgLXusoRYSgVepNrwhE22l9CBy17XcMXs=; b=afhFStSZg7FeiqroZeQqN93pTAtxXRH0P/1ZNRiX83IeFNiZhWp2W0WH6mVkX6AYnU ckNJ7cUsZk7EyG+rG+G9VPbbO8jZw7JkNuM1ChRqe5IkLRbVePeAXfIdaePJQKiGoSVg 8z9ktXBStS9k5+5sXXM4LJZ4ITm+z/xoLvhw5r8b+XUX0yHFuORaxNQg3RVUBsgT41fd 5Uf1I4BgfCVw2PBdatwe+nC1wGDQFGFhToMnNqIByeDFCAsoaOAJijwxOiki7KaUyAfX 7iiNWikvzyY0LUNxMBKbAFxilJN+M3kQZtNOyZoP9t8qxJgHzgyXuNbfK5IByR+Ur/gZ pkFg== X-Forwarded-Encrypted: i=1; AJvYcCWrDlOnOYcnzLefY0iw2S7KGgGhY72OFYpPoLYq9yOoTJiMFL+Cs0drc0AaXTSqFBlKjTC7P+wq9w==@kvack.org X-Gm-Message-State: AOJu0YwIcVY5mxXVokMaTgP8Spq6FkzcFeV3mTeEPMAWDI8h6Tg0j65l RtbS4599wXTBWwyj8BMNpLBTTg5nV6dgndwX/abm5bxu7tSSQb4M2lonFo8Wgg== X-Gm-Gg: ASbGncuATu4SJhgDdH4L7C/fGQAnkwfy/C6TWYJZTlkKnoVBpILtBY3AnOXBRK4GtR4 FJsmeiJdKqNQ5dDslBpWQiKjNjMWCDcK6Y8ZWPVzHhYnc+nKLtvSb+Qfxsw6Nv4tFTpZ6Abl9NK qCWXdp7NvYJtTnHba4f3UmI5UIouQhAnguX+fwcicVgznveAQnDBipYVtIUVUoWE0beEwkohgMS iuSui32GbeNLWLsu52h5niHlUKIYxywMqa8BcewI2RDpdwNoFJZKn6g4GjmRiH6q3UscA2wl3j5 13V+7bVl6TQyTydaWQ== X-Google-Smtp-Source: AGHT+IFt4PulwuqoCTMVnGkv5xAvCqNUHqsd70C5Y0GLWGm62Ls+ETbNspIM52jswK714tvw53XBug== X-Received: by 2002:a17:902:f68b:b0:21f:1096:7d5 with SMTP id d9443c01a7336-220bbb209camr155241455ad.17.1739508777155; Thu, 13 Feb 2025 20:52:57 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-220d558591asm20952015ad.224.2025.02.13.20.52.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:52:56 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 07/17] zram: limit max recompress prio to num_active_comps Date: Fri, 14 Feb 2025 13:50:19 +0900 Message-ID: <20250214045208.1388854-8-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 31850140005 X-Stat-Signature: qrdgiqpwndr47i4xkyszhsnd1zmktoys X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1739508778-940154 X-HE-Meta: U2FsdGVkX1/+w8cPFhzy0JykAa2ijReqB5Tachtykvs/VcftR4J4QiYsUj6isPtspxut+vUjSk+6AQN7Ijk5Pok89PCXFOx447MB3BJa++1NMuBMbni4oljHD8xE9TU3jIRqcgo+uQBUruAJAnPtnQZd8ZVX8qAqkncwJu/WazglPjI9xjCh1K5Bp21thv8aeVoqfktRiJYfJnIwVHRy6odTeY8bUSyPnq9/zK6yXU8rj4nGSC0MSrjNhAJzHB0lFE97s7DtFwwyVXJK19yDtD9ZJCqUvl6xCw9b+4imHMn3Hi71VmG17sJrnbr6HORMqkGQ1YvN+2+M5hgKA56L1vj7ZDeL6p86lBsZtEkZvG+rePHYONwyOIlrl+glxPL5dzzEiaVlIStn2QasnldO3VQljzLOAd2i3OqVFFsS0hRLFmio0KUVu9eqFv/TzcQyCsbLGDhwdXJ5A291dGmr11Ec+Lk+vq7bo6v3NWHq2AOdP/TSmKIiTLmt2k/aJfOxYz9CjJ9zkb5n48nN+rJb80HJkhv2PtN4qPMb9ll9yo1r63AYnZ47Z1yxEf4kxfFJ2BnmnWqFbpF/0WAnjsWFiVqQp+VrAgUR2sXnjB/ayK2i4MsxeqrmJwNiFuFLlmaQZa+4Uge92SgKXdH9clU4s5SOdkgqrV+aOeMoZjdmM0lUk0ROUr2ppbPL20SPY+tWC1COCUiWD6IsGdhKi/ZVyLLa+LSeI7zUqx/oJMtL8HGLLeard1qKg7JNIz0WdvfF9hALJ/2a5POCjL2rn9+Zj2FI67GsKWYH94hH1MtZ43deHon8lOujnIbxKECXXCd7s9M1CyThiC0VY7GikYfu+Y95FTPskzCGepins40j95w++0NcZUOD+w5fHLhEh6nRVopqea9QTFhrApYssB6doxzdz1Ma5rgnhi++Pa43q8DWgDncV2++MNMQ2zqImN2QwSpCS6qYoidaID8qIUj jldqOtIk BFEgGmaB3+bKViZukyLKxma9wqvUe5AfNFhtZiGFcITxenqe9Zra64ZVrkxYHMWdNfIDzlBKx0dRw4DOkj/m5D4BAMhKtC3iVO1AIP06SNNOijSdLf5KK6J0UENTqIBtJp2FLvjZdWteZXWjqx2o9j+fOZ7WWnXpfGkZQzL3j86HSqkvfPBd2U8satOIkbmm0kO2Bppxme7exLP4Vfza5FkbIZEjgOauC+Ee4KFEoWB3/1bBvYOvNQetdCTNJRZHGVGLX0x8xiTb8e0bQ0vNCQ6K8wN3lKoSH5JhAlRNOM0j9+iD3a6VdCypUZeMwBbOoZo+i1preYzqaeN46H1E5Qt7F8zPqXV3/vNZIM6Bf+0Xph4o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use the actual number of algorithms zram was configure with instead of theoretical limit of ZRAM_MAX_COMPS. Also make sure that min prio is not above max prio. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index c28e1b97bf92..c11ed2dc23aa 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -2068,16 +2068,19 @@ static ssize_t recompress_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t len) { - u32 prio = ZRAM_SECONDARY_COMP, prio_max = ZRAM_MAX_COMPS; struct zram *zram = dev_to_zram(dev); char *args, *param, *val, *algo = NULL; u64 num_recomp_pages = ULLONG_MAX; struct zram_pp_ctl *ctl = NULL; struct zram_pp_slot *pps; u32 mode = 0, threshold = 0; + u32 prio, prio_max; struct page *page; ssize_t ret; + prio = ZRAM_SECONDARY_COMP; + prio_max = zram->num_active_comps; + args = skip_spaces(buf); while (*args) { args = next_arg(args, ¶m, &val); @@ -2130,7 +2133,7 @@ static ssize_t recompress_store(struct device *dev, if (prio == ZRAM_PRIMARY_COMP) prio = ZRAM_SECONDARY_COMP; - prio_max = min(prio + 1, ZRAM_MAX_COMPS); + prio_max = prio + 1; continue; } } @@ -2158,7 +2161,7 @@ static ssize_t recompress_store(struct device *dev, continue; if (!strcmp(zram->comp_algs[prio], algo)) { - prio_max = min(prio + 1, ZRAM_MAX_COMPS); + prio_max = prio + 1; found = true; break; } @@ -2170,6 +2173,12 @@ static ssize_t recompress_store(struct device *dev, } } + prio_max = min(prio_max, (u32)zram->num_active_comps); + if (prio >= prio_max) { + ret = -EINVAL; + goto release_init_lock; + } + page = alloc_page(GFP_KERNEL); if (!page) { ret = -ENOMEM; From patchwork Fri Feb 14 04:50:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974470 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64DD1C02198 for ; Fri, 14 Feb 2025 04:53:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EB1E06B009C; Thu, 13 Feb 2025 23:53:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E3AC86B009D; Thu, 13 Feb 2025 23:53:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB43F280001; Thu, 13 Feb 2025 23:53:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id A5F4D6B009C for ; Thu, 13 Feb 2025 23:53:05 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 29C5981C46 for ; Fri, 14 Feb 2025 04:53:05 +0000 (UTC) X-FDA: 83117330730.12.B7C4B60 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf02.hostedemail.com (Postfix) with ESMTP id 3B78780004 for ; Fri, 14 Feb 2025 04:53:03 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=jw0Qulpd; spf=pass (imf02.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.182 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508783; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=u8qIWvBqgXd04+Bt3tanZdpv0pd6FYzP6TsmdRGZtIY=; b=AOnmOn3vBJYqoM+ryrq5f3jB2AXo5O3UF1eCprg3jq/2KTlU7jEKT+c1g5TgSrBrz8xk67 joq/2SuLv/sirLfxOSSVYOzMgAdq649MVzY2ASA5mvpIOrnPkobACzz7nxhN+nEUao+Wds HRhRFhyKzwqJJJWZBeJfJzNU4ROjHSo= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=jw0Qulpd; spf=pass (imf02.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.182 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508783; a=rsa-sha256; cv=none; b=KJmuaz+I9O/lMYIM4ISpL6W4DGfWxOmswUox/7MuAtvJwbWOuHFwVUZdaaaewedOIwNfIZ 9gfEIasNSfhh+uC32GH0cSLEe2XaoN1ggOegPRF2py10JAnIK8C7jWqBhbImdCy6iYzrxd KfcJUk/IvTpQhhlY28KO7/WKjiGj6rc= Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-220c8f38febso28737515ad.2 for ; Thu, 13 Feb 2025 20:53:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508782; x=1740113582; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=u8qIWvBqgXd04+Bt3tanZdpv0pd6FYzP6TsmdRGZtIY=; b=jw0QulpdsJXduAHW5J3V/TcDShsu064F/iRogsutF3fNbcEnvaMXfsUoggsxnp0elF G3rMe+4EzIpAiaKjgHr2RZYUzg3Mb3cIg+EMEGgSMzfF4EkP+rsTE9eSJXH+qcPLeyc2 T3b4WypeoN0ZEIlGEdbhFkC6dEwEOY33YtFfo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508782; x=1740113582; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=u8qIWvBqgXd04+Bt3tanZdpv0pd6FYzP6TsmdRGZtIY=; b=a4a+dpt61ta4m6HBKHJFqmaT7utXhVggyqE+xXLA6eGjuDlTjD1RkMI1bgBT8oUSbr IovNuOM6XRHpvUJkLx9NcFh2V9jFqm7lKhmteMH4irZyN+u5M4k0QjzGOEtab140c2sw N53rHcqYbz//EcRabLF31EckaMzi1AA5H7HPbfFRVYRJmiOHhFxhdWvIXiCvQbQV5Ndd OgixNhdGNlzKnSaU62bJOF7uS7cIksduk2xII4jjTN0e6Fezy/YWXVNUmg8lYZ/jc9Ik +0/Nls2x64Xan6g1eWF2d3dpZvqhR3g2b7SEkEghpChIFiKJ7gwYlbitWQhbDnc5+vks T+IA== X-Forwarded-Encrypted: i=1; AJvYcCXXYW33lkUyu7OJMo7gEkRCmZpic7qHmFqBW/jzu6Y+dBM0ml7fO2DIIqU1zZGZXeVV3PzwMx3oQQ==@kvack.org X-Gm-Message-State: AOJu0YzR7VREPmRb7AEQkjQmHAwfncIaRpkdBx+yCpiapGPATqt7VK4c ICSMmBDygTXzmk9Xib2vCA6wKBFWQ2z8DEUYciUQtI1nYbd9MCONfrnwKy2dHg== X-Gm-Gg: ASbGncvppf2NM4Rz8qYVMwtgnefdXyuA0N4snYhOPMx3RtRiV6ivYo8i3GyEDWMeoBh e3syuajGzyyjZVrYhxqtDjIvPKmP85Gbg3xL4Vpnrt6XbJc5sVRuIV6VyvEdVsoFTseMh77zIj/ OxsAelSsnHW0cZFP/qxjuLczcSkwRuI1GJR1fZp8+16+fyojkS5sdoQRz8Zllp7T4mbmZHVhBIU WfVY1QLLYtXOtghDCuKawoKqVcXlxlOFV8QCE+MrjUbNvy58n/fRfA11KPMsu/rm276nmarmyoL XCS6kaIq1PRi0K86uw== X-Google-Smtp-Source: AGHT+IHUcQM2DJmewLKGuE+CMu76WyyV+1GS8gtwww+ICNVj3SW61vpHXjXYA/HCtx5vMpfKv1a5mQ== X-Received: by 2002:a17:902:f707:b0:216:725c:a137 with SMTP id d9443c01a7336-220bdf56c72mr172323375ad.28.1739508782199; Thu, 13 Feb 2025 20:53:02 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-220d5349203sm21043715ad.29.2025.02.13.20.52.59 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:53:01 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 08/17] zram: filter out recomp targets based on priority Date: Fri, 14 Feb 2025 13:50:20 +0900 Message-ID: <20250214045208.1388854-9-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 3B78780004 X-Stat-Signature: 1r3d5zpasg3q5eeejmgyhrobdqntuscz X-Rspam-User: X-HE-Tag: 1739508783-150829 X-HE-Meta: U2FsdGVkX1/jgMhgrO73095uXLVoLBmHtr4los3K753Q+FIu4lWOFb0WYQXmIt6m7x0l9uczji0uFICVB1ZztH6K50IIJt18gnog6cE7kgJBzpIhVs21lIj6rpzspuKSEnwFfwXRmFyf34j6rwvG1zEe6K/WfHO41Q+d7N08LXpMR34UjHenJuvD82V6I2lPKJCYEPeLxhH3YPoKFcu9gLRkHj+4saacEJPZniCZaPAtGIYeHmAMxzJCdr/5VJUKZpKkiN4ThzBlKNZatKZxdlaqxcZP7sWHJdvPpTzY538w8zSUfv8UNq4cbHaDQTzqoI4e+tosfk+miMbvRdAtE7MDrgbgpNpMIlwcgAj4l1DfvcxftVro6qNBUSqSsl67QAZcp7ZVTBC7XGI7zanpj7PXt/DiPGUvEQs108vEh1mORe6m1/itEx36B32a+wO2aew0kIhT6dsXo12yczbE+wIWvFOlvYxpi0Gc+DGFNs0so/5P2Xs+GiohwDcBVW2ehJsN67YUsNnirE7beH3jijQa4EJy/5JC6mfC7Ikhrl3WrpwonIh5trmOCtcDcRay5LA+8qfmUkYoGouuOh9xHKzPSjFX9YMEYuwu1IF0EimwH1/sgiATBzSGzghRK2tfYyJwVRKPnbgHDC8gjbVTADU7/uouTwO1lnbOw85D5L7nB7T/ZeG64YpaSMyH7EusesbXRVOdp/StCf2iCYeDeSLrYLWgGCSG4SeOR8oRXU1MglLNn+/sngMPvZaJPIZEK4MFJdKDTfb2gAkRuXidozsMc2H7E8u+8uwebObsAbNzXI929NK48ZudTujL1VkeCGEg6Gau3OGN6tJ28Se2YupUwR0UMkwFyXlQbSaUZXPZxlZm5/nE0+aYYWzC0bKMnqhftoqD+ZnYL4RMs1e0amSmNs6ESLGqVl117IuvpwSnWEKEx2fjUREhwNzeyf6BgLp8KKH6O2U0+K1AFsa /ekR1Zic eyAvpmyx7IBLfviY81iera4m9VkNK9daU3JxDUOYhTZnut5tM9TVN+B/v6FVydH5hen+kwVFluwxRPBKo7Aj0AhTWEticA4TPeZlLsIk8WOYUEENjutbdBjajFtFXZHCh9gc+ZOr7KgXQhA4j9EMOWgw+u1kDjJht4DWhwj7xsoybueAg81wrAWVYK+Ppk/EUASaiTLnDFVdJ+AmDxCMOjIqu1mc5X3RHBewN0ec0iKJVQhOfGd2Z4zcuJK7GFmBr+dHS94GnjBInnzA918/JCGbEQEa5gHLxS3xaankN/C5ehyKQVhS4swERDA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Do no select for post processing slots that are already compressed with same or higher priority compression algorithm. This should save some memory, as previously we would still put those entries into corresponding post-processing buckets and filter them out later in recompress_slot(). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index c11ed2dc23aa..5f8e963bd513 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1864,7 +1864,7 @@ static int zram_bvec_write(struct zram *zram, struct bio_vec *bvec, #define RECOMPRESS_IDLE (1 << 0) #define RECOMPRESS_HUGE (1 << 1) -static int scan_slots_for_recompress(struct zram *zram, u32 mode, +static int scan_slots_for_recompress(struct zram *zram, u32 mode, u32 prio_max, struct zram_pp_ctl *ctl) { unsigned long nr_pages = zram->disksize >> PAGE_SHIFT; @@ -1896,6 +1896,10 @@ static int scan_slots_for_recompress(struct zram *zram, u32 mode, zram_test_flag(zram, index, ZRAM_INCOMPRESSIBLE)) goto next; + /* Already compressed with same of higher priority */ + if (zram_get_priority(zram, index) + 1 >= prio_max) + goto next; + pps->index = index; place_pp_slot(zram, ctl, pps); pps = NULL; @@ -1952,6 +1956,16 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, zram_clear_flag(zram, index, ZRAM_IDLE); class_index_old = zs_lookup_class_index(zram->mem_pool, comp_len_old); + + prio = max(prio, zram_get_priority(zram, index) + 1); + /* + * Recompression slots scan should not select slots that are + * already compressed with a higher priority algorithm, but + * just in case + */ + if (prio >= prio_max) + return 0; + /* * Iterate the secondary comp algorithms list (in order of priority) * and try to recompress the page. @@ -1960,13 +1974,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (!zram->comps[prio]) continue; - /* - * Skip if the object is already re-compressed with a higher - * priority algorithm (or same algorithm). - */ - if (prio <= zram_get_priority(zram, index)) - continue; - num_recomps++; zstrm = zcomp_stream_get(zram->comps[prio]); src = kmap_local_page(page); @@ -2191,7 +2198,7 @@ static ssize_t recompress_store(struct device *dev, goto release_init_lock; } - scan_slots_for_recompress(zram, mode, ctl); + scan_slots_for_recompress(zram, mode, prio_max, ctl); ret = len; while ((pps = select_pp_slot(ctl))) { From patchwork Fri Feb 14 04:50:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974471 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7472BC02198 for ; Fri, 14 Feb 2025 04:53:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0F5886B009E; Thu, 13 Feb 2025 23:53:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 07F5E6B009F; Thu, 13 Feb 2025 23:53:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E3A64280001; Thu, 13 Feb 2025 23:53:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C2BB56B009E for ; Thu, 13 Feb 2025 23:53:11 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 75ECE161D20 for ; Fri, 14 Feb 2025 04:53:11 +0000 (UTC) X-FDA: 83117330982.19.07BD38E Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf24.hostedemail.com (Postfix) with ESMTP id 9F84E180007 for ; Fri, 14 Feb 2025 04:53:09 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=D7wM0s3E; spf=pass (imf24.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508789; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=l/xbb64LFlIEuFMcJ/r7e+VdeaEJuKPUYJ75LZ96nQ8=; b=hgriNYCWwsNQZ76IcU86xM8H5Kglxdweqr2ej/J2iXF3sjJ9JYVvUIowJd0mdj5baPEFhM ohdqNXaWT0lIQy5KNe0Ccntvv+vHwZaxpWHDqtaIzyMDNreYm+uwjn2prit5NMUN+gj4N6 cQXdP07JIih3xZLlRHD2c9SNd0ydNhM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508789; a=rsa-sha256; cv=none; b=4+T5JvjhnLgNztiu0RtD7voMq+6PV10b+yKleyQZ40r7cePXWEDPtuOEf9yi1Hhe1tvw9p xkxPztDyfbvq2jNGPX0tXGHc7KSA33z2mjyvMnBzWGCbSHgoyJH9c6tK4NeHwHTBNUB7RT OnO+ngRxVXqeHJ8oXN5ZOshEFsnRg4g= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=D7wM0s3E; spf=pass (imf24.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-21c2f1b610dso38141585ad.0 for ; Thu, 13 Feb 2025 20:53:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508788; x=1740113588; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=l/xbb64LFlIEuFMcJ/r7e+VdeaEJuKPUYJ75LZ96nQ8=; b=D7wM0s3ExnDqFwnZv3bdorcxgo9OXwAb3YawsjOpC7BNJWh7g+jxwbbjomWwjEVl1a f/t7etPeqkhBzqcFrDEQxlJMDU7GtDYLI3Sz/gBZfS7/0HDN/iYUKGWMpKfwel8mWesM 7qmRCx8BKpnFNb/H6xw+qAsP6GaF1emMjIbCU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508788; x=1740113588; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=l/xbb64LFlIEuFMcJ/r7e+VdeaEJuKPUYJ75LZ96nQ8=; b=Qjp95kpxJ2PI1Iba0u+KZhy3aQMUCkx9ub98mY333lurCKRQTvQY6lvxVfBvizAy71 +ltiHm6mAufTTuHiM3YsM+gJN9/9C6Bn2R4zjrP4g3t8vHixHCoIPyNir9f+AdqNBAhK TUa8DLGKJ23OVG5Y0i+hp5N3Ufibg8oG1IFWJyRMcBlGCqrQrcT5BVoWqFB4BqZH4KfM 27LY//tnyGZOtXyfrsY/H4sJzWfimxMCMt/1OzTk6Aa2Qnkh4BuvxCErPOn7Ey07VQVf 2vCLn752QQUj7PcKo8NZUryj9YUvv4ypxarzJxiDilJU837MdJwsO2yGNTwIpjlwAAEM ZVvQ== X-Forwarded-Encrypted: i=1; AJvYcCUT1TU4Du4MUvGau0kCxNm2xDHIcuVgIhdaXVDm3CBtlB0JICpLBW5cFJkxkcBWFL+4Ek4bMMk0XQ==@kvack.org X-Gm-Message-State: AOJu0YxlM9PQPbF8yqUgtoXVEqrmW4Y3/2syUNH+DeKYhMjTozIb5V0Y /r6AK56IyMpbWETyglB+tQqFdhdXqhcKBFlv+rZL7E3H6FUlYsONMeMgq1EWIA== X-Gm-Gg: ASbGncsWh1hyLok0KqdNFMNYvyT4O4gRa6mCnec35CxIs1OQmuKIoGiVndJfGGtoe71 L6rZ6qTCtzt4X+JuJZx7QSbiMQliyky61JH1/U34u5806iQ8k10J5V8UPpxpC/hSo25DNobnITd 4/JTcIXuYeGDopz3oqPI5bxSvlQ7HcfZi4mp+rpZO74+9CJgx5aXbEMyFvabZZwmwg39tMyD5IA 3DGFovBSpHL3KMSzUjDB8/qZc/XJVlcsy+DTYn8zfVxpDxnR5sxDkiH2GxoHHXSmeX3cFKiRU61 4/pNA2yPr4ZB6oMOKA== X-Google-Smtp-Source: AGHT+IH47sqL4VstyP3CQTqYF0OisImdj6UBrruloe47Ytl0rlHucdtsjUQ25uJQt/qj5N2jpQbI3w== X-Received: by 2002:a17:902:dad2:b0:21f:baa:80c1 with SMTP id d9443c01a7336-220d2364ac7mr109012865ad.53.1739508788597; Thu, 13 Feb 2025 20:53:08 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-220d556e15esm21001265ad.190.2025.02.13.20.53.06 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:53:08 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 09/17] zram: rework recompression loop Date: Fri, 14 Feb 2025 13:50:21 +0900 Message-ID: <20250214045208.1388854-10-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 9F84E180007 X-Rspamd-Server: rspam07 X-Stat-Signature: uxsngbfmsszid8q3f88gf9uczmtg8sc9 X-HE-Tag: 1739508789-205127 X-HE-Meta: U2FsdGVkX1/xuS1O2vEwdYw9GwYBmlaDsaOp+qKgsIK4jhst0ZUFYm99qAFFTeXcaSBs4t8SXC5em9BYLtFOhKpm7KdHhrRu3g61Y6mZtrbSrflnDWlDCMhc4uK8EGq7dnpxKXPewmuRMC099Fg4ZDL6vIWP4Cg3lIR4fRBuSy3l99kYw1JjW4jar0GcVMx6E1vj/YmET3RxQhHgCrlJ+m+WUxpScV72AE2nsnexQwuQKuFa/piNh25VgsxtNK0LgmHORDRHIw9c9+uDyR7d3ZxO93QhC6T9Gy2K3g1vOpZIu+b710ovzxElQzs4TPlMO9zzm/Y2MMNs/WQlJ8g9mEyR1+8gn4zGPjFEf2lyylZcLuq0RkDA6TKSDj1uNz8+kVDFZSpTkxyjyP8zIqb8if9Vjw7acupm3IQcsT0mefUQcvrhF8Kq/FI9zJaud+FlmDDGV4SPfrmARLz0zd7COCUncrby0gJEmzrznEB3rbkKOfL9FysIF+wZofyNoDRLJPWvA2LsweTIujD67XDtRQaKYvojQOYqf1fY9/MKd1/FGAC11Ge1Pf/ajXp7l0Br/2z/GzjP39PzGbZa4sbfGnADciD7f0++sciThZtwvp72qOiXKSV05OqEq6vw5w1M7rlTvbOrLM6S+0jSItZwERsOCHYMew6NVWTHCjc1nuTh24SrA87/NgHtsFAzqFda9bq8ufusH6kmVRKh3y1VzGAN+TzVCYbGkUSoIWlPfmDf6phhMjewqDBItvrClM9VVBu7NuxUIflqCnSHbEVCkNGRrkBQG1ZEkL7IVRHrGujeJENRZAnHwnIUjvgKLANO0fq9ojoHD3tZOd7KBLAppqYPxfYzGfVtdqdxqefRN2WBUnmXTJnbYBFf2Gc8oNYSmrDyL2vRYkebMbKnWgXXAHyjR/ryJi1obGEeaxxOj7LV9bfXInpuTkPwkinNQXIxhBhAs4hht9jcEWqJcqw BmZpJBKF 8TMx3jMXDYQBqBpVrlKAI7YixI79QAhFMZ4+F2gV0vYjLoWLVtibGbBrNe7wAxOk2tfi5zjrYuzvqzeZDU0jCFhyZKF+dTF54CLBKeKzPE66ElJb3tfiRU8/VjZ89VXYQm9HrI5EmS2HVCNVJwUy8NuoEg7V4R9zeSFS+Q8uVu+6flrsOR3Ak2UlDxaKl+KfECa2Gz4jvwD/2AkxID0yyxP5A0JeVtfUvRrT7qq8Q3heBNaOXNJxs5TJ53S4UTDTMDc6tiwAG4ldjs5fEnbu9NzHy+XivFNazUidGEOBPYU4IiUN4n+xFEHjpHQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This reworks recompression loop handling: - set a rule that stream-put NULLs the stream pointer If the loop returns with a non-NULL stream then it's a successfull recompression, otherwise the stream should always be NULL. - do not count the number of recompressions Mark object as incompressible as soon as the algorithm with the highest priority failed to compress that object. - count compression errors as resource usage Even if compression has failed, we still need to bump num_recomp_pages counter. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 53 +++++++++++++---------------------- 1 file changed, 19 insertions(+), 34 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 5f8e963bd513..2e46ea542ecd 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1929,9 +1929,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, unsigned int comp_len_new; unsigned int class_index_old; unsigned int class_index_new; - u32 num_recomps = 0; void *src, *dst; - int ret; + int ret = 0; handle_old = zram_get_handle(zram, index); if (!handle_old) @@ -1974,7 +1973,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (!zram->comps[prio]) continue; - num_recomps++; zstrm = zcomp_stream_get(zram->comps[prio]); src = kmap_local_page(page); ret = zcomp_compress(zram->comps[prio], zstrm, @@ -1983,7 +1981,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (ret) { zcomp_stream_put(zstrm); - return ret; + zstrm = NULL; + break; } class_index_new = zs_lookup_class_index(zram->mem_pool, @@ -1993,6 +1992,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (class_index_new >= class_index_old || (threshold && comp_len_new >= threshold)) { zcomp_stream_put(zstrm); + zstrm = NULL; continue; } @@ -2000,14 +2000,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, break; } - /* - * We did not try to recompress, e.g. when we have only one - * secondary algorithm and the page is already recompressed - * using that algorithm - */ - if (!zstrm) - return 0; - /* * Decrement the limit (if set) on pages we can recompress, even * when current recompression was unsuccessful or did not compress @@ -2017,38 +2009,31 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (*num_recomp_pages) *num_recomp_pages -= 1; - if (class_index_new >= class_index_old) { + /* Compression error */ + if (ret) + return ret; + + if (!zstrm) { /* * Secondary algorithms failed to re-compress the page - * in a way that would save memory, mark the object as - * incompressible so that we will not try to compress - * it again. + * in a way that would save memory. * - * We need to make sure that all secondary algorithms have - * failed, so we test if the number of recompressions matches - * the number of active secondary algorithms. + * Mark the object incompressible if the max-priority + * algorithm couldn't re-compress it. */ - if (num_recomps == zram->num_active_comps - 1) - zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); + if (prio < zram->num_active_comps) + return 0; + zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); return 0; } - /* Successful recompression but above threshold */ - if (threshold && comp_len_new >= threshold) - return 0; - /* - * No direct reclaim (slow path) for handle allocation and no - * re-compression attempt (unlike in zram_write_bvec()) since - * we already have stored that object in zsmalloc. If we cannot - * alloc memory for recompressed object then we bail out and - * simply keep the old (existing) object in zsmalloc. + * We are holding per-CPU stream mutex and entry lock so better + * avoid direct reclaim. Allocation error is not fatal since + * we still have the old object in the mem_pool. */ handle_new = zs_malloc(zram->mem_pool, comp_len_new, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + GFP_NOWAIT | __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { zcomp_stream_put(zstrm); return PTR_ERR((void *)handle_new); From patchwork Fri Feb 14 04:50:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974472 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24E16C021A4 for ; Fri, 14 Feb 2025 04:53:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BA4DC6B00A0; Thu, 13 Feb 2025 23:53:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B2E22280002; Thu, 13 Feb 2025 23:53:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 958B6280001; Thu, 13 Feb 2025 23:53:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 7354F6B00A0 for ; Thu, 13 Feb 2025 23:53:20 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E2881141DAB for ; Fri, 14 Feb 2025 04:53:19 +0000 (UTC) X-FDA: 83117331318.08.6E8C7B4 Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by imf26.hostedemail.com (Postfix) with ESMTP id 0E8E4140004 for ; Fri, 14 Feb 2025 04:53:17 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=mMTG3v7L; spf=pass (imf26.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.45 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508798; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nEsYVf2FanYsVWFazbZFyY/NjoKwvYYj/znay1eygdw=; b=wNkSQOlDeSueKWA6iojigwzDAqdFxZuvk80ytVrRssqM+DpyikCYcb8zZVxW8VhZv3lfpu nAc+rAclLEVQOR6ghTPSg4onTWBOG55R6pQGvzINHyL1B+wyu/GfQeXw8VC8i1hLpe7Iwh MF8XYRZ3QbXAAPxa9mqWlo9EggPuSq8= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=mMTG3v7L; spf=pass (imf26.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.45 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508798; a=rsa-sha256; cv=none; b=G9oeUk4T6hYpnHPmrNHXxX5rTwBb4PKY4epRQ2y9MFZbEOykkzvY0RhYsnz+HsQ4jA5rGb QbZqLed5zEq6QPP5T4bsfQjCmPA+/keNt7+g4PmvkZCCCAdRe4WoqYaox8TyQ/AkPlZ3tc wLvkpi7ksNywJYFeDpLY5bRzDm4Sahs= Received: by mail-pj1-f45.google.com with SMTP id 98e67ed59e1d1-2fa1fb3c445so2434736a91.2 for ; Thu, 13 Feb 2025 20:53:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508797; x=1740113597; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nEsYVf2FanYsVWFazbZFyY/NjoKwvYYj/znay1eygdw=; b=mMTG3v7Ljp8Ado/dUX1idE/cbbKvPXlbaB1z1y/5FMfeaCCqmnY36Wn/BIA7sf0c3I 0HHAvK/FxUf/Fz1ANPlKmq8IHT7qKCqTwCsIFbHVSm2mXzUVxlr4cz4H9GSnmKF6iuRF Jlyp2uqTHGN6ZCML0RXCNZjlVq89giU9i/i4o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508797; x=1740113597; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nEsYVf2FanYsVWFazbZFyY/NjoKwvYYj/znay1eygdw=; b=RMadukvL7GkbC6xllGXlFBi+7nFyFDLgDxp62azFsxjY7vyQO5CezJ72sNecp7VXWs xneBQn7qQ8KuWIHFB8Vol48oglJ+0W68UWRCIQ2SNMweA1r5P+z3A829i0tZ2jr0xI6e yPpuIzR6H4KaRY4nHvkVl2UTeNuBD55//4qmXSScPx4m0kWK466GK7KxtfGugYpfpVB5 2igV/n9y9Zaz5hsbszuT9VWdqEukQN4+l+06eEDST25/SC4qB/IDDoYtLOOWrd46mMsf i5YLy96Vj9ZgJgoJcqgOEJVn58ADRHYKGwnd3857dxz5zKSk4HtS6SCuhOHTXIB2TqT7 hvEg== X-Forwarded-Encrypted: i=1; AJvYcCXfbl82Uk3pEtMxXeSCVAwgBLa8aOc5BYU1rDRFIQCtesLSkBnvwu+vDF3WI3z2s57Rw8nmWDM34w==@kvack.org X-Gm-Message-State: AOJu0YzywctyvJ/wr+QvzqjGXVAMOev9xxhBiA/8S2b1L0GiNwfIMarP 0BTMiyPMzezih9hGO5Xj9hVbaBw/Mu8C372SP4Z+PhAY7SLUx5ZDGUDhaEThug== X-Gm-Gg: ASbGnctNVVV3a4SF1RQnN2T+GfGRYjXARgGBGvdRdDGi+JKRxCtC79jyUZu5PcjGo1X Tk7uCpPrs1BMPY87m2z+a/lc3TTrqigHqoPYjF6UdEQp73nlfLlBEZHdHLCa6i40vNSJIDqI95S NxpDy6SbfMe8L7uTa90J9h4l34xN1a0h8PjAvoVoD+rj+H5U9EMRwaNsx4vGJa42UGona72p7Lg NknfWSPwZWIYQ8Kr7L+JHyaFaGT6rgbWw3Ql+0IV5/H9QgcULSuYyiMTXwgvgYCvKYisJGBmW7O tFpiQZzCJ4US3Ea1Hw== X-Google-Smtp-Source: AGHT+IGwpYmFxf0CUvgBN28Mu9LR6po8v4fVBJ8EuK4NGdlYOspY9jkFKQ6vUxoKOF+lj1g+wrgKpQ== X-Received: by 2002:a05:6a00:8c8:b0:730:7970:1f8f with SMTP id d2e1a72fcca58-7322c5a9058mr14486712b3a.12.1739508796987; Thu, 13 Feb 2025 20:53:16 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-73242764cd1sm2272338b3a.149.2025.02.13.20.53.14 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:53:16 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 10/17] zsmalloc: rename pool lock Date: Fri, 14 Feb 2025 13:50:22 +0900 Message-ID: <20250214045208.1388854-11-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 0E8E4140004 X-Stat-Signature: gq6ppbqkzn1gra7zzugcsusfake6bcs5 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1739508797-645382 X-HE-Meta: U2FsdGVkX1/yciUbgLN+OZUnTt/gEdpB4AdqhHbh0YM9AK9lVzUJZ9czYKMsf4U8i906q9MAf8MZjmL1mKAenHJlqLF67MlUYCXm1IKKJjWF1Zde7wdIq9fwAZYBRAZ9Vdwm9WFXkwisUWKVeiUSeNdQbJF05+vvMvvNTeU9wKrtkRYNZwo2IxvsyNDQrLIpRh6ygOqWNcSyF+vWGpeDUpZk/P8H4wOxufM4jkhcq35v4mnjGK3Mv++9OWPatfukQu3jwE9z3XWXzc26q0v/yPvp6sjBpY/0JeQEWRnlJ+i6cJVad6oTzqK21S75uyv6TTgaHcs2RjdCC3IHRFIPA/m/RIygL9EplRaYsVwe7qPZOTga9aZUDCkBjC/Nqjwx0CDri1dBUbF0qtDxcmr+dDy3emmN4cyd0Dt3/Mpvo1cCUTZfmKzkr0prx6bjIr8TM008DDHBLaFiil5xZsj6qnESexQJIPuGzwUiAKNBVW+hQA5MEyUEWY6Bt+gOxs83ZBa/a1leC7WFft4XVYlVX6wBDwTfrxWzAXUy9O908daVqBsSVymQBoKkpPN87H1F3Hhppt3nJu3f03iNXVsTrg8vprNyDT9ZoOJOG9cdMXdRP07QXRTkFgXI1IANselm/6GPwdCx/BH6o+ACBoW4vrqo70rU3Ox/fSwuZNa3ljXPvmx9e08jerREMx29d27CDw+jTCU5tqhHSZtmHxV1L3NIMEk76E0s1c7fZ8b6ZB6WbqHKEfli+qwOqYRRRjN5Pj1GiKJB0gz1EmGU7jo0VQMq/DirLzdQham+ncoD+sG1xOzw7/9x+r+/wt22/PFQJiH1HdyWnKFH305rsNegP5HIhztuHYAUgTHotfrUteaS2GL+VBKVo0bkYjs4f7iuSjUR27oRvcRFUNvbr9eAsRQ+x21xQ/gtWaI9pEfT4+I2GzqcmTad7QhFh2RopqXpwIR0eHM1njIe/DQY6Hv /5DmWASa hKZyXKHTnTmpHBwWQVX0oUPT23UGeA8mQQ/HYGJOrvF6pfeYEr6CQ0bqAJD/SDMa/yoh+0LZH8a5u8F/DwO6Jc+/lcXg8w/FqyFHLvV28I8rTAMu1EqzTiI6W77ImyiTaD75/wOYuW5CJ2ohhkfXIFa1QkQfFkZO5tLLPcRhW3fAsX3YZf8oRRoKRAxD6CxA4npq8ixNbsuEjc4Sy2yicUv+KSXWYBpj9Gp4ls1OlHM3pm6CceiPvci7oYw4/nYqPTM9LrfAgxKFWwHKFowpg34UZJgHA+Lt0b1yVw4mwOWW8LUiWXFtTs+q8TA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The old name comes from the times when the pool did not have compaction (defragmentation). Rename it to ->lock because these days it synchronizes not only migration. Signed-off-by: Sergey Senozhatsky Reviewed-by: Yosry Ahmed --- mm/zsmalloc.c | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 6d0e47f7ae33..2e338cde0d21 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -18,7 +18,7 @@ /* * lock ordering: * page_lock - * pool->migrate_lock + * pool->lock * class->lock * zspage->lock */ @@ -223,8 +223,8 @@ struct zs_pool { #ifdef CONFIG_COMPACTION struct work_struct free_work; #endif - /* protect page/zspage migration */ - rwlock_t migrate_lock; + /* protect zspage migration/compaction */ + rwlock_t lock; atomic_t compaction_in_progress; }; @@ -1206,7 +1206,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, BUG_ON(in_interrupt()); /* It guarantees it can get zspage from handle safely */ - read_lock(&pool->migrate_lock); + read_lock(&pool->lock); obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); zspage = get_zspage(zpdesc); @@ -1218,7 +1218,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, * which is smaller granularity. */ migrate_read_lock(zspage); - read_unlock(&pool->migrate_lock); + read_unlock(&pool->lock); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1450,16 +1450,16 @@ void zs_free(struct zs_pool *pool, unsigned long handle) return; /* - * The pool->migrate_lock protects the race with zpage's migration + * The pool->lock protects the race with zpage's migration * so it's safe to get the page from handle. */ - read_lock(&pool->migrate_lock); + read_lock(&pool->lock); obj = handle_to_obj(handle); obj_to_zpdesc(obj, &f_zpdesc); zspage = get_zspage(f_zpdesc); class = zspage_class(pool, zspage); spin_lock(&class->lock); - read_unlock(&pool->migrate_lock); + read_unlock(&pool->lock); class_stat_sub(class, ZS_OBJS_INUSE, 1); obj_free(class->size, obj); @@ -1796,7 +1796,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, * The pool migrate_lock protects the race between zpage migration * and zs_free. */ - write_lock(&pool->migrate_lock); + write_lock(&pool->lock); class = zspage_class(pool, zspage); /* @@ -1833,7 +1833,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. */ - write_unlock(&pool->migrate_lock); + write_unlock(&pool->lock); spin_unlock(&class->lock); migrate_write_unlock(zspage); @@ -1956,7 +1956,7 @@ static unsigned long __zs_compact(struct zs_pool *pool, * protect the race between zpage migration and zs_free * as well as zpage allocation/free */ - write_lock(&pool->migrate_lock); + write_lock(&pool->lock); spin_lock(&class->lock); while (zs_can_compact(class)) { int fg; @@ -1983,14 +1983,14 @@ static unsigned long __zs_compact(struct zs_pool *pool, src_zspage = NULL; if (get_fullness_group(class, dst_zspage) == ZS_INUSE_RATIO_100 - || rwlock_is_contended(&pool->migrate_lock)) { + || rwlock_is_contended(&pool->lock)) { putback_zspage(class, dst_zspage); dst_zspage = NULL; spin_unlock(&class->lock); - write_unlock(&pool->migrate_lock); + write_unlock(&pool->lock); cond_resched(); - write_lock(&pool->migrate_lock); + write_lock(&pool->lock); spin_lock(&class->lock); } } @@ -2002,7 +2002,7 @@ static unsigned long __zs_compact(struct zs_pool *pool, putback_zspage(class, dst_zspage); spin_unlock(&class->lock); - write_unlock(&pool->migrate_lock); + write_unlock(&pool->lock); return pages_freed; } @@ -2014,10 +2014,10 @@ unsigned long zs_compact(struct zs_pool *pool) unsigned long pages_freed = 0; /* - * Pool compaction is performed under pool->migrate_lock so it is basically + * Pool compaction is performed under pool->lock so it is basically * single-threaded. Having more than one thread in __zs_compact() - * will increase pool->migrate_lock contention, which will impact other - * zsmalloc operations that need pool->migrate_lock. + * will increase pool->lock contention, which will impact other + * zsmalloc operations that need pool->lock. */ if (atomic_xchg(&pool->compaction_in_progress, 1)) return 0; @@ -2139,7 +2139,7 @@ struct zs_pool *zs_create_pool(const char *name) return NULL; init_deferred_free(pool); - rwlock_init(&pool->migrate_lock); + rwlock_init(&pool->lock); atomic_set(&pool->compaction_in_progress, 0); pool->name = kstrdup(name, GFP_KERNEL); From patchwork Fri Feb 14 04:50:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974473 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8ED5BC021A4 for ; Fri, 14 Feb 2025 04:53:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 27C3C6B00A2; Thu, 13 Feb 2025 23:53:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 205326B00A3; Thu, 13 Feb 2025 23:53:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 07F936B00A4; Thu, 13 Feb 2025 23:53:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D6D4B6B00A2 for ; Thu, 13 Feb 2025 23:53:30 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 93E581A1C7E for ; Fri, 14 Feb 2025 04:53:30 +0000 (UTC) X-FDA: 83117331780.06.CB8199C Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf25.hostedemail.com (Postfix) with ESMTP id A80A6A0008 for ; Fri, 14 Feb 2025 04:53:28 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=N0VnJd0g; spf=pass (imf25.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508808; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qnJprTwBbnPgr4JVTYJPq6LS7kls9Uwm5mA/DEYKF7Y=; b=25kb6U8+ZvtWh9BXZMdxY9/9xa+5x9c9kQQHezpwaz8VcD1/NRtOSUpOBK7teCF3iyyHDs /0CENPVh14O4BnV11LYj9VBQinkPl/HLsR5xc2fLagR7KPe6sJjyKNvATA5mE8GkEPRv4U b3CZRCRLm//ovAo+P8SACpgtVfFmFWQ= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=N0VnJd0g; spf=pass (imf25.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508808; a=rsa-sha256; cv=none; b=T4O/TPv8pdHymNKmBwJrKo0weMRrWT+DjLz8mslAj15qdT2yHyv/ZwlZQ+gTyOKoGi0DTs a7yqCmlurStDn91bg5ETbvQwRBofy6j/GQoH39jPIQ3/GIteN08KWUIXPcac1oCg/vyT5A 5Fh/KpPchB0lWs+OZ2H18T7dA/iB07U= Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-21f6a47d617so27995645ad.2 for ; Thu, 13 Feb 2025 20:53:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508807; x=1740113607; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qnJprTwBbnPgr4JVTYJPq6LS7kls9Uwm5mA/DEYKF7Y=; b=N0VnJd0gohFwiCWGaZJPTJ1LbHcT0vFVmEB2fsvkc7sdnEb7Q+v2IBH7yyKtcGXznS uTW+gQ90+dcyDzbIzSVRbd26eqn1G2TB5Y4pUoqGCAr0w68vuHuWtZta6sY7kbsU7TWZ 9PWwzjS7HGKvre/N/1kIu+jk1Q+9hmEOETKuw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508807; x=1740113607; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qnJprTwBbnPgr4JVTYJPq6LS7kls9Uwm5mA/DEYKF7Y=; b=aZSOyao7JxSFkrT7mnnnufQZkhfdTuLUfhgrAEntsMpJcTSHe2fNZWFF34yoE07ilU 6KVDy+WGecaRIPyZQ3lWhAlgw517TYCuUK5bSdcP/QIK4ewd9WJXtHe0ZfUIp2VdVA/7 76CTRn6gLLK5geTOAlNwXbPjZnl+E4aDV2DKm8SvE2sm2/aq0Pln6CT4fSjlwYbPNyGm 2SX0jWZ87vlbaGAo3sinciUQeac+Addhr3/QmZylgh1TqZwXNpC3/Hm4CrehfLma2ZSn UiXDVx7pJrgjQUCP1Tnahn1o8CHv+Ve4tBQXtIpkWO7UB4Wu6os+m/SsEkVVwgXDyzzu nQgA== X-Forwarded-Encrypted: i=1; AJvYcCVq11vtnKTqcj7UHn012YXE5HGYm69JhXtCdvMXYk3EFb3pnQBkdL8gw/X9qOn90zvA8Qdx8MLpJQ==@kvack.org X-Gm-Message-State: AOJu0YwBHR3o/fXH0RRVIdOdcZfvFJnqQyybVNieUwRRCaKHJ8ycCZyN ePu0f1fX6qRGCBMai6QzU+pFifQUqOC+uQZXQJ3JM8QKbqHnHnqVhpuD58ugn+kMku1RozMQD2k = X-Gm-Gg: ASbGncvaWh3LrHin3yJeyEuZAwASZtW4JuhIbRiD+7aVpK4qF3JDtMqmEaqfwjIruIp yRcCMs+X33isigzGrHHiJgfKNtSiI5tjnOrokn4ri0wDipcRhQZz10+nw7bszR9yp/ZINqmOm2d cm+2wyoDPf6Yx0pwC7taM4JhMQ0uKLqLEX1fT6aevivkFvRTeCjoAuN4RIiPgM1F/z1Ycrp+WoZ YP+7pCqXrO+jL0++sdUaH4WXYzUiw0pDnbi/+yoyXIOUGH4XoUw4SuBMnJzGqzaHag8/lS183wg 0VikqwMbf4kd/dcunA== X-Google-Smtp-Source: AGHT+IFFD5AmsVRlFVnjhldrCVgoR4+3BSYYM094ysV/C/O/38mcNtXqChtRYmpdp4CXK407opgp9w== X-Received: by 2002:a17:903:32c8:b0:220:ea90:192a with SMTP id d9443c01a7336-220ea901a5fmr37761435ad.5.1739508807544; Thu, 13 Feb 2025 20:53:27 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-220d53492d0sm20892945ad.35.2025.02.13.20.53.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:53:27 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 11/17] zsmalloc: make zspage lock preemptible Date: Fri, 14 Feb 2025 13:50:23 +0900 Message-ID: <20250214045208.1388854-12-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: A80A6A0008 X-Stat-Signature: f373ypytrdbd1squd3wzt1yynyoms11r X-HE-Tag: 1739508808-715924 X-HE-Meta: U2FsdGVkX18cHdZkKqXKyc1Mlt5E+J5PtSCEt2FEzBzfRKfUT5uCsiOSvkGs83p5vQR2/u0hQN/XnGz5bbCDx0xC2jnVdLXA3W7r+In7blcEKGXeEf0MvPrFCObCzvffRSj3aLVrxP0GYxPZqVyuwKxGN4btfnhWXu1K6zUExe1bpAc47yhIE42wStlgXe8wwXN0zoDFFcKegl0ibRVItQpiNiSeUXl9ai3I5ZhHfpjsuJxlXF151RatGk3n6iZ6a523XqdICS8of8UWl1xzjrvL4OzQYctgs6MW//QvLhmolhduL2F1+uXNtJPXqr1kMIXy1M+YZV0rQiItar3ZrOxdsDR39kqBgjudQhFv+VuzmMcE7It2EfPjt8uNAaKPyGYeMLB5vsPAnytoMZN23GuZ4dWFhy48LXple1qjUSNlZSYnwfpbPkwz0M0AzkvL8I1bt8vPorvH2RbLXsxCu7l3ORL/K+t7oTGpk2aSjlLFkBfHrzro+R36Yl+2Ff1TKynHWPCPKL76bTE0T9t6PEwjTtb45dLKPABWy1z9/zNQTY0wh2SEegtG9fYmEZzE1UNTrw39yvAL5Zq63r28EpY+SvE+JA9lBjaeeQcjLHlJkWJH3ZqFfSurDZ43YxcsChNV5Kbk3EQOQ6JgFrxUxEA4MxKlE3k1H25xMY+SDx+lwTdhwSCSdeuBWJ4wtXzXQSb7Mz3c7m+Zebhpj64GxpCk+OsjxIRqKtz9Pix6ldGUNRars4kCII01W5mxRDwIcSNWGOPQTVuZo8xCujNhSD4emvg5xlKQzXLX095AbANkHnldmzUN9adhO9Ku6t5T5I+r/MNnQyg6/zqSjW45cKn4WPZwwE/JEJ2ysvsIfxIN2MLnQiIYtzQIEFQExn5QERE18I8e/7D/bEBNoOBvU/LMPA1Wcs5vTls2v38A6PJK/+9Ge7ZPjqeSmISlytSGy1xTblCE4DycBcwzuSi NNuqqcSA qAP3s74hMGH/rUr4JGfa7kYUxgzV55alqNR7rczIcpZurc2n5/Av8SlSkvsHa+2RNG1gAYfbKDdXZkvwT0zHuYKEHfBTJysbbevpBWDOKpIRhBrnb33bwQbEsugDnnnJfsu/1SYS+2NJ9MBXxhKW3VSvxaMT8ULiTuZfM00gAbiRB2DR+22rh3Bm/ATcHdX7O3PLWmJG+6Xr+Bq83lj1Fl3YFU8+d+m0xfIkJ99DBVRqypEacuQAp5ZP0jqwbMYW8H/NFO/EOfTSJgYNCBhA+9zWQsZbr3vO7HGKHrF7ni2njwZzfXRIOT4JQ/E9IVZUptqMdgRMVo2C2rTJFzC5XpcASYPvTmApfOG9apffsoDyQyy51docOf1rb6+svJzpIshtEEiSPJ0QRLckLSJjV2MHbjw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In order to implement preemptible object mapping we need a zspage lock that satisfies several preconditions: - it should be reader-write type of a lock - it should be possible to hold it from any context, but also being preemptible if the context allows it - we never sleep while acquiring but can sleep while holding in read mode An rwsemaphore doesn't suffice, due to atomicity requirements, rwlock doesn't satisfy due to reader-preemptability requirement. It's also worth to mention, that per-zspage rwsem is a little too memory heavy (we can easily have double digits megabytes used only on rwsemaphores). Switch over from rwlock_t to a atomic_t-based implementation of a reader-writer semaphore that satisfies all of the preconditions. The spin-lock based zspage_lock is suggested by Hillf Danton. Suggested-by: Hillf Danton Signed-off-by: Sergey Senozhatsky --- mm/zsmalloc.c | 246 +++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 192 insertions(+), 54 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 2e338cde0d21..bc679a3e1718 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -226,6 +226,9 @@ struct zs_pool { /* protect zspage migration/compaction */ rwlock_t lock; atomic_t compaction_in_progress; +#ifdef CONFIG_DEBUG_LOCK_ALLOC + struct lock_class_key lock_class; +#endif }; static inline void zpdesc_set_first(struct zpdesc *zpdesc) @@ -257,6 +260,18 @@ static inline void free_zpdesc(struct zpdesc *zpdesc) __free_page(page); } +#define ZS_PAGE_UNLOCKED 0 +#define ZS_PAGE_WRLOCKED -1 + +struct zspage_lock { + spinlock_t lock; + int cnt; + +#ifdef CONFIG_DEBUG_LOCK_ALLOC + struct lockdep_map dep_map; +#endif +}; + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -269,7 +284,7 @@ struct zspage { struct zpdesc *first_zpdesc; struct list_head list; /* fullness list */ struct zs_pool *pool; - rwlock_t lock; + struct zspage_lock zsl; }; struct mapping_area { @@ -279,6 +294,148 @@ struct mapping_area { enum zs_mapmode vm_mm; /* mapping mode */ }; +static void zspage_lock_init(struct zspage *zspage) +{ +#ifdef CONFIG_DEBUG_LOCK_ALLOC + lockdep_init_map(&zspage->zsl.dep_map, "zspage->lock", + &zspage->pool->lock_class, 0); +#endif + + spin_lock_init(&zspage->zsl.lock); + zspage->zsl.cnt = ZS_PAGE_UNLOCKED; +} + +#ifdef CONFIG_DEBUG_LOCK_ALLOC +static inline void __read_lock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + rwsem_acquire_read(&zsl->dep_map, 0, 0, _RET_IP_); + + spin_lock(&zsl->lock); + zsl->cnt++; + spin_unlock(&zsl->lock); + + lock_acquired(&zsl->dep_map, _RET_IP_); +} + +static inline void __read_unlock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + rwsem_release(&zsl->dep_map, _RET_IP_); + + spin_lock(&zsl->lock); + zsl->cnt--; + spin_unlock(&zsl->lock); +} + +static inline bool __write_trylock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + spin_lock(&zsl->lock); + if (zsl->cnt == ZS_PAGE_UNLOCKED) { + zsl->cnt = ZS_PAGE_WRLOCKED; + rwsem_acquire(&zsl->dep_map, 0, 1, _RET_IP_); + lock_acquired(&zsl->dep_map, _RET_IP_); + return true; + } + + lock_contended(&zsl->dep_map, _RET_IP_); + spin_unlock(&zsl->lock); + return false; +} + +static inline void __write_unlock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + rwsem_release(&zsl->dep_map, _RET_IP_); + + zsl->cnt = ZS_PAGE_UNLOCKED; + spin_unlock(&zsl->lock); +} +#else +static inline void __read_lock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + spin_lock(&zsl->lock); + zsl->cnt++; + spin_unlock(&zsl->lock); +} + +static inline void __read_unlock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + spin_lock(&zsl->lock); + zsl->cnt--; + spin_unlock(&zsl->lock); +} + +static inline bool __write_trylock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + spin_lock(&zsl->lock); + if (zsl->cnt == ZS_PAGE_UNLOCKED) { + zsl->cnt = ZS_PAGE_WRLOCKED; + return true; + } + + spin_unlock(&zsl->lock); + return false; +} + +static inline void __write_unlock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + zsl->cnt = ZS_PAGE_UNLOCKED; + spin_unlock(&zsl->lock); +} +#endif /* CONFIG_DEBUG_LOCK_ALLOC */ + +/* + * The zspage lock can be held from atomic contexts, but it needs to remain + * preemptible when held for reading because it remains held outside of those + * atomic contexts, otherwise we unnecessarily lose preemptibility. + * + * To achieve this, the following rules are enforced on readers and writers: + * + * - Writers are blocked by both writers and readers, while readers are only + * blocked by writers (i.e. normal rwlock semantics). + * + * - Writers are always atomic (to allow readers to spin waiting for them). + * + * - Writers always use trylock (as the lock may be held be sleeping readers). + * + * - Readers may spin on the lock (as they can only wait for atomic writers). + * + * - Readers may sleep while holding the lock (as writes only use trylock). + */ +static void zspage_read_lock(struct zspage *zspage) +{ + return __read_lock(zspage); +} + +static void zspage_read_unlock(struct zspage *zspage) +{ + return __read_unlock(zspage); +} + +static __must_check bool zspage_write_trylock(struct zspage *zspage) +{ + return __write_trylock(zspage); +} + +static void zspage_write_unlock(struct zspage *zspage) +{ + return __write_unlock(zspage); +} + /* huge object: pages_per_zspage == 1 && maxobj_per_zspage == 1 */ static void SetZsHugePage(struct zspage *zspage) { @@ -290,12 +447,6 @@ static bool ZsHugePage(struct zspage *zspage) return zspage->huge; } -static void migrate_lock_init(struct zspage *zspage); -static void migrate_read_lock(struct zspage *zspage); -static void migrate_read_unlock(struct zspage *zspage); -static void migrate_write_lock(struct zspage *zspage); -static void migrate_write_unlock(struct zspage *zspage); - #ifdef CONFIG_COMPACTION static void kick_deferred_free(struct zs_pool *pool); static void init_deferred_free(struct zs_pool *pool); @@ -992,7 +1143,9 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, return NULL; zspage->magic = ZSPAGE_MAGIC; - migrate_lock_init(zspage); + zspage->pool = pool; + zspage->class = class->index; + zspage_lock_init(zspage); for (i = 0; i < class->pages_per_zspage; i++) { struct zpdesc *zpdesc; @@ -1015,8 +1168,6 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, create_page_chain(class, zspage, zpdescs); init_zspage(class, zspage); - zspage->pool = pool; - zspage->class = class->index; return zspage; } @@ -1217,7 +1368,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, * zs_unmap_object API so delegate the locking from class to zspage * which is smaller granularity. */ - migrate_read_lock(zspage); + zspage_read_lock(zspage); read_unlock(&pool->lock); class = zspage_class(pool, zspage); @@ -1277,7 +1428,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) } local_unlock(&zs_map_area.lock); - migrate_read_unlock(zspage); + zspage_read_unlock(zspage); } EXPORT_SYMBOL_GPL(zs_unmap_object); @@ -1671,18 +1822,18 @@ static void lock_zspage(struct zspage *zspage) /* * Pages we haven't locked yet can be migrated off the list while we're * trying to lock them, so we need to be careful and only attempt to - * lock each page under migrate_read_lock(). Otherwise, the page we lock + * lock each page under zspage_read_lock(). Otherwise, the page we lock * may no longer belong to the zspage. This means that we may wait for * the wrong page to unlock, so we must take a reference to the page - * prior to waiting for it to unlock outside migrate_read_lock(). + * prior to waiting for it to unlock outside zspage_read_lock(). */ while (1) { - migrate_read_lock(zspage); + zspage_read_lock(zspage); zpdesc = get_first_zpdesc(zspage); if (zpdesc_trylock(zpdesc)) break; zpdesc_get(zpdesc); - migrate_read_unlock(zspage); + zspage_read_unlock(zspage); zpdesc_wait_locked(zpdesc); zpdesc_put(zpdesc); } @@ -1693,41 +1844,16 @@ static void lock_zspage(struct zspage *zspage) curr_zpdesc = zpdesc; } else { zpdesc_get(zpdesc); - migrate_read_unlock(zspage); + zspage_read_unlock(zspage); zpdesc_wait_locked(zpdesc); zpdesc_put(zpdesc); - migrate_read_lock(zspage); + zspage_read_lock(zspage); } } - migrate_read_unlock(zspage); + zspage_read_unlock(zspage); } #endif /* CONFIG_COMPACTION */ -static void migrate_lock_init(struct zspage *zspage) -{ - rwlock_init(&zspage->lock); -} - -static void migrate_read_lock(struct zspage *zspage) __acquires(&zspage->lock) -{ - read_lock(&zspage->lock); -} - -static void migrate_read_unlock(struct zspage *zspage) __releases(&zspage->lock) -{ - read_unlock(&zspage->lock); -} - -static void migrate_write_lock(struct zspage *zspage) -{ - write_lock(&zspage->lock); -} - -static void migrate_write_unlock(struct zspage *zspage) -{ - write_unlock(&zspage->lock); -} - #ifdef CONFIG_COMPACTION static const struct movable_operations zsmalloc_mops; @@ -1769,7 +1895,7 @@ static bool zs_page_isolate(struct page *page, isolate_mode_t mode) } static int zs_page_migrate(struct page *newpage, struct page *page, - enum migrate_mode mode) + enum migrate_mode mode) { struct zs_pool *pool; struct size_class *class; @@ -1785,9 +1911,6 @@ static int zs_page_migrate(struct page *newpage, struct page *page, VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); - /* We're committed, tell the world that this is a Zsmalloc page. */ - __zpdesc_set_zsmalloc(newzpdesc); - /* The page is locked, so this pointer must remain valid */ zspage = get_zspage(zpdesc); pool = zspage->pool; @@ -1803,8 +1926,15 @@ static int zs_page_migrate(struct page *newpage, struct page *page, * the class lock protects zpage alloc/free in the zspage. */ spin_lock(&class->lock); - /* the migrate_write_lock protects zpage access via zs_map_object */ - migrate_write_lock(zspage); + /* the zspage write_lock protects zpage access via zs_map_object */ + if (!zspage_write_trylock(zspage)) { + spin_unlock(&class->lock); + write_unlock(&pool->lock); + return -EINVAL; + } + + /* We're committed, tell the world that this is a Zsmalloc page. */ + __zpdesc_set_zsmalloc(newzpdesc); offset = get_first_obj_offset(zpdesc); s_addr = kmap_local_zpdesc(zpdesc); @@ -1835,7 +1965,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, */ write_unlock(&pool->lock); spin_unlock(&class->lock); - migrate_write_unlock(zspage); + zspage_write_unlock(zspage); zpdesc_get(newzpdesc); if (zpdesc_zone(newzpdesc) != zpdesc_zone(zpdesc)) { @@ -1971,9 +2101,11 @@ static unsigned long __zs_compact(struct zs_pool *pool, if (!src_zspage) break; - migrate_write_lock(src_zspage); + if (!zspage_write_trylock(src_zspage)) + break; + migrate_zspage(pool, src_zspage, dst_zspage); - migrate_write_unlock(src_zspage); + zspage_write_unlock(src_zspage); fg = putback_zspage(class, src_zspage); if (fg == ZS_INUSE_RATIO_0) { @@ -2233,7 +2365,9 @@ struct zs_pool *zs_create_pool(const char *name) * trigger compaction manually. Thus, ignore return code. */ zs_register_shrinker(pool); - +#ifdef CONFIG_DEBUG_LOCK_ALLOC + lockdep_register_key(&pool->lock_class); +#endif return pool; err: @@ -2270,6 +2404,10 @@ void zs_destroy_pool(struct zs_pool *pool) kfree(class); } +#ifdef CONFIG_DEBUG_LOCK_ALLOC + lockdep_unregister_key(&pool->lock_class); +#endif + destroy_cache(pool); kfree(pool->name); kfree(pool); From patchwork Fri Feb 14 04:50:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974474 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 939DEC021A4 for ; Fri, 14 Feb 2025 04:53:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2A39C6B00A4; Thu, 13 Feb 2025 23:53:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 22D516B00A5; Thu, 13 Feb 2025 23:53:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0803C6B00A6; Thu, 13 Feb 2025 23:53:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id DC1B16B00A4 for ; Thu, 13 Feb 2025 23:53:35 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 9AB1B161D34 for ; Fri, 14 Feb 2025 04:53:35 +0000 (UTC) X-FDA: 83117331990.17.C5FF88F Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf02.hostedemail.com (Postfix) with ESMTP id C1F6B80004 for ; Fri, 14 Feb 2025 04:53:33 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=gxnU7RjN; spf=pass (imf02.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508813; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QjapF9ytftJrXMbeHEPYkCcc21hA6s0G0LeFdpvgJ8k=; b=jQ/XwV0aX5pfFfBI4vZanyD4SVPZSIjgpUfUxKF1LGPvniTKRPNyc6fOpOUFL6gRQmcV9T j+5dtZqBkkT5hVrnxfVqqczOJfate+ewMJGpoBQVPS+3XC+fqVtARVT181a4Zt0hUeU+F0 CXQKoEwPgdlIzsh7PPFtAD4F96gyAXQ= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=gxnU7RjN; spf=pass (imf02.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508813; a=rsa-sha256; cv=none; b=c2enB/MUc/J0k1YJLNQU3XNoQ/DI4g8MiITWuAyFkrN3H//vkfGnH/tiZgGr5Ooc8Perdw Ya/9e6tE8lqRom20vKc1kmNxu/yuo57xeCQbJkBMPmxscXR/f5lv8+5rzuMEIyCj1hhBqL RXAAwo6qrUcHbepx4w4h8BrbVtJWs0E= Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-220c92c857aso25950835ad.0 for ; Thu, 13 Feb 2025 20:53:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508813; x=1740113613; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QjapF9ytftJrXMbeHEPYkCcc21hA6s0G0LeFdpvgJ8k=; b=gxnU7RjNFzod0ZWrboargt+a/21AQ6MGqr4qJLEaDXqz1iVIeElZcDKwYw5I+yG8R2 9koozTP511L0W3Onxwid53DGdFRaJOxGMWSbcae8QhcAB5Kt9V+kOfdqrjo0u1WjM2ls eVe640Z4Jo1g1cxKfr2uaC2zW8xdbqdjElJAc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508813; x=1740113613; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QjapF9ytftJrXMbeHEPYkCcc21hA6s0G0LeFdpvgJ8k=; b=Y798nhUS+286YSuZ3QgqJh96YuU5RSCw5kzFMCD+hOAY5LZrTJeHWI4iKMk8v5ZjLD /BWElQEBERPhjAG/2pDJaSl/RtVwya97ZwtjOGfjGByjOd16LDRVfHT+IYetB74ORYFq U9Dv5crU07+oY2US4JO4EddDUUu6VWd1y9PYO7XndPLalrmrR9ltqxZIINCJi4YGertc /ydBNqMdIrBvNYR8XqdBbHhvrRD7yin7dsrIZ6IvQ7idi1SIoD/z9amQczilq3bj2oEF +cCBguCGkKTHLhlkVW4hkiShTdgSUVeRXO0ZWxS/UyhlQXy8UxO0DCGIUO5A0GZqPqdM tXkQ== X-Forwarded-Encrypted: i=1; AJvYcCUWIPx8ZjapwajUy0fbxcfFsDbkNuwVcYzZwF3B+hqvf8v7qaYpsKB9DY3z5cEjzR+sqdp+tu0hyg==@kvack.org X-Gm-Message-State: AOJu0Yx0XP3Xlzg5aYkjvKgbVJXLkaGgkfw14qritnRnl0ejHqGqFEEt c/mRWGKLGFL31nhuX6BKcOsq7ioYdUwuvbMlfPoB5LH7O9MDyXVT0STPTnVvgg== X-Gm-Gg: ASbGncvQdNSgYz3ZS9aTox40zj9ep3nh8VlbAza8D/wMIp25nZapfxDw4pyoq3wHtK9 qC4Pmxgda2g4OAIXqB1aXwG6+PSlnIJ2xw+IjdyYtMrLYY9aqvDoqupNK2R+Y3k23XtW9vUy6qD 0Fck612gcS6tsUi9NVibO1cxFE1E5OsWuAwP6opfurPlHsIEY/mb9i4xsGYCHMSh2us8w0tkQWR g0prjmdwh6+elJCXh9ayedZjZszcw6lQr4eNuNWHWyi3ii+c27CNzByfklvyxerTNiwVZyvXnDh 6pP6A5mMp1XPBbGfOA== X-Google-Smtp-Source: AGHT+IFqSB8efCw3bZ9x/H3wNB/h7WT1Sozgx+u9wqyq8bV1nirLfbUOM2pE4SmRpm8pCa8TR9oGIg== X-Received: by 2002:a17:902:ef05:b0:220:e91f:4408 with SMTP id d9443c01a7336-220e91f487dmr29258795ad.22.1739508812669; Thu, 13 Feb 2025 20:53:32 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2fbf98b4c45sm4198281a91.2.2025.02.13.20.53.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:53:32 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 12/17] zsmalloc: introduce new object mapping API Date: Fri, 14 Feb 2025 13:50:24 +0900 Message-ID: <20250214045208.1388854-13-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: C1F6B80004 X-Stat-Signature: 6cjuga6jcp3d4zb3yefa54w89w331w5i X-Rspam-User: X-HE-Tag: 1739508813-188817 X-HE-Meta: U2FsdGVkX18+QrjE/xVnt0Y1n+UrhAcMJYRDMtxy+p4gFa3pt8X5DaETnoSyoxluY55DVD5aJF18nQ7cbZQ1z8t+8HiJcn/QDm+228C2Xsc25B9aVvI8CjPuCHUWUtpPPh2HmOO3QmdoqfdujaEF7ToUqODXwkCniJ42CRo/0gN+d7fdbgtDr1U/lCS/s2IMuXCfndhZVoz3RvkBzR5dLkICckk60SCypSPLIOYxJHhCk/boUu88BLBTMOkBn+qAGoSy0lC805d/s8F14pmrxfuD++hSbc4n+43389Cwnb7XWsbNbFp/H/L7T7Rbydqn0g4nlcmGKHHmBdr1Kn094BHzONPGGL8RYU4FDxOhBR81etJmBmG7aLGJM8YtX0jeCeh23NasGrCDnTBdXYUaoul7tE3kg77lNIN2WaTJwvXJW19bGosR2DeboIpSmFAAKetjg2QDuESZNEcBcjYGPpG/G8uJoxs5SZ0u73mJ3A8Bx3q05FNwMxApD+MTOpkk3hnK/+88lFEmiKw6oKANXzfuD3qD1ierbcELKOmU/k0i06b5CMTN8Pw56JAqV6CO1SkHxwmX6SFbXE3f1rz1YI/irYbAJgwfOVPWLbr5deQLgfdhx8Fc6pHju0p4dRX55jO6B8zwVbZW4YsyxbWsu7bWwHvv+k8p567znpCek6QxR+j9+pyJSGNdzIXH5Yzr0niRkrrjW9JPmM1H8SHod4VAqlmj7royRK7ZBbqVdm950HNL1mE0YBM6J3ee4F9zKTzT6csATCB2GdDVAXEYwJVjwyj6e9G38UGsoJJex7f0AFQcmhGX7HS7x1KuQZMJxFY0bniQlhXrxLVFgU4Ha/+zSHIsIRXgLzsbB2zw57P7LyCABV9ybrs4xFx2NTdWitIA0is/uoNsKFhXfnD42jZ245eNMRRkkim4W3o71G4QH6lvHc3tiQvLHojUAcnhRsHoH67JTqJs8RYLRGm wS9b1sgT gA9dyRmAMEaPhT+Oyc+wzOEQUcqAqzJw5PcL6mRMRCPf7rpkSf69/cuQ4FBGmCrXCYcLq/voPPIIR2nUmD38mxFC6yv8uINWaPpythmwWDgLmfa4Mkl/FaOAh2Het6ex/llBIJcsu6Gq9O7pKlOm2JuiLZq+xxMoynx5AE8N904vCLINCl6FAKsOfKU73ecpyw/Q4KxSikMGI7b3DXckrRxCFRUD+tVc96937JbsZRvlvCbkBKNH0b1ZZ0NWrhHhxDL/mWy9VrKTqAScpvFOTwWJBDXitlhDUV5t3jFIiZX9d6EUIEyNcYgM+NQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Current object mapping API is a little cumbersome. First, it's inconsistent, sometimes it returns with page-faults disabled and sometimes with page-faults enabled. Second, and most importantly, it enforces atomicity restrictions on its users. zs_map_object() has to return a liner object address which is not always possible because some objects span multiple physical (non-contiguous) pages. For such objects zsmalloc uses a per-CPU buffer to which object's data is copied before a pointer to that per-CPU buffer is returned back to the caller. This leads to another, final, issue - extra memcpy(). Since the caller gets a pointer to per-CPU buffer it can memcpy() data only to that buffer, and during zs_unmap_object() zsmalloc will memcpy() from that per-CPU buffer to physical pages that object in question spans across. New API splits functions by access mode: - zs_obj_read_begin(handle, local_copy) Returns a pointer to handle memory. For objects that span two physical pages a local_copy buffer is used to store object's data before the address is returned to the caller. Otherwise the object's page is kmap_local mapped directly. - zs_obj_read_end(handle, buf) Unmaps the page if it was kmap_local mapped by zs_obj_read_begin(). - zs_obj_write(handle, buf, len) Copies len-bytes from compression buffer to handle memory (takes care of objects that span two pages). This does not need any additional (e.g. per-CPU) buffers and writes the data directly to zsmalloc pool pages. In terms of performance, on a synthetic and completely reproducible test that allocates fixed number of objects of fixed sizes and iterates over those objects, first mapping in RO then in RW mode: OLD API ======= 3 first results out of 10 369,205,778 instructions # 0.80 insn per cycle 40,467,926 branches # 113.732 M/sec 369,002,122 instructions # 0.62 insn per cycle 40,426,145 branches # 189.361 M/sec 369,036,706 instructions # 0.63 insn per cycle 40,430,860 branches # 204.105 M/sec [..] NEW API ======= 3 first results out of 10 265,799,293 instructions # 0.51 insn per cycle 29,834,567 branches # 170.281 M/sec 265,765,970 instructions # 0.55 insn per cycle 29,829,019 branches # 161.602 M/sec 265,764,702 instructions # 0.51 insn per cycle 29,828,015 branches # 189.677 M/sec [..] T-test on all 10 runs ===================== Difference at 95.0% confidence -1.03219e+08 +/- 55308.7 -27.9705% +/- 0.0149878% (Student's t, pooled s = 58864.4) The old API will stay around until the remaining users switch to the new one. After that we'll also remove zsmalloc per-CPU buffer and CPU hotplug handling. The split of map(RO) and map(WO) into read_{begin/end}/write is suggested by Yosry Ahmed. Suggested-by: Yosry Ahmed Signed-off-by: Sergey Senozhatsky --- include/linux/zsmalloc.h | 8 +++ mm/zsmalloc.c | 129 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 137 insertions(+) diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h index a48cd0ffe57d..7d70983cf398 100644 --- a/include/linux/zsmalloc.h +++ b/include/linux/zsmalloc.h @@ -58,4 +58,12 @@ unsigned long zs_compact(struct zs_pool *pool); unsigned int zs_lookup_class_index(struct zs_pool *pool, unsigned int size); void zs_pool_stats(struct zs_pool *pool, struct zs_pool_stats *stats); + +void *zs_obj_read_begin(struct zs_pool *pool, unsigned long handle, + void *local_copy); +void zs_obj_read_end(struct zs_pool *pool, unsigned long handle, + void *handle_mem); +void zs_obj_write(struct zs_pool *pool, unsigned long handle, + void *handle_mem, size_t mem_len); + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index bc679a3e1718..51d8080f187a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1432,6 +1432,135 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) } EXPORT_SYMBOL_GPL(zs_unmap_object); +void *zs_obj_read_begin(struct zs_pool *pool, unsigned long handle, + void *local_copy) +{ + struct zspage *zspage; + struct zpdesc *zpdesc; + unsigned long obj, off; + unsigned int obj_idx; + struct size_class *class; + void *addr; + + WARN_ON(in_interrupt()); + + /* Guarantee we can get zspage from handle safely */ + read_lock(&pool->lock); + obj = handle_to_obj(handle); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc); + + /* Make sure migration doesn't move any pages in this zspage */ + zspage_read_lock(zspage); + read_unlock(&pool->lock); + + class = zspage_class(pool, zspage); + off = offset_in_page(class->size * obj_idx); + + if (off + class->size <= PAGE_SIZE) { + /* this object is contained entirely within a page */ + addr = kmap_local_zpdesc(zpdesc); + addr += off; + } else { + size_t sizes[2]; + + /* this object spans two pages */ + sizes[0] = PAGE_SIZE - off; + sizes[1] = class->size - sizes[0]; + addr = local_copy; + + memcpy_from_page(addr, zpdesc_page(zpdesc), + off, sizes[0]); + zpdesc = get_next_zpdesc(zpdesc); + memcpy_from_page(addr + sizes[0], + zpdesc_page(zpdesc), + 0, sizes[1]); + } + + if (!ZsHugePage(zspage)) + addr += ZS_HANDLE_SIZE; + + return addr; +} +EXPORT_SYMBOL_GPL(zs_obj_read_begin); + +void zs_obj_read_end(struct zs_pool *pool, unsigned long handle, + void *handle_mem) +{ + struct zspage *zspage; + struct zpdesc *zpdesc; + unsigned long obj, off; + unsigned int obj_idx; + struct size_class *class; + + obj = handle_to_obj(handle); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc); + class = zspage_class(pool, zspage); + off = offset_in_page(class->size * obj_idx); + + if (off + class->size <= PAGE_SIZE) { + if (!ZsHugePage(zspage)) + off += ZS_HANDLE_SIZE; + handle_mem -= off; + kunmap_local(handle_mem); + } + + zspage_read_unlock(zspage); +} +EXPORT_SYMBOL_GPL(zs_obj_read_end); + +void zs_obj_write(struct zs_pool *pool, unsigned long handle, + void *handle_mem, size_t mem_len) +{ + struct zspage *zspage; + struct zpdesc *zpdesc; + unsigned long obj, off; + unsigned int obj_idx; + struct size_class *class; + + WARN_ON(in_interrupt()); + + /* Guarantee we can get zspage from handle safely */ + read_lock(&pool->lock); + obj = handle_to_obj(handle); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc); + + /* Make sure migration doesn't move any pages in this zspage */ + zspage_read_lock(zspage); + read_unlock(&pool->lock); + + class = zspage_class(pool, zspage); + off = offset_in_page(class->size * obj_idx); + + if (off + class->size <= PAGE_SIZE) { + /* this object is contained entirely within a page */ + void *dst = kmap_local_zpdesc(zpdesc); + + if (!ZsHugePage(zspage)) + off += ZS_HANDLE_SIZE; + memcpy(dst + off, handle_mem, mem_len); + kunmap_local(dst); + } else { + /* this object spans two pages */ + size_t sizes[2]; + + off += ZS_HANDLE_SIZE; + sizes[0] = PAGE_SIZE - off; + sizes[1] = mem_len - sizes[0]; + + memcpy_to_page(zpdesc_page(zpdesc), off, + handle_mem, sizes[0]); + zpdesc = get_next_zpdesc(zpdesc); + memcpy_to_page(zpdesc_page(zpdesc), 0, + handle_mem + sizes[0], sizes[1]); + } + + zspage_read_unlock(zspage); +} +EXPORT_SYMBOL_GPL(zs_obj_write); + /** * zs_huge_class_size() - Returns the size (in bytes) of the first huge * zsmalloc &size_class. From patchwork Fri Feb 14 04:50:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974475 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABC67C021A4 for ; Fri, 14 Feb 2025 04:53:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 42BFF6B00A6; Thu, 13 Feb 2025 23:53:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3B49A6B00A7; Thu, 13 Feb 2025 23:53:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 22E30280001; Thu, 13 Feb 2025 23:53:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id F2F666B00A6 for ; Thu, 13 Feb 2025 23:53:41 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B0436161D40 for ; Fri, 14 Feb 2025 04:53:41 +0000 (UTC) X-FDA: 83117332242.07.3CD8AF2 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf12.hostedemail.com (Postfix) with ESMTP id CD6204000C for ; Fri, 14 Feb 2025 04:53:39 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Hmwjhs9O; spf=pass (imf12.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.169 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508819; a=rsa-sha256; cv=none; b=zJ5OPQSW++1rAilf6mQUNq6KZpJREhq7+Ig5VNcKvKkOgdtUob+P2CiDpUUToS3LcmGG1x hdXbz1fA2+RkpYfTK7/d8jzRKZttU84EKIAnxLcWLJonqfqqJEj36EQqmCtWE5OrmMG5qh LsfvtdOGP6kKAfFHu41rrYBy96ETjgU= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Hmwjhs9O; spf=pass (imf12.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.169 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508819; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=g8MSYx1r6vOiRHwxjnCDNpKvIAk9Qp1UAjAoAUDj4z4=; b=pHxeS86xC1DiTtxwJyFjGAbQ3xQEYDva6cF2cPrnXLuietGUt+IMcZEjcuozgobU2o6utW Jz5SR7rVCE3SMNAe4R10q0O/hhEd3h3gtLwHgLH3qEoOChNhVtDbm4fskPi2aokyng4VYv 07DD9g5NCBUUMwR1PvfMPiBE3BrhcW8= Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-220c92c857aso25951675ad.0 for ; Thu, 13 Feb 2025 20:53:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508819; x=1740113619; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g8MSYx1r6vOiRHwxjnCDNpKvIAk9Qp1UAjAoAUDj4z4=; b=Hmwjhs9OzIxaQbWczXCQIKsah/syruFEeZmVBclYvQ9nKvRzFa2Y1ORbqVJbfpBicq ku1K9HavjrLQODeeL23LNajNE6ryc3j8sIxW9d22hi54+pfK7cXG1ojkrZY5xxF5MZYR llajou0tHTmWYEb2/m+gVXPoUPIkSkQOaCyZQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508819; x=1740113619; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g8MSYx1r6vOiRHwxjnCDNpKvIAk9Qp1UAjAoAUDj4z4=; b=D2tfUVO6IuqdYyeLhGyVIGQrBnwSP3Lafo/cQXslEeadK8sDjOcExdDUtU/lixGm8R JKAqo7f6TIp8TaZXGTVQxsTLSLdy5GGxNZyaDhW+UNLTZE5A4Pf1no5+6vKPZTgTrj9Y m7kJ/9O1wqlJrlIV2Tms+TkO5lxDzKZzsysdOygdZrKQtjza4yQ6DbsJnALA8m7mf+qT h9EOB2t3PrCS6vtgUyufQn79XfebEkdRveY2C2xiJoGF8I1+azbdq3/LZhrhhGVWHzn/ tznb+bl0Ud9AU6w5EWyB5hKyLmlZYqvwx684i2p1UdME8M0uJtKIa7Q5jzUMDfNRF3KQ zDXg== X-Forwarded-Encrypted: i=1; AJvYcCWBcdtXGw3Qe6/55giGjRrcORUfcYtUI1q2KPDEcTeTNDb1tWbgVSRg9jQ/mwuAp8EI+wu/Qfht4Q==@kvack.org X-Gm-Message-State: AOJu0Yxo242DSm4Lh6Tlb9+fCY9cmByIiqQBZAvhDG+yyuZZDKmbuWUS Al3YGWIvXAFpS8klMVPhLWG8tvNHyfe6pZIXc23GRkWmlY84Otpf+IKu4jVa3g== X-Gm-Gg: ASbGncu6t13getje4nFGH0P5dK+SfOIP8I3K3ie3J7T3lbS3IH2lsykFPVf/Z8yJely n1/+TwFEetZzPS5YcuzTc3WUQv2KDpiS+Ds0i8s4gMBvzysSatAkB+yqRNNiKbndCcZQGgoZEC9 0p/YIfvWuylfSsKIAdcO/Wck+jx/HcrQ/pfrWWkVN7Ta/5sfzAyMc/Whcg6fSoWKaFkHBJ6Dc6y BvvZhmU5wial0Ar3dsjDEoKZSYBHTCl1D9wFiL4loGdzkFRZB8CezVldSAybgpWfSTq02XA6kjZ Ai7RFlN+d2/aMLexfw== X-Google-Smtp-Source: AGHT+IFj0J84blWWRorX7QC5QTsPm4yxqb/UskRAxNtWaOKmY3i/R9l8RGftlk8+B1lrXJZdO59P7Q== X-Received: by 2002:a17:903:18a:b0:220:e98e:4f1b with SMTP id d9443c01a7336-220e98e52b9mr44389295ad.0.1739508818756; Thu, 13 Feb 2025 20:53:38 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-220d53492d0sm20895595ad.35.2025.02.13.20.53.36 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:53:38 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 13/17] zram: switch to new zsmalloc object mapping API Date: Fri, 14 Feb 2025 13:50:25 +0900 Message-ID: <20250214045208.1388854-14-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Stat-Signature: 7kbdge1ccatz5yu4fwtucn63p7hh8a7n X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: CD6204000C X-Rspam-User: X-HE-Tag: 1739508819-597541 X-HE-Meta: U2FsdGVkX1+5VPC6vaEyASmDFCZej8flF++CRhlW/J4rVY8bu6TKPply903VG5fUInnFnNE9gwPGgRmO/IakiPrxSb6EQixLbXoRwkWBK6h2Gell2I2vwLsDkfv50LLKSyp9Pq5G4YQhD7gSSxjGd4+2aifN/gRT0+e1W87+C1qN748cTW28Ux8hgzVlj6GNcDeo4R//kjZsDURZDYriSyYzgP6g/XzgZGkHpTJ5rLjBNCxGpu0QCwpe2gIZoDrmr4/ytYGz6f8mGYRocLqK7hKyZv/eIAeKc9tjwwiAffLYQc5xK6Ikad4NBEDCObQ9mH+vsvjAy4/zvpV0gN2UIh/bGIaLjDbZ6Y93iB3Kj0klaYi2LZDOCOKco99y2lLxJ7iRFgUchpZiO3ioB4l/Zw80YdwyD+cjGaPaPdJqnnSJkLqGY/aeXpQq3zQEX8TMpE564/PDjc9bYV8Hv7xLkt2hT3bABb1H8YCdDIX/ECnMSnLgSCix4RD/33yKHA9J2bMRzqpW0fM8kEBTrG9GlrhXC3pzDSJGl3s7YfDVAPW5i/lP745gTyEOf+Vvdf5CgBwL6lQXs1cxyPC2mbR/Ha/0eIh0akl4aOakkuO7bgs+pbMGCa0lPDMgmsgtNlrrkdbRNHZtbZMOgrX5MzCdXkn+eEy3XSdPlfCwcWpQv49KSr8n07xcd3yO/Amd7aiUw6BIEch/x6Mp40qwsuD4jbD5Gcc/4JqJWbpDkkILaMOKZ+0XWCJwuUzqB06Tw1DqvJ3MeE1l7b4+Gkfhkd7vkaxee2lFjeE0bZhppSvMh9YowA4u77fF2NlizUbzP4TvnTlAzCKfPFIfYzvaw+voMcxQUjGtj8itiUb26YPn59rHrrfRzdo6J3nBFCe1yCOwbfFUyJ/50YEcgepc5TnDTjNCIZhfgrdp89LD73fgvCMV1Zit7RPJoekr/D48K91yVuGLKZkVf2+S5d7FAbZ 8V0iWghW PMlzRHE3E2Kbo3+54250PpHgnFUqKSXWcBY2QTzVJCA7yZUIV06ulvzrHk2Ej89vZ94W6XMrZjUtPtCUCjbCz86cDoNaS+rpeKLYRhHIH4BRdfD4nLw7DAswCQ+EbH/tuCkjFAU/vsKt8bgdU3KLKLlGvBRnavqGHh5BuCX6ggBqzfBwzCqN2/3DJKqzetZ9WstOpqMUbdj0EZvx3emKHn/nEfGcFtC/FemYDEgLwJQMK/CG4q6Nf1CWypA28wIZOj05LEpv54jX73T+nwhrfMkXdf4oxyjj/XGtsJr4Nu/uA6jL6iC5JgZJPUA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use new read/write zsmalloc object API. For cases when RO mapped object spans two physical pages (requires temp buffer) compression streams now carry around one extra physical page. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zcomp.c | 4 +++- drivers/block/zram/zcomp.h | 2 ++ drivers/block/zram/zram_drv.c | 28 ++++++++++------------------ 3 files changed, 15 insertions(+), 19 deletions(-) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index cfdde2e0748a..a1d627054bb1 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -45,6 +45,7 @@ static const struct zcomp_ops *backends[] = { static void zcomp_strm_free(struct zcomp *comp, struct zcomp_strm *zstrm) { comp->ops->destroy_ctx(&zstrm->ctx); + vfree(zstrm->local_copy); vfree(zstrm->buffer); zstrm->buffer = NULL; } @@ -57,12 +58,13 @@ static int zcomp_strm_init(struct zcomp *comp, struct zcomp_strm *zstrm) if (ret) return ret; + zstrm->local_copy = vzalloc(PAGE_SIZE); /* * allocate 2 pages. 1 for compressed data, plus 1 extra for the * case when compressed size is larger than the original one */ zstrm->buffer = vzalloc(2 * PAGE_SIZE); - if (!zstrm->buffer) { + if (!zstrm->buffer || !zstrm->local_copy) { zcomp_strm_free(comp, zstrm); return -ENOMEM; } diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h index 23b8236b9090..25339ed1e07e 100644 --- a/drivers/block/zram/zcomp.h +++ b/drivers/block/zram/zcomp.h @@ -34,6 +34,8 @@ struct zcomp_strm { struct mutex lock; /* compression buffer */ void *buffer; + /* local copy of handle memory */ + void *local_copy; struct zcomp_ctx ctx; }; diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 2e46ea542ecd..87ce304a60aa 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1603,11 +1603,11 @@ static int read_incompressible_page(struct zram *zram, struct page *page, void *src, *dst; handle = zram_get_handle(zram, index); - src = zs_map_object(zram->mem_pool, handle, ZS_MM_RO); + src = zs_obj_read_begin(zram->mem_pool, handle, NULL); dst = kmap_local_page(page); copy_page(dst, src); kunmap_local(dst); - zs_unmap_object(zram->mem_pool, handle); + zs_obj_read_end(zram->mem_pool, handle, src); return 0; } @@ -1625,11 +1625,11 @@ static int read_compressed_page(struct zram *zram, struct page *page, u32 index) prio = zram_get_priority(zram, index); zstrm = zcomp_stream_get(zram->comps[prio]); - src = zs_map_object(zram->mem_pool, handle, ZS_MM_RO); + src = zs_obj_read_begin(zram->mem_pool, handle, zstrm->local_copy); dst = kmap_local_page(page); ret = zcomp_decompress(zram->comps[prio], zstrm, src, size, dst); kunmap_local(dst); - zs_unmap_object(zram->mem_pool, handle); + zs_obj_read_end(zram->mem_pool, handle, src); zcomp_stream_put(zstrm); return ret; @@ -1725,7 +1725,7 @@ static int write_incompressible_page(struct zram *zram, struct page *page, u32 index) { unsigned long handle; - void *src, *dst; + void *src; /* * This function is called from preemptible context so we don't need @@ -1742,11 +1742,9 @@ static int write_incompressible_page(struct zram *zram, struct page *page, return -ENOMEM; } - dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); src = kmap_local_page(page); - memcpy(dst, src, PAGE_SIZE); + zs_obj_write(zram->mem_pool, handle, src, PAGE_SIZE); kunmap_local(src); - zs_unmap_object(zram->mem_pool, handle); zram_slot_lock(zram, index); zram_set_flag(zram, index, ZRAM_HUGE); @@ -1767,7 +1765,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) int ret = 0; unsigned long handle; unsigned int comp_len; - void *dst, *mem; + void *mem; struct zcomp_strm *zstrm; unsigned long element; bool same_filled; @@ -1813,11 +1811,8 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return -ENOMEM; } - dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); - - memcpy(dst, zstrm->buffer, comp_len); + zs_obj_write(zram->mem_pool, handle, zstrm->buffer, comp_len); zcomp_stream_put(zstrm); - zs_unmap_object(zram->mem_pool, handle); zram_slot_lock(zram, index); zram_set_handle(zram, index, handle); @@ -1929,7 +1924,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, unsigned int comp_len_new; unsigned int class_index_old; unsigned int class_index_new; - void *src, *dst; + void *src; int ret = 0; handle_old = zram_get_handle(zram, index); @@ -2039,12 +2034,9 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, return PTR_ERR((void *)handle_new); } - dst = zs_map_object(zram->mem_pool, handle_new, ZS_MM_WO); - memcpy(dst, zstrm->buffer, comp_len_new); + zs_obj_write(zram->mem_pool, handle_new, zstrm->buffer, comp_len_new); zcomp_stream_put(zstrm); - zs_unmap_object(zram->mem_pool, handle_new); - zram_free_page(zram, index); zram_set_handle(zram, index, handle_new); zram_set_obj_size(zram, index, comp_len_new); From patchwork Fri Feb 14 04:50:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974476 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 831F3C021A4 for ; Fri, 14 Feb 2025 04:53:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 207B76B00A8; Thu, 13 Feb 2025 23:53:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 191EF6B00A9; Thu, 13 Feb 2025 23:53:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 00BC9280001; Thu, 13 Feb 2025 23:53:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D344F6B00A8 for ; Thu, 13 Feb 2025 23:53:48 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 7AF38C1D68 for ; Fri, 14 Feb 2025 04:53:48 +0000 (UTC) X-FDA: 83117332536.06.115CF28 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf06.hostedemail.com (Postfix) with ESMTP id 82DAA18000D for ; Fri, 14 Feb 2025 04:53:46 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=OCWfwOGJ; spf=pass (imf06.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.179 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508826; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cvgHEnC+lRfOrRLv2RIVOYlOBk4EIx2tIzOwTFt2Qag=; b=kipk8ZhRcUhrMlY6gqbAy1JwWiYimC0berXd0xxzQWdYWqVh9ML9KORYLfNa1usotIhG+m LiUBk3SHyGGGhc3I36rpP+NjSeSU6PftGLoz3qeZVuoEoAL3sk2iWleLYJgujDvzuSuWv9 0tSgWXJVVJYU8tO/90Q7d0yMTeQiczw= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=OCWfwOGJ; spf=pass (imf06.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.179 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508826; a=rsa-sha256; cv=none; b=rkR6OApFpmpB23S0LS9pbPRoHPtc1CMKjkDAqtl1usZDLFlGXnpX2XnJpPUGBmfA7+h6KY 35tcBHsmOmMAM9Mgw6w7doHZNyZWpUk58RseKpim1/O3KoS/n+s0g4ZWtSKzW3KHM4RFKN JGppItMJeZ+VEDOn8spXa0JW1K2/WO4= Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-21f3c119fe6so39161805ad.0 for ; Thu, 13 Feb 2025 20:53:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508825; x=1740113625; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cvgHEnC+lRfOrRLv2RIVOYlOBk4EIx2tIzOwTFt2Qag=; b=OCWfwOGJKwS2N4k4f9N+YWzXPjwbiCMCQNpst4mCKe26Y/v+Q7f6hsfiLm3EacoaCh TnXLtxY/OKS1jPBS4eojVmpcCUa37YjeP7ARVHrrTFGjMXxUach6mNX9bL4tJq9r4End VFQl4wE6PtqXjTFBAmAfjmYcyUFjG9RxXTDEI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508825; x=1740113625; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cvgHEnC+lRfOrRLv2RIVOYlOBk4EIx2tIzOwTFt2Qag=; b=XlzePVZ/L8cbXjKGIZM99zxkR5DKQka3WuYpnFGFSg1fKdvRhO61RiqXEOhtDQ3ruO MBDy5VtI71EatMjJedh2+BSg5oPtsbg+jNZnaIWudnTiw7mU4VQtOg2LzquulajIPr2b NqKWu993uLxpLD+vC77hHNCzUbS+Uh8ZzStFIUp6JNpJ+Hh2A4+QgiYxYX6AfNcJjTfX umK/f3FOydzHIBaXdyEG9pI3/InBfPW110M3iPuJdRaYq7Olvj28ApqssGVjUyz7Er2D 3W23oIT7c7KtjsMaQOSpEaafUczUiPxs3rTXOfzstgljOo2LAOHynHd2rPHRSeG7xQ54 6B1A== X-Forwarded-Encrypted: i=1; AJvYcCUYfJzF6O+tduXkdpmrcebSbEMfalHlNYXJvYmk58OHm2DuUptiv8TKqKX2MY5lBuUbkMU4cBwrYQ==@kvack.org X-Gm-Message-State: AOJu0YzZuiCEQiDphSWFlpc1nRi+YClkMFj/VTD9Ztz4COuIR74WcZsO IvJc5YBWtwPHmqazQJgBpA3z27TF2Ve4zyfT6M1rKd+9wk+0i3vUb9DXltLt5w== X-Gm-Gg: ASbGncu7VD82Uv9GV9LT1vGKImCrZpJnCVU5F7pVBlqFXR8aMXf0kRYeRUx8Ui0e952 1KNey/t15HEzO5/cjAVY1/ZWv0I3bdTLOQyVpdpvAdKYoL2hvUQ+ExdIFvwpG/M69uMFxuBLBxf F0lZxCNJMrIGsGUHGJobqHdbOhOgsl3n1pJe1eesXYtVTdxLw+YQGuidgJSM39336njlnQiDA9Q 3TMdZRs99W9s6IzU74NiOARReHCG1KuCwI8t2hyoMnWmFPZzv2TDLuRlhs5nyn0PGrCqm/ojHnq i0Q1TxIIXGNdO1yvGQ== X-Google-Smtp-Source: AGHT+IFtBGqgz6iEWt7LMNhyXXy7Ao2/LISUPvugqZVTStAhfwtsv9UYgCngz+ynDyppB1uJogIN/g== X-Received: by 2002:a05:6a21:6d85:b0:1ee:85b2:ef73 with SMTP id adf61e73a8af0-1ee85b2efcemr458480637.34.1739508825453; Thu, 13 Feb 2025 20:53:45 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id 41be03b00d2f7-adb5a92c6d8sm2017586a12.65.2025.02.13.20.53.43 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:53:45 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 14/17] zram: permit reclaim in zstd custom allocator Date: Fri, 14 Feb 2025 13:50:26 +0900 Message-ID: <20250214045208.1388854-15-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 82DAA18000D X-Stat-Signature: 1knj44rbzjn6yriem94brnojxcjj4p3k X-HE-Tag: 1739508826-396217 X-HE-Meta: U2FsdGVkX1/drRGpfUa0wj4c18p3I5AVOtlQ3yQeUO4vz7pslSlr5Z+EdJdTqNuEo3Z33E6dRvLCY8+1mc1b7AECE9Lm/ikHW7Vcj8hsbgivAWkuxbzzUuzZyi+JHEGUnifKKyQfnEcyrEJlNmkAcX9xPVIzLh2fnKEXVLmzJZb+IHEsL0fYvLga9a1MwGQvAu6hHky8BztQiMTnT3t4Uo3PuueSFJyjju+FECGi9Afcx7ubQ3kYS86xkqMe4HftLPIa8PVtJWj7tGHATV4AoZo2Lo8/Ys5Xp9k3k9T1CGq5JeijoBXBzE890H7HG8YHE1n7Ld6B6Djw4M611QC561vRjB3WuJgtA+ui4R8/yHpYNordrpThIh4Y9g4/EIuHDBa9GywBLnH+CTONHJwz2ne2a03HsxIysS0n9JMFhgMaFqRXUtzg5gE7d82Nn4G5xiM+po7A3b9Rogv7/Kheqpf3QXREE3wNX7fF0C5q55n2Mh5t0Y0F2po3J5gHd6LxGkM3Ti/4TSVFBQKd7k2G9xfCzEnlf5eCG/yMnb4FEkNaPB9sfEK8tsprV47qKvrT4fMOiToijaDzdmkNyOiuUCstp5aSMn0yIdt73mYWKY1/EWvp6tSx0XoudIV1jf1ljkIbhzngtvfRcVYu0F8SOhgSr4EaUsV+5tAiwoMuX0QAAUBJOuL9PEbSSGnaPnvPfAJ5mSSaI7J2RlwkpBobU/UVj1t/BeczXH/Drt9avNJsCHPhUhgWbJZpwYrf1ZcE5OcXkxTVgKVtd4xGyF9p6D+vHdBgkER6Bcm/osbDb7G0Lft0mKbIe+XA0lqPc/OVl2kCfwV5Ah5K2ZrRYOTl1iI1Et74xiluDtHl6khtVmS48P8gEUAwkXIJYL59t5clTjNEpcZevy2sS7K7r7+i8gnZJtJuk1p4x4w+eIXt3bUFi4CYzI0SxpL8t3rSb0lBSoHaDoT23Kw0D32tjwc t1UG9jGK K2OiNXC1/TnQWw8IjMkzV19maFD7DpUdW3y3yJ+izPbx3+uix3y8WEr3NUGmaxU2fILqm/eCOMiMUpxKV46f/gDB4LMaV2UGGvc4umnYTINk89ycUI/skAcQ5uXnGRA/ev9zLmhdvZAmm3lhUfymDqPE002xgW9QbeMDp04eLO375YxikfdlOJwvZd3IXK0G6Lqz5+rs2MCIc3Oq3c9sH1Wp/+OwILUMiwDZqk5ttw8nlcbKv5FPVVVNhg0kVgdR6VOxAjq/DMsFddr9uvtmolv+HrTgt8m0nklx/0ZXV3C9BkQGyKlAE/1gnkg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When configured with pre-trained compression/decompression dictionary support, zstd requires custom memory allocator, which it calls internally from compression()/decompression() routines. That means allocation from atomic context (either under entry spin-lock, or per-CPU local-lock or both). Now, with non-atomic zram read()/write(), those limitations are relaxed and we can allow direct and indirect reclaim. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/backend_zstd.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/drivers/block/zram/backend_zstd.c b/drivers/block/zram/backend_zstd.c index 1184c0036f44..53431251ea62 100644 --- a/drivers/block/zram/backend_zstd.c +++ b/drivers/block/zram/backend_zstd.c @@ -24,19 +24,14 @@ struct zstd_params { /* * For C/D dictionaries we need to provide zstd with zstd_custom_mem, * which zstd uses internally to allocate/free memory when needed. - * - * This means that allocator.customAlloc() can be called from zcomp_compress() - * under local-lock (per-CPU compression stream), in which case we must use - * GFP_ATOMIC. - * - * Another complication here is that we can be configured as a swap device. */ static void *zstd_custom_alloc(void *opaque, size_t size) { - if (!preemptible()) + /* Technically this should not happen */ + if (WARN_ON_ONCE(!preemptible())) return kvzalloc(size, GFP_ATOMIC); - return kvzalloc(size, __GFP_KSWAPD_RECLAIM | __GFP_NOWARN); + return kvzalloc(size, GFP_NOIO | __GFP_NOWARN); } static void zstd_custom_free(void *opaque, void *address) From patchwork Fri Feb 14 04:50:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974477 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EAEA1C021A4 for ; Fri, 14 Feb 2025 04:53:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7FE41280002; Thu, 13 Feb 2025 23:53:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 784B6280001; Thu, 13 Feb 2025 23:53:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5AF91280002; Thu, 13 Feb 2025 23:53:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 366A2280001 for ; Thu, 13 Feb 2025 23:53:55 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id E8D10B150E for ; Fri, 14 Feb 2025 04:53:54 +0000 (UTC) X-FDA: 83117332788.25.40D2003 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf10.hostedemail.com (Postfix) with ESMTP id 02BADC000F for ; Fri, 14 Feb 2025 04:53:52 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=eaDbH0j7; spf=pass (imf10.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508833; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dOTlfVzF3e2Wm9f7wmP5BEorbNSbvme8XOlj2WM+zhc=; b=Rl8Cg6fTt7feonSy1guvnnDVXxDYGz/GvX1l5uj5BalTlPRfnCWeyG3OBEav37g3N75sKd 719NVJCwgA96kGBORIhtwZuB2qdHpD5VtnvZ1TBI7JMOWonDUwd/pgcVbTwj5c2QAmG2I2 E0mOrRxAXiD8i5m+vU+r3MjApmzKgSg= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=eaDbH0j7; spf=pass (imf10.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508833; a=rsa-sha256; cv=none; b=sdqOKuJajuasi4TByRK2x7DmFLtqhoI9H9IN9q5Syk91q0SwfzWi3dieP818DddzKPMFE3 QMknLCedNrOWtx40Hp6Zz3O/oFjtOT+WIW/vOgNdrzRFoP9HtmVUFeDYWIBJnpAFDB1S4R e2q00PGdGnC8eNMWSDHHK1lxWGutzE0= Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-220d28c215eso22108585ad.1 for ; Thu, 13 Feb 2025 20:53:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508832; x=1740113632; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dOTlfVzF3e2Wm9f7wmP5BEorbNSbvme8XOlj2WM+zhc=; b=eaDbH0j7eUlv9WQNLDtE06NJE0mzHb6PDp2QUZ7V5Ie0c4uwa2lQU9HyLmViTLfvTJ jbLCC0t87yWe78RRnIPSpNf//blGThlVbwsWd3/ael0ggSguLUqXp/L2BcBchlkn9vTU aW5wiksHu9kB+IJnDfPAzseFqj8T3sjkWFE8k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508832; x=1740113632; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dOTlfVzF3e2Wm9f7wmP5BEorbNSbvme8XOlj2WM+zhc=; b=F5Z1W5Uk1qeSlGohV/YzU/kdCtW51ErioqKqH6rEW+qV6t11DfmTkyv+fy8DnyC6lM vkpJjxWby43kzt7fRLmZF5ZYkrd5okr9+q5ER91SlCjWxX7xzYpQ9bJdvM0pW2MCDtgl F3QXkbHw+l9MUDc6s2HW23CZFQJiYtP6ZA1gJ6aLonxz3OyS+PRxto43dLO9qU//f8YA m1QIGF3kupEVuwmWzxTFcbG/WPjalG3ubQqH+NFezOfxagNz7BfftEQ/G9Xl/cU3EXiu xhn5ZV2i69OGWtgGrlmrOFJ4x4AUtpsP1iffCGS+9RqylqaTpSIHk9ycbuRAijNOtgiG T7uw== X-Forwarded-Encrypted: i=1; AJvYcCXA/RDTOvX0EZe5lyv7mJSmBsmlt7qzLx+w4iV7Vgy84vWh8Vhg9+9s6oD+VfgriGsmj8ECJ9UxRQ==@kvack.org X-Gm-Message-State: AOJu0YzhWxDevn8KcAwcrve2bq38JgMBtLUxjXW7twbi2QTHWK8d2Sr5 Ag+FsATHn/2OXa1MJcLrWIMPMqWk8Q3l78P8cqEpH/E9Izl1HBw1pSQEsG7VFw== X-Gm-Gg: ASbGncsFaE/0wklXaKiM4fVnpTSjoeinOyHlZ8ZF+4yo67Hr1hhDQPbpjvPWrK+QskB cUrFGiydPS7hlb7zYr/bPYZlquWJrIxHnMt1B5P42FBwkc+zVPePep4VXIOkeKfGAeNDysN7hQs wli4kIEKGowHCbvijlnFTLsYqMfzoZ5Gh85LiyHGVCtDdBiuPtVfrG9zsueaBJqasD4TbLATgby C+2fQCX2Lv6e/21avcxyN5qcmA72P71M3vCdfxvL5Qpw0/LiyscOxCzSuGIN3S40I1/N/QjV8pp HeM1Bzue4ul6EySC8w== X-Google-Smtp-Source: AGHT+IHKsmcyZIg17GYlLEbKgjy5fMkUQ7BmQ1HUJMxSkfACov/TMDteDP157pPV1tX1h7GlE/qMmQ== X-Received: by 2002:a17:902:d48b:b0:215:9bc2:42ec with SMTP id d9443c01a7336-220d216ce9fmr80849875ad.47.1739508832004; Thu, 13 Feb 2025 20:53:52 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-220d5585fbfsm20932385ad.226.2025.02.13.20.53.49 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:53:51 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 15/17] zram: do not leak page on recompress_store error path Date: Fri, 14 Feb 2025 13:50:27 +0900 Message-ID: <20250214045208.1388854-16-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: g8yft9k8x4j945axned68t161x1zxehq X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 02BADC000F X-HE-Tag: 1739508832-458603 X-HE-Meta: U2FsdGVkX18bdEd7ju30JEcQnGwPOL5dgZ1QfTKbKXzUA3bbKEl/TfmVZWIVBWr3yUpQ7FRzDVOk4QMyfgG5DgJ4f3PltwrIQZ57R6leyAgfaxeY6J43FfMLB3ZXEc6z2WHqo67megxq2VouI6Z3uNmecbw538bEp5LQGJdZSc/yL1f0DMkamdG3vYtzKzGs0yWIs9ihdVQw/uGFKt1dXGJt2Cvg73qIdombjXHWPmogXwbqzJDGO8oKD+RVDgaOJctgYDNNhauCfhJtYAZM616lHYz78nHBhA1qG1YdNxpWy9u3SG9qrFEPJRp+69pCecYTV5kHuMoQfWcIN0zRSuh/VxwkIz72vfrds/xCRhkExoAxR68yggwTZ4bOyqkTj9FVx+vJO+vDCL0cx0h9aY3IC3NKYzcijLACaZBGVGdH/EUlFFWy46+JRbPQo60Y85RZRLYkokeOu9cv5VKP2bUM6wD3KiMyPb+/OnLIgGgIS64sR2mQpZsu4JaatVxEKGbwDTlxu3prlehXT2rhO/OWPV/gi6v/iwPAEq1NEcOMYDGFwy/lx1WQc1sycmf0oRjwCjQVzNNt2bES3yEUZPLba6ZQQ4fKD2PSKgeM0C0ZT14kKZJYCR2Jyt+/1GGKHFqEz2R8LIML64Y8cs87lNwcFqWYI8xKUrf7ReTQOZBH6k/Je9aXMgUGlrgcj/8DkHaew5jisex8kafhk4uaNEiMiGJDM600tjW/aLkReHKY/jKqqoKUBuxyEmfZHNHSYy2mZ6GRifOS/ArrtCRzuRDUEoLmcvWRhMqCQ7n9tLTZYAXVtozoEjbulCQzB+1nTuQcUg5faym2frYHfUJFAJeaS+HuWUXG1XJRbfMMLI8dcXNrSkpCnD1plwrMcgvxzY53PpM0X/iWPr6dL4LH9PWa/nHcAxMm6ubWsPFwYFuVgP50ksR6kKusJh7CJNahep9Vf+NPqmX2XRzJT8O FoHVQsNN 2qpnQohMzQiBvHFwJTD7Hoz8EJ8828PVuZB5U/4nnOc3A/1HyI0BZ0KtQ1mUZG60OyKigW27CCjZl3uc0a5SlxlwbuHg1z0rdGBt1Zu+1BgQuyytEQlPomXxl0zaYMSKFT7DYzQOX5psj9p1N5NUVTTRL3uOX7IbmMEz7qZ1BAzMdk5jJFo8InXDIp5yvXw4i7yeA/4BSLuESAJeJreiJeUeooc5+qUgkPbB21kPm0aIr97wXVO6D43wZQkNZYpvV/GKbE4Q1ybzUY21a0GB5DAqJ+HsmoNcy56JTUOV8oUqgRCMnHPzbtNz1a28gNQ2glIkmCNBu4WP72+TIJsl/TDQ2owzBAzV7UtNq2DyrelkIYUI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Ensure the page used for local object data is freed on error out path. Fixes: 3f909a60cec1 ("zram: rework recompress target selection strategy") Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 87ce304a60aa..b4661de6bc29 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -2059,7 +2059,7 @@ static ssize_t recompress_store(struct device *dev, struct zram_pp_slot *pps; u32 mode = 0, threshold = 0; u32 prio, prio_max; - struct page *page; + struct page *page = NULL; ssize_t ret; prio = ZRAM_SECONDARY_COMP; @@ -2203,9 +2203,9 @@ static ssize_t recompress_store(struct device *dev, cond_resched(); } - __free_page(page); - release_init_lock: + if (page) + __free_page(page); release_pp_ctl(zram, ctl); atomic_set(&zram->pp_in_progress, 0); up_read(&zram->init_lock); From patchwork Fri Feb 14 04:50:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974478 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2D92C021A4 for ; Fri, 14 Feb 2025 04:54:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4C8E0280003; Thu, 13 Feb 2025 23:54:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 45029280001; Thu, 13 Feb 2025 23:54:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2CC9D280003; Thu, 13 Feb 2025 23:54:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 01B51280001 for ; Thu, 13 Feb 2025 23:54:00 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id B807FC1DCE for ; Fri, 14 Feb 2025 04:54:00 +0000 (UTC) X-FDA: 83117333040.09.A861509 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf05.hostedemail.com (Postfix) with ESMTP id DB892100002 for ; Fri, 14 Feb 2025 04:53:58 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="hpk8/NK2"; spf=pass (imf05.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.52 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508838; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=c3vcsQRfcy+wCE7pS6Fz7pnfmcjhxh9cIXH2LP2gfks=; b=DICok6Ms/2a7IuemOX/wqJ2qUVG/tfJvt7Dt3NQWYNPUpTYiSQkOeB47sDWnJcHHhEuzs7 hQwrxa73eLDCXj09GhYeZCu39TFR4e+gz1NNrGHNy5gmT03XvattcnET2GHz/Z1IeamBm7 Ahw4QJ+EI/DV1gigMzdcDL/DMrYEWY4= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="hpk8/NK2"; spf=pass (imf05.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.52 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508838; a=rsa-sha256; cv=none; b=Uvrs7+G0iT5/32fKawle34cKEKlz21BEZxCc92P5uPJWSFsMmapy59VpxzgY6z2Xnm+OBh fKT8zYxSf4kLBadtVooQxbCk1A1e0PtcGhDTHyRehJSTnhLifVgqg2QC8W72YqZ50tNq5P z2IRJkVuzq4rkuxuRHI3QwlKxD6Y7o4= Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-2fa5af6d743so2649986a91.3 for ; Thu, 13 Feb 2025 20:53:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508838; x=1740113638; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=c3vcsQRfcy+wCE7pS6Fz7pnfmcjhxh9cIXH2LP2gfks=; b=hpk8/NK2hpowv/J+kJn2GPX0EkMzRqagKEiplONo3umj5g+IPJZY9ebKquZ2v4Av5T SnLf6G5oH5RzI47FMuSu8OLbIb+JWnfOGv58XMm56p6cdbZUucDkzAH0mL7oAUanpODE +INScQBixHyyOdbgy8IRixH8zv1o2tFhAjRwo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508838; x=1740113638; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c3vcsQRfcy+wCE7pS6Fz7pnfmcjhxh9cIXH2LP2gfks=; b=seB1Wj3CVLgTrTH7WHlYqeqM2p+YN9duIwEnYBTMqYHPpSTMBWQY8U0I2FeIvZpk4X EEkaniU2mREqoYpqTJ++xo8BG38+kwPXGlw6frzcHYj3rt/Dg62Sc8nOOlZSSgXmrLaM Osxi5x7Jh180rpFDuuXtsQ66wkDOcW3B8PZjAxRnYdQPYpV9Yleh4On48qiHk0lSz3jn dAe83yvBmdqZzgrypIqkBFo1k9DLHKAK8Ov5aQGMSPKhdSP89gsY7IPlr+kZ6nShqMKE IbgKBwRO317TRz3J5d7zcv2bo0CoDjS+wgG2w7DzJUFlhyMAPoQHKO3AJlivCX0bEiJV XOyA== X-Forwarded-Encrypted: i=1; AJvYcCXcX50y3ZitPQftr0T9+KbwqAQciIDcgy0/LRFegR0cgEd7Ol6Bb+BVeLOtSZkmd7GxmTyXtdK4yA==@kvack.org X-Gm-Message-State: AOJu0YxBqSYwENmgvn5EjmMkHlU+mVwaq/ZPZs8d+ngu+0wQ9+fiCOFt zZN1yEeXq2fy/qOAbLt22UtupHekIb+oPEiDT3zIOMFaogQbZ00Apii+eO/XCg== X-Gm-Gg: ASbGncvKKF590a7z98rTdUxiPgT5rxbDwYg/XYs1JvN/chuyNfYircEgK1WlFvZG3Z8 7t9uI2LlaYFBnyoqEyxH5hfIVKK0LepscsjpqLrCMsEC6MqTHNwuBKGvGxmGu/aDUONp+BG8DtG /PTpQ9ABd1jqUEPGEXPQtAKLQ8c3a+0fdUX8fUyaPrr439hhzl7p+Pf0amBbCRYJUq+zKLOWNYg lOqyLo7V4m6BDnPwRRmvvbRD9YpanhC/85e1b35clQFDmcZAxMw3jjTopQ/a1t56xtIbleyF5SM 638kO3VERZqMEJiWNw== X-Google-Smtp-Source: AGHT+IHAcm0g7zYqkHRuT4x03coHX+AA6wb7u2m15AhzZy57LDAWmkRXZjtrdEJDwTr85W6SV3VPqg== X-Received: by 2002:a17:90b:3c48:b0:2fa:1a23:c01d with SMTP id 98e67ed59e1d1-2fc0ea881c9mr7990978a91.21.1739508837865; Thu, 13 Feb 2025 20:53:57 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2fc13ba6ac7sm2214490a91.45.2025.02.13.20.53.55 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:53:57 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 16/17] zram: do not leak page on writeback_store error path Date: Fri, 14 Feb 2025 13:50:28 +0900 Message-ID: <20250214045208.1388854-17-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: gmiom75d8iqkynsy3juezjuc8hdfa5b1 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: DB892100002 X-HE-Tag: 1739508838-508622 X-HE-Meta: U2FsdGVkX1/NQ2Uj1fryKL3u6RT2YedcLzR0juhSMMe9Mj2km0GynToaSTTZpfB3K9TfmyHV4N35P2WFODqK+YF6VihltiEs6VrmCwra9dBcy9yWWpT5ihyDIqJYPIVwkIM0MzUE/hnPzMUqNR84/b+MaE4ADAVyIcPpqIHkqc4voRIgIwpuAtzUSrhZ8bCAnqdHjx9PH3EgIAWos7P40LHmZNZKocwX1Yt65zyV3HzREphLlbGUvt1ud4tevUkKefKQZPchtL3Y8LfZCiuF0yeyP3nqBKYXukEXLcTKtmt4XuZE93SOmeaUJV1oiPIN0QNV/0+IT/5Em1pDzue0lmCx+zdn54E7kvJbLglmSB4wd2ydc+e3mVAT9CuMvXCxDPP+rPx3vi4alfrTBgJ4ceFVF3e4IFLc8OzMjFO3d4yuNjDODffLfrnP5bXjlN9Rt6EiHkMvimbl2o70/O0MlvwxcyZ5ljMAbdE5al4ubI7gRfTUo0z8Oht2Zbbn+ukd441ByezSC9mSZUvRMJ0pLwxGVqWnIyV+fZrNt+BiniGJKN71OCOO6stTjvHXbR5rnS6imFjalh1xVpbTk9bzOnKaxsyCL2TuJtxnhRxKjVL+sBLEEQTIGk4MrTjf6q3ibiF0oWj6F200NwZlCNZYaGF0P0i+pr/v2e6G2kdjI6Dr0x5EstKJozuE9KGCv5O1XH1Af0MrG2eWuRmuyjeI83/RmZ7SYIALf+AJ3m5YR1LfdiV4+UqjCm10irWgilkQgXiMBR2nIPazXDMM71K2et1SW0nxjWj0gxU6liGlgeVubpkkI5KVrjanPS7iI/DaqqriiNQXPo4Ih1/Hv00rMTcR/OTSrVHpITEOA5nJfBelWREmnyQmoWtQmqKTBKySCDEKz052OfFCRvr5YAmHbM/R6V0zLCrvkwJeo9L8/u3PcO6/kBG1XRzvesb6OP0L5zbIXFtkNEtnQtQwfPJ J6Ya33Hf pQsZDeHLcQpLC50zdmwH04Wt+hP/mpjrDytFiaCQ9Xl1x9JupFmvqzL/od/Bj0x22PW2aOJpZ9KVaYad0LREXONt2JXHyIJ7wezaM1GbKarjwBbRL3g7/YR2LwdwmCU2xIM6j/VMXIO5MKhYDwS6aeeeUYpEDNKSpXIrowdjfH4JFYmmmyVBb9g7OVWarhycApJWhqIrnJiBZBexvcB6UCsNgxzaZOfTESRSfcabyVlHH5i1ziMdoRNkAkF3te08/blGG+hcJUyHly3oI6tX0zc9+/SPFU5F/c6W4SWGNIhpPVbQK3WF8vBToBf23otCjxAsBsBiylchY61A2rK29xue93nUwy/Sb/61r+b8Xhw5iMKs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Ensure the page used for local object data is freed on error out path. Fixes: 330edc2bc059 (zram: rework writeback target selection strategy) Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index b4661de6bc29..6adaad56f4ce 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -829,7 +829,7 @@ static ssize_t writeback_store(struct device *dev, unsigned long index = 0; struct bio bio; struct bio_vec bio_vec; - struct page *page; + struct page *page = NULL; ssize_t ret = len; int mode, err; unsigned long blk_idx = 0; @@ -971,8 +971,10 @@ static ssize_t writeback_store(struct device *dev, if (blk_idx) free_block_bdev(zram, blk_idx); - __free_page(page); + release_init_lock: + if (page) + __free_page(page); release_pp_ctl(zram, ctl); atomic_set(&zram->pp_in_progress, 0); up_read(&zram->init_lock); From patchwork Fri Feb 14 04:50:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974479 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9E94C021A4 for ; Fri, 14 Feb 2025 04:54:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5B3E4280005; Thu, 13 Feb 2025 23:54:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 53B30280004; Thu, 13 Feb 2025 23:54:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3B44F280005; Thu, 13 Feb 2025 23:54:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 17C03280004 for ; Thu, 13 Feb 2025 23:54:06 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id C0BDFB150E for ; Fri, 14 Feb 2025 04:54:05 +0000 (UTC) X-FDA: 83117333250.21.5D13041 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by imf13.hostedemail.com (Postfix) with ESMTP id D0EC22000B for ; Fri, 14 Feb 2025 04:54:03 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=UQRcTqXy; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf13.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.174 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508843; a=rsa-sha256; cv=none; b=a8J8mi6MG15BZIKmMcwW6O1k3+Nmv9qnKYTmy5q9pULsNXtJWr7YjM8WcSahzoBTXm3a7d v+iQeEFQMqdmv9x8VSqE+YY90aWPW2lHwTPY2ZOC7FWEAdbru0Z9uBfNAy1xUqPxcTHDmo wi1ZATtZa1/RvxTSDkjm7/24Ye/WNwY= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=UQRcTqXy; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf13.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.174 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508843; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sFE2Vkh4W4kK9IrN6erQ+4Og/P/7Lb06X2GD4vSsp54=; b=F7qGJTxlcXdQnZhvF02HMtD+Bt9pp1Cj0EKM0V5PLrdp+LqLiG+qQgNYvu3FgQkwkfiYAG 7pQVVJ1fjwE3cUy8kH1X51RkPU6jvidfu2AsqkZARDkpAtybq9SBDl+gSGifROfdg6DY08 is5HcaQSIms7AsEEoCnBK7buWHYSoeQ= Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-21c2f1b610dso38152575ad.0 for ; Thu, 13 Feb 2025 20:54:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508843; x=1740113643; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sFE2Vkh4W4kK9IrN6erQ+4Og/P/7Lb06X2GD4vSsp54=; b=UQRcTqXyXjfddThwJaEofqAgm/P/3z7vinr5lvp2ihKfyqPAVXFtR19Yh/NIYqUdZL f9VJ+zpDbZB3PR4L+uY8jZPP7Er6x3dTYRd/w1yXDCCssGtOFP9y3TeRXUbDu3kPD1Wh JGo0xkmFVwduKouEmcz20gaLBs5JRGbrMa8sw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508843; x=1740113643; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sFE2Vkh4W4kK9IrN6erQ+4Og/P/7Lb06X2GD4vSsp54=; b=SgNZAHDappOyUB9B42cf9405GvwlZ5anbLFKnOXs+ok0REulAI5RKZFyHdrMtsHFOl /Kd1zUQUG6OcQ/pPLfsN0ceO1k0ROjqFlAkXh1BlJW6jNtMWHs3X9osuzYHnyd3V1ZdE lwR2DfHmzHEDhQZt4hry/o5Sk9PZivE88DbtCq4PwvTHZat97q4Ca00WUl4y/YHsDkHZ JXvIvxlVEt2KoE4Cx4CavZIwWwKYNcj1pXd26b0iIO5HCKPruAC/dL4tJU+h6B1x4+Ld QNoJXCz1sJiw7oKnlKzwHB4o1yCTxd/Z5Y6Mdx1Svk2zqSwHunKzah/MEDc2kOEYLIM4 nqqw== X-Forwarded-Encrypted: i=1; AJvYcCVFiTjmKiTapEmSK7ehtEulGfBHkCj8EYp3FwMycPIU3R1F948U8uTEz+CoLBYk7a+ShjfliiitDw==@kvack.org X-Gm-Message-State: AOJu0YyqEJdulRLimtXciPvps/Uo05mqEh5nVxwBIz2BFkadZbo+mQGS 1HneNqd7DoD2YEyfVPc49pPwujc3qANx9KWn2y+FHvHs5Tzp81G1sXLo0KQT1A== X-Gm-Gg: ASbGncvZtINbE4tZP8Ow3NrYpR2AmHLKedGH1sk1+RtyIAKwomCyaMgTWhJE1sOOl7a GxQJHJ1WIDQoYk2xW3j7e79emnbiyRl1S5rUgFXO8PCUQb3Q7ImRi6aNKI36EkiIBR3SYZxUhZ4 Fp/IHhDHdK1x9H/fkqD1AX059gC66LqbCmhawVTTHTjh3xHnNe3UPhgnHBBQaYlZCpnS3OjBuFC P3Y7dPdbER33hny88jSvgwVWwwOR91XtAPUZljmQmMFd9OcaYOlvNKwO6uUrAv5GUBuj025vZOa jB4RNxCJtDrsFWkm/w== X-Google-Smtp-Source: AGHT+IEdA+mk+O5VEtAV37EuQ9q5WsoauseiUVXxS1yqw/ceGVJW5mSRJN4cdhO3tV9GqvV3fWIxjg== X-Received: by 2002:a17:902:f550:b0:21e:ff3a:7593 with SMTP id d9443c01a7336-220d1ec17dbmr77579475ad.6.1739508842864; Thu, 13 Feb 2025 20:54:02 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-220d5349394sm20869245ad.49.2025.02.13.20.54.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:54:02 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 17/17] zram: add might_sleep to zcomp API Date: Fri, 14 Feb 2025 13:50:29 +0900 Message-ID: <20250214045208.1388854-18-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: D0EC22000B X-Stat-Signature: ibx8ohnnjfwixbd6hrgujguznwqatukg X-Rspam-User: X-HE-Tag: 1739508843-274625 X-HE-Meta: U2FsdGVkX19+RAke96tsDMszxgtp0OCcPcjcXddcQc1MAT45JG+LhE/2ghISxklkILlGhB1kYsRJ7YrRmA176UuqiQpNRxgbwSdDViM1ZkOCJH//GMMVzMiDqx+UZx5ge1rVxPNoL+BeXiT7fVRQnYnQowy5MpLIgc8ucvaAs1CIrs9pU3POR+tHMR2qQX0fHn/buoPx2CUUp3p1661KpFczJZjc4UXcnqQUzVo1bsj6ZCJZyEDEOiIb7EqASdj0qowTeo7VeAvOrqfNcoa4Nw1hOEyJiKaJRtJWHDDLPnbt2iXA0tmZyqX7stEseJvWEuHFqRNkNMyeLjkepQ/H2IFQW1pOj4FFteidoBtYenY6+OZ9jXjq5XZjvKmvsB/qAb/23MAIaGY4p/hd5aKCUevPA5BkDv1XZkASMhlwuS5qzEDEDzd84EDknjUe3/Q/a+hGjxVT58BGCz8S0VDh12B6349lNAcBoqffgwVvrAnD1Q/GwcLyDc9+3SS8WaXB+ENlWUFtY5D/C4zW+OE/c/tCBrcnWzDu1oXgD+R1Ml58R/RXi8Cdiu4fyqpvS5JUj3L9r927+pVDlop/jmvmUhc0s+mWT4cx57fjdb6iSzJXXsR3j16ZnKVSpydBdXvhEC1dx45/2nQZRikVIWjV89RnsV7+gmXo3P4j8tI24Ny2pTIdTAjD/EpxQZNeD4FiJb2sTzoMlrwOgPSOQmhaLT8mQNpmRnW8W6VvDT3nKsBy4IoVIxsIw1xKEUSlGDPm5docTACAhPHZqf8fm2sx4hPzsMReoy44TQ7moqzLPndLN/rlrsAJRkbb39TEMA22j847KGw2XnpK48/9mdXzxfoP/Fhq0Il5xiITPby1c1FaMk4Qv6E2wOiiw+f/zBFTmAS3adncRVsog70hv8mRV9xdNobOEu69wYyLraJPDS6TAb7ukoDNS+YQaOukaaty+Yc6Mt7TcJV2adFM8Jw ZDhW8p5B IJHFb/Rob8JjTJGZmJJ9TAZWZQD88xGW+AjtbP4/xBd2ilKreoXY+ruLCkAw5ZGGzTCpJwsw3pRw6VeZPhXoxlVdUD+sQwhtt80iJklcR9eFFzCZEFWl0hzhIjMmLQbzKySCrAaj9JuKMFDR+NbbMgokwe33TDZs404v1hjk4PBKwVnHu0AGUaKbP+2KnW3K2EQBx4i5bvReDJtPJGuBf9lmNNzVA+l6Ospvp6noucnjQ3veXwaTgy1C6H5WRnR+OfG14yJ6SBJMMRrMZSJK1xYykjjp7lkFngu7rkVzdnO6LKDMpFQJzIAeM8bx9tCb2NhAvOM4j3B/nHn/BLdphdWW29Qjc2yY2Ae8quv0Gq9fNqAg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Explicitly state that zcomp compress/decompress must be called from non-atomic context. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zcomp.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index a1d627054bb1..d26a58c67e95 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -146,6 +146,7 @@ int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm, }; int ret; + might_sleep(); ret = comp->ops->compress(comp->params, &zstrm->ctx, &req); if (!ret) *dst_len = req.dst_len; @@ -162,6 +163,7 @@ int zcomp_decompress(struct zcomp *comp, struct zcomp_strm *zstrm, .dst_len = PAGE_SIZE, }; + might_sleep(); return comp->ops->decompress(comp->params, &zstrm->ctx, &req); }