From patchwork Thu Feb 27 04:35:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F877C021BE for ; Thu, 27 Feb 2025 04:36:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AB2C0280002; Wed, 26 Feb 2025 23:36:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A3B456B0089; Wed, 26 Feb 2025 23:36:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DBE0280002; Wed, 26 Feb 2025 23:36:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6C6726B0088 for ; Wed, 26 Feb 2025 23:36:40 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id E21EE8020A for ; Thu, 27 Feb 2025 04:36:39 +0000 (UTC) X-FDA: 83164463718.02.F512662 Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by imf15.hostedemail.com (Postfix) with ESMTP id 0AA19A0005 for ; Thu, 27 Feb 2025 04:36:37 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=MgVbIZXy; spf=pass (imf15.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.45 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740630998; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NR62rHb3pebtfvQHNCZtoXwLqkDdn/SbiRV6/pPxB7w=; b=ArixSChD3mwzHNKDVkvdfDPktLtWO8dADqNdUa2cOuUFaqSl20ImF95ArYrajQ+tF8Hgv+ bzwJzUsj6EYNRLbeYgNXSwTdojUsRSmLXq21SjQh57Kqo7DqX+tO+kDxUEolG/z7OThLfC tCMlRQL2u+IBjmJ74i4HH2W/U+WiLHU= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=MgVbIZXy; spf=pass (imf15.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.45 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740630998; a=rsa-sha256; cv=none; b=0HEKM4mLbeEjx25eYT/gXFIJCOUkzAzbnQmDD5Cwepymf/13DHGQ/oTEw3gKjvlSfOu9qq +MvsmxvI13MiVPRESBzK8Vjfo07xQWw64w97QI6TG7pO31TtvEh1IKPaeXKaF776Z3Lerh yFvkbUFh/cUaMpMWA2vBlDgNuxWPn5E= Received: by mail-pj1-f45.google.com with SMTP id 98e67ed59e1d1-2f9b9c0088fso994432a91.0 for ; Wed, 26 Feb 2025 20:36:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740630997; x=1741235797; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NR62rHb3pebtfvQHNCZtoXwLqkDdn/SbiRV6/pPxB7w=; b=MgVbIZXyD//nwHpZGG9t866jhlXbKW18U8SBcwx0ymN+COMOf+yNNYPMilxpEMu7sC 5I5Merz/0JZCTrQryy35X+/2YGIAm2Z+rPoNPfN1jo8VfOD/pQle8O1NUR9y2lVddDzl ifANB/u7LnGUWjjcrzu/kUP1X2kDyuNnHCvyA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740630997; x=1741235797; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NR62rHb3pebtfvQHNCZtoXwLqkDdn/SbiRV6/pPxB7w=; b=lbLKbCm41d8BnmIuyxMRZMGQxuMY0NXgFRMjkTkVj5gHpK2rnZhCUwPYZErz61gj9U cat1AizsxaktYlt8IPKtz0oY/h8c6a16bCa4fYTZ4UceAte76nBzV8vBNe6/IFuh3F+Z IxgbyY+NabRiIKZ071nbML+LAWzWWgWJNi7G3rDRx284x1XrdYsmsSgGqTsyLXkvC57b bmv/P2Zq193Q/U/oPpfEoWpuAMPw8FXmDJ3bna02McdtMKN/LU7Is1mw+lzXEqwRMkOY YYCgjKNnvhyc5uzR7zLfBunndMqyBdfdshhE6Xa0d5Z5q5qziO5utNPeqkz1529e0cWt eAaQ== X-Forwarded-Encrypted: i=1; AJvYcCXrprRd+vEaRQxysDR+d8MkuGMEGvP89tK7M5zTAeuPtTWU2GqUFreAziyKzoxf0hx4faLg8jc46g==@kvack.org X-Gm-Message-State: AOJu0YxrfHop26unL7eziecKV7+3jhpZwlW4Pa69ZLh+IqcRmOXexjgU afKr52409yxAt77cwbk9qNBvGI/GUHB5+BVlg60uyV1M/Pgdff8Lj/fpFm0tmg== X-Gm-Gg: ASbGncuRppdr2MYvmQMi6ThMsOB177+GDXfVHVuuO6hbYL/QlkCgKO9eOTMtTuKyXr/ zWnUzDHykBdTUw4tPk1o9Xl7EhRzTT0z42Y5EPAPf2uawHkz0r7VVMr8IqXXHhPPB+FIC6dTwfD SOkvMB3p5xE11yxTt3U5rGerui/1G0LmOHXYmQRHCBAECyzIzJl+3H9LcxzFx37C4cQNeS+eFXB j2YSc+bilPvWfjGeMFWqpKbbQtg3Fd7PzDYyO5DNKmovBIcYNtp+7muRytjPdSCQ0DoterEuPWy jjKvRK29VhYB0ouAUytonPsNufBG X-Google-Smtp-Source: AGHT+IH/UF6t7IzFfea6GJp3N/IG0r3W9xlvoHcoiOhurhjeGRrnZ4qGrFjng8ecS9ungkxHCpZQfA== X-Received: by 2002:a17:90b:184e:b0:2ee:c30f:33c9 with SMTP id 98e67ed59e1d1-2fea12f4accmr3176743a91.14.1740630996801; Wed, 26 Feb 2025 20:36:36 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2fea6752060sm548358a91.3.2025.02.26.20.36.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:36:36 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 01/19] zram: sleepable entry locking Date: Thu, 27 Feb 2025 13:35:19 +0900 Message-ID: <20250227043618.88380-2-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 0AA19A0005 X-Stat-Signature: n7116jcbzj3ow1k9uf9egiirmnquwr85 X-HE-Tag: 1740630997-415374 X-HE-Meta: U2FsdGVkX1+Ph/V8ezQqclvXHBM4PFngH6hNGhvZI5k1rUCdmRRATO9gy047ywtKGL+hUsbMFtcbHyIxGIyKxida2Oymhtr9B6UrPSWAgW3Wp70UUMb6Hfu6dIxvKFXyPCEj1ltEt9hLyCFBRN9wQxMY57WonFEy/TAxond087BnHt2M2SrDOuilk0r8eGvDaC9TCBELdt95fsbl4oE7dPbnwUzzBecXKd3/7iNXOnga5lTrnuYONo6Biv0uFnc5TRSR8SK90yL7bzPqr6zEpQX/a3K3l7Y6MH31bSzK4RGSiB5raoQP5uIntkd3+j9lpjbnSf0ETX5Pk3Xv767Nbn00Dc7Tm473Hfpd0uFygA+UowzpWe1jG71mjd9DG1aHtq4VSEwLNW4oy7gjc+pMjDsNA1kclyluBF9cTzXNkO9zI9BkTa71w3QzHmyf4grfBIOglGhE33HTKsT+QtB7wFtwqGz2nQqYshudMt/G29nEkF1IiiBvD/6GUPRvE5Qx61Qsx+D+om95EvsNIHahamF0xsqyqU4pEEsCmDUph6wqFEPviblhi+8N6mZW7hyGIPJ4Pyjb32fO3A4u96tP+xq7fcUL4rZBAdTdGNh32L8YBlLZzrNDCn8lpOLiNhWlNf8+3vCp+q3pKGvPFv3R1GZyx7S1spnhDwh+DzbaHtnd1UWcfnvP2ooH34pezfzV291dtbl3hU0emcxmD3Q3g6uEnO1ZIcuPWI6t9kyQl9dsGdZp9rT5ofLHUiGJTpknYatw/js/M+z0IleVasihLOphDPjZMrQnUunZyhXnl0RwE5FZAy2FX38ftADvCrMJHNFB6ytzQMGjgMRFNZ+SIh3Tc6Rvgw6R4bQeuxZ3goH+5Ng9HvFL468yQIzzDsdJ2HY8TfuQYvUHYkqRiwHs5C9wjMx0eIDrKKxsbcmKY0booD3ugC0IVrTi6OdmxHLuOQeg6h5tFkZcWsJgf25 xtKLp7F2 +4UrKq3eSYDE/ZioYYkxoKYuFESBoxVGNyyIjdurwZcd4MsBLoIP2kJwwWkJ0lGaQevIqUQSJziF0kJpU7kO6lG7feYP9+ryrhSmQMqgeaUKqoJxUDdKVEkszPPHkKP267mgHy5cqF19xOn7sCfVQm10/94mhtpL44mQ5x+q7DX4kl/3k++8mhQB4r66xR53OhHdGlH83QTVQd3+HnThg1G3Op8ongIpnAB224CgFX4oW5Fi8F7pNrFEOEf1Rx0rHgA/IuVRO3zrChMX3YSVqY+GICwUjNeBhm28eOYavAsrWWgQZGsirlvKnpw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Concurrent modifications of meta table entries is now handled by per-entry spin-lock. This has a number of shortcomings. First, this imposes atomic requirements on compression backends. zram can call both zcomp_compress() and zcomp_decompress() under entry spin-lock, which implies that we can use only compression algorithms that don't schedule/sleep/wait during compression and decompression. This, for instance, makes it impossible to use some of the ASYNC compression algorithms (H/W compression, etc.) implementations. Second, this can potentially trigger watchdogs. For example, entry re-compression with secondary algorithms is performed under entry spin-lock. Given that we chain secondary compression algorithms and that some of them can be configured for best compression ratio (and worst compression speed) zram can stay under spin-lock for quite some time. Having a per-entry mutex (or, for instance, a rw-semaphore) significantly increases sizeof() of each entry and hence the meta table. Therefore entry locking returns back to bit locking, as before, however, this time also preempt-rt friendly, because if waits-on-bit instead of spinning-on-bit. Lock owners are also now permitted to schedule, which is a first step on the path of making zram non-atomic. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 56 ++++++++++++++++++++++++++++------- drivers/block/zram/zram_drv.h | 16 ++++++---- 2 files changed, 55 insertions(+), 17 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 9f5020b077c5..ddf03f6cbeed 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -58,19 +58,56 @@ static void zram_free_page(struct zram *zram, size_t index); static int zram_read_from_zspool(struct zram *zram, struct page *page, u32 index); -static int zram_slot_trylock(struct zram *zram, u32 index) +#define slot_dep_map(zram, index) (&(zram)->table[(index)].dep_map) + +static void zram_slot_lock_init(struct zram *zram, u32 index) { - return spin_trylock(&zram->table[index].lock); + lockdep_init_map(slot_dep_map(zram, index), + "zram->table[index].lock", + &zram->lock_class, 0); +} + +/* + * entry locking rules: + * + * 1) Lock is exclusive + * + * 2) lock() function can sleep waiting for the lock + * + * 3) Lock owner can sleep + * + * 4) Use TRY lock variant when in atomic context + * - must check return value and handle locking failers + */ +static __must_check bool zram_slot_trylock(struct zram *zram, u32 index) +{ + unsigned long *lock = &zram->table[index].flags; + + if (!test_and_set_bit_lock(ZRAM_ENTRY_LOCK, lock)) { + mutex_acquire(slot_dep_map(zram, index), 0, 1, _RET_IP_); + lock_acquired(slot_dep_map(zram, index), _RET_IP_); + return true; + } + + lock_contended(slot_dep_map(zram, index), _RET_IP_); + return false; } static void zram_slot_lock(struct zram *zram, u32 index) { - spin_lock(&zram->table[index].lock); + unsigned long *lock = &zram->table[index].flags; + + mutex_acquire(slot_dep_map(zram, index), 0, 0, _RET_IP_); + wait_on_bit_lock(lock, ZRAM_ENTRY_LOCK, TASK_UNINTERRUPTIBLE); + lock_acquired(slot_dep_map(zram, index), _RET_IP_); } static void zram_slot_unlock(struct zram *zram, u32 index) { - spin_unlock(&zram->table[index].lock); + unsigned long *lock = &zram->table[index].flags; + + mutex_release(slot_dep_map(zram, index), _RET_IP_); + clear_and_wake_up_bit(ZRAM_ENTRY_LOCK, lock); } static inline bool init_done(struct zram *zram) @@ -93,7 +130,6 @@ static void zram_set_handle(struct zram *zram, u32 index, unsigned long handle) zram->table[index].handle = handle; } -/* flag operations require table entry bit_spin_lock() being held */ static bool zram_test_flag(struct zram *zram, u32 index, enum zram_pageflags flag) { @@ -1473,15 +1509,11 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize) huge_class_size = zs_huge_class_size(zram->mem_pool); for (index = 0; index < num_pages; index++) - spin_lock_init(&zram->table[index].lock); + zram_slot_lock_init(zram, index); + return true; } -/* - * To protect concurrent access to the same index entry, - * caller should hold this table index entry's bit_spinlock to - * indicate this index entry is accessing. - */ static void zram_free_page(struct zram *zram, size_t index) { unsigned long handle; @@ -2625,6 +2657,7 @@ static int zram_add(void) if (ret) goto out_cleanup_disk; + lockdep_register_key(&zram->lock_class); zram_debugfs_register(zram); pr_info("Added device: %s\n", zram->disk->disk_name); return device_id; @@ -2653,6 +2686,7 @@ static int zram_remove(struct zram *zram) zram->claim = true; mutex_unlock(&zram->disk->open_mutex); + lockdep_unregister_key(&zram->lock_class); zram_debugfs_unregister(zram); if (claimed) { diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index db78d7c01b9a..8a7d52fbab4d 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -28,7 +28,6 @@ #define ZRAM_SECTOR_PER_LOGICAL_BLOCK \ (1 << (ZRAM_LOGICAL_BLOCK_SHIFT - SECTOR_SHIFT)) - /* * ZRAM is mainly used for memory efficiency so we want to keep memory * footprint small and thus squeeze size and zram pageflags into a flags @@ -46,6 +45,7 @@ /* Flags for zram pages (table[page_no].flags) */ enum zram_pageflags { ZRAM_SAME = ZRAM_FLAG_SHIFT, /* Page consists the same element */ + ZRAM_ENTRY_LOCK, /* entry access lock bit */ ZRAM_WB, /* page is stored on backing_device */ ZRAM_PP_SLOT, /* Selected for post-processing */ ZRAM_HUGE, /* Incompressible page */ @@ -58,16 +58,19 @@ enum zram_pageflags { __NR_ZRAM_PAGEFLAGS, }; -/*-- Data structures */ - -/* Allocated for each disk page */ +/* + * Allocated for each disk page. We use bit-lock (ZRAM_ENTRY_LOCK bit + * of flags) to save memory. There can be plenty of entries and standard + * locking primitives (e.g. mutex) will significantly increase sizeof() + * of each entry and hence of the meta table. + */ struct zram_table_entry { unsigned long handle; - unsigned int flags; - spinlock_t lock; + unsigned long flags; #ifdef CONFIG_ZRAM_TRACK_ENTRY_ACTIME ktime_t ac_time; #endif + struct lockdep_map dep_map; }; struct zram_stats { @@ -137,5 +140,6 @@ struct zram { struct dentry *debugfs_dir; #endif atomic_t pp_in_progress; + struct lock_class_key lock_class; }; #endif From patchwork Thu Feb 27 04:35:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993732 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E3CCC021BE for ; Thu, 27 Feb 2025 04:36:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0041C280003; Wed, 26 Feb 2025 23:36:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EF5D96B008A; Wed, 26 Feb 2025 23:36:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2275280003; Wed, 26 Feb 2025 23:36:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id ADBE26B0089 for ; Wed, 26 Feb 2025 23:36:45 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 2FF4F120172 for ; Thu, 27 Feb 2025 04:36:45 +0000 (UTC) X-FDA: 83164463970.11.911E8E7 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf20.hostedemail.com (Postfix) with ESMTP id 4EF8A1C0007 for ; Thu, 27 Feb 2025 04:36:43 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=bI89CVVk; spf=pass (imf20.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631003; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=k3g8UcvdgyQ3xyw+tSoX3H8q6cI5L8QKqPIepd+Unng=; b=KVqjBJ5RfaE58Iz/wIWQk05b+CXW9L3GosH+uaQDYGCMslxp5uqtStPX8A5DKg/VSSryzs skUevj5BAxCR4uP16mIdDGzjzAPZv51/pLKjGHlo4kkTR7p/i/E1S22QvrH8WX+v8Nh8Gn xEebOaD1pzYCxalbNa3b69tYjYAoL9A= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=bI89CVVk; spf=pass (imf20.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631003; a=rsa-sha256; cv=none; b=158VNawvNOA7ZIP0Cc9oxNUohmmr3GOzM7gBJdVtY7cD9cHW9lTkUmNzYTj66Y7mDxm7t3 /6WeXYZ0Fge0BRtKPPfzFwpWl+/YHZHOlOF+MNBeUhdemlWNtwVAX5akQW79ZMtsCjbkk3 ngcwerCssLlX4UoOBjvXzvQq0a+Ouh0= Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-22339936bbfso7232975ad.1 for ; Wed, 26 Feb 2025 20:36:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631002; x=1741235802; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=k3g8UcvdgyQ3xyw+tSoX3H8q6cI5L8QKqPIepd+Unng=; b=bI89CVVkH8C09SVyAVhOnGV9hdJMW8ZhHaXsbfOrWTbsy+zphatN4r7RI1Wjfm9FIR ZQaU7ekLESPUAZSqkD8ENg8ElGFGl8aHRIWTlGQfwKhZLVpy0QfK4SwbolUTiNPFnVDJ uc0GP3mNYOD/41XqUZg6U9sVaeUzujmBlYFyI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631002; x=1741235802; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k3g8UcvdgyQ3xyw+tSoX3H8q6cI5L8QKqPIepd+Unng=; b=oB0PsTrIfxMlV3qgLmJgFMg8ffe34n0697K/udKZ0KFP+Vt/XuSieqTs9CVpDlSQDh W+WeTd/l/N+/CjlU9nFLqUc+a34aL+NXbnQka4B7IfJjQjHUISxfS68VX5Hy2brfy8Ll ooCpVkVNEptSgb+qTY+nBzKtdvZ8CkAXwv56i5U8W8xS1lHPZxlvLaHt1+GnolvfLEcj Wvi2+XpSzGA4UT14qtSDfbAtsreHBX6R0vRpsFl43qg+8NV3tuXqBHYbhVtTiayipmNp k+vUJI9stvmkLA3R3LsQEe2+BIU0jWKbWiUzAHIpP2GnFl9M0JnwMSfJzSf1tCMbaefS 7WLQ== X-Forwarded-Encrypted: i=1; AJvYcCX93jIwfzfrV+In5fi5bOZ5N7Y8U5YV789nX1+0NgU9bWL7SeITH8XbOyXErNlnlAjN52OZy26dAw==@kvack.org X-Gm-Message-State: AOJu0YxwEmI0MAp+kkPjqdmN0HO96WiUygn35/V5nGAaKiFTLMnkdoou ye1kBI7bf9Tmp+ULVuEV7gAcqIMI01UFEz+CtHfHIcEI84byTat05iT/CN71zg2vS01T9FdKEWY = X-Gm-Gg: ASbGncu6pup7h53SefAaPCOsq5vShQQlpBseObLqfe1XdqDcqJrJ6JqjWH6iuz6uxJw qVYfbxGf7Canu5C9wyvbM9PfnikYP8ZYh2sJn9a9Xj3NBVVLYhe2LvQXVobltLWV+OMKYO3WeNT 18RIcqOcRXQYEnQSYUUbGhjZGf/s8aTiOHwCUJEPXnHFovW3FNu+w8eMuYvDyo/is05rbCEUV1e zyheuG8mNPgqXUOZ01spqWslbLj7ichwX0dYrPAhVIgsymTw9Mq6FoSHgl5t15iXuZSfZ3T10Nn K/6TT/H/uM+nZMfzbvdf6xe3QlSq X-Google-Smtp-Source: AGHT+IFKqiVpX1vg0EKtsAPxxL19rWKpvI+zaBubIUtbgTf7uq8NxOqv9BQnr4f0Yj2mw3OI5Ik1WQ== X-Received: by 2002:a05:6a00:23c1:b0:730:4598:ddb5 with SMTP id d2e1a72fcca58-73426c8d300mr33904409b3a.2.1740631002104; Wed, 26 Feb 2025 20:36:42 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-7349fe4cc2asm515402b3a.52.2025.02.26.20.36.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:36:41 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 02/19] zram: permit preemption with active compression stream Date: Thu, 27 Feb 2025 13:35:20 +0900 Message-ID: <20250227043618.88380-3-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 4EF8A1C0007 X-Stat-Signature: pykwswh6hsy8w5n8j7d75mda9qk15oin X-HE-Tag: 1740631003-42795 X-HE-Meta: U2FsdGVkX1/WFAq8HRpiZcThSUfT3do+/otz36hvaV3nvzjecqftrc75hPt83ACTsb06GhGwWejTKvvUdBOaGqWlINjLGpXbkHT8DFc5h3xA0/GVy/O3OuaaZN1p61OFwm9tqSRY7iNAZGe1tCPkifUAEYBEMiuHCsL3umllj4IrOX90hAMGJsCqsbtmCul56Ahui4J36rannlXH5piTHUsEtHAfXLpgGoF0GmH2Zvp15VRcn7Gt4hs7IrlEMmUv7TDhHc9MibQXGxunVHGJKlLskYoZijHomUdTitSSutrphQlTx0xVTx5OKDJVjKGQ2NofBS9B71vtfPuZza9joJysV/WjTeDYKVlzDYfEIKqD76QLx79GDezow+9uvayMLZnAy4DH/C3wuIX2kUT2ru7O5QxIn1ajQVl74/klnIFdBK3pbux9f56RfFabwz0WfKdDalDKBe2UrpAyRBsDNixlo122Abp2ZB7nvQZx5pR4QD2XV6TgMigoPBnZu7FhHtPOW2kb7zKjYtKClAA9CeePC8cXPwsxvwjG25v8M3Zjq9C2/pKH2+t7jSZ1zQQ4P4YhgpYu1nivJHuW+YAFWnCbL3Jna07T5coS6qju7UWTZleHU0AkRuBcdmwXWtHymD4uI8WeQdHuI+ObxLtifsBzdmmw2NalE3mzLPQJHDkDnRRb41C8brDTwk1JLWB/EubdUsTjqN7UtrK6UdG9a7U+Pm8Mta9FlAr7QKpw9YutfmKtN5zOQ3Zs4cmbyy6elMNU1rDg/6MiIt+jLwqBVQvonho0xfc+OiFQTNXDEuQIUZnQrElNwwZ4+pUD1eWChr2rIg7ukNwPK28hZfjE2SzXmpzytlNWb8RO0jjKj/qmtpHnlIjzUiJbCeA2YOtN/mOUGSDcqrtsTGLemIrA5m9XDN4h+nkfyQprNSZW6deQzYvKY9CYQu5mjRftWQ9MPFY5M009MN69Vix1d1M 8+qcBVWa xo0bVFs502IVO6JJwznNRxyEJ+DJvHtKgK1CiXunnlsS7zYnvjy3Yfpgtn8hZQK6QQ6uwExBZ43ZK4SH3jYby7d5b+yX6ssWVktdOIegWV8RLoeD7hguU/WUjESEo85+b6k6UN1VRn0k6JJAcOwu63HG1lJ6JVnNO3FIhGGysLQr5MbKnrHunsNQRnFsyE0TJDkWBCBfzECslCicbsmjVkxJCH3Nb+7gQGlOcuABl//B/FQpr1EW0NcgCB2HDEObovGwk2qkTkZm/OHiTdRubWn0eYAjGrCE3ZDvPLjjimnMFfj+Ry2hCy8kHjA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, per-CPU stream access is done from a non-preemptible (atomic) section, which imposes the same atomicity requirements on compression backends as entry spin-lock, and makes it impossible to use algorithms that can schedule/wait/sleep during compression and decompression. Switch to preemptible per-CPU model, similar to the one used in zswap. Instead of a per-CPU local lock, each stream carries a mutex which is locked throughout entire time zram uses it for compression or decompression, so that cpu-dead event waits for zram to stop using a particular per-CPU stream and release it. Suggested-by: Yosry Ahmed Reviewed-by: Yosry Ahmed Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zcomp.c | 41 +++++++++++++++++++++++++---------- drivers/block/zram/zcomp.h | 6 ++--- drivers/block/zram/zram_drv.c | 20 ++++++++--------- 3 files changed, 42 insertions(+), 25 deletions(-) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index bb514403e305..53e4c37441be 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -6,7 +6,7 @@ #include #include #include -#include +#include #include #include @@ -109,13 +109,29 @@ ssize_t zcomp_available_show(const char *comp, char *buf) struct zcomp_strm *zcomp_stream_get(struct zcomp *comp) { - local_lock(&comp->stream->lock); - return this_cpu_ptr(comp->stream); + for (;;) { + struct zcomp_strm *zstrm = raw_cpu_ptr(comp->stream); + + /* + * Inspired by zswap + * + * stream is returned with ->mutex locked which prevents + * cpu_dead() from releasing this stream under us, however + * there is still a race window between raw_cpu_ptr() and + * mutex_lock(), during which we could have been migrated + * from a CPU that has already destroyed its stream. If + * so then unlock and re-try on the current CPU. + */ + mutex_lock(&zstrm->lock); + if (likely(zstrm->buffer)) + return zstrm; + mutex_unlock(&zstrm->lock); + } } -void zcomp_stream_put(struct zcomp *comp) +void zcomp_stream_put(struct zcomp_strm *zstrm) { - local_unlock(&comp->stream->lock); + mutex_unlock(&zstrm->lock); } int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm, @@ -151,12 +167,9 @@ int zcomp_decompress(struct zcomp *comp, struct zcomp_strm *zstrm, int zcomp_cpu_up_prepare(unsigned int cpu, struct hlist_node *node) { struct zcomp *comp = hlist_entry(node, struct zcomp, node); - struct zcomp_strm *zstrm; + struct zcomp_strm *zstrm = per_cpu_ptr(comp->stream, cpu); int ret; - zstrm = per_cpu_ptr(comp->stream, cpu); - local_lock_init(&zstrm->lock); - ret = zcomp_strm_init(comp, zstrm); if (ret) pr_err("Can't allocate a compression stream\n"); @@ -166,16 +179,17 @@ int zcomp_cpu_up_prepare(unsigned int cpu, struct hlist_node *node) int zcomp_cpu_dead(unsigned int cpu, struct hlist_node *node) { struct zcomp *comp = hlist_entry(node, struct zcomp, node); - struct zcomp_strm *zstrm; + struct zcomp_strm *zstrm = per_cpu_ptr(comp->stream, cpu); - zstrm = per_cpu_ptr(comp->stream, cpu); + mutex_lock(&zstrm->lock); zcomp_strm_free(comp, zstrm); + mutex_unlock(&zstrm->lock); return 0; } static int zcomp_init(struct zcomp *comp, struct zcomp_params *params) { - int ret; + int ret, cpu; comp->stream = alloc_percpu(struct zcomp_strm); if (!comp->stream) @@ -186,6 +200,9 @@ static int zcomp_init(struct zcomp *comp, struct zcomp_params *params) if (ret) goto cleanup; + for_each_possible_cpu(cpu) + mutex_init(&per_cpu_ptr(comp->stream, cpu)->lock); + ret = cpuhp_state_add_instance(CPUHP_ZCOMP_PREPARE, &comp->node); if (ret < 0) goto cleanup; diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h index ad5762813842..23b8236b9090 100644 --- a/drivers/block/zram/zcomp.h +++ b/drivers/block/zram/zcomp.h @@ -3,7 +3,7 @@ #ifndef _ZCOMP_H_ #define _ZCOMP_H_ -#include +#include #define ZCOMP_PARAM_NO_LEVEL INT_MIN @@ -31,7 +31,7 @@ struct zcomp_ctx { }; struct zcomp_strm { - local_lock_t lock; + struct mutex lock; /* compression buffer */ void *buffer; struct zcomp_ctx ctx; @@ -77,7 +77,7 @@ struct zcomp *zcomp_create(const char *alg, struct zcomp_params *params); void zcomp_destroy(struct zcomp *comp); struct zcomp_strm *zcomp_stream_get(struct zcomp *comp); -void zcomp_stream_put(struct zcomp *comp); +void zcomp_stream_put(struct zcomp_strm *zstrm); int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm, const void *src, unsigned int *dst_len); diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index ddf03f6cbeed..545e64ee6234 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1607,7 +1607,7 @@ static int read_compressed_page(struct zram *zram, struct page *page, u32 index) ret = zcomp_decompress(zram->comps[prio], zstrm, src, size, dst); kunmap_local(dst); zs_unmap_object(zram->mem_pool, handle); - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zstrm); return ret; } @@ -1768,14 +1768,14 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) kunmap_local(mem); if (unlikely(ret)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zstrm); pr_err("Compression failed! err=%d\n", ret); zs_free(zram->mem_pool, handle); return ret; } if (comp_len >= huge_class_size) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zstrm); return write_incompressible_page(zram, page, index); } @@ -1799,7 +1799,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zstrm); atomic64_inc(&zram->stats.writestall); handle = zs_malloc(zram->mem_pool, comp_len, GFP_NOIO | __GFP_HIGHMEM | @@ -1811,7 +1811,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) } if (!zram_can_store_page(zram)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zstrm); zs_free(zram->mem_pool, handle); return -ENOMEM; } @@ -1819,7 +1819,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); memcpy(dst, zstrm->buffer, comp_len); - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zstrm); zs_unmap_object(zram->mem_pool, handle); zram_slot_lock(zram, index); @@ -1978,7 +1978,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, kunmap_local(src); if (ret) { - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zstrm); return ret; } @@ -1988,7 +1988,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, /* Continue until we make progress */ if (class_index_new >= class_index_old || (threshold && comp_len_new >= threshold)) { - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zstrm); continue; } @@ -2046,13 +2046,13 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zstrm); return PTR_ERR((void *)handle_new); } dst = zs_map_object(zram->mem_pool, handle_new, ZS_MM_WO); memcpy(dst, zstrm->buffer, comp_len_new); - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zstrm); zs_unmap_object(zram->mem_pool, handle_new); From patchwork Thu Feb 27 04:35:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993733 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 181A5C021BE for ; Thu, 27 Feb 2025 04:36:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 92CB1280004; Wed, 26 Feb 2025 23:36:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8DC6E6B008C; Wed, 26 Feb 2025 23:36:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7561D280004; Wed, 26 Feb 2025 23:36:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 50B436B008A for ; Wed, 26 Feb 2025 23:36:51 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C7F26A152C for ; Thu, 27 Feb 2025 04:36:50 +0000 (UTC) X-FDA: 83164464180.01.C5FB020 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf11.hostedemail.com (Postfix) with ESMTP id E8E1840006 for ; Thu, 27 Feb 2025 04:36:48 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="c/npoxui"; spf=pass (imf11.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.52 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631009; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=56O4ZNY5k4tishEKpit5ayGqrbKDM0LZTadKzr6R7CY=; b=2AnfW5rhK5LVNPFHUXvmkHMynSVcpt+YCwQ4jQ8v1pGKaig6RgNMpeVhw4VxlY9GvP7onL +eb7amDP+xWrqa7KcG4VSyA7f7NOKG0p+omKa50RiRjW0aU3ztr+7LAAKrD6QvkaIMt9pr Yq3nha6ZSGG6xkNbz3u7+CTPFAFrjqA= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="c/npoxui"; spf=pass (imf11.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.52 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631009; a=rsa-sha256; cv=none; b=vhjl+HhqXNO50vC0/vanW2eOxcUof8mvTZ/QfuOzL2zbzvdziNpl7QxY6z/KOfKDqw+7Y0 yDxJzadXWMarsLkF3z49GP6k1cLiQB2Mwb1LQ8aE4vw7LB+Tx1GEmx5pK5TWiiw/rXa2ly uqs8TYT0fU3t5TEyQvA7lVASSBdQ7VA= Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-2fc291f7ddbso905544a91.1 for ; Wed, 26 Feb 2025 20:36:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631008; x=1741235808; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=56O4ZNY5k4tishEKpit5ayGqrbKDM0LZTadKzr6R7CY=; b=c/npoxuik5ax7NtjrZ/7lfAhjHYMf+L229gxtuI3szbCZVJgw13l8QdUcnqHG3Fc+C Nr/fClmBj3yh7rlMAZf0gMmXikb2aTBAAK/GN2NFZRn5ulE74O0zOUVqppIt0sbMxYh/ KtPgQoyW0Wi0voH5jhtEIpsPIjMyAcvFDJKEI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631008; x=1741235808; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=56O4ZNY5k4tishEKpit5ayGqrbKDM0LZTadKzr6R7CY=; b=WI1MVy+5lccPAeUhxVZusqHLSJLs2KHoM/soTn94QDFu26b/pT/LBxNsrE8AZjjqss 47bTAVVw3/SCO7JCNVVUulK3RLA5qu6zggpEpWYc227hBneTUFgducj5GYnT5u4VvdFX A/wNfdF/YgSw6CnweyXxxBPtmbhQaq1qjrP18eseTbJCmMr2uR8JUbEnuJ4HDV9EnMWa EAgSmMhyIP63MBS/rheQ7FnLtZ8xRsn4iJ4G2OHZd3IWKfje3uzN3+aBQ6bZVBW3E0Dg d/DcDwUS6Ped14ae2HnKia6RxlyIrn+ySCgPtq+E4hz3bHJXV02KNSSodc1JZzZzL7hz 9gQA== X-Forwarded-Encrypted: i=1; AJvYcCWmERQ5+oBPzDmx2DHWd9H+WAOhbx5CEwjc0YddfG0//Ne68Dpl+9BMdKSoccq89xW1L1WUGAZrNQ==@kvack.org X-Gm-Message-State: AOJu0Yyai0lkI67S9QrpBx2mFUGIwNIIA7VwEJuq85wO4J6Ly9usAcVZ nIXwAukuah9asD112pasBgF4UbOT8RbDMMTpXre6EB2Lk6xhMmqSpKiB23sjn35KTVkZowABjys = X-Gm-Gg: ASbGncu+Y+6wcqglwvGQlmFPWFq8gjzE9u4awILmcAmYKWqLDknU1L6AEUbPDY1dAht ifsTPllJlK/LH0nJYaGfeu20HrcvsE/k32bxBz9qhcJBaffcsr7ZkaSfc1ud3r7yGGzlqeW6WTf oViIwF9wW1jFcRUPmWqdawthwmMWbJmHEMtNfKE6unUnmgcYAZGH1Wnl41HW4KwXjrOc2Db5iYV 4aAr0igAXefIUaOk3juhkOeXj2YYr5fmE4MWut45d9X7fIeXaCORwhgclrQku6i1opwx4vUnNAy zngVe/fgnXjOSaMHu46pFjiGNqBC X-Google-Smtp-Source: AGHT+IGMIGgkdrOdgCZypYyXDBfzfhMe1xi1OCaXSVRhVGjTG8sqvOZYXxtnCpOaAmEsvUhAKgiqvA== X-Received: by 2002:a17:90a:c2c7:b0:2ea:3f34:f194 with SMTP id 98e67ed59e1d1-2fe68ad9a18mr16247712a91.10.1740631007917; Wed, 26 Feb 2025 20:36:47 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2fe8284f064sm2587811a91.40.2025.02.26.20.36.45 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:36:47 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 03/19] zram: remove unused crypto include Date: Thu, 27 Feb 2025 13:35:21 +0900 Message-ID: <20250227043618.88380-4-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: xpoqx6dki6k89d69qj7g5erifwt4xf8a X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: E8E1840006 X-HE-Tag: 1740631008-621060 X-HE-Meta: U2FsdGVkX18puzZVSUScVwNLZeQqygHymIGe1xYneAvuy72+qeoflPSNwLBkLEIA9qpErLn8+PSzkR9X2I5UGTKDhQbzPwRzd9CG7bz024nZW0ph7/tcrrYTFUNHFQ/RR3itHdl5ZNbla5d7Y+drTnpNG7uNrCR3u0xtm7ftW/k8k5dJH1Qj2R8W8+ZsQ8VgkXBqAfNfuTENVo+ZeLyYUvNmworitM+e34X+ETR4b+DXlj6db1tMuOUjr//Rj+pMiE4NaqeuYfgwyjNquhnaTHXR+wSFKszFaDUSLvzie5yM03iTx9zTAarPBf5/lV0uFeVy4HZVNiFN58hurFPIMgVVWHp08OWT9MpFoekc/YB3TrXnyoPglKXmKzlJKGDltS2FasQXeC5FSVyXMTEm9r/RGxYpMe2zlqpBs4w9pKZ5qXztbzB/eh5GyaWCY3a4cHRyZAz0j5S6KoUBGJ4eA0oF39aKs5A+qrCO9cNP9glz9Tc9gmD5C1L9Fh6DDduv7NPaxZM+bSpwm0WONuGyXqieIH/tDf8pB6vexkOb5Nwu0UhZ2uJZEEPvZ7Myn9ugLrQMYS+RPRLU7pp310SuHEVMDH7Li16iHDPHhlgDw63BPCYZ3olYFvWaSc8BScNsnMaTcJMAEr4krZyDcN5RzVIcLTD+fbjuJBrQbTNxuv56+AviNtefw2CW2ldH+D6V49nvZo0EL7GRW/67iES2cVYgbU6yH9JFWRO7j5cOTtu5upaahpyyL3QPB5UdyK3Fh7eS8vGGWlOd7DcZKqMfuSU6Umu4RIURMRNybWIxk8TfTNDnnRLI4KTXulunmPumc0Zd2TFyZaDw4MXLAKYzeI6xRSBI2GCI5l6xOuxIMbVnMKePgeBGn8iw+G3J7Ypm4SBxng/DFXAElFAnDaM1fhrz+Ow6CvybscT+bvn3/NHFDmo6x4ghr728FWqM74y3zErz+HIRBKO7F/RDtj0 Q6BEUUTt HIMayLAykwTFh2Y97LfbP3pbTJrj0Ue2qoEvp3mNpLq6aAZcfPzZv+XsT0D/r+VQz8nZCIuu9ejLPM05DzphMu4vD6gNz1QLo8zZnLu1oFrgOMzeAQZtjsaQGhGrQNWdPJm/LRnEmsgXrnKlaw4K64WhNksYXc2/vXz0ZGFdcdEvLImwCGdm/VPPmD4tma0b6+BqaJ8r4VUI+dEHedKtGvA1PUgQR50A3rUoOvgPeeI12XemriiCpZO228pJUTHUFngQIJd/qQ+26RaA+XqO/YLWZYSo6pvsFjpKyOpxZrdQfpB4rQcQH1v7MZTRo2o2KmK4DQUZphz3TiW8FgTx8kwcyFd6WNnUm9BPB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We stopped using crypto API (for the time being), so remove its include and replace CRYPTO_MAX_ALG_NAME with a local define. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zcomp.c | 1 - drivers/block/zram/zram_drv.c | 4 +++- drivers/block/zram/zram_drv.h | 1 - 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index 53e4c37441be..cfdde2e0748a 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -7,7 +7,6 @@ #include #include #include -#include #include #include "zcomp.h" diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 545e64ee6234..ff662144b98b 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -44,6 +44,8 @@ static DEFINE_MUTEX(zram_index_mutex); static int zram_major; static const char *default_compressor = CONFIG_ZRAM_DEF_COMP; +#define ZRAM_MAX_ALGO_NAME_SZ 128 + /* Module params (documentation at end) */ static unsigned int num_devices = 1; /* @@ -1148,7 +1150,7 @@ static int __comp_algorithm_store(struct zram *zram, u32 prio, const char *buf) size_t sz; sz = strlen(buf); - if (sz >= CRYPTO_MAX_ALG_NAME) + if (sz >= ZRAM_MAX_ALGO_NAME_SZ) return -E2BIG; compressor = kstrdup(buf, GFP_KERNEL); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 8a7d52fbab4d..8b66af0d162d 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -17,7 +17,6 @@ #include #include -#include #include "zcomp.h" From patchwork Thu Feb 27 04:35:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993734 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6076EC197BF for ; Thu, 27 Feb 2025 04:36:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE67E280006; Wed, 26 Feb 2025 23:36:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D9786280005; Wed, 26 Feb 2025 23:36:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C37C4280006; Wed, 26 Feb 2025 23:36:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A3F36280005 for ; Wed, 26 Feb 2025 23:36:56 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 126051C7045 for ; Thu, 27 Feb 2025 04:36:56 +0000 (UTC) X-FDA: 83164464432.10.FFFE64E Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf16.hostedemail.com (Postfix) with ESMTP id 23BFF180010 for ; Thu, 27 Feb 2025 04:36:53 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=BOemDG2K; spf=pass (imf16.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631014; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DWBUXpwpeVKwHwztQYQNBbUUc1WcNoYn31qV1KKEn5Y=; b=mPmY6a2bImm3Zf9YZgjESt/6wCr8P5/VsO+2R04cN16BhDijUUTcV2OlEmbj5VFmLwv6P3 KeNEvzMTvxrf1dM3VslkIN7HZzyRfHdDLExrVZm1Ia5IdHYTojKqG+Uobw6DbVt2JUzaAY WS3bb0WiYtCJqW67SLMY47KljvNuifM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631014; a=rsa-sha256; cv=none; b=dKdZP8YcxpETvE52YwNyIpxswqJh2tFcJZWqNEKqzRTkADF96n7jIGhWbdRY5rCLYLT7zn s2mQ7HaqJiB9trxtlLdO7taQK1rJcZmh+eIk9sZKFzQ7UxEVX/wcDPFgxQbNSwCnQPlKXu SWTuxG2tvEeDhrk+CmRHYZqcY043v3M= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=BOemDG2K; spf=pass (imf16.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-2235908a30aso364045ad.3 for ; Wed, 26 Feb 2025 20:36:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631013; x=1741235813; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DWBUXpwpeVKwHwztQYQNBbUUc1WcNoYn31qV1KKEn5Y=; b=BOemDG2Kq1I/MYpEJ9HYHRRUVHZUJ4A/cTKkokaXllaRTYN5zUyXxhlZEEW6n6KGlW KyYq3SdoHspS1h/h1Q7djUMeDNfaxKjUeZ1tuzR01H9Xlbm4P8f8f9UOFscz6vpp7bW1 VC+D37uq9pgeyyDUgoBFNdaZncoK6/f16s3gU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631013; x=1741235813; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DWBUXpwpeVKwHwztQYQNBbUUc1WcNoYn31qV1KKEn5Y=; b=Nsg7V0767wWeNpauVf0yE0QnoAXHUkxsEfmEDgyPCNZ8RUf95pTvNMe94VioOOOWDi KSid4118WzFbazgQ1cDpTAQlFVc3b8V3F+6urj/yC6WI6SPYRMVLtIL32q3HQ6MmsLf4 3IP5r39AhCdhmZV4eDnOBDfhsLpilPDmbTYN49YuouSFncARsPTpy/rpRdHudPYCkhf+ VIYUmok1tdaBR0q0+FLFT7UPN2zC11oePXScylb1zfW8WV/3SkvGe3K7W1E+ODCbW/EG jczrAl/mNXGu5UhCt7GaftYG+6L5gt+GrlkN/P28tXP7XbmiHvvGyvVGZ9+6y0OId/XI FBng== X-Forwarded-Encrypted: i=1; AJvYcCVwM0GU1OSoKPXVILspElyASkqKfJGngKzzdfeIaqzvGocCiQ8tuMMQv56gIRrlGUBREpxHBFqgdw==@kvack.org X-Gm-Message-State: AOJu0YwcQN0neB83w6V3sHaSwt46lxq5Lkyf+7oOesWwBq9wOX5imvJO MvXBPB+sjgm9mJqWPYJv7WGIpz0MHnYA9oCsjpDBfMawNhv/QIo/4JjowHfMXw== X-Gm-Gg: ASbGncuW9TmjGFnNWEASknXhgq78Itkvkz92pxKUHUJSp+JKvGXneTEycwqn5ueJ2bQ WB4qvJsQ3Ji7F2cj0dBYv0kFLvrLY+X77zmL3inFm6ffh6kNBAbXnM+AyGItQOq9Wl4l7qZrMSI 84YtE8ofUnrk8R8flFpOi9Wh0GJ4P2TKNwj79znXmywn4fZPZnw0KuIGXz88vzs6V+JkrE8pFY2 nDYLqLKYF16sivRVaKMpQxvVlwdTey9MgU96ZW2030afmTMzbR3qIqScO7hMe4D8UpbaIFQUHTT +XSSIvQE8371sqi/GZhkIoeUr315 X-Google-Smtp-Source: AGHT+IHsEmBFpQZ3F93ddNTk7TR/BbwMCem5lOBEnuVdbTxvBpCIY+MLGxmXXQpOcN29GcZXld6Agg== X-Received: by 2002:a17:902:d2cc:b0:216:3e87:c9fc with SMTP id d9443c01a7336-22320082b84mr97602405ad.5.1740631013020; Wed, 26 Feb 2025 20:36:53 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2fea6985226sm462165a91.40.2025.02.26.20.36.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:36:52 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 04/19] zram: remove max_comp_streams device attr Date: Thu, 27 Feb 2025 13:35:22 +0900 Message-ID: <20250227043618.88380-5-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 23BFF180010 X-Rspamd-Server: rspam07 X-Stat-Signature: otmpgr8zdkzy6u3mir456scb564huq3s X-HE-Tag: 1740631013-392979 X-HE-Meta: U2FsdGVkX1/2FkiOLVACoFabZ0PJ/yshhK1PqNLkP/Q1yrLmzPUbKAu/c/qAGFZbBFtls1vTlfStgtDMDtxcCJ9urt0MUWzZWwKYD/RvwqfsRPU8zMdvabmUkt04UE1LdODlLH5cLGn/c7qv42+uGYB+GkeIcLYSsSoj8hgdeS4/rREOH2F8o4UyBpXNQwg/GNMKca+Z1xh5Leo0wm+QfDsPP3m1B0PQX/mMkP0/wOgpvqq57DxVXkXNXGtRU6A4CCfhP9WBx6hVlPUj4GH+l1HVBo3Nzt1c7G0PBsl+lhEmXsGf8xdVdg2wF7MEYTpDC4X/y+Z5zugciiIqotPlO5l3asssQ+H1SBXyfGSl/EgbAPRXlXMOS6vuoUpkmIMQoVyzcL1FdXOg25UBpvpttR4YVLQ/bcWebVvhKpp+Npeb6Rne1MP2PB0EFqSLiGcNnccBUoHQ05fYBwW3vbVHli2m8z8zf5w0JoX92+nh3hPanfpGr8L+ejwe9iSbtdOtYAZV59zJQ30tjZKcGz3LZv84mKjFBWEEvy/v5SyXzVVV65r6MKzIMMF+UqR7pOlV2qdC7eYQfaoO2Vtmnblg0dWK7CvkDnD82GusbaWWGYTKVpoF6TEMx2wzNgG5KfCXkBIHwIY5eniI+txQfDXCDy2hFH/Y0zyfkSyEjeuc7f3kOLXW75GPSAUIoQyHxhJ8ZLBHk2Sy8b/w9Hp8dFBCA8Q749XyJMOPL/tnBVqPJoVM7bzSrdTy5MAykfpQn7GGa4mQHzsKfQnV+kahFUrAFYRI/tLj6PmzqhGTzNWKDN+TC/g6411SG8Ee6W7N1kALsz33ypV05GO9J/X+riTm5mG0+acv0Xh2YhMt2oB+CMmSOYxIaxrYJdw3zsgh3r24CJMgwGNg6hYsOqzO+BFeFAI25SYk6KEPwMIAC31y4uTROcgFETk91M554RtMcBadlOt7PeT+ZWYHTVcZ18k yTpj0vT0 BxgfSUV5rm0x5g/pqpn+/uhACvowtJLVjpMQETaHAExqSkxBFhvjrDpavqpCnNTcIxZjgGUgGBeBPWE6hdof9c9TQi33SzEh5q576IkpBWC40992uSa9nuxyYaSIIiKI1LQ0P02O7ZDi1RP9lrpd7O13wolTTsH5yh705jxmeU4nP+3LVC4a954pq7YE9W4hZp1m3N4DJxGi8z2rBN1m3lJEQwOEjl4b9KEkWb5LvWa9wsvbgko25ntF9oe2/pNH+a5Z4NpCJ0MBYuXPGWHdZSRo2pLA/7thQMOr4o6aN3h83GSPzOPebAtTGmOSdjGpS2XYO9uJYTc9eSdsH6++VTWr1jz5S+dvuk29dnCJIqEIjDRed3HIwAhACJVbyvvgVFIEi X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: max_comp_streams device attribute has been defunct since May 2016 when zram switched to per-CPU compression streams, remove it. Signed-off-by: Sergey Senozhatsky --- Documentation/ABI/testing/sysfs-block-zram | 8 ----- Documentation/admin-guide/blockdev/zram.rst | 36 ++++++--------------- drivers/block/zram/zram_drv.c | 23 ------------- 3 files changed, 10 insertions(+), 57 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-block-zram b/Documentation/ABI/testing/sysfs-block-zram index 1ef69e0271f9..36c57de0a10a 100644 --- a/Documentation/ABI/testing/sysfs-block-zram +++ b/Documentation/ABI/testing/sysfs-block-zram @@ -22,14 +22,6 @@ Description: device. The reset operation frees all the memory associated with this device. -What: /sys/block/zram/max_comp_streams -Date: February 2014 -Contact: Sergey Senozhatsky -Description: - The max_comp_streams file is read-write and specifies the - number of backend's zcomp_strm compression streams (number of - concurrent compress operations). - What: /sys/block/zram/comp_algorithm Date: February 2014 Contact: Sergey Senozhatsky diff --git a/Documentation/admin-guide/blockdev/zram.rst b/Documentation/admin-guide/blockdev/zram.rst index 714a5171bfc0..7ad4c86f8258 100644 --- a/Documentation/admin-guide/blockdev/zram.rst +++ b/Documentation/admin-guide/blockdev/zram.rst @@ -54,7 +54,7 @@ The list of possible return codes: If you use 'echo', the returned value is set by the 'echo' utility, and, in general case, something like:: - echo 3 > /sys/block/zram0/max_comp_streams + echo foo > /sys/block/zram0/comp_algorithm if [ $? -ne 0 ]; then handle_error fi @@ -73,21 +73,7 @@ This creates 4 devices: /dev/zram{0,1,2,3} num_devices parameter is optional and tells zram how many devices should be pre-created. Default: 1. -2) Set max number of compression streams -======================================== - -Regardless of the value passed to this attribute, ZRAM will always -allocate multiple compression streams - one per online CPU - thus -allowing several concurrent compression operations. The number of -allocated compression streams goes down when some of the CPUs -become offline. There is no single-compression-stream mode anymore, -unless you are running a UP system or have only 1 CPU online. - -To find out how many streams are currently available:: - - cat /sys/block/zram0/max_comp_streams - -3) Select compression algorithm +2) Select compression algorithm =============================== Using comp_algorithm device attribute one can see available and @@ -107,7 +93,7 @@ Examples:: For the time being, the `comp_algorithm` content shows only compression algorithms that are supported by zram. -4) Set compression algorithm parameters: Optional +3) Set compression algorithm parameters: Optional ================================================= Compression algorithms may support specific parameters which can be @@ -138,7 +124,7 @@ better the compression ratio, it even can take negatives values for some algorithms), for other algorithms `level` is acceleration level (the higher the value the lower the compression ratio). -5) Set Disksize +4) Set Disksize =============== Set disk size by writing the value to sysfs node 'disksize'. @@ -158,7 +144,7 @@ There is little point creating a zram of greater than twice the size of memory since we expect a 2:1 compression ratio. Note that zram uses about 0.1% of the size of the disk when not in use so a huge zram is wasteful. -6) Set memory limit: Optional +5) Set memory limit: Optional ============================= Set memory limit by writing the value to sysfs node 'mem_limit'. @@ -177,7 +163,7 @@ Examples:: # To disable memory limit echo 0 > /sys/block/zram0/mem_limit -7) Activate +6) Activate =========== :: @@ -188,7 +174,7 @@ Examples:: mkfs.ext4 /dev/zram1 mount /dev/zram1 /tmp -8) Add/remove zram devices +7) Add/remove zram devices ========================== zram provides a control interface, which enables dynamic (on-demand) device @@ -208,7 +194,7 @@ execute:: echo X > /sys/class/zram-control/hot_remove -9) Stats +8) Stats ======== Per-device statistics are exported as various nodes under /sys/block/zram/ @@ -228,8 +214,6 @@ mem_limit WO specifies the maximum amount of memory ZRAM can writeback_limit WO specifies the maximum amount of write IO zram can write out to backing device as 4KB unit writeback_limit_enable RW show and set writeback_limit feature -max_comp_streams RW the number of possible concurrent compress - operations comp_algorithm RW show and change the compression algorithm algorithm_params WO setup compression algorithm parameters compact WO trigger memory compaction @@ -310,7 +294,7 @@ a single line of text and contains the following stats separated by whitespace: Unit: 4K bytes ============== ============================================================= -10) Deactivate +9) Deactivate ============== :: @@ -318,7 +302,7 @@ a single line of text and contains the following stats separated by whitespace: swapoff /dev/zram0 umount /dev/zram1 -11) Reset +10) Reset ========= Write any positive value to 'reset' sysfs node:: diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index ff662144b98b..5f2c08337ba9 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1103,27 +1103,6 @@ static void zram_debugfs_register(struct zram *zram) {}; static void zram_debugfs_unregister(struct zram *zram) {}; #endif -/* - * We switched to per-cpu streams and this attr is not needed anymore. - * However, we will keep it around for some time, because: - * a) we may revert per-cpu streams in the future - * b) it's visible to user space and we need to follow our 2 years - * retirement rule; but we already have a number of 'soon to be - * altered' attrs, so max_comp_streams need to wait for the next - * layoff cycle. - */ -static ssize_t max_comp_streams_show(struct device *dev, - struct device_attribute *attr, char *buf) -{ - return scnprintf(buf, PAGE_SIZE, "%d\n", num_online_cpus()); -} - -static ssize_t max_comp_streams_store(struct device *dev, - struct device_attribute *attr, const char *buf, size_t len) -{ - return len; -} - static void comp_algorithm_set(struct zram *zram, u32 prio, const char *alg) { /* Do not free statically defined compression algorithms */ @@ -2540,7 +2519,6 @@ static DEVICE_ATTR_WO(reset); static DEVICE_ATTR_WO(mem_limit); static DEVICE_ATTR_WO(mem_used_max); static DEVICE_ATTR_WO(idle); -static DEVICE_ATTR_RW(max_comp_streams); static DEVICE_ATTR_RW(comp_algorithm); #ifdef CONFIG_ZRAM_WRITEBACK static DEVICE_ATTR_RW(backing_dev); @@ -2562,7 +2540,6 @@ static struct attribute *zram_disk_attrs[] = { &dev_attr_mem_limit.attr, &dev_attr_mem_used_max.attr, &dev_attr_idle.attr, - &dev_attr_max_comp_streams.attr, &dev_attr_comp_algorithm.attr, #ifdef CONFIG_ZRAM_WRITEBACK &dev_attr_backing_dev.attr, From patchwork Thu Feb 27 04:35:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1334C197BF for ; Thu, 27 Feb 2025 04:37:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 53559280007; Wed, 26 Feb 2025 23:37:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E479280005; Wed, 26 Feb 2025 23:37:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3389B280007; Wed, 26 Feb 2025 23:37:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 11D3B280005 for ; Wed, 26 Feb 2025 23:37:02 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id B84DB1C7A08 for ; Thu, 27 Feb 2025 04:37:01 +0000 (UTC) X-FDA: 83164464642.08.F1AFE5E Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf11.hostedemail.com (Postfix) with ESMTP id C667A4000D for ; Thu, 27 Feb 2025 04:36:59 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=od01zQUK; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf11.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.177 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631019; a=rsa-sha256; cv=none; b=B1vjs1jJth/NoKo/bbb36n0IaveXxJPuZEYqB1MBXffSH80jfyBhkPy9zvlCbpBaTqqUzR TTCmUJgRR6GVMQAkr3EjlNBUhbX5IApSzZAutsEYU+VP+GD0K9whgQqEV39iD0ZY5aLSv7 r93K6RDhRrv7Ctg7BQ4ADy9sCGCXVmw= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=od01zQUK; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf11.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.177 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631019; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Be7BmKV5Rl+tSCSL+PenBDpRFLFJDOVjENq1jnQl0Z8=; b=ckiCYJusXb9/N8tW8BWKK6J979RPtG2h/92xE5f5G37xMDIGCzAuCC7cqAZ54PosY59Xte ZrBZQDB6NABqwpmSrwGGeSLsiIizTuJ31ZmDhRAt1QTbzT01XloiSf7KvGa9OhCoK2Sluo VBxGQ4WBM/SEbk5zkqAJzdjb4r48NW8= Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-22355618fd9so1975165ad.3 for ; Wed, 26 Feb 2025 20:36:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631019; x=1741235819; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Be7BmKV5Rl+tSCSL+PenBDpRFLFJDOVjENq1jnQl0Z8=; b=od01zQUKJws+q19B5kLMmBGtp2RjseAe8MyIIXt+4OKN42DLDBA3+oStU0lw+mfHtH wGVT/7E/+WplA8km9L/ssG9nqtwoLLqNj1QV6akgehWQWFYkZ2VJF5h3IS2iUl1mEPwn uZOXJFhjJiAttYlazLSnqmireKSyzuUCgy6oM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631019; x=1741235819; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Be7BmKV5Rl+tSCSL+PenBDpRFLFJDOVjENq1jnQl0Z8=; b=u+MQ5n1dahwRxSBO+nFUviCbfnnnl12RrJCekFTsR0NpoXjmr6WbSWy6RbU7RM+1qD wUDJ7YscnlIj7wnE8riIs9qZiN8Frl5+37rwXTeQeS9sCF/1TC5XKy5F8TPX/7/lnxX5 5I9+1+6z3bZ4Iv23BvcxEDSkknzUMG6ngN+lwEKrZvIbw9xriq2BgCEHttZKPK3/gWyX S7EUfOIkzg2LzUMtk6+KnHVVbVvX/EhM7c9z0Spu9bWtKOXSF/79QxscjNtxISMd7hzA h53C8OxwKWEVQHY1Q3c+nmxxyQilwkb5yZEp1v1VxNtH/iMzWQW8JpJ0eoQStjodLsmr 6w+A== X-Forwarded-Encrypted: i=1; AJvYcCVwz/CfBFKJEY8I1J+dWsI/CgjvjHFDEJMg4BpGq2Y/P8KlT/rGaCF+OJGqB/+zG8UbRMI6mciIYA==@kvack.org X-Gm-Message-State: AOJu0YyxCPjo5HtDvKpkNzNnPsX9YZnGLFxavELUabIc0y/fvtGzGk5l fy5XQ3XCBXDwhwjT/BmLYhBQz6MiH3CjngKQPeutUk6sNiOINg2tmcBd3MiKng== X-Gm-Gg: ASbGncsYhW8EYj1jgqW3PKWW3CokWzTWwRstVga8hfZWbFq4Yky7cbIEMIfsJvPrFDZ 5YBZQT8pSYfv21UgI80AG/CqTsuJf/yoWm7WLXuGOUssYOhQuIKXyuI+fYH3/a58I9IwV21T7wc OfnHDmyF2Qpd/Sidh1ieK9/mQypjHd6TemBt3N8tWJurKZPleBQFBK37gBN5QULtSLDWNwoAelN lUlcJM6LSMiQlQ0w07V0oIVkgPA9tC0iUqQcbbQSix1Prc2TD5lGSorDkWyYLE34cWm6WoJRL2j jVw1a2aW4JQlbSOOyu7caTzbGT8U X-Google-Smtp-Source: AGHT+IFJgaRFQVylaLj6I4IPqbcJ3f5L/ef1xGtQVqtC08AgMT8m0YWQWIDG6Ie7Da0oM3f1AlVfQQ== X-Received: by 2002:a05:6a20:72a1:b0:1e1:ca27:89f0 with SMTP id adf61e73a8af0-1eef3daad8bmr44973568637.37.1740631018819; Wed, 26 Feb 2025 20:36:58 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-734a003eb70sm485399b3a.124.2025.02.26.20.36.56 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:36:58 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 05/19] zram: remove second stage of handle allocation Date: Thu, 27 Feb 2025 13:35:23 +0900 Message-ID: <20250227043618.88380-6-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: C667A4000D X-Rspamd-Server: rspam12 X-Stat-Signature: 54h8fytspnd41kmcei4439ub6bjy3ef9 X-HE-Tag: 1740631019-759641 X-HE-Meta: U2FsdGVkX1/0WZcOqZkpeRscgbacla94E7S3OUHgAYgdmIeQuGHy86mRFuz2UlhiEARo/KlsZRgXqNtZC6yLEk+jgpjRMfs2nnYTZ7UvwuYj+Ca/IYk0+GQkb6x/LIE+AaxejO8VCTZdBPft5+5nswS+Yw5u1XQm7rGU4iep9m4C0vSazHoZtiYWiWKYSYDVmq3agFnEa+352YqRR1zYsRTfMTT2biXwwhxfQHW5cjwFZAjaHdSnbNW+zK4ASmszrwfBqxeROsWL8diCQF2GhakZ5+oHwwifM6TeTXSJoCxZkPxbV2D3F4oNMBbxISeZGR54th3PeBM/ypLoFhvCAWJ1LpwvG/MJasS44smwxNqYfjCCQ092c93hOyrMLIUXEs60TB3L+l+Eh1hFfzouog6g/dZPGcuyvPg7BBKw4acrL/7heeHhnHIcpAw3f+gVTcBrwTdhCIrr2/4ocQDBHEL2t1M9vG0/RxccNjxDxFaYxoIfrXKGeUFkxUCK1Ua8vax7nPlJhVjk2OPYnX+l3D9HiHp53Abx6Xpo1cD7d8cA6QufRJELdIqOcE9JSCMA/xIVwNoWXtkUWSs4T71NvmeXelTLgC3/gSD8NavarkhUoHRKQqSXKod8ttUIfDCcG8Aueyzxok6/VrIRuJ01keyw6CilLwtC0GnbcODGsLkmlRc7DtONRIIpAPtp+wwW/ejDQLDZ8UPeMNZ+3v/fqMufcqFJdWfQOibEKUSRNgY0Xb8NW1r8fCCc/4hRYdXtUZdn4QiVHVnPa7ffeVlJ8nVhk1xxhYDE6pI+GOsOAzmWxGyKHC5sXehpTDDOztDEjRM1SerAiaKzI/s06u76Rc1gytMv8Tayt1spmICpI2q6Z5/jg5XuivpEK8jueZ7QLuvk5u1V+hYeUyEVD2VxM2KFw9LE5UozSePjxrSySiBMxYBf8i8hciLmtQcz6UaX3VrPY6QGcrLciWvsH7/ +roNunNx XA6Y01EU6JPWNz7nDcOEdM6Bpwin8RyY5S7LKghiwuRrBW3/ddvi/bAvBt00O0t3NC/THMhov7JC127EGM65k/DtjxZ5EtAnLQabHiBe7bSR2ThFa4UruOagQKXxo8DGtskaPzcJQ0GdHlTApv5W4jjdDsD0QhjmnADyT66vji6fTrK9sc7Jyy0hGF0bsTqm9+JUzAfDIvyT6C0SIZu8CXwZPVUVUASPsGLVnyR0Wb9HS1bBqoqVp3qLShvfxC6ewU+Rlk82R5Khcu9TrVcqa/kvne9YfmoaxSCLUrfcKL9d9sQxt5PkUFvwZ2A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Previously zram write() was atomic which required us to pass __GFP_KSWAPD_RECLAIM to zsmalloc handle allocation on a fast path and attempt a slow path allocation (with recompression) if the fast path failed. Since we are not in atomic context anymore we can permit direct reclaim during handle allocation, and hence can have a single allocation path. There is no slow path anymore so we don't unlock per-CPU stream (and don't lose compressed data) which means that there is no need to do recompression now (which should reduce CPU and battery usage). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 39 +++++++---------------------------- 1 file changed, 7 insertions(+), 32 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 5f2c08337ba9..28aa1c437d83 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1723,11 +1723,11 @@ static int write_incompressible_page(struct zram *zram, struct page *page, static int zram_write_page(struct zram *zram, struct page *page, u32 index) { int ret = 0; - unsigned long handle = -ENOMEM; - unsigned int comp_len = 0; + unsigned long handle; + unsigned int comp_len; void *dst, *mem; struct zcomp_strm *zstrm; - unsigned long element = 0; + unsigned long element; bool same_filled; /* First, free memory allocated to this slot (if any) */ @@ -1741,7 +1741,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (same_filled) return write_same_filled_page(zram, element, index); -compress_again: zstrm = zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); mem = kmap_local_page(page); ret = zcomp_compress(zram->comps[ZRAM_PRIMARY_COMP], zstrm, @@ -1751,7 +1750,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (unlikely(ret)) { zcomp_stream_put(zstrm); pr_err("Compression failed! err=%d\n", ret); - zs_free(zram->mem_pool, handle); return ret; } @@ -1760,35 +1758,12 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return write_incompressible_page(zram, page, index); } - /* - * handle allocation has 2 paths: - * a) fast path is executed with preemption disabled (for - * per-cpu streams) and has __GFP_DIRECT_RECLAIM bit clear, - * since we can't sleep; - * b) slow path enables preemption and attempts to allocate - * the page with __GFP_DIRECT_RECLAIM bit set. we have to - * put per-cpu compression stream and, thus, to re-do - * the compression once handle is allocated. - * - * if we have a 'non-null' handle here then we are coming - * from the slow path and handle has already been allocated. - */ - if (IS_ERR_VALUE(handle)) - handle = zs_malloc(zram->mem_pool, comp_len, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + handle = zs_malloc(zram->mem_pool, comp_len, + GFP_NOIO | __GFP_NOWARN | + __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) { zcomp_stream_put(zstrm); - atomic64_inc(&zram->stats.writestall); - handle = zs_malloc(zram->mem_pool, comp_len, - GFP_NOIO | __GFP_HIGHMEM | - __GFP_MOVABLE); - if (IS_ERR_VALUE(handle)) - return PTR_ERR((void *)handle); - - goto compress_again; + return PTR_ERR((void *)handle); } if (!zram_can_store_page(zram)) { From patchwork Thu Feb 27 04:35:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993736 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 188E5C197BF for ; Thu, 27 Feb 2025 04:37:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C93F280008; Wed, 26 Feb 2025 23:37:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9786F280005; Wed, 26 Feb 2025 23:37:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81A87280008; Wed, 26 Feb 2025 23:37:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5FCE4280005 for ; Wed, 26 Feb 2025 23:37:07 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 214EC12025A for ; Thu, 27 Feb 2025 04:37:07 +0000 (UTC) X-FDA: 83164464894.04.165D524 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf14.hostedemail.com (Postfix) with ESMTP id 33D76100004 for ; Thu, 27 Feb 2025 04:37:05 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="LvF/1qBy"; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.176 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631025; a=rsa-sha256; cv=none; b=QCQNO1a/q1kuIbifZlBGdEvSH9Q+VfVt1UdJmolvDPgr2ZYKPnFFYzDuJ7zZ9J1ZayawXw UuTQS9fJBUJaWmz5+uSYsmlb7Hg7fOEcLbcY6p3dkDrHMv8PO2lSN1OsToFzWZoZOI9RZn bGv9hylI1tCXIAtc28AMFkaDmfpNul0= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="LvF/1qBy"; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.176 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631025; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Vr9FTPnbuS7T/s7kIrPcS+bzGN5VDCIRcXkQbYpH7iY=; b=jmt5jLPSYDdmVJAqOAiWCngYa+XRAH8ulAslv3IWogJHKb/CMT1JNEK8tsiX1/IralmJjx eZ/ZJyle40UipfkpF0qW+tEWiJ4GRHiPSZWU5YHcZFXSb3lV5KRCQcee0pqc8WPz5WafR6 8bcKSOC9U7itNE2UFklwGZJsu455y4A= Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-22349bb8605so9126275ad.0 for ; Wed, 26 Feb 2025 20:37:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631024; x=1741235824; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Vr9FTPnbuS7T/s7kIrPcS+bzGN5VDCIRcXkQbYpH7iY=; b=LvF/1qByJbD1M8CaCVJ1OnDYLEOg7pH816BoAQGSDkJovI2LL9KZo7QH6gNblWOB5A ralSySBItnqdGltsqq9UIzYol9l9mqlbTkGaUwMUCYe5+rbnVXBCJCxZXjbWVuXUlwYt f8ExrSFfaltiHexB1hyTDdmCBSesBiR0arV7g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631024; x=1741235824; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Vr9FTPnbuS7T/s7kIrPcS+bzGN5VDCIRcXkQbYpH7iY=; b=txMntaGyhwkAXmBzo3OjXNCHZ5tJmCTSb0l2MtND+0Ui20vNThU6Zpwm2XF7x5633u A6Wjf4b9SoMTDBcF0CG+fubhWKOfbEMRLSPTx+mZG066x0JAgfY4Ss7E1EhDBvPBXs3A QOXOXXWaoav6jO1vNIiIb2oJgFXy8YWESA2dxPOJYB7JoSJKlcy1nBiXZDWKWTqD2PWN cb6fvmIlKYwFZ2d1J7qHu3Yn90ve2NiZp/eYtU+H+8q/7FBe34vDVfLsfuMo+23zMVrG f0+Bux5UCEK7iOKMlzqW0l2PhTcEZQxDh2WSnCdmSVxI6vXHPBcme/nY7jqQUmYHnPBP DdXA== X-Forwarded-Encrypted: i=1; AJvYcCXuwoUwSW72/5MSH1W0Pf4BmpSaxhNHB7APLr0JIoQUBA+Fjb44naf7FlXQ4CRxeHYNRqu25rToIA==@kvack.org X-Gm-Message-State: AOJu0YyDujekbch3UsDHZY+6M7HoV3rZLj8U3yliXbZzp5EE1Zo2qLay 4bquzEya4BhH8NjPHuFob+cNEF3AbBIeeRUxdTNzhbo8Y1xGwtCN9gpcXUqv/g== X-Gm-Gg: ASbGnctFb1tCt0RAKhdBTqGDsqx/6Fziw1QEuVMSA4lwgeXtY4XyY44WmcjH7wvqDqM 6crESDFPWPPnYJ9krsq2s+XDAeeGt4kml6NYJzNUdte3bKNspy6H4N2epTi+adWELy/QUfOtf94 omsrZm3RJkp++ImH5b1+w7RuguQw8e6RKPSDq0HiInoaCHaWhbLVSBPVJi/8dj910tpw8owsfMv kQvSX6v34EAkOFg8mvPn74knwlzY2aCsnicrZxoMaAS5Qq4d6ssjrZ+F4+ij+CPKMXd0gyy4V5R 7xFhlIqvVsm5NAw/Qeq//f86A9PR X-Google-Smtp-Source: AGHT+IG7lkAWlRB5fX++3hFsx9jC6B8UxClPPlZOhyWNT72+u/H7C4m/UOugG8H83NzmmgRIdsxc9g== X-Received: by 2002:a17:903:22c7:b0:220:cd7f:cde8 with SMTP id d9443c01a7336-22307b5b218mr158250195ad.29.1740631024159; Wed, 26 Feb 2025 20:37:04 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-223504e4d16sm4908435ad.189.2025.02.26.20.37.01 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:37:03 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 06/19] zram: no-warn about zsmalloc handle allocation Date: Thu, 27 Feb 2025 13:35:24 +0900 Message-ID: <20250227043618.88380-7-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 33D76100004 X-Stat-Signature: piqitbgbzn191rdecowxem5pi1ypquji X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1740631025-152618 X-HE-Meta: U2FsdGVkX183f0CODU16MXrHeMiAfRpiOCOoIMDmn6hGfHTkMCbKDTbyb8Fn/noB/VdkPJmG6flG7qpBfFFxjUkE6pHtnCp4t8XUzkSZz8kfUivU9fgN3qrYkzryhUibrWXN+YNjzyxWo6SEtWi/ApsYrxmb8FcD1aSK3wyxNxAVWSo/0AGYHahyRQUR96kuod+Ek8RYyZtUvZsWo6pYdaTHFlKFk/Ss51XaKiMnvBqtFUvRL515pkX/12yjPOe2hiqb71hrIaal6mCQ3ekMiFb1dhSBJNW43twc54TO3mHRFqaSwJ3fpR9mvlqjSVex5RwC3U1ig34x7yl3VL5I9OpuoRufABpmsoCxbWD2su/M0929VlCiKaAcB8OoDnLeBtksrWy4nbfysq9KrSLX+yd27R5vSqE1SCNW0Vr1CQEvZXxu9fHHvEm14RWN0ofAsKQN/IpvjTXlNWmIlcm2uo4NhWiJCMmhss/9jptQC/xRlfEetOgfeKjviVpkZ9TFaxQHifQugsG/7LMaEpM+aS+GGpdckQgxp7z6AhTyVpxMtcHsEAebP+frN8oSt7Nqt7/7YvjwSy5Y2ctWKGuD5EWamHSdDv08O1JJEbeK2+xj5ztJqBEf3a9teffv32+6jgBS2bRozulLVMo9WsEDs8cNJW3i6HdKt7k6oLHhgmlxvbLyD40WCycEaJCQLW1pvtBzPxPeMFG/X7uHewkZrjHBBTpnOCzRmU92KUOhD57w63dy8acZmH8u675QvLlh+SzV2rr2sGsno5xgZ4r7cAbiFp5zozd0yNQPAWHgvhCunJClGnSUaf//TF6mCIwXw4N2c6XBGve9d1j5uuXA0/xpcaGZaVLYjxATsHSFQR3Z/GRwCUxvS9b76YFMthvwHS8G09JLzmsGtn9d6iH6l7w0XcJzCiBrbqAII6oqvImdpKj313IHGLk8ph4hK4jzzzjrVjstvjCXH6QCFbS HqlXeFZN FtM3zkvLplkVND/gD1ynUGOuAtO6gqAXbG6ESM9StKXhzHHHCUaFkXNhbtJMlbZxndEGK9JGbfU2DgF/dgsgD1M6gmalEHDuxK2VC9bQkRYgoA+dxIM6LdC5+qTKVN7PS/d//E+IsyJMMu2YRqlvyJ2lnDkjs5eFCUVuG2b//SGCenbkqpkeZdmJkPtGNQk8KgMJDF6eAtbk2OnTU1/OsrVysZeznF3HO8VYONoQFBFZ8S8Nu4ZnkQ+2BSfDyR3P0YUu5YwnB87/x33LiHlOt58NC9UsyQND+r7r0xYX638A1JjIjuuAb27IfdQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We normally use __GFP_NOWARN for zsmalloc handle allocations, add it to write_incompressible_page() allocation too. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 28aa1c437d83..08fc9821d71c 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1691,7 +1691,8 @@ static int write_incompressible_page(struct zram *zram, struct page *page, * like we do for compressible pages. */ handle = zs_malloc(zram->mem_pool, PAGE_SIZE, - GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); + GFP_NOIO | __GFP_NOWARN | + __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) return PTR_ERR((void *)handle); From patchwork Thu Feb 27 04:35:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993737 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECF94C021BE for ; Thu, 27 Feb 2025 04:37:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7CD00280009; Wed, 26 Feb 2025 23:37:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 77CF0280005; Wed, 26 Feb 2025 23:37:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F6D9280009; Wed, 26 Feb 2025 23:37:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 3EC8B280005 for ; Wed, 26 Feb 2025 23:37:13 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id E97D04BF89 for ; Thu, 27 Feb 2025 04:37:12 +0000 (UTC) X-FDA: 83164465104.30.F3EE26B Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by imf15.hostedemail.com (Postfix) with ESMTP id 1A0F2A0006 for ; Thu, 27 Feb 2025 04:37:10 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=dIXSAUTu; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf15.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.50 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631031; a=rsa-sha256; cv=none; b=NsR3E3kOfSipU0Y0UveJtX7deJ0wDrzRC2drVzpnZepYxgrpl+/Xt08F9cY8oKECZAavZE T5m/V77t1GSQYkOm2AymedGKmnk0QuuJU8QXJp5lCPBbKZJtQaJzOaveQUa4hZSrnqc9B9 G1NO6mwIAm+8g7RucnWJi21jj+KWk/Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631031; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jjK5z/f2m0DncP7Ixpif51js64ejx6xQxz7bzy27nuY=; b=CNA5RT1J+wMmAFbvlBBpPNeglcjWaIF7x8hLdGsjUDxTfjSpA6fWI3DJL31e6Hds4tLLKZ PHhAsxjEgR74TTjz7x0KzAvFYyDcwnqZB6ujaGF2f8krj8ERo0FVQxGL9FYTaS+mrf0TF8 91zvDOU9ZqjlM6bbc4pjUEdaWihLGJ4= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=dIXSAUTu; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf15.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.50 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-2fe77285e12so1049739a91.1 for ; Wed, 26 Feb 2025 20:37:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631030; x=1741235830; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jjK5z/f2m0DncP7Ixpif51js64ejx6xQxz7bzy27nuY=; b=dIXSAUTuFXWKs2nr6Yfl4ZLKrI4ZK5MZ+apD29b+eM3qaNrftyfI8VkG9vepOCDQaa h6Yau38QUk2qUdoZbtY0CYlbhJi1sgnKXk0JmWrTmKDYtnZ/i2pAp197GhB5BFss1jga /Y+x/2GfcVAKp7OxIrYZuWj2iqY+m3Pl+i8UE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631030; x=1741235830; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jjK5z/f2m0DncP7Ixpif51js64ejx6xQxz7bzy27nuY=; b=BgYWpz8r8sz6ab1oLwlwopNiPc8xSVBuZszznnq5+RsataJeTvDXGXqAf1HyijNvsx JSyfKzFylSFqKJxysyvJo0qrKKjAY9iLrO0fD6S7/tZzVf+5I2tc7zAskIYZhqg1b25J jkv4JM9a1irkyxeYgi+Lbyoj3UDGBLc/xNNc4DwcC6qkxOVk5in9w95lfIBGZojW3RPF NJBuuURIeOPXSEviGuSG/lVSqL5m2FKACTs5m4LrgFOVuHuDyRfsDAxXSitEXxYKVl6z QLXSwlj98OYEJHVDMCNjdZEpnbv+eAvdwKz0vSEfqo1W98Fu92l2TvsGA9RnADSwGgt4 kGlA== X-Forwarded-Encrypted: i=1; AJvYcCXjimta88Tix5oqijT3085wVMOUWw2Qh1rD+WxPNOSZy4DYYysh6fxXYabLx9bkdxzJMbsoeQBxQg==@kvack.org X-Gm-Message-State: AOJu0YxduyDp9aUCcXNrXACD0EEyC7iyPVuRN3zCaO0/+z2A3TNBlLDz KwcwNmOJtfiMPOyD5XQaI5hDwjSMe6lbnBLxgYQetiI9/5RT65r9McAnE3rFBeVoERWbjBhre8s = X-Gm-Gg: ASbGncsDK/PWpK9w2cZbAD48dk5qgTB9QKtSgefOiRi3f/llJHnzCfQaJQ06w/9uthr dEKKmz1fHTjby825zg9QR97YbZV1xWT0ZpDC/eUkMulpj9l+kiEirWr95l0oE4+IiCo/9fdKnsd Gw1uZ8gmrVeOV9npVj9LOcW8s8iBM/OSp/tzG3/H3tiOT5NkJ7X7Kw8Q7nmxNkjSkKgVE1JbGKp CRaGIst6NMDD5qYmIG+zmIlRwRyZzwZqRU1kKK/M9sXo37CyLJ1I7rVpMgXGFjeHTi+OkNguGpK vxom0+7YAASR39ixlHsTkpjvWcYx X-Google-Smtp-Source: AGHT+IFGQH7QQ3h8Q2RsubEbMnBDcSEsStmitP3pSesQC9VI8bjIeDpBSGzc6iv7/4S9G/lG5bzkYw== X-Received: by 2002:a17:90a:c2c7:b0:2ea:a25d:3baa with SMTP id 98e67ed59e1d1-2fe68acd797mr14877858a91.5.1740631029968; Wed, 26 Feb 2025 20:37:09 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2fea676b133sm474944a91.13.2025.02.26.20.37.07 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:37:09 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 07/19] zram: remove writestall zram_stats member Date: Thu, 27 Feb 2025 13:35:25 +0900 Message-ID: <20250227043618.88380-8-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 1A0F2A0006 X-Rspamd-Server: rspam08 X-Rspam-User: X-Stat-Signature: o6yantyf613jwz8at4a3rbe6epbcgic6 X-HE-Tag: 1740631030-891441 X-HE-Meta: U2FsdGVkX186kGHU5822mjW7daoTInZ/rCHlSRdYEcnzt3nIXXV86LpMsRWlzoHKXMbngZycjQSBotd0i+/fedHzWW4RDEwDRS5RAM/5g08s9GrcvSXr8pNSyFU3UXGH/l3034DyxG//+Php56jtUTS2GCsfuSIBAsvBW11QmepNEXD6pt25wz0AbMYJx3eA4ZIyeo0FLolG027hXkjeN2VbrqmPfE2d3oVRD9RrcVEEMebo/e9lP1oSr349s0s8Ut+jXBq/YAQRMCteOkruKuHK85jJSSxGpcS8WZFwMy7YqIiPvBJN2bXWHUHSqN+ZgzZ/hDxjZ+1zzYfFn0m4fJo443w6VH5C9jeEWubqwYgeJe5ORI++wK4iyRazkSaRFnM6PF/VcX1sYFXugd7YvYoGvpWvhJafQ5hVd5rj2vIHOPvp6cGIZxLSvXozBTOsaJGrC1Y/C3WXOr0Oe8+nr+FHqAkukf/pggx0smewU8VJynPtHmVSg+QiBVUnr+EDqzi3/FRIzOY7MsBvXEA01v+FGXyunJgFljWEVfWcyjPhHsSxfRfeAvBHMK8nsZVl96ECBP5dDUH/mtbcJifkFT/uZ+8BrutzdYPFAm7o6dpjCdU8R3TPeJFKBQvnfyahB8mSyTjfoltoBpAvmWvAbwyf99MOIQn3hvtuXStTSh0l+Xg+m/et9huJBzvEoXqNzinJOknrcacjmqber42Z/Xg4U5RzElcg+rZNSVOUvLWf1WcZlrZBvaZQ1x2Z5AnYvZCEluER0uq/g9IwdRu5RXDiYthj/d82BlMtXrCfgSc5kKVUDLSlTn9XtVBo8abbjDkgzFZZVDYxngWAyCwweQOYKuf+01oUz9mx6EWIhNCuqUVFvt3k2B8RzBdgEGVxAUO5HrGovE/sqwIuHBB8wFODEQLdsKKyvrgF3OLy1YeLGKne1g8l9al25XdE9ItkEAM7P/XR+WSggBQFc2i 2J9QrVCC wYqmWhBitCTBP/SYngIMLLe/emaZEye+dP3AQcyonpGuhYxZ369Qax22FZeL0zFw4nMQeN6WAZ9F7xFXdUuQGQD05W/KYIrLRB4pmyY9vSq4hCwqA5H7RLKEB7R2fysbVvNZw79IFZznnXe2qCZO16vIfbePkvohY2IxC/plHvSnx3vPJmLY3iIXIUGtT2cFjAXKfalYxla1PnneL3tlHf6wIF57inHjkkGgqe18+JXRTz69pmF/Cjbty74x9gOS93ZmdyxVz032lPXSfXrUeROypmrttHk/9TdVZRh6K4CESKE0zwlCT2RhIliNnHOrCcXS7YofjkiY/3Hy5VvI3dgbfMHJIpqnXyySS34l6Wz3RnrA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There is no zsmalloc handle allocation slow path now and writestall is not possible any longer. Remove it from zram_stats. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 3 +-- drivers/block/zram/zram_drv.h | 1 - 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 08fc9821d71c..6266884de1d0 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1437,9 +1437,8 @@ static ssize_t debug_stat_show(struct device *dev, down_read(&zram->init_lock); ret = scnprintf(buf, PAGE_SIZE, - "version: %d\n%8llu %8llu\n", + "version: %d\n0 %8llu\n", version, - (u64)atomic64_read(&zram->stats.writestall), (u64)atomic64_read(&zram->stats.miss_free)); up_read(&zram->init_lock); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 8b66af0d162d..5ca87554aa26 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -82,7 +82,6 @@ struct zram_stats { atomic64_t huge_pages_since; /* no. of huge pages since zram set up */ atomic64_t pages_stored; /* no. of pages currently stored */ atomic_long_t max_used_pages; /* no. of maximum pages stored */ - atomic64_t writestall; /* no. of write slow paths */ atomic64_t miss_free; /* no. of missed free */ #ifdef CONFIG_ZRAM_WRITEBACK atomic64_t bd_count; /* no. of pages in backing device */ From patchwork Thu Feb 27 04:35:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993738 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B65AC197BF for ; Thu, 27 Feb 2025 04:37:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AB49328000A; Wed, 26 Feb 2025 23:37:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A635C280005; Wed, 26 Feb 2025 23:37:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DE1F28000A; Wed, 26 Feb 2025 23:37:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6E0A4280005 for ; Wed, 26 Feb 2025 23:37:18 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 2C62CA15B5 for ; Thu, 27 Feb 2025 04:37:18 +0000 (UTC) X-FDA: 83164465356.22.1023E57 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by imf23.hostedemail.com (Postfix) with ESMTP id 4EB45140004 for ; Thu, 27 Feb 2025 04:37:16 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=czxF1PRv; spf=pass (imf23.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.181 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631036; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zkdYQ64vEruCdUV2Hjk4Cody4qJgi72PfTkPf6ILG94=; b=sYjZAFqFaCelzEdPeGzXdVAF5ZcZ2MV9HVhUM+WmWmv3ba3KJ7H+EvO8EU893ZHxwk7x3B Zv79k3TV0bXsNt1VXl5IrV71kHehoazVSvH++HH+e6XW4WpOQxy5Akous98R83gMBie0uQ eqTRpxHTweV1RdTFLzLgFiGOEEn+/Fk= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=czxF1PRv; spf=pass (imf23.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.181 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631036; a=rsa-sha256; cv=none; b=JmrjoVe3gJhbLKHbPSW84QjVdBHXJOVqFERCmn3l4r90xShCyYkGF8CIkKfxrcPy/4jMC4 r6NXAkulW5zf6I4AeDcBW86J79wEPc+kyBew8TdIm7X60kdf1++6NxHMw9lOC2QrONvJQ0 ImqyU4QL3hg1zkfPASX3jFQO1mdizbU= Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-219f8263ae0so8209315ad.0 for ; Wed, 26 Feb 2025 20:37:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631035; x=1741235835; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zkdYQ64vEruCdUV2Hjk4Cody4qJgi72PfTkPf6ILG94=; b=czxF1PRvtXqu1OpfgQsR3YB+OorFbQSgtRxiL99cFi7ot1xen+tdEKbPPyFN0dhHeE LMyFKQFDYCAuoDTuQYY4H0Qh1cS7nHrAPYbHP8RZGev6lcSl9sCIad+Jd8cGe+UzRr0W 5wgt+ZFps2e9c5F9ahi56Um/gEZcaGA4FncgA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631035; x=1741235835; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zkdYQ64vEruCdUV2Hjk4Cody4qJgi72PfTkPf6ILG94=; b=MoI7PPepFVr9T+wdv9KRJcgXsbDshM04NbwgTcYdg6KF2hNX8tQjzHO6a5ZmzM8u7p kbxdg4DLPYlz6NJ9VvIukQlacZFguFzsDognkqaygcU1ruleIWGM+o6ilwWSgefEI3Fc PvLCtgm4alWmOpXU3vZU179KER2Zd9dv6ZrXL0sPT1e7xvpZ+F/cxNHwAPwCVWRwbAMf OFKEgYeDFBeeIO8wkHkUjpaD0PvnEKMJg7WxC8nuy0Z6Vh9aSwW4wEHJGdprK7o10FWm mSMBB4PN53fbMe0aL/bReq5FxaupqiL8b5xHWcA/BMCBEryXoE2Tn2d67RiFP1DpaIc1 AHEA== X-Forwarded-Encrypted: i=1; AJvYcCU3yiH9/OCse984X+1ODiDokVh5kbc7KJc7gvju0Y6WIlC2/kWaoj1UMORPLltrEyiGmY+E/AsMcQ==@kvack.org X-Gm-Message-State: AOJu0YwqMBOdoYf5y65wJjNKcpRRd7MsKIrLBMSHg926kzVHjZNWJML3 u0rvnNWsYDqdhRuEmzypxLVmXdy13x6FGG/+fXezORc3OBCl7AyWgJslr/tneA== X-Gm-Gg: ASbGncvVxjUGoRTwW/r98GSs3xqb2mrYUvkmMtOzvFCYisNmSFC6EoNjHdTKaVeyKv8 n2ja+wYSOHPFHXY2GjFWt2SO1Wi751//mZAAD1ZiVxsyGPLQjA+/nMCS2tge80Mv8U15VMUq5px hDReHfKv77yBghJjdFo+QcIkcdycvVOaMB0ZJwyQBqvMZoN0mTdYjjy+eIdSIv0EFZXvrjw/2RJ HCrxrtM1Wp9mpjS3EAYbL/bzJ7cDpqCFGmhVxRxzqpdUyRLPq7cMaHu9VlxuOevIEBjrA+pIgj1 gr7ElOEHsICZUBYew5LiaC4EG3xv X-Google-Smtp-Source: AGHT+IG7ihHLtTFv8mxg66ATtd7LlmNyi9F2oOAdH3uY5mBvX7qlHeuZ1UTku6FwKKmo5fi5m1btJw== X-Received: by 2002:a17:902:f681:b0:220:cd9f:a180 with SMTP id d9443c01a7336-221a11db956mr390915765ad.53.1740631035318; Wed, 26 Feb 2025 20:37:15 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-223501f98a3sm5049295ad.63.2025.02.26.20.37.12 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:37:14 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 08/19] zram: limit max recompress prio to num_active_comps Date: Thu, 27 Feb 2025 13:35:26 +0900 Message-ID: <20250227043618.88380-9-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 4EB45140004 X-Stat-Signature: zqwiq78g4x6rqgfwe3s6moudg73iojde X-Rspam-User: X-HE-Tag: 1740631036-895312 X-HE-Meta: U2FsdGVkX1+Yf+A799NiJn0gwE79iQQnHLNAGjr/uja1WIwpWLd4WMPP0AL5a2qUiI3LRqM6QhSqqxsaYd5UV0YybkZE6tEKf6mrR5DPgUUeaWOIDACb1hfeVGyBgHGHCfTScp9gDn0NyqelTqwdOpayVPKOxlW6DVyAOVAuPl0ir0SDcS0euF7C/NBFP08RCUeUkEz2xIn7LQUxQo80OkYv5CqVMyZI0TQLP5PkokvQNUUBNjr2Y18q7LxvaMSl2TPeJfNGT4idGe1EMXAKUxObzkhqrqRZAMHV61sKDqDX3nJPtmU0TU94vDF/7Yn3/Jy2/nduhMWqz9QXcqa0JTY2v9G3qalAELUMO+IP/7Jm+tlIeZpluH3SwDvm7c9aqaV8XJ6VBE/rKAxCKxaXj0zY/crK4v1yKOprnKXOruzt6ZnmkN/hBe795DeQ0vIwLc+B+2Etu/jSJ2XOPlYz06/Pj3rabkUJWVGFBRJUfT4BEVsra++4QmXlsw6yVGXw9wrb///nqiR6vQCNc2PzV+/qp0LlzsiiokJSs/Pl6JEfmgXojLyp5J3Gqwq3PSDQJ0jo8NzSnfVI2AVmmWmjfEzNV6jAhRGY5ql6yHex8PcrOHZqdm+WlzXxo4S7etiIXNZTSvlPssQwzJ+KmsYlIDSzpWD1U4/2OGh+rQe8F/huib0AC/VPnHt1X0pYSfokKv2udDJgkDLfMaGfjCp3aN8uXcgWQD2NsAM7Hc8V6yL98KCKDcWcAE6KPCQyE1+R35Sn1QG9xIQ7+1kMducQswCSsY3ig2Tk4RaNOwDAiahQGhKmUQKgzl9xxvZE9X6sMMia9T+jFAcFRK72NGDfyHSoRRosaNDtqpCaL9O835guA7clacerpX5YG6cQbohS90zHD+JxGJXq63J/QUc3SNiZn00yugFjmJ2KHZL+EIyChuehpDWsmNU1+YcAO7vvO6ehxc1wHXWNkVNtpcm kodcEy4/ voVY9O3dUt+u3ik7tMAD/heORl3WmKmkY/83IGY7/7CivJqCVxvZECDFyG0459Au5C3t5S6XTqhCENC7Xkex8gwM1D3CsXiWB5hP96j6fbPa9KZRlNq/EXY6w+hHfxSGDpTl9KJrS7x6ussR6M0pk9qQp09pI9g1K0IkO6Cx4/4hKj2BOeOvLPNAYwEObAd12Y4v/KRua85UntXOEgEY79MODPrFVyiTy/XX1aoAJpxLTybsp3vgfvxHYWbJSCJAY2NQaZ9BSv+EUEj2hb/RoLl6LDwXnFrwXhpKlYbVMr6SQQFm0a+EbWGE91/3HSB22S/8QJ2dZgNX2WwCqPYWA0sSAnWvUxm4QORbZ8zL/Mom6evo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use the actual number of algorithms zram was configure with instead of theoretical limit of ZRAM_MAX_COMPS. Also make sure that min prio is not above max prio. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 6266884de1d0..12b99a557667 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -2027,16 +2027,19 @@ static ssize_t recompress_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t len) { - u32 prio = ZRAM_SECONDARY_COMP, prio_max = ZRAM_MAX_COMPS; struct zram *zram = dev_to_zram(dev); char *args, *param, *val, *algo = NULL; u64 num_recomp_pages = ULLONG_MAX; struct zram_pp_ctl *ctl = NULL; struct zram_pp_slot *pps; u32 mode = 0, threshold = 0; + u32 prio, prio_max; struct page *page; ssize_t ret; + prio = ZRAM_SECONDARY_COMP; + prio_max = zram->num_active_comps; + args = skip_spaces(buf); while (*args) { args = next_arg(args, ¶m, &val); @@ -2089,7 +2092,7 @@ static ssize_t recompress_store(struct device *dev, if (prio == ZRAM_PRIMARY_COMP) prio = ZRAM_SECONDARY_COMP; - prio_max = min(prio + 1, ZRAM_MAX_COMPS); + prio_max = prio + 1; continue; } } @@ -2117,7 +2120,7 @@ static ssize_t recompress_store(struct device *dev, continue; if (!strcmp(zram->comp_algs[prio], algo)) { - prio_max = min(prio + 1, ZRAM_MAX_COMPS); + prio_max = prio + 1; found = true; break; } @@ -2129,6 +2132,12 @@ static ssize_t recompress_store(struct device *dev, } } + prio_max = min(prio_max, (u32)zram->num_active_comps); + if (prio >= prio_max) { + ret = -EINVAL; + goto release_init_lock; + } + page = alloc_page(GFP_KERNEL); if (!page) { ret = -ENOMEM; From patchwork Thu Feb 27 04:35:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993739 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A422C197BF for ; Thu, 27 Feb 2025 04:37:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DC15528000B; Wed, 26 Feb 2025 23:37:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D7099280005; Wed, 26 Feb 2025 23:37:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC2D428000B; Wed, 26 Feb 2025 23:37:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9DB38280005 for ; Wed, 26 Feb 2025 23:37:23 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 60C6212027C for ; Thu, 27 Feb 2025 04:37:23 +0000 (UTC) X-FDA: 83164465566.13.A81B1F6 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf09.hostedemail.com (Postfix) with ESMTP id 81DA2140002 for ; Thu, 27 Feb 2025 04:37:21 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=k4uHkA26; spf=pass (imf09.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631041; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=k2tj7S14ByHudFFBWtU70rdMhsyqSn6b6QKduoDJaoI=; b=LPgzMVbCpc1153i63s57uV2EDUe9Cti799eElW5eYA3LKzuDNro7wPM7Iu5vSyJ4Db0nT3 GgbYr1elKJGcdX+I9CDXiMqQFvdO5TkXxcf0923HKSyJiFKQlBd4yYbUcbqdkpB1InCXcB JS/nliCpY5m+G+mpKNHdTiGOgsrQwfk= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=k4uHkA26; spf=pass (imf09.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631041; a=rsa-sha256; cv=none; b=smG+UlgD2c8n52r+B8/mdu5eS2wjYhl6+PDIUbHfCTKrlv3riNnZwfJmR5YiiPBDDsd7JI AYyaKEJj+S2zypzOsX5Wfh+Eyjd5EAgQpbMhuqRyapnzM5oQyh+Z9sqBbA++6IOzRsCzx8 XSctcHkvlRLkL9Rlbh4OY/KXSlw5ALE= Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-22113560c57so8812405ad.2 for ; Wed, 26 Feb 2025 20:37:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631040; x=1741235840; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=k2tj7S14ByHudFFBWtU70rdMhsyqSn6b6QKduoDJaoI=; b=k4uHkA26Y+pyyb5fVnlYfh94fKDG19roU3NjyDMnmnc8TUasKy2k+S+Tu0WVc91O2f DBWAb7xbScC4m/7vRgDJ6HAVBxMwP533QXuBG0Le02eEJtCPiZx5ouW4RpmANS0MikQ5 Rr61/qb9yIXK8ZnEMKe/TnhQpyFzGjjVsSmJM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631040; x=1741235840; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k2tj7S14ByHudFFBWtU70rdMhsyqSn6b6QKduoDJaoI=; b=KYp4ROrVH+KFAU75UpCWEdOWCZDe5xEvjM54EWkS7Wwq4G+aMLtcmMpaqJ3z4RmRVS 0NnYxPdgQpz0xc+nyYGaFIV0jhV3Sp6b31eLkiW5+6VuU1NbUPS2ut0+AOCAnM6KmHlp HpbTOoH1ql28YcUCGw2LycshTC7Znok/4pKnDNGA0caMu7bJggjCyUtIHqep9WmZJF7m nEKl6T9463prpCfCS8h4nK4hvgWKw9Wt0dUVvE9tQwJOkgoRZiEuWliiKk+OlGirjc4/ XR8zd7na6Dqjn+Ym1mQCI5j3uWeYjM4SuIhKLFLiwdMbmB7lraRkbELmwoDEyQ2O522i 4pjw== X-Forwarded-Encrypted: i=1; AJvYcCVcdOLMuv7h6Tfw76dgbcRwVa/XbMPkgzvIG+h73gAL3pRrdCGrMW6DCuTM+GOQKpag/ttq9JFsDA==@kvack.org X-Gm-Message-State: AOJu0Yx/UOCYAAT55psGOkzA787BN9gQiT5TiCTkVOMf2r4plpBZ0H+5 EnBx+8YjFx4x2u0PvdwYCtqvEddh+a2E2QavmhMMCBLQyYa4Z/xNhPFBf17wmA== X-Gm-Gg: ASbGncsvhGMkwBZiM9O8WtUEmWCEUgt67aA5r13Ju8NZbxXrpM+nnC5lqorB9qNKk2F gW7YPtP9w4FGhmKCQmV5Ql7V59fE+uaJ9A6/m4C84zlzp17j6PxymW45QGZNkVXf2PrtslP6148 PdVIw0snm1Ud1L7CjyIn811oz/3ojlEftsxL3/Wcz6y2La2BxyQhOKqrHG6rgQOke0/3rFG0DN+ rv3EC/1GM/06jH75JSiAE2clyRY7k1GT7K1xSrGAykbmtnfuCAqggHmod7PBh0xO9oE7zT6Ak3I I0QJIMsbs1LGDHvhOXwv2DZfIB5F X-Google-Smtp-Source: AGHT+IEN+RE7KRyg5caAuRJqhyxwCNu+Sch604nCRIaPUWr0NHMHIZpiZPkRYwWAzDLUylZbTZZtGw== X-Received: by 2002:a17:902:d4cd:b0:220:f795:5f14 with SMTP id d9443c01a7336-221a10de0c0mr360128185ad.27.1740631040453; Wed, 26 Feb 2025 20:37:20 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-223501fb1f7sm5039955ad.98.2025.02.26.20.37.17 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:37:20 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 09/19] zram: filter out recomp targets based on priority Date: Thu, 27 Feb 2025 13:35:27 +0900 Message-ID: <20250227043618.88380-10-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: wuggb9dmyx74ykbrrhgcngytttqzdgjx X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 81DA2140002 X-HE-Tag: 1740631041-471164 X-HE-Meta: U2FsdGVkX1+ESkcCfaXTIAwU5xXLRG9aMrd9QKvYmMEWNXoGbkHv6xBXL95xmOX+WzVi49yVWc/+lBKRUzkfa3wn22QVVlOChCzhKX0pNG0tEUq8ZrAclxiW17r8BykpwiIuClCiBQDGtOLkBMqycZqIFp8l091lC9RVz0zOObnOUllSYmnQLqXCuYCACVZlvu5xz4VjE03yN8VePqOv1k2dmKiNhku1HBbSqzfbK91nC4ZinPvlgcsTr0XtHLFh4atsOFWuNtMqe0c/qb/8+/DSifH7f/c/C3CM8k99BiZMLeC8IBHwGHOOu1BxbNU3Znn+a8XdwiDtE86/1OVguAlohBEXEJprM6ko8OhSzZyIGzu/E9p4nbSE4r29AGiDrU2VUFZ/V7Am5EPEdZm3RQ208E2Ek6K0DCYrlisAycI/WQ5CPX4haLDY+Q4h5L2Ma0xm5L6Bd80xzgOQhgEJjQGUumeCEUp2wow8OViq4N2VkHIdcsj0/9jZtfcl1vBxS0r6Lxt3Aifmit7hgXIT2Q8qJqiNTAz/Fj8AGpUg1jgB6kW+cJmv84pDj/fDL5+eDxUyO/uUsN/JtKrkpStAGLqxR8E1dxstyTqM29w+PG1js0FVCjZSIUkQ3/OsUXQbK8n+EvjjNml5m2etMG52zyhOUJH/Z0N654WrOt1vOgQZgHq6J58maTxPqfLQhxwUdA/ztFzIpbj1/8WyAkYwTlbCAtgpBJbYm2ufW9mvsR38oxlEQ/M/OcqMNxdHVtno1lxxvEXwLsh2XlhunkngG6dPbQemz5FHfGredUCkwIKGyVvK0KkKG89/863/+SUaX8qy0a8Uf6jdBBQFp1YXPwCcz2kt2ERK+zykPoH5RBqN7o0oGVAEpaLbgJ263u19u1vjjdDrFC1S92ePzqHtsr+3hty2Ef1aPldFJJNU++rY/GXz3WssvrMQOuCXKI0rP+dUetSKCVs+zX7Piub pSFEmtNY 8i9UAcincEXQJ8J4GYsiwqvBt7S0p6+iGycWK7c8sTIg2D01/ibyIUdg6zslY3zWloyOFe0+wxAAZf2u90UfdqCV7CrjtAXobLgZxYhKbo33vIHOurgR1lnA3h084h2vaMxEExfpCM4CwA6S19rjPm96S73L/wRFDMk2AWVbSVs6MkoJObAReBa87DUDE6nw1642iaK8dbfqG4doAdmnSZNclffbAztS3VVWNujrjY28qhhd1mNr9yext+L8UT9Y9e7Y97Lex9qY2vFqDDOonJZTz8gNUXaYEwlJZiVpXiBX3S52X3xsHDW5lrg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Do no select for post processing slots that are already compressed with same or higher priority compression algorithm. This should save some memory, as previously we would still put those entries into corresponding post-processing buckets and filter them out later in recompress_slot(). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 12b99a557667..3a23dfed9542 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1823,7 +1823,7 @@ static int zram_bvec_write(struct zram *zram, struct bio_vec *bvec, #define RECOMPRESS_IDLE (1 << 0) #define RECOMPRESS_HUGE (1 << 1) -static int scan_slots_for_recompress(struct zram *zram, u32 mode, +static int scan_slots_for_recompress(struct zram *zram, u32 mode, u32 prio_max, struct zram_pp_ctl *ctl) { unsigned long nr_pages = zram->disksize >> PAGE_SHIFT; @@ -1855,6 +1855,10 @@ static int scan_slots_for_recompress(struct zram *zram, u32 mode, zram_test_flag(zram, index, ZRAM_INCOMPRESSIBLE)) goto next; + /* Already compressed with same of higher priority */ + if (zram_get_priority(zram, index) + 1 >= prio_max) + goto next; + pps->index = index; place_pp_slot(zram, ctl, pps); pps = NULL; @@ -1911,6 +1915,16 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, zram_clear_flag(zram, index, ZRAM_IDLE); class_index_old = zs_lookup_class_index(zram->mem_pool, comp_len_old); + + prio = max(prio, zram_get_priority(zram, index) + 1); + /* + * Recompression slots scan should not select slots that are + * already compressed with a higher priority algorithm, but + * just in case + */ + if (prio >= prio_max) + return 0; + /* * Iterate the secondary comp algorithms list (in order of priority) * and try to recompress the page. @@ -1919,13 +1933,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (!zram->comps[prio]) continue; - /* - * Skip if the object is already re-compressed with a higher - * priority algorithm (or same algorithm). - */ - if (prio <= zram_get_priority(zram, index)) - continue; - num_recomps++; zstrm = zcomp_stream_get(zram->comps[prio]); src = kmap_local_page(page); @@ -2150,7 +2157,7 @@ static ssize_t recompress_store(struct device *dev, goto release_init_lock; } - scan_slots_for_recompress(zram, mode, ctl); + scan_slots_for_recompress(zram, mode, prio_max, ctl); ret = len; while ((pps = select_pp_slot(ctl))) { From patchwork Thu Feb 27 04:35:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993740 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1CDDDC197BF for ; Thu, 27 Feb 2025 04:37:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C8C028000C; Wed, 26 Feb 2025 23:37:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 97961280005; Wed, 26 Feb 2025 23:37:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8197C28000C; Wed, 26 Feb 2025 23:37:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 62635280005 for ; Wed, 26 Feb 2025 23:37:29 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id DD6D4801E6 for ; Thu, 27 Feb 2025 04:37:28 +0000 (UTC) X-FDA: 83164465776.17.8AF30C8 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by imf20.hostedemail.com (Postfix) with ESMTP id 0DE2B1C0010 for ; Thu, 27 Feb 2025 04:37:26 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Zy6Vjk4p; spf=pass (imf20.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.170 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631047; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=s6NLD2jL1GvsoHkQKXZoOwiQJj4KiHrs9duVpBtevTk=; b=h2SwHRgUxqwcpxsuZ5QB6fhV+9yb4l5w7FieGh9Qr0WYvHM0Z6NB8dwGUf6/353doHIW3U uOZCIKW1TKydJ7GbAC94WXOZclLgpTnhwYshz2LaZtF5JH5aHq4ApfaMsSuAkGUsjrXMR5 t2+KUeZj2gBPDVb4QTpHc3zoX62NS/8= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Zy6Vjk4p; spf=pass (imf20.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.170 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631047; a=rsa-sha256; cv=none; b=O/XC3xzMwwnGMktZJw+uYyKW2TEUIbFn3gocs2WnmmAY0JmubseNE27cyvUpQELglbQTH0 2iET5kNu4WNmjI4ajACYtwo1y8ENI7XDTFOvaZo1ZbECygtVu70chE7ITfWZXandABKO6v rY9pTqFin+z2LRG3xrMsabhLZOTib+s= Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2211acda7f6so9980775ad.3 for ; Wed, 26 Feb 2025 20:37:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631046; x=1741235846; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=s6NLD2jL1GvsoHkQKXZoOwiQJj4KiHrs9duVpBtevTk=; b=Zy6Vjk4piQJWnDa5p/2Hykr23qYV+WxALu9EoWHrwTBhJoWn25gvvUGxtQD9sI+JHW iDrierL6cIVd5BtWL9kYuDmXtaBjKzdYmXS9WE0NpaachiY2zpTxzjks4dNLvspXBko/ NVygkU/vgROAzfI2wbep420FShgZAsgCw9KUY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631046; x=1741235846; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=s6NLD2jL1GvsoHkQKXZoOwiQJj4KiHrs9duVpBtevTk=; b=Flqmo0dT/46cMWLs/YKj0AgNBaP/NJKMZlpzlz4yGpbiTId8XYXFiWCnMuNequJ/GT SCiPKbKTH3ejxwBfLObBld4RG1uDmN+mtAh/RE1E6JKAYXpkyXVBJAvYZHc5YihpaLV7 7l3JuCf/yAVbmZZiLlFNmPmsMSB2ro9zjz3o+t1oPjSrbGvEJE3VA80H8fQ46D2cL5kf r3ndFExGA5jcQkjRSTqJuSgrBPEJX7+4yrZlesp+L1EQIP8iNL3sZs7netB4xOFadOSE HyTGTkTOG95oPrfyE3MtFOvzj/8DsKNAO1pxvrqpjNrsDPWhzAxWlLHE2Z9pzAe7TfnS K3ew== X-Forwarded-Encrypted: i=1; AJvYcCWRu1XhskA50rSVuGKDzHUYl5ZNiiDmI9bQigKhw+29BC5C0c08DJ7d2OZm3JIuxf6tRtnv+KbuSw==@kvack.org X-Gm-Message-State: AOJu0YxRnnRjq8QDP190OSSo4AVcTnxC3T9WDr1/5ccBZ/9wK1TlQVWn MNGEPxDNBMG2VqNAEs4y3zJBte6nWwupx2bOS+dNvak7R5vSnBKhOTpFogCIGA== X-Gm-Gg: ASbGncs49v4K6gVIjDMEV5TkpHBB4NXJzTbRdo2N8uLRtM1Ip7TRhMdPy1AM4NnMvlK RsiWayyJls0J7nFMWRr0xEHGdiZPgsj6TGydKnr9VtYH9uQQwoY/hfVirLerk5nmrMhobis0Cv3 pAeuxAoaXFLrnBQoei57tAg08unN/29l9CqV14M3E4EHwC7Yle4AqZYexd0+x0JQeS+NXzwiqj/ W/PK2KnwXLl4p9YB/AdewpOYyEkxzmTVRhj3Bn5+KOvg+aC+f6ncWp/H8Z4wpRLLxqi37q9wPOn Z65Qfl5Xn6yZH3b52WKUqd1erQjC X-Google-Smtp-Source: AGHT+IE7XcAdc9vNrlMBc7Oz2yCA4ADvv2oX4w4TjS94Lh2tcodEasjQb7dK+y4SdRXQWBQQi/bf9w== X-Received: by 2002:a17:903:22cd:b0:216:2bd7:1c2f with SMTP id d9443c01a7336-2232009991bmr68248845ad.18.1740631045980; Wed, 26 Feb 2025 20:37:25 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-223501fa918sm5062645ad.69.2025.02.26.20.37.23 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:37:25 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 10/19] zram: rework recompression loop Date: Thu, 27 Feb 2025 13:35:28 +0900 Message-ID: <20250227043618.88380-11-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 0DE2B1C0010 X-Stat-Signature: crn8t8rgimgm16o7hga6fugzt44eyaec X-HE-Tag: 1740631046-491145 X-HE-Meta: U2FsdGVkX1/HLDW3oCryWQGPidkQYi1++832p5Wc9ufcBepmxqxCLoNkQoioAfa2kKTZwoGGroS/vK8s43ziRmInUFhcidRV/M3DxwJQ5luraESyI6lq+C+uKhWUlZzoHHSalZBbQPvNcXE/TSa9zOioyWbikuF9kh/BSc7cblzJWtaZg990V5HMQD/7oVyJBgcUwzTdZpCqYGBBFKT/c/MymVyNSViDGbsJdlOCxEGJt8U8srgP/rfHhmgjULmYnogsNZvv7G5qzJNqS2ckug1kpGnDGpfXHeCW+0/RnSVJKB6p9KBxlLfV91ioOBCq0/o/WcGPGUX6rbPq5TGSntfSZfmBYWwewk9lktHwd1bWsxV2r2Af7IGALnvmmzLv6IYJaIBgRzMBV+9VZeT9h33rKdsDbhqvLPCmLEl3+Ha+7euJSiXkwRx7YWxCmWM1D/iTil1ddRRB18c0Xf6ng9BqsrUoWMSApKk2eVWafUezZWwKLsI25q3/qOsuPMeZGWZQfIr+wzuK42w5GEYmyfocTdse91HOOuv5vctGRT+nfSDwRKwH6PnMkmadH3iwP0K5fGuTAi0nIzRxUN21N2CktN1hoHz/lbRGQnzIv3tOM6DY4OGRCMnaa0P3/a9MHWyUivUwtf/4Erzy7SqTlXC8hv8rRm18nc4S//Zn2OytSIhu4rU31D0qzyJf/pb46joemwI3+3bJ8soo2BLWt+7b7wfSiRAiCQdtplj6Krwi8Y7QvX32o/E7V/CeiRYjs+WkDblnNE55XUGPwUsLLTjx1GyUslXsRW/5sq3HW3/Y6Mi5tqjCiVLwaxx4s243zOsGXbnJ7OZwG4pNbUj1oTJY1NbATXwg+iiH2jw/18Jb5UO8Gjz0pjKBMUjUz9KDhgORM313UGBdGMupar6bAqeC9fSjWLKGb1u2/aHdv5tEmm/aI6CQqFGwVx11SVFGA2I9EoPomgenYLCXZiw fCiqNKAh 5h6XCNbWtn8b35gJpg+oCKOTzP/J8KVeFVOsanVJ5L3B3Rls00WIM2I1indc1omWnDD6vE4lVHiha9qrP8bS6U3Z9MRJv5518mg/6jM9vpOcL9UN6AwoLBuVQtrPvMgDmd8FWpOkpCKEA7nteT3ss1AEzs8pZEvLgbtoisyoJMigWL/MUPY7/woTnQTxWqsm3esA7uj2XkF/fj8dw9dWNTQA8hykLnsdWdqAj4sxE8C3UwltfewygYKCjuvXIPSxNS1gIW4+QGBRXPL5Qg1DsceLCGI8slWJb6jMhFwH7DhYHrYFX0gOjDZ6o2g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This reworks recompression loop handling: - set a rule that stream-put NULLs the stream pointer If the loop returns with a non-NULL stream then it's a successful recompression, otherwise the stream should always be NULL. - do not count the number of recompressions Mark object as incompressible as soon as the algorithm with the highest priority failed to compress that object. - count compression errors as resource usage Even if compression has failed, we still need to bump num_recomp_pages counter. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 54 +++++++++++++---------------------- 1 file changed, 20 insertions(+), 34 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 3a23dfed9542..6755ca90f737 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1888,9 +1888,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, unsigned int comp_len_new; unsigned int class_index_old; unsigned int class_index_new; - u32 num_recomps = 0; void *src, *dst; - int ret; + int ret = 0; handle_old = zram_get_handle(zram, index); if (!handle_old) @@ -1933,7 +1932,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (!zram->comps[prio]) continue; - num_recomps++; zstrm = zcomp_stream_get(zram->comps[prio]); src = kmap_local_page(page); ret = zcomp_compress(zram->comps[prio], zstrm, @@ -1942,7 +1940,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (ret) { zcomp_stream_put(zstrm); - return ret; + zstrm = NULL; + break; } class_index_new = zs_lookup_class_index(zram->mem_pool, @@ -1952,6 +1951,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (class_index_new >= class_index_old || (threshold && comp_len_new >= threshold)) { zcomp_stream_put(zstrm); + zstrm = NULL; continue; } @@ -1959,14 +1959,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, break; } - /* - * We did not try to recompress, e.g. when we have only one - * secondary algorithm and the page is already recompressed - * using that algorithm - */ - if (!zstrm) - return 0; - /* * Decrement the limit (if set) on pages we can recompress, even * when current recompression was unsuccessful or did not compress @@ -1976,38 +1968,32 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (*num_recomp_pages) *num_recomp_pages -= 1; - if (class_index_new >= class_index_old) { + /* Compression error */ + if (ret) + return ret; + + if (!zstrm) { /* * Secondary algorithms failed to re-compress the page - * in a way that would save memory, mark the object as - * incompressible so that we will not try to compress - * it again. + * in a way that would save memory. * - * We need to make sure that all secondary algorithms have - * failed, so we test if the number of recompressions matches - * the number of active secondary algorithms. + * Mark the object incompressible if the max-priority + * algorithm couldn't re-compress it. */ - if (num_recomps == zram->num_active_comps - 1) - zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); + if (prio < zram->num_active_comps) + return 0; + zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); return 0; } - /* Successful recompression but above threshold */ - if (threshold && comp_len_new >= threshold) - return 0; - /* - * No direct reclaim (slow path) for handle allocation and no - * re-compression attempt (unlike in zram_write_bvec()) since - * we already have stored that object in zsmalloc. If we cannot - * alloc memory for recompressed object then we bail out and - * simply keep the old (existing) object in zsmalloc. + * We are holding per-CPU stream mutex and entry lock so better + * avoid direct reclaim. Allocation error is not fatal since + * we still have the old object in the mem_pool. */ handle_new = zs_malloc(zram->mem_pool, comp_len_new, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + GFP_NOIO | __GFP_NOWARN | + __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { zcomp_stream_put(zstrm); return PTR_ERR((void *)handle_new); From patchwork Thu Feb 27 04:35:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78766C197BF for ; Thu, 27 Feb 2025 04:37:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0085F28000D; Wed, 26 Feb 2025 23:37:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EA935280005; Wed, 26 Feb 2025 23:37:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D231C28000D; Wed, 26 Feb 2025 23:37:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id AC670280005 for ; Wed, 26 Feb 2025 23:37:34 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 61B0C80204 for ; Thu, 27 Feb 2025 04:37:34 +0000 (UTC) X-FDA: 83164466028.25.03E2328 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by imf16.hostedemail.com (Postfix) with ESMTP id 8C0C818000A for ; Thu, 27 Feb 2025 04:37:32 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=APJoduKT; spf=pass (imf16.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.46 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631052; a=rsa-sha256; cv=none; b=rKnAzGQcX+ZxjSsBkiCyQhRk+mFGEDusf4WHvA4oXR6rwnUcFSMDad0HGbalhkjaC81TI7 2gUPl0auJjUtLJmQiqwA05+7uX+bh2Dc1WXBEwXFgc2Li680XyWHqAuOgQSywJLoNZ6u+Y vx08p4MHVJzd1RAJpHP+28Eld4tkjKw= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=APJoduKT; spf=pass (imf16.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.46 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631052; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=atSj686udXXb8naSP/ilX+V3ED7QvGuQ33PoSFmI78o=; b=gLLkeRrYcV95OiS/n7V04T7GfMVkwMnKUQF7e0tT44hz/MY88er1miK0MSnEt2QJ1sh/CW PsoBrfv4DHYUkpQOywuxmQOoip3B/Tx0/ujMI4m/KdR7AxODjL+mQ9yhfhl1w6EWpeZ86l q1Tf+B9yZpDj/PVipaM/C18d6gXE3M4= Received: by mail-pj1-f46.google.com with SMTP id 98e67ed59e1d1-2fea795bafeso257178a91.1 for ; Wed, 26 Feb 2025 20:37:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631051; x=1741235851; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=atSj686udXXb8naSP/ilX+V3ED7QvGuQ33PoSFmI78o=; b=APJoduKTTNHHWFMpH8IM8ppjazeSI3UVeaEZzSWhaYY1y9B4MWUXIyGW8838a9lsjv dcpoxB31Waipd7W1ufp4MY7HVlhuPcTI6eR0rcfLjwQRlFWFCJZiJ344F22+rS1c86b8 Sv2hZe+iyDAwifXegCVPX4l6AWvFIq3+eCRvU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631051; x=1741235851; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=atSj686udXXb8naSP/ilX+V3ED7QvGuQ33PoSFmI78o=; b=CsBHpjwZoMKlIlPhi7F+iz0x1I9qPLEfNXmX4bkUyxq7otcFSlgx5c5ZX6rR4GV9NR 6+Bd2B7WSCK8EzuCVFG2IdD7gfdJmUZzpEkGhgGfuZUuC80QVjMiA0/BKTEdZ0t+2bSo agUr0fuGaAa3swlglKTimsaW1P2zzAlMqzGK8G+WtctuPWYNTTRghLB/O1XIllCpd9pY Gk+wD0m4wM4nOBpNqXzL1X+G6ZUY7Yi7S9n5N4Q362jI9pyemx0UxYNpJlE/ufIGKFi8 +rTZ80yNKA8YkUBILBULSf2N5qAp7hWBKTE0uPeIkF7R7E+sSikRDraJax0TNOYN5Sfj JTyg== X-Forwarded-Encrypted: i=1; AJvYcCXIB80o0VJNyYrQPENBeMgtSp37xcrNZwoATpAnA0wIOwWLcT4xOSLV7ibIOoqA298sRPCEEe3eWQ==@kvack.org X-Gm-Message-State: AOJu0Yz1tbqJAjklNWiDE+sVWJ11fu4O14fElgKYGolciezereYQ4inH dcT9omliBKHtWXI27wR6HjDG5zq7wbBwVMyCRdjjcQOYlqmcX5yZnBzKA56M5Q== X-Gm-Gg: ASbGncvv6N7aMHjGs57Kjj+uT2Bj76fDhtZziW8wXFLHQyuHiEHbGNA66CnOcUn74R1 4M/z97MXM3xc5c/fkO1G0rVe43xsJ1yf+pU8oh2WA0ayrBoHnlrSk7hClJrvEbAE8xUNeGPzIiF iclld2/qUIa3ijk9Me1Kwa7yhgYVPo8AlA59bBhyRuA6aD9DZhLH7Q461poWOIuQCb6BMJtIuDZ 0EehhnEFEh1QI4RNv/q3lqh+QoJts3OQ0cqKYbxS3OudiFFDJ8zRdrW+B9pv1gQY7/b149oPI1m 8HkFiI4Pln0v8i1/vmiiMyUjJBlT X-Google-Smtp-Source: AGHT+IEJkvjK9xJSO0CePo42cX0OoaGvyhAPmRkoUPST+KqNGlEPU6xv/EJnWdmaTpIOPBA38FE11Q== X-Received: by 2002:a05:6a21:1515:b0:1ee:c8a4:c329 with SMTP id adf61e73a8af0-1f2e36efeefmr3287648637.0.1740631051528; Wed, 26 Feb 2025 20:37:31 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id 41be03b00d2f7-aee7ddf246bsm411855a12.2.2025.02.26.20.37.29 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:37:31 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 11/19] zram: move post-processing target allocation Date: Thu, 27 Feb 2025 13:35:29 +0900 Message-ID: <20250227043618.88380-12-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8C0C818000A X-Stat-Signature: zoy987p7mjmji8nyfbcwhui6y4fahynk X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1740631052-461336 X-HE-Meta: U2FsdGVkX1+5yMf8PgrqMFRwMd1H9KTfkP/5OqND9+nx6Kw/9AUVABHE3skbuP4xifeSmJWFjLqO26NTm+mml5yQw9OLpOqW1mwBsDnTWx5Z9IjvjN+94EcHdWeWrID7b0U5QVToPKvwIOUg1pVNXGKKVo1tSJeARdDPF9s5YdZYXn4uTn9XhZG7INTlL3cBOBqUTZc8MlRV8WEnLZ1ujryG4SBs6RWz7b9ce27wu3BBtnVaRFu6xYAQtU6aFNGUICGEcF0mRi0qrNPIgrhiQnYpu0U9Ro9KQ3yOR84M1UUYg2MVIZeGjVMlsHP3RIv75ksqxOzxePZr5wJzvaIn3biUm8nmeXDcRDTWws6OaPVRylKRaPbVd4Mbs8EHsezNaQRQKBquE4XfPZ9JZuXqTxSUBjV96LxkJg0w9KBLbdocucaRKIAj0XenUMGy3RHR+MUNLlvR4LEpJZX+pD1/VbnW1CdobORnkgxh/busf0hWKZ7tUl9qIzqb1G/QfP4AORGmqV8yIO+by80BeKnD/HM9Uatv0GzaxUo/4XdEzrdZE4GXQbVkA3WbVDlQVGrOCaPUAnYupFceyHi5SwnZOZ7nuQQz/6sYDqsRtvEvVaw5R+dEPAz8awZAtIu1uwH58YZNPGrOd+nNSDrOYmUjMCGRzPxrihre6ANXqWJlGCS4LUVFZb70XFB+rrmdz4MOyq2kEVDcCRMmsNjl8fMdwKiDnBB4tft7KuhbrdiJwwevVkWtRF4vLS4ET2LcWb+LHb4D2CWlHm5/eNK2DodCTxmkKHu7nJqRCK984mIZc+d4kEJwLu5ADrYf9FKGjpeBBtCZ8vgAV99UdFYntfynwPa6rk+ybxizC9VicKTj+fPfQW+GF3wWEHZbgFJYfoBE/5sX7InmmsPxaQPybuXj6bs+C6vawowf24hJbrO9sUON0JpPji8yL4FDb0L8p0fRSb6FwAkWJ/SHFg4jdv0 nK1MrffX Md2ly9rhw1VzR38NpYIP8nB/Fr9TExGVEPanYboQqzNtaRa4qtm6a5F2VRyH90z5EebuakkCNiSYRO111o9/11cnb/TA5oizT1kzpsozmDvpr4Mg7V8dUx7ZRz02rRmr9nLGiBv/FCRvOyKi9AGiolUN+pR7vAHX1M7NQ0uKZ0RLef3f0qPq3do9wtA+9ad8mK9VNQz4WXzTheB+zr1jf9Qunb61beX7nBDFgBNbrETF/I2a3qTilzpmK6rowfGECIkeLHHNny6sNJW2BkxwY74vSkjqk+bspSKu7w5huMD+ulpafJFpYSZdGRtA8qZeVLdY6OU0tdP4H8twcjklr2NQrZojZjpXlZplb X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Allocate post-processing target in place_pp_slot(). This simplifies scan_slots_for_writeback() and scan_slots_for_recompress() loops because we don't need to track pps pointer state anymore. Previously we have to explicitly NULL the point if it has been added to a post-processing bucket or re-use previously allocated pointer otherwise and make sure we don't leak the memory in the end. We are also fine doing GFP_NOIO allocation, as post-processing can be called under memory pressure so we better pick as many slots as we can as soon as we can and start post-processing them, possibly saving the memory. Allocation failure there is not fatal, we will post-process whatever we put into the buckets on previous iterations. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 50 +++++++++++++++-------------------- 1 file changed, 22 insertions(+), 28 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 6755ca90f737..7ad89df9a304 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -295,15 +295,24 @@ static void release_pp_ctl(struct zram *zram, struct zram_pp_ctl *ctl) kfree(ctl); } -static void place_pp_slot(struct zram *zram, struct zram_pp_ctl *ctl, - struct zram_pp_slot *pps) +static bool place_pp_slot(struct zram *zram, struct zram_pp_ctl *ctl, + u32 index) { - u32 idx; + struct zram_pp_slot *pps; + u32 bid; + + pps = kmalloc(sizeof(*pps), GFP_NOIO | __GFP_NOWARN); + if (!pps) + return false; - idx = zram_get_obj_size(zram, pps->index) / PP_BUCKET_SIZE_RANGE; - list_add(&pps->entry, &ctl->pp_buckets[idx]); + INIT_LIST_HEAD(&pps->entry); + pps->index = index; + + bid = zram_get_obj_size(zram, pps->index) / PP_BUCKET_SIZE_RANGE; + list_add(&pps->entry, &ctl->pp_buckets[bid]); zram_set_flag(zram, pps->index, ZRAM_PP_SLOT); + return true; } static struct zram_pp_slot *select_pp_slot(struct zram_pp_ctl *ctl) @@ -737,15 +746,8 @@ static int scan_slots_for_writeback(struct zram *zram, u32 mode, unsigned long index, struct zram_pp_ctl *ctl) { - struct zram_pp_slot *pps = NULL; - for (; nr_pages != 0; index++, nr_pages--) { - if (!pps) - pps = kmalloc(sizeof(*pps), GFP_KERNEL); - if (!pps) - return -ENOMEM; - - INIT_LIST_HEAD(&pps->entry); + bool ok = true; zram_slot_lock(zram, index); if (!zram_allocated(zram, index)) @@ -765,14 +767,13 @@ static int scan_slots_for_writeback(struct zram *zram, u32 mode, !zram_test_flag(zram, index, ZRAM_INCOMPRESSIBLE)) goto next; - pps->index = index; - place_pp_slot(zram, ctl, pps); - pps = NULL; + ok = place_pp_slot(zram, ctl, index); next: zram_slot_unlock(zram, index); + if (!ok) + break; } - kfree(pps); return 0; } @@ -1827,16 +1828,10 @@ static int scan_slots_for_recompress(struct zram *zram, u32 mode, u32 prio_max, struct zram_pp_ctl *ctl) { unsigned long nr_pages = zram->disksize >> PAGE_SHIFT; - struct zram_pp_slot *pps = NULL; unsigned long index; for (index = 0; index < nr_pages; index++) { - if (!pps) - pps = kmalloc(sizeof(*pps), GFP_KERNEL); - if (!pps) - return -ENOMEM; - - INIT_LIST_HEAD(&pps->entry); + bool ok = true; zram_slot_lock(zram, index); if (!zram_allocated(zram, index)) @@ -1859,14 +1854,13 @@ static int scan_slots_for_recompress(struct zram *zram, u32 mode, u32 prio_max, if (zram_get_priority(zram, index) + 1 >= prio_max) goto next; - pps->index = index; - place_pp_slot(zram, ctl, pps); - pps = NULL; + ok = place_pp_slot(zram, ctl, index); next: zram_slot_unlock(zram, index); + if (!ok) + break; } - kfree(pps); return 0; } From patchwork Thu Feb 27 04:35:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993742 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47FBCC197BF for ; Thu, 27 Feb 2025 04:37:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CD6A428000E; Wed, 26 Feb 2025 23:37:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C5E30280005; Wed, 26 Feb 2025 23:37:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AD8E228000E; Wed, 26 Feb 2025 23:37:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 8D37C280005 for ; Wed, 26 Feb 2025 23:37:40 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 58D6512027C for ; Thu, 27 Feb 2025 04:37:40 +0000 (UTC) X-FDA: 83164466280.21.D09CD07 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf26.hostedemail.com (Postfix) with ESMTP id 7D439140003 for ; Thu, 27 Feb 2025 04:37:38 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Od9Byk5t; spf=pass (imf26.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631058; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZEqWbULFJyDHZ0uxeHUHEx9cW/XUYWqE9opCIKYGE7k=; b=UJfVjXoaL61KunOIfyCUCmQHTPezGT8CkZ9XQCGo/iPkDXeFaclzxl56VfnIMi6uX+icFG dwmxa9lOq6x/gaZGJcXRA94v40UdoaYxypEbpSIBDur7PPdzEVABRrkK+wFcuWABEAlU/a VB0yPLljAtPxWcc7K1jiXQXadiUNuNc= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Od9Byk5t; spf=pass (imf26.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631058; a=rsa-sha256; cv=none; b=waClyY4HV2JwnWhGX2O5EnjtIFagM+i5Gov8zH296pJbt4iujbNwQQRdjzqlVOMOvBv0H3 FALxEHnyt3FlTi9DpCI6/0n2RTI23yeI96nP6kzlpAUvK0z2IsZl41+Tk2lrp2SbmXIr2K mmTP4HTQ4g4phjl99PQqtodSfDqB7VM= Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-22128b7d587so8489835ad.3 for ; Wed, 26 Feb 2025 20:37:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631057; x=1741235857; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZEqWbULFJyDHZ0uxeHUHEx9cW/XUYWqE9opCIKYGE7k=; b=Od9Byk5tMoNJUSF9ny26N6rmRxJcLJQbcnMVaAhJBTGhNUO5SqZYpYbcp6vv9Behf/ Suss62ce/eWTg8RIloAtrOoenX8zciwIVrVKbqZ2vKDvEzImVf7sb9luMj6yXFW19P3H 3U4keyzxWmier2D+PhBxZ38EinEwAuU+7EzB8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631057; x=1741235857; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZEqWbULFJyDHZ0uxeHUHEx9cW/XUYWqE9opCIKYGE7k=; b=oR1BxoWkxNa0My9x8IePHaq8xr3oyrHzkFpoXO0pPmPhQbRGUqdjHGgC/eaOv87A+O 5Hv4juPfQjeySmuRWY2niGfLwlTe171rHfFd8caxfeUgACI6dLStKxxGBcN0I0EVXURP 0sxPF5wwFaCcMOOTM1B4PA+Al+42cocKNBN/BJs2bTHWw7+JEzAaI2T/xSe8kbGvLdze e06vb4KGcAG6KmYwWeDDPD6Qo8fZqdQrOx1RIazmR6+uZmsaRvIKD2XgTqWVteSQf/tv +85OOu3a34fuhkWsJAzn1sRBmaReD7EOtTW7eUW5lRkLdB8t+VOCaFDRxoBWJSrF7cE0 /Bag== X-Forwarded-Encrypted: i=1; AJvYcCXw2CM7oGQzKPd9/pYpC7iYE6WlNLtIJfdSKHQBI02OJZjlDVn9jJA7JUgMVBhAtDba++CJZAlEVg==@kvack.org X-Gm-Message-State: AOJu0YyHARe9CVllZLn+PlNGuaI+pkXxEYipIFBn6tkrJUewLCUl6nEg 7hhSMfnHtIn0XqR6JfzcRR/+SatOEb3OV6h0fca5eWoLdvk53fLXNevzTRe0eA== X-Gm-Gg: ASbGnctho0G2c27+UnOOCaEIf3xf6Y0LUemOrGF+ciIIaVLvpoYXTzcRpgJ13OwMPC5 HsDyRWlFnFDKgt5xq0VSxxLGLxqd8YmxVYPbDBCF66wxShnngZDcj5DTBHsIq0GBGRicDBGTjnc yTTHBXEijPAA3PrCAzqiobDXr3qfSOu5Oor+S0JBX3FwgnRZ1HclbFboXKUqWVQNERCrWt/8o8T XWc2iV3e4dBMQivdYBJC8V2+5qrGc63GjFfSfZpI0LowqwtIzX6wv30bOATvGPWxdsQ8BhkYDNo 0cQ9EV+EwRspPVDu+GrkQBfExGHu X-Google-Smtp-Source: AGHT+IGjRG1+LdYUr60qs0ikZWZDjzWHfWUJZUn2QU7B7wuT5jMjcjNdqY/z652mUps+J+R2kLTanA== X-Received: by 2002:a05:6a21:48d:b0:1f0:e7e2:127e with SMTP id adf61e73a8af0-1f10ad3ffabmr9855132637.23.1740631057375; Wed, 26 Feb 2025 20:37:37 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-7349fe6c1desm500488b3a.73.2025.02.26.20.37.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:37:37 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 12/19] zsmalloc: rename pool lock Date: Thu, 27 Feb 2025 13:35:30 +0900 Message-ID: <20250227043618.88380-13-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7D439140003 X-Stat-Signature: jnsu3rb85g84ii8s9uj9f6zf7pjm77s6 X-Rspam-User: X-HE-Tag: 1740631058-117609 X-HE-Meta: U2FsdGVkX18FXvOfaQu0sLLf5LVCwEBFWPdxXSOxgi2kw5Q5PmTALCx052Dizk0JYsjJ/ON3kjkufBi5W/JroWTIpdY8hz0uuaHiVL4sSrxMRm1Eq4mFMjo7usny2m7GH/BbKGjWwvyjFprmISyS2BJb1udMDh/a6J3vHqwy9MCCZxoFWnJh+pwB0xJNImff3zWJMrBv9BCyHPBJW+h0RoGBLKVOT/LpcjQ0tj2Y87C1dP8q0B9oLlN/GdeTQCccfBYdy/CiT3N6JopYWW5gCtzxKPQu8NEpnpv/hWDaPKyWADMXV4RO3WoywxWA6v5oY3ANQjmgGbHgFsU0mp6sw+lMqSWXwfnlVlumu6/3vmrbO6AR4Ms87wEf74xBLFkLRHANH0KCwXA4WPdX5brZf/mBJcwvSTDrsyRyhX7dVMc8XR1MOEIlThhOLrtxq3IGrkuUSWnS9GMX8NKwI2bI3RqU4pyuxc/Ghl2PDPGCCWV9PqpyIJbl/j0ZmFuKURJmUAllnqKec5UEbcyPQ6u9mkF9Zg8mDOqiVa2Je7DZ8GvRWj8AlHWCUeZj+/ElW3MWc9IH37QCbCPoeJMk3ufaYazv3qszobxcEFel7byCz/Oqq3hIQNE/pgl0zijmHiLLQupByogxyWxLN1IKV0KK4V+530U0WJHqjteyeEjP7PCWB1YCF2+VtcR0pAgvCQns7+46KerwDpSV/Ua7ugoZjYy/Ua/sFzzyQ01Y/o+258YR+iV6FFhSWQWCqfhS7C6TFAfThWN7Xq29XIWuq7oXy/w5MhhT9ylDCfGzXxaMKkWGRmGPLz5ZuR7geitDtcDkZdVsW1dJtmP+nE6ZHekGotHFeTwwILYmpTi63VIjjM9mFLyLefWkTFZ00lMBLZQ2CemOPnUXlehytQVWwtsJYt9/SrauSZyjIhJmYLnCDdGJWbwFgPatzTQikiX4AdVR9TxAgPuUyvnQPV6J0mR KXgcUDNN FVRQ57m5WZ9UuruZxNf9yItqQBPBF50fHkJlAlW8bnqkf671XVbATbBX2r22wRDTQC5+lDLLzE41ITOrgPy2zMIWexA+jb1+ThR6W+qNu9Wnp/6knZLwxfHYil+CYJgPixly+H4tZNdH4D/f/K/AIJME5Q7uYVuclo2SbYXSioaPdIQrKxCvHTRyCuXv9xJQIU+1oKU1kRLkgIFYJ4OoDhsS2kgxBuwEI6Il34eS1YdgJObsFjUux4l1hbDtjpPGWKupAK/j7MFAhhHQPlgZFk/49PHYW3EocifqjwG40Xp1kP9GoNpImxoS04A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The old name comes from the times when the pool did not have compaction (defragmentation). Rename it to ->lock because these days it synchronizes not only migration. Reviewed-by: Yosry Ahmed Signed-off-by: Sergey Senozhatsky --- mm/zsmalloc.c | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 817626a351f8..1424ee73cbb5 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -18,7 +18,7 @@ /* * lock ordering: * page_lock - * pool->migrate_lock + * pool->lock * class->lock * zspage->lock */ @@ -223,8 +223,8 @@ struct zs_pool { #ifdef CONFIG_COMPACTION struct work_struct free_work; #endif - /* protect page/zspage migration */ - rwlock_t migrate_lock; + /* protect zspage migration/compaction */ + rwlock_t lock; atomic_t compaction_in_progress; }; @@ -1206,7 +1206,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, BUG_ON(in_interrupt()); /* It guarantees it can get zspage from handle safely */ - read_lock(&pool->migrate_lock); + read_lock(&pool->lock); obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); zspage = get_zspage(zpdesc); @@ -1218,7 +1218,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, * which is smaller granularity. */ migrate_read_lock(zspage); - read_unlock(&pool->migrate_lock); + read_unlock(&pool->lock); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1450,16 +1450,16 @@ void zs_free(struct zs_pool *pool, unsigned long handle) return; /* - * The pool->migrate_lock protects the race with zpage's migration + * The pool->lock protects the race with zpage's migration * so it's safe to get the page from handle. */ - read_lock(&pool->migrate_lock); + read_lock(&pool->lock); obj = handle_to_obj(handle); obj_to_zpdesc(obj, &f_zpdesc); zspage = get_zspage(f_zpdesc); class = zspage_class(pool, zspage); spin_lock(&class->lock); - read_unlock(&pool->migrate_lock); + read_unlock(&pool->lock); class_stat_sub(class, ZS_OBJS_INUSE, 1); obj_free(class->size, obj); @@ -1796,7 +1796,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, * The pool migrate_lock protects the race between zpage migration * and zs_free. */ - write_lock(&pool->migrate_lock); + write_lock(&pool->lock); class = zspage_class(pool, zspage); /* @@ -1833,7 +1833,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. */ - write_unlock(&pool->migrate_lock); + write_unlock(&pool->lock); spin_unlock(&class->lock); migrate_write_unlock(zspage); @@ -1956,7 +1956,7 @@ static unsigned long __zs_compact(struct zs_pool *pool, * protect the race between zpage migration and zs_free * as well as zpage allocation/free */ - write_lock(&pool->migrate_lock); + write_lock(&pool->lock); spin_lock(&class->lock); while (zs_can_compact(class)) { int fg; @@ -1983,14 +1983,14 @@ static unsigned long __zs_compact(struct zs_pool *pool, src_zspage = NULL; if (get_fullness_group(class, dst_zspage) == ZS_INUSE_RATIO_100 - || rwlock_is_contended(&pool->migrate_lock)) { + || rwlock_is_contended(&pool->lock)) { putback_zspage(class, dst_zspage); dst_zspage = NULL; spin_unlock(&class->lock); - write_unlock(&pool->migrate_lock); + write_unlock(&pool->lock); cond_resched(); - write_lock(&pool->migrate_lock); + write_lock(&pool->lock); spin_lock(&class->lock); } } @@ -2002,7 +2002,7 @@ static unsigned long __zs_compact(struct zs_pool *pool, putback_zspage(class, dst_zspage); spin_unlock(&class->lock); - write_unlock(&pool->migrate_lock); + write_unlock(&pool->lock); return pages_freed; } @@ -2014,10 +2014,10 @@ unsigned long zs_compact(struct zs_pool *pool) unsigned long pages_freed = 0; /* - * Pool compaction is performed under pool->migrate_lock so it is basically + * Pool compaction is performed under pool->lock so it is basically * single-threaded. Having more than one thread in __zs_compact() - * will increase pool->migrate_lock contention, which will impact other - * zsmalloc operations that need pool->migrate_lock. + * will increase pool->lock contention, which will impact other + * zsmalloc operations that need pool->lock. */ if (atomic_xchg(&pool->compaction_in_progress, 1)) return 0; @@ -2139,7 +2139,7 @@ struct zs_pool *zs_create_pool(const char *name) return NULL; init_deferred_free(pool); - rwlock_init(&pool->migrate_lock); + rwlock_init(&pool->lock); atomic_set(&pool->compaction_in_progress, 0); pool->name = kstrdup(name, GFP_KERNEL); From patchwork Thu Feb 27 04:35:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993743 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 509A0C021BE for ; Thu, 27 Feb 2025 04:37:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DC16B28000F; Wed, 26 Feb 2025 23:37:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D7139280005; Wed, 26 Feb 2025 23:37:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BEA4528000F; Wed, 26 Feb 2025 23:37:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9EBF4280005 for ; Wed, 26 Feb 2025 23:37:46 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 5C01A140245 for ; Thu, 27 Feb 2025 04:37:46 +0000 (UTC) X-FDA: 83164466532.19.4BB8620 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf07.hostedemail.com (Postfix) with ESMTP id 7EADC40007 for ; Thu, 27 Feb 2025 04:37:44 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=G8mwJ3w5; spf=pass (imf07.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.169 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631064; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GwrDBXsyO2EvEhOdvagCAu0rolY3IippWT1+2LO84LM=; b=flfoMnFLiHYAuhrNazLZ+v4vTNFI2K1jUxT12ob9D1Jd1/wxlq3DUV04hMXbm5Y2TaRkmL rQqB183hkZ5cxfi3yhQfhqbNt2jDNW2USrsU3aPNj0xTkMesJMN0F93QrB6rNziswtx4E1 8byxatfGgsoQakcPMo42mDK/dD3r3EA= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=G8mwJ3w5; spf=pass (imf07.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.169 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631064; a=rsa-sha256; cv=none; b=x/SawPBARnpl6z8sBQcFW4TxX0gr1oLQP4a8jToVsgPuvWubawnfH9/7YewlVvAuDr4/8z MS9RQFyZPoy9M/tnhxTldf9echxk7gW3dB3YvqIsp/8KTK4IfP7tt0TVpaREddpCvmIY1O cyABJuZU1CTYNqfOa0FSfAr9EwqhPUI= Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-22104c4de96so7263555ad.3 for ; Wed, 26 Feb 2025 20:37:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631063; x=1741235863; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GwrDBXsyO2EvEhOdvagCAu0rolY3IippWT1+2LO84LM=; b=G8mwJ3w5Us239D3Z6oxhO/ypJ4yjOoMXA9COFap0kes3b3CqYO2FnEA14uiUBuLPzR pmeD6JRAALhmwBlglXwZfYM1sqoAPSGAHbW92TPrIz2l/lBgQZ7yoJFf/A8ZMajyCBMf 3Z+HAG+eBCBvdhng0sZ9BFriKLt9yy/qjhqTY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631063; x=1741235863; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GwrDBXsyO2EvEhOdvagCAu0rolY3IippWT1+2LO84LM=; b=MYPYSMjKCadrLgwfBeH70fX+kqGa0lZCXcbAlZC36AnjY0cdCS75bFnaBjVmtei4An GzfwyG8iR7HyUrmdEYKu/2OF9gmNx5f3v/40XehOWyYswBSiG8ZnouyA8WvmLrNK0Fez 7Qks6ROCIR2dcHofYtIcKh8ppTWDzTczYZB7Ukq23TPqFdq02r8VEW/oh88TXhF82rSu FdkQGMQjTtlMTkJ6Hu+k9cGTbzdPIdB4ErMd53JJX8RtVHZdt+sC34yZMJUagpD/RzDH YjTkkwsjeCo6zk4NSylpzAD4JIfs5pgeMV4RVZeP4VU5yersfeiGVAjT4964qImbIXfD hGOg== X-Forwarded-Encrypted: i=1; AJvYcCXvvrT6AdEr/Eg6yBYJkkSiSsR0QF+v0DbEj1jPQ5Q24H0NSgYIc6FJCn9qLk5JATBUlX4c7NKDGg==@kvack.org X-Gm-Message-State: AOJu0YwkZudMPgSz1eECL5Ch7WuJxzCd8NAdwOQEG4YOhoEvv2z+MOB+ 0YTKU3uOYoPpodFoerF56Axl1StkyGMN1NYXBRrVZKlcvV+97R9CqT6JbrnC/oXfRY61IfNnmD0 = X-Gm-Gg: ASbGncvOLp2yf2NlxuRANdZ9xPRJSgFY468e7dLF4LzWX5To8JEywCsTNdzh7eu5LKA avrSjbTdo5wlcV24Y9krRMVV0hYsrlCxD9cTSw+vvDmbdquMK6PrHnzrvosQIVlFW1LcR93YIwx TEzrqX/EtiD4ZQrptdcpt4RIkxzKxoKDP3y+Ad8Y88am2DFPWvUw9rCXgnvMzoNa19OUFSgVP7M 782cNWPNX6/4heY2LKLdwUr9OiveWlUrt0dvTwcb5trwDs1mKSe/ul2dFZ3TZlPFARgEj7C2xGa VYetqBym57i9mboqHljiuIkv9WJd X-Google-Smtp-Source: AGHT+IFrOSZSmfTMzWd5HqkTvrPS3qIgNLqZdoN+0693MnsV/OS8jz0BlediKk7QkapJlhesjLvlCw== X-Received: by 2002:aa7:84c8:0:b0:734:918a:4ecd with SMTP id d2e1a72fcca58-734918a5063mr4878646b3a.15.1740631063305; Wed, 26 Feb 2025 20:37:43 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-734a0024bd8sm501433b3a.91.2025.02.26.20.37.40 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:37:42 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 13/19] zsmalloc: make zspage lock preemptible Date: Thu, 27 Feb 2025 13:35:31 +0900 Message-ID: <20250227043618.88380-14-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7EADC40007 X-Stat-Signature: kznneycyywdrzpz3ua5pdnkhxrqnizff X-Rspam-User: X-HE-Tag: 1740631064-552108 X-HE-Meta: U2FsdGVkX19VICNQRogxmbGfU2BRmK4+fU6i5HHKRmMXkdcN4I6VKFEL91GUY60Rsy1v4eQRFqKsv/IYQZ8ehgPubzl4pA01CaEiDIvxmWOx8rc1m8nf7ABNkcwLM3Zz5JxEokcdh9MD66M1BRC5ksQupHGh1Z4bm8UyZdLYuNQ/qIqMn3UKjXqhf83Ad8SgDKKi2xE96lrwjEtEJ989aWiwbChogiCBvmY7X69+pM7QEOkYy2waJup/s0g+X61yo3W4lFA5sqOIE0Mu1RPZOp736cZYa7FD4rRjnFPEqqaWcCPfnOznHd3SyAbSRKeU4PRUkbJVFBpzd9AOdy33jHKPCIsGCeK7F7C4pQgqZ/uFXf3YDusnPmWyVdhHVxMEoU8YBhS8UR5KqXvFcl1QnlWF1LNfbE2gOQ0FhecZ7/xZXJpeeLkgnM1fM2COqgWgqsk9SEuM4/Frqyv3j1JJl0LRZKE4fe6QBKIiJVALGCm2UxQpb9g07bKajDaGoCBUho321cjfSYEDCb/nxxJ2aYXDYT5ZwVKOGWzseoR8TRtflodXiqHsp1agYwxBA4uOE3zdp+GmTicCkCyEh6NL/qiofgrJd15Vi8H6e9vMq8izBFd6oF3jlpblMD1qtInjKzOA0ykQGW3vkftzUQ3aY4GtGg0RfU0WqEvdOQv1KvnlAKTR68to4Cc7FPNqh19jrOmbMGx1dKwp99RMHvIjTGf8RV/qJhe/iVVty99mRQyFgA5SVsG8gSsrCOQz0Oml+upJl4QfBL4OgliSTQGN6PG8PPHLjViiU0xfsH4rZXwGcCjvlfU7rxk9/nKhOWmPXL8Bewc5RK90rQs3MVpvTtn/Zw3f8QmpXEYDk9WWL6B6KETw/juseJf0mbUU1pOxudEhl7xRmRIhvxJSVhW/F0ojMj6GM7sM9rt+mI0m9vO1h9wzWAxBIm4w2gNm8j9YC/KkB2Borj3yoSXdsYD 18FpDLXq eddUKt3L9xv5Ro2UogolZ1yTUqaOp04hebvd5iTkjb7PFfdRck73s+T+q5nIn/LhyZcd1OOPq4C4U24wGcnqmg/X9/BPyXxnCGjhCUjR6cv9RSHUPvVFbJu1KOX9JK2E81g7pNuDQbdSpcGSTj1bdJxLujTprqrR8A8TJRLcpGdQ92h5FtjK5dGpSx5HJd2KxVp2783HHWQsJT9PTnKgn6+XVbdI1XNCTXoDYCFsvady5+/sR8JXE+LjWRA/QSNnD8Wh791qfEkVpEAG7XrF1HZv6IW0BM1aRa/K94/jMshOsng/lid11bnKUrlzLzn/bTWmnTYnyjU2JRlo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In order to implement preemptible object mapping we need a zspage lock that satisfies several preconditions: - it should be reader-write type of a lock - it should be possible to hold it from any context, but also being preemptible if the context allows it - we never sleep while acquiring but can sleep while holding in read mode An rwsemaphore doesn't suffice, due to atomicity requirements, rwlock doesn't satisfy due to reader-preemptability requirement. It's also worth to mention, that per-zspage rwsem is a little too memory heavy (we can easily have double digits megabytes used only on rwsemaphores). Switch over from rwlock_t to a atomic_t-based implementation of a reader-writer semaphore that satisfies all of the preconditions. The spin-lock based zspage_lock is suggested by Hillf Danton. Suggested-by: Hillf Danton Signed-off-by: Sergey Senozhatsky --- mm/zsmalloc.c | 171 ++++++++++++++++++++++++++++++++++---------------- 1 file changed, 118 insertions(+), 53 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 1424ee73cbb5..74a7aaebf7a0 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -226,6 +226,7 @@ struct zs_pool { /* protect zspage migration/compaction */ rwlock_t lock; atomic_t compaction_in_progress; + struct lock_class_key lock_class; }; static inline void zpdesc_set_first(struct zpdesc *zpdesc) @@ -257,6 +258,15 @@ static inline void free_zpdesc(struct zpdesc *zpdesc) __free_page(page); } +#define ZS_PAGE_UNLOCKED 0 +#define ZS_PAGE_WRLOCKED -1 + +struct zspage_lock { + spinlock_t lock; + int cnt; + struct lockdep_map dep_map; +}; + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -269,7 +279,7 @@ struct zspage { struct zpdesc *first_zpdesc; struct list_head list; /* fullness list */ struct zs_pool *pool; - rwlock_t lock; + struct zspage_lock zsl; }; struct mapping_area { @@ -279,6 +289,85 @@ struct mapping_area { enum zs_mapmode vm_mm; /* mapping mode */ }; +static void zspage_lock_init(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + lockdep_init_map(&zsl->dep_map, "zspage->lock", + &zspage->pool->lock_class, 0); + spin_lock_init(&zsl->lock); + zsl->cnt = ZS_PAGE_UNLOCKED; +} + +/* + * The zspage lock can be held from atomic contexts, but it needs to remain + * preemptible when held for reading because it remains held outside of those + * atomic contexts, otherwise we unnecessarily lose preemptibility. + * + * To achieve this, the following rules are enforced on readers and writers: + * + * - Writers are blocked by both writers and readers, while readers are only + * blocked by writers (i.e. normal rwlock semantics). + * + * - Writers are always atomic (to allow readers to spin waiting for them). + * + * - Writers always use trylock (as the lock may be held be sleeping readers). + * + * - Readers may spin on the lock (as they can only wait for atomic writers). + * + * - Readers may sleep while holding the lock (as writes only use trylock). + */ +static void zspage_read_lock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + rwsem_acquire_read(&zsl->dep_map, 0, 0, _RET_IP_); + + spin_lock(&zsl->lock); + zsl->cnt++; + spin_unlock(&zsl->lock); + + lock_acquired(&zsl->dep_map, _RET_IP_); +} + +static void zspage_read_unlock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + rwsem_release(&zsl->dep_map, _RET_IP_); + + spin_lock(&zsl->lock); + zsl->cnt--; + spin_unlock(&zsl->lock); +} + +static __must_check bool zspage_write_trylock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + spin_lock(&zsl->lock); + if (zsl->cnt == ZS_PAGE_UNLOCKED) { + zsl->cnt = ZS_PAGE_WRLOCKED; + rwsem_acquire(&zsl->dep_map, 0, 1, _RET_IP_); + lock_acquired(&zsl->dep_map, _RET_IP_); + return true; + } + + lock_contended(&zsl->dep_map, _RET_IP_); + spin_unlock(&zsl->lock); + return false; +} + +static void zspage_write_unlock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + rwsem_release(&zsl->dep_map, _RET_IP_); + + zsl->cnt = ZS_PAGE_UNLOCKED; + spin_unlock(&zsl->lock); +} + /* huge object: pages_per_zspage == 1 && maxobj_per_zspage == 1 */ static void SetZsHugePage(struct zspage *zspage) { @@ -290,12 +379,6 @@ static bool ZsHugePage(struct zspage *zspage) return zspage->huge; } -static void migrate_lock_init(struct zspage *zspage); -static void migrate_read_lock(struct zspage *zspage); -static void migrate_read_unlock(struct zspage *zspage); -static void migrate_write_lock(struct zspage *zspage); -static void migrate_write_unlock(struct zspage *zspage); - #ifdef CONFIG_COMPACTION static void kick_deferred_free(struct zs_pool *pool); static void init_deferred_free(struct zs_pool *pool); @@ -992,7 +1075,9 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, return NULL; zspage->magic = ZSPAGE_MAGIC; - migrate_lock_init(zspage); + zspage->pool = pool; + zspage->class = class->index; + zspage_lock_init(zspage); for (i = 0; i < class->pages_per_zspage; i++) { struct zpdesc *zpdesc; @@ -1015,8 +1100,6 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, create_page_chain(class, zspage, zpdescs); init_zspage(class, zspage); - zspage->pool = pool; - zspage->class = class->index; return zspage; } @@ -1217,7 +1300,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, * zs_unmap_object API so delegate the locking from class to zspage * which is smaller granularity. */ - migrate_read_lock(zspage); + zspage_read_lock(zspage); read_unlock(&pool->lock); class = zspage_class(pool, zspage); @@ -1277,7 +1360,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) } local_unlock(&zs_map_area.lock); - migrate_read_unlock(zspage); + zspage_read_unlock(zspage); } EXPORT_SYMBOL_GPL(zs_unmap_object); @@ -1671,18 +1754,18 @@ static void lock_zspage(struct zspage *zspage) /* * Pages we haven't locked yet can be migrated off the list while we're * trying to lock them, so we need to be careful and only attempt to - * lock each page under migrate_read_lock(). Otherwise, the page we lock + * lock each page under zspage_read_lock(). Otherwise, the page we lock * may no longer belong to the zspage. This means that we may wait for * the wrong page to unlock, so we must take a reference to the page - * prior to waiting for it to unlock outside migrate_read_lock(). + * prior to waiting for it to unlock outside zspage_read_lock(). */ while (1) { - migrate_read_lock(zspage); + zspage_read_lock(zspage); zpdesc = get_first_zpdesc(zspage); if (zpdesc_trylock(zpdesc)) break; zpdesc_get(zpdesc); - migrate_read_unlock(zspage); + zspage_read_unlock(zspage); zpdesc_wait_locked(zpdesc); zpdesc_put(zpdesc); } @@ -1693,41 +1776,16 @@ static void lock_zspage(struct zspage *zspage) curr_zpdesc = zpdesc; } else { zpdesc_get(zpdesc); - migrate_read_unlock(zspage); + zspage_read_unlock(zspage); zpdesc_wait_locked(zpdesc); zpdesc_put(zpdesc); - migrate_read_lock(zspage); + zspage_read_lock(zspage); } } - migrate_read_unlock(zspage); + zspage_read_unlock(zspage); } #endif /* CONFIG_COMPACTION */ -static void migrate_lock_init(struct zspage *zspage) -{ - rwlock_init(&zspage->lock); -} - -static void migrate_read_lock(struct zspage *zspage) __acquires(&zspage->lock) -{ - read_lock(&zspage->lock); -} - -static void migrate_read_unlock(struct zspage *zspage) __releases(&zspage->lock) -{ - read_unlock(&zspage->lock); -} - -static void migrate_write_lock(struct zspage *zspage) -{ - write_lock(&zspage->lock); -} - -static void migrate_write_unlock(struct zspage *zspage) -{ - write_unlock(&zspage->lock); -} - #ifdef CONFIG_COMPACTION static const struct movable_operations zsmalloc_mops; @@ -1785,9 +1843,6 @@ static int zs_page_migrate(struct page *newpage, struct page *page, VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); - /* We're committed, tell the world that this is a Zsmalloc page. */ - __zpdesc_set_zsmalloc(newzpdesc); - /* The page is locked, so this pointer must remain valid */ zspage = get_zspage(zpdesc); pool = zspage->pool; @@ -1803,8 +1858,15 @@ static int zs_page_migrate(struct page *newpage, struct page *page, * the class lock protects zpage alloc/free in the zspage. */ spin_lock(&class->lock); - /* the migrate_write_lock protects zpage access via zs_map_object */ - migrate_write_lock(zspage); + /* the zspage write_lock protects zpage access via zs_map_object */ + if (!zspage_write_trylock(zspage)) { + spin_unlock(&class->lock); + write_unlock(&pool->lock); + return -EINVAL; + } + + /* We're committed, tell the world that this is a Zsmalloc page. */ + __zpdesc_set_zsmalloc(newzpdesc); offset = get_first_obj_offset(zpdesc); s_addr = kmap_local_zpdesc(zpdesc); @@ -1835,7 +1897,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, */ write_unlock(&pool->lock); spin_unlock(&class->lock); - migrate_write_unlock(zspage); + zspage_write_unlock(zspage); zpdesc_get(newzpdesc); if (zpdesc_zone(newzpdesc) != zpdesc_zone(zpdesc)) { @@ -1971,9 +2033,11 @@ static unsigned long __zs_compact(struct zs_pool *pool, if (!src_zspage) break; - migrate_write_lock(src_zspage); + if (!zspage_write_trylock(src_zspage)) + break; + migrate_zspage(pool, src_zspage, dst_zspage); - migrate_write_unlock(src_zspage); + zspage_write_unlock(src_zspage); fg = putback_zspage(class, src_zspage); if (fg == ZS_INUSE_RATIO_0) { @@ -2141,6 +2205,7 @@ struct zs_pool *zs_create_pool(const char *name) init_deferred_free(pool); rwlock_init(&pool->lock); atomic_set(&pool->compaction_in_progress, 0); + lockdep_register_key(&pool->lock_class); pool->name = kstrdup(name, GFP_KERNEL); if (!pool->name) @@ -2233,7 +2298,6 @@ struct zs_pool *zs_create_pool(const char *name) * trigger compaction manually. Thus, ignore return code. */ zs_register_shrinker(pool); - return pool; err: @@ -2270,6 +2334,7 @@ void zs_destroy_pool(struct zs_pool *pool) kfree(class); } + lockdep_unregister_key(&pool->lock_class); destroy_cache(pool); kfree(pool->name); kfree(pool); From patchwork Thu Feb 27 04:35:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993744 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66FE0C021BE for ; Thu, 27 Feb 2025 04:37:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E5DE3280010; Wed, 26 Feb 2025 23:37:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DEEC1280005; Wed, 26 Feb 2025 23:37:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C6104280010; Wed, 26 Feb 2025 23:37:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A2C52280005 for ; Wed, 26 Feb 2025 23:37:52 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5C64E1A0276 for ; Thu, 27 Feb 2025 04:37:52 +0000 (UTC) X-FDA: 83164466784.27.29C44F3 Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by imf17.hostedemail.com (Postfix) with ESMTP id 7F0584000A for ; Thu, 27 Feb 2025 04:37:50 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Ofq+4Kmo; spf=pass (imf17.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.50 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631070; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DBpeAJmv3dkwKu8bLvx37fcH2/07qN2DhOmgxltefRs=; b=aP3RgkQWr1SS5B8+WOWn+alfug5g2uqI91FGmu5h1ghHrlzR62RyL7mmauGxWpiEKBxK5J H9qPPd7JA+6So2TW/obT+jPjBKMDkF3rFOJSs21v5ou8RJRAewVClXsAJkEQH5G8VCJYbn N1bFUSL/iJQ+dWTDPhwWi8NyC7yL4Q0= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Ofq+4Kmo; spf=pass (imf17.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.50 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631070; a=rsa-sha256; cv=none; b=aH7FFRuQhoV0n+cfJ2f9FRT5de78AvzI3XRC1EFVGUJ2Wspb64DOPRtxiRjmIf3kkMBs5h Wk15YMZiaX7pt1YF3tqtzkRgdvGcGoPFy6Cl5kualZhnriZNMQo5DASMlFMpbN7GHan23w COdKwLRQtipk/V2MhleuD8pwO5wQoWY= Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-2fe9759e5c1so925411a91.0 for ; Wed, 26 Feb 2025 20:37:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631069; x=1741235869; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DBpeAJmv3dkwKu8bLvx37fcH2/07qN2DhOmgxltefRs=; b=Ofq+4KmoEKZaJHnMpaXPK1dHtTUb5PVHQTCM7LDsN2L7IAS4lcd7GVQpTRWBC0O+gY A+acCQDzJgqLu7NppYKv9BbN+zeiyuafl9wahXoY1GJVwU/4OhB3ol8jsMW5wHHVSH9/ u5s8Df454vCAw8LLNjUd3/++trTzs0Ug8iLNQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631069; x=1741235869; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DBpeAJmv3dkwKu8bLvx37fcH2/07qN2DhOmgxltefRs=; b=W9ax0sdyM5cUGbFJtPFNeRM8tiyOcwIry6E0cOIOkYGUw8VgjuDISrHsD/wLmKmELS f3hBpLURJeqiTjB3OgzPuNV7YWZiF8lY79b91aWw9jh60eaTVsvqTNZGS8n7cFoQud4r amDeOcZ/8xvYqYUSRwBxRUcAA1bbXz8tl/Cr/n8oF711skisCkeaMAs+CW/obupPVvPn kShcOQtUKiG3S2OGz4t4dXvQzFCn1i312NMqnQv4YLgzf59+6xi54KPW9PJYG9/h1XDM wMcu3YOtY7BnuxLxpNIvAlVi84ll/IUTxvQqR3TR0TYgcNLhyL86IuJ6NE3RFC8qto4M bA1g== X-Forwarded-Encrypted: i=1; AJvYcCVfOvQhzW9v3u6BeF9pYEUCCOdD67L+NhT5OCvZC1iNwLu4mDm4+zzX6E6r41I6twktAPQhlDufjQ==@kvack.org X-Gm-Message-State: AOJu0YxgiR0OATz8Q3S67ewueZNk4/BXwzn8Sh28UbOV8Ao0hLz9awJM HK162ZdrS8FCUF6eASHbM4DdlrR+3wiGw2rjyCdwIWdlG42ddhK9cAEFB5cw4Q== X-Gm-Gg: ASbGncuKC5Kst/+hRg35/iLNzRQrIpd3qxv92yyuQyVv2a31vj62M8Wa/V1FHwxMM1g cfrXqdlb9a6W1rdY/T5nk+A1rbtL2GwgVNLkFqSMcd+0o2SjjpjdzNiaMJBTxqTNnDtSUK1w3hn CXBBTWGCuDFntNBkZE1FoWrHNs4CSx54ioiynET4mVM6zFqFuTB5riq1fFgUCwEKYRjRE0Lg6sn XXBbbDKCvW8x5ud89kt/YhM6hxOzRccAMZJ6FoEm4F9BT0dsGfPcAZufmljZl3HXTxY1i+6gDV8 7aPkmr4g6sGjsA5POFIArs6Au+fK X-Google-Smtp-Source: AGHT+IGT9IFUdagZ07dcKsUPA3Ga8h0xkyUlcDu4WDHvdo9Rq1PU1sFyBt6xUxtOAdS37OHnC4HqBA== X-Received: by 2002:a17:90a:f28e:b0:2fe:7fae:59f7 with SMTP id 98e67ed59e1d1-2fe7fae5c01mr9272618a91.33.1740631069336; Wed, 26 Feb 2025 20:37:49 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2fe6ded6ebfsm4534899a91.1.2025.02.26.20.37.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:37:49 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 14/19] zsmalloc: introduce new object mapping API Date: Thu, 27 Feb 2025 13:35:32 +0900 Message-ID: <20250227043618.88380-15-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7F0584000A X-Stat-Signature: 3e4pygcd5ihh65kotogbkf5mj78jgh34 X-Rspam-User: X-HE-Tag: 1740631070-565746 X-HE-Meta: U2FsdGVkX18Wt3uwVeveTiZW3+MJz0EwvhTB0Uh3gQQ9Ifhdw2lAevJtyQDb8xFkYMy1qpSvr5RWJLYvoeGR+AZ6br5Rh3GjPAyYDHwz+6AvfPHzKOzl7TreEd3NADr8xX3sKYFr2+ZYUzoVPTUOwLfEAeS/1AzKsFTVSyyr4GujNo4V1eO5nwIwGltgGBVNQyk3ieLO4p8I0p3naBsWHy9Gs89D3L9lhI3IjaUaHvPz/lEQfQrsaRrwXK4Jo/g6S52TIQBJAOr/zl8V2xBzcZrCsveUysn4A5XsCZmRVttqL5s+RLAOWo94/q7MzoGQQ/PbmErM7uI0VoDftMpIs88xWOlK8dicQy/ojAXaSY6uv7SidyMh/QM/pURTbClixjiXOEc8Xg57uY7nKmTRQhkqMq4KV8xBNm32r/O1VrzIMX6B3HdewGR+6WzBOvy/o8NqF1a0D62Z6J2Z82AiuXWqsQpmOzf6+U27V14VxsqggZd3upr9d9jikm9zDcmZ3xhF6Fqf0CS5BnJti2q4M6QluwZLGiAWT6/Td1m77jOOB/7wAgGjsNotwvuAh1H3HNCxhG2Ys+Mf6065GV/SS5ftA1g8uSlBvmkXv9RWNn6wx9bJwXvkYa6j8c1uN/3444q1Dh0ED1J9KXSRas6TGXBUVrmRXFncwPeAgQJz1gWVmSX+kbAPl5trAMfjzPYOYuZzYPeGvOwn2wkkKH0FizJh7NkwkgcKJdZ/HgV3Eyou0+KeEg76qkDQ2j7ktPYZVX4CSzwcT12RsMVwvvwrbI14leTmBJoHOm8+guhtwaROIm2mjD3y6aaePETv9HRQCn6ZEtY3wZS6vrAC8VOzNNHvaqAbmeB/HN5WU1ACqmo5WTdMLTle9DOW1Ka3TC7mVNFaE9PuUIwc/2jm3aAQG/FK8tok0AXf+cML23H3Osx/DUkoloHCGSD0HqvDZ3Y9T8N9AAMVX4SFfkx6+vM 8CvL43xO 9ZsPak+kJ5A7QAMHX0pQxTWviGENI1BS2SHq4v8v16kcpeAxx5TXoJXX/G0artnEDi4yya198b3BJchYbaH0klXkaEbIVqCBcCdw+VSZQn2X/aIVg+J1apG0MsMTxINXEhyf8cmk+GBW8iAYEiEvlz1JOQu58AotL3KAjj61Wd5UVs5fIM5bUWrz9oTnDDEnC9IQiDAf0vEuXVVQdN2t+9hae6P7jKm0eJqMADfCYEC0L/jLVMeSYw+DR4GGoxb4JBKHFtFxobheNnQMETJsrqUKnuRcyEEmDn6Ec3OmwyltjGmFdh0rsGYnKHHAvtEdZy3B0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Current object mapping API is a little cumbersome. First, it's inconsistent, sometimes it returns with page-faults disabled and sometimes with page-faults enabled. Second, and most importantly, it enforces atomicity restrictions on its users. zs_map_object() has to return a liner object address which is not always possible because some objects span multiple physical (non-contiguous) pages. For such objects zsmalloc uses a per-CPU buffer to which object's data is copied before a pointer to that per-CPU buffer is returned back to the caller. This leads to another, final, issue - extra memcpy(). Since the caller gets a pointer to per-CPU buffer it can memcpy() data only to that buffer, and during zs_unmap_object() zsmalloc will memcpy() from that per-CPU buffer to physical pages that object in question spans across. New API splits functions by access mode: - zs_obj_read_begin(handle, local_copy) Returns a pointer to handle memory. For objects that span two physical pages a local_copy buffer is used to store object's data before the address is returned to the caller. Otherwise the object's page is kmap_local mapped directly. - zs_obj_read_end(handle, buf) Unmaps the page if it was kmap_local mapped by zs_obj_read_begin(). - zs_obj_write(handle, buf, len) Copies len-bytes from compression buffer to handle memory (takes care of objects that span two pages). This does not need any additional (e.g. per-CPU) buffers and writes the data directly to zsmalloc pool pages. In terms of performance, on a synthetic and completely reproducible test that allocates fixed number of objects of fixed sizes and iterates over those objects, first mapping in RO then in RW mode: OLD API ======= 3 first results out of 10 369,205,778 instructions # 0.80 insn per cycle 40,467,926 branches # 113.732 M/sec 369,002,122 instructions # 0.62 insn per cycle 40,426,145 branches # 189.361 M/sec 369,036,706 instructions # 0.63 insn per cycle 40,430,860 branches # 204.105 M/sec [..] NEW API ======= 3 first results out of 10 265,799,293 instructions # 0.51 insn per cycle 29,834,567 branches # 170.281 M/sec 265,765,970 instructions # 0.55 insn per cycle 29,829,019 branches # 161.602 M/sec 265,764,702 instructions # 0.51 insn per cycle 29,828,015 branches # 189.677 M/sec [..] T-test on all 10 runs ===================== Difference at 95.0% confidence -1.03219e+08 +/- 55308.7 -27.9705% +/- 0.0149878% (Student's t, pooled s = 58864.4) The old API will stay around until the remaining users switch to the new one. After that we'll also remove zsmalloc per-CPU buffer and CPU hotplug handling. The split of map(RO) and map(WO) into read_{begin/end}/write is suggested by Yosry Ahmed. Suggested-by: Yosry Ahmed Signed-off-by: Sergey Senozhatsky --- include/linux/zsmalloc.h | 8 +++ mm/zsmalloc.c | 125 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 133 insertions(+) diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h index a48cd0ffe57d..7d70983cf398 100644 --- a/include/linux/zsmalloc.h +++ b/include/linux/zsmalloc.h @@ -58,4 +58,12 @@ unsigned long zs_compact(struct zs_pool *pool); unsigned int zs_lookup_class_index(struct zs_pool *pool, unsigned int size); void zs_pool_stats(struct zs_pool *pool, struct zs_pool_stats *stats); + +void *zs_obj_read_begin(struct zs_pool *pool, unsigned long handle, + void *local_copy); +void zs_obj_read_end(struct zs_pool *pool, unsigned long handle, + void *handle_mem); +void zs_obj_write(struct zs_pool *pool, unsigned long handle, + void *handle_mem, size_t mem_len); + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 74a7aaebf7a0..147915ba04f9 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1364,6 +1364,131 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) } EXPORT_SYMBOL_GPL(zs_unmap_object); +void *zs_obj_read_begin(struct zs_pool *pool, unsigned long handle, + void *local_copy) +{ + struct zspage *zspage; + struct zpdesc *zpdesc; + unsigned long obj, off; + unsigned int obj_idx; + struct size_class *class; + void *addr; + + /* Guarantee we can get zspage from handle safely */ + read_lock(&pool->lock); + obj = handle_to_obj(handle); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc); + + /* Make sure migration doesn't move any pages in this zspage */ + zspage_read_lock(zspage); + read_unlock(&pool->lock); + + class = zspage_class(pool, zspage); + off = offset_in_page(class->size * obj_idx); + + if (off + class->size <= PAGE_SIZE) { + /* this object is contained entirely within a page */ + addr = kmap_local_zpdesc(zpdesc); + addr += off; + } else { + size_t sizes[2]; + + /* this object spans two pages */ + sizes[0] = PAGE_SIZE - off; + sizes[1] = class->size - sizes[0]; + addr = local_copy; + + memcpy_from_page(addr, zpdesc_page(zpdesc), + off, sizes[0]); + zpdesc = get_next_zpdesc(zpdesc); + memcpy_from_page(addr + sizes[0], + zpdesc_page(zpdesc), + 0, sizes[1]); + } + + if (!ZsHugePage(zspage)) + addr += ZS_HANDLE_SIZE; + + return addr; +} +EXPORT_SYMBOL_GPL(zs_obj_read_begin); + +void zs_obj_read_end(struct zs_pool *pool, unsigned long handle, + void *handle_mem) +{ + struct zspage *zspage; + struct zpdesc *zpdesc; + unsigned long obj, off; + unsigned int obj_idx; + struct size_class *class; + + obj = handle_to_obj(handle); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc); + class = zspage_class(pool, zspage); + off = offset_in_page(class->size * obj_idx); + + if (off + class->size <= PAGE_SIZE) { + if (!ZsHugePage(zspage)) + off += ZS_HANDLE_SIZE; + handle_mem -= off; + kunmap_local(handle_mem); + } + + zspage_read_unlock(zspage); +} +EXPORT_SYMBOL_GPL(zs_obj_read_end); + +void zs_obj_write(struct zs_pool *pool, unsigned long handle, + void *handle_mem, size_t mem_len) +{ + struct zspage *zspage; + struct zpdesc *zpdesc; + unsigned long obj, off; + unsigned int obj_idx; + struct size_class *class; + + /* Guarantee we can get zspage from handle safely */ + read_lock(&pool->lock); + obj = handle_to_obj(handle); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc); + + /* Make sure migration doesn't move any pages in this zspage */ + zspage_read_lock(zspage); + read_unlock(&pool->lock); + + class = zspage_class(pool, zspage); + off = offset_in_page(class->size * obj_idx); + + if (off + class->size <= PAGE_SIZE) { + /* this object is contained entirely within a page */ + void *dst = kmap_local_zpdesc(zpdesc); + + if (!ZsHugePage(zspage)) + off += ZS_HANDLE_SIZE; + memcpy(dst + off, handle_mem, mem_len); + kunmap_local(dst); + } else { + /* this object spans two pages */ + size_t sizes[2]; + + off += ZS_HANDLE_SIZE; + sizes[0] = PAGE_SIZE - off; + sizes[1] = mem_len - sizes[0]; + + memcpy_to_page(zpdesc_page(zpdesc), off, + handle_mem, sizes[0]); + zpdesc = get_next_zpdesc(zpdesc); + memcpy_to_page(zpdesc_page(zpdesc), 0, + handle_mem + sizes[0], sizes[1]); + } + + zspage_read_unlock(zspage); +} +EXPORT_SYMBOL_GPL(zs_obj_write); + /** * zs_huge_class_size() - Returns the size (in bytes) of the first huge * zsmalloc &size_class. From patchwork Thu Feb 27 04:35:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993745 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84F4AC021BE for ; Thu, 27 Feb 2025 04:37:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1B3CE280011; Wed, 26 Feb 2025 23:37:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 16340280005; Wed, 26 Feb 2025 23:37:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 002BB280011; Wed, 26 Feb 2025 23:37:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id CD5D3280005 for ; Wed, 26 Feb 2025 23:37:57 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 94D838022A for ; Thu, 27 Feb 2025 04:37:57 +0000 (UTC) X-FDA: 83164466994.17.E33ADD8 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf17.hostedemail.com (Postfix) with ESMTP id B51D24000F for ; Thu, 27 Feb 2025 04:37:55 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=jmCYQCR4; spf=pass (imf17.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.171 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631075; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HzxKzH19wN3flX6Re4AXRoS/RZlbQkKv5ZSRzF7j4DA=; b=S6ijobUb6C3NkoJvQnecjV9Stscf2z975Y+U6UoRAFJLSHbF5GoT+e0DI8ByTJk104LaEO fQot4VEJWwq1FR72L6Qs6U4DY1y0B1rlfpy0IWAZKoIwa74sCuXdLu1ZxoevPS09GAxWVi hNSpa8vW2LJ90/iynOuXT6CkEgwEFfI= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=jmCYQCR4; spf=pass (imf17.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.171 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631075; a=rsa-sha256; cv=none; b=5eLSQfdAsPxbfmDFzJA4tQGbEDHOi14RiBMRHBpXxOmGj7ZuzjEDRXL10O2LVw09Gw+SYn Qj+9IwcfCi9LpAp5o0Y28CdO2EDcR4ZlbaqVqBflnTXbcJ4t3lKOqPuZhQhsSO4uYJ1RDJ pBlae4T3XCxXaz4A0iBgo5JjfvK4pBE= Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-2234bec7192so12476545ad.2 for ; Wed, 26 Feb 2025 20:37:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631074; x=1741235874; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HzxKzH19wN3flX6Re4AXRoS/RZlbQkKv5ZSRzF7j4DA=; b=jmCYQCR4I8Yq9f/xyCNhWUZlbRT843VnLBUseVtmA45/atWG+KX2Kjw/RU93jjvGsE HopvIUaaRSCqvGMK1djkxZnRClNPhZAU5WDxWS+MviClX+X2nJuHcS5E0uLyR/jxD1O9 H1/J8eSc3zL9tgkK7zameK2hKgCwtxe2DtFhA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631074; x=1741235874; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HzxKzH19wN3flX6Re4AXRoS/RZlbQkKv5ZSRzF7j4DA=; b=iMC9bbARmjH9cgHO9cdw2sq3DaDthnl2lKztFpyKPVAq00Vdt/WiW0oFDtacYnF8A1 UJRAAjDZMS8tlLi4YXS/ClylVHVvfs3KAD7rknQV6mtZ8KhFxBnjz5V2QKxoHo0feIuD nARmgyLTtijI9eT5j3aXjIz3iIuUbbZDuueRhjKuzyn1e/xCRdJXpRtOzjBPO2l5OEWr ycjwW3oH+cDc4FljH8cJapgQsC7MSxJIsqaj5JJ41VLMAxTiIBlO59T91bnWpoM9FSB/ 8CvxA5PGs9x+Wqt3CrBzn1MKZDHnx0/psQ8ZpddFdc4ZbBNjzOLqjyypq2z1SxI5kb6W efnA== X-Forwarded-Encrypted: i=1; AJvYcCWWIdQ91L2UR8VU90JrG+PuQg3e77aqLYMirvp4CIdRBTjlaf2QzYP/jaFbd3lRutFtcXyRSpKwzQ==@kvack.org X-Gm-Message-State: AOJu0YzAjGQL2rqt59X/UgNRiWDqfeo3IYViEsIiPGMV3MKMl5cZtUVY acfFIK+r3oIx2Aa7rMaVH0zyyElqYMXg7BvZM51BVOSNizasiCgVpNdNrTNhOA== X-Gm-Gg: ASbGncubEO/FRJQ0ilFxpaZOc18kyxIVrokoxqvY+f9RM8peLchJ0pYnfaukwkv7u4f sVa/H/p6yQpNVgJ8gXxb3zyDpaXWOf9Ne+EUA+wr7m/G/2tfX9i/pL5viCe+BOW7hyVBr09eU6R oGZpf3mdP6s6FxEnwUwRjfmY2aAcPO5CyZ0JH7oULE/wDfIQWtqvLt5KNzERkKMARxBuSnUclOE M2ByQ4BQuTAEOMYv9H5tGYv7+Nf+AHpT4yeW2LukEcugRSRHk/+eOVk6MHi3m7Fj/acBQr+YsF+ xVJyUWCyJkkHwhYj9hdMni8HhAcL X-Google-Smtp-Source: AGHT+IFihPm3YLRz0ki1R3h8+MO8xOX0re37Dk+wIL69kSAnv/pKGoOrc8itNvxpjLwlfq4K2/CuWw== X-Received: by 2002:a17:902:f541:b0:223:26da:4b6f with SMTP id d9443c01a7336-22326da4d3emr76424375ad.14.1740631074685; Wed, 26 Feb 2025 20:37:54 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-223501d5247sm4996945ad.26.2025.02.26.20.37.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:37:54 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 15/19] zram: switch to new zsmalloc object mapping API Date: Thu, 27 Feb 2025 13:35:33 +0900 Message-ID: <20250227043618.88380-16-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B51D24000F X-Stat-Signature: wdzfh3wjiufwj4ep8monkhw1677songu X-Rspam-User: X-HE-Tag: 1740631075-540023 X-HE-Meta: U2FsdGVkX1/gFd93e72jhTPYQwmkcTjMBZL3eTzKT0EzMKnqQZ8KJ9MichlgG9ufwD4pZErS+mD1LlvXiHn5g4/b5/2g3kDe3uC9GXa1I3oFB9H+GVqjn7Ms3yHaLycSsuUG23DXKukUEVjpMgmulVbNBmvOqYj/jI3ieBEfZ4QD8yZDtK0Lmnmb+P6LBLe3l0ednKDIxqnFUfugL1Tu1LD80KrupsgBQVzKYyGj3H8XfJ6zTOPADgNhElY26zDO2OwiCfdmBXiUqqw1W3hB/SMH29AViGRZ81g7izPB5Xjhiosh+UkjE12jFOVOM96/y8p8R4PLrvR9E+h4tcUIdw+5bFPGVo1eGs3Q3gcSiqAue0sMoh4ucsgcm3pyC9sM3JLJybMHGj7FR18UcsNdehoTIDCSaaMRWr5smJLK5I36LkLXPBdAYFhtzIG6W9MXuw2pBIV+aGhLc942rPduee/vhyt2C/WeuhHguRBFnnRPU54O0HWW3a/qu4qB5Xxc7USSX0GuLLgIbECafJa/gNqIa6aVvfBruLxr7gqdyzYGAzCEIpehuJu7m41Uhs6yWRs6/WFaCRZFhowQmvmuuEdo5MQVU7efzIYOR7xQrzzlDH9W7/wUXcODjxcisVBUNqHc6/t1pUUEZy7d69qCTRRP8l1xl26IqSae82wPyR5q8m9Eecxers1HmyoWBFVXliIKxiJY/GFNAiEc/8U42l6xhKDEyyJUi8yvd9FkOdohKaXv/OASr27VqisSNceeRIZl624wulVKSl+8vu0Pa18e9aL4toX+SVAGJ8CRgJCTjGdtppQAB2gnm7tkMcKaXHIkZCIYgguxcwFg8vtbp550sd/jMkhcnd9PapVLJ6l8sTCdlY3Dc5GnxdVrzcuuub52iLotTX0Mo4l0S0/2TWT2Bfwuei6fcnrIOk3d/3YuEVbs0CUOxCz+UWC36v5BrDrRFsqILJOtyYhdwu+ ffgnTTx6 zx46TXINBh/3dfZSZkMRBlR8WnzF3nAJHJiH6QPH84XoDUrYIaaZr12jvwbb+VmhU2vq7mCdtEscdrOM15qj09W/j/M3eLyLx2PRjDZlvCvpRIwJbr4H07kg1j8ygzy6VO/Iva+z8HOEYxV84jY8ptXBsaLBUvkRR8BmM2Wp376lM98Y0ZfezmC3yBhJ3EseGKKv7Tyrb0zkQPhVvHOBfcwiwtjL/GpjYGHm09giu6CAs5ysW5kMG6AVDNITXWPQyel6ygbQYPhtT0C7407yoN5Tg4vZK3kOVZEmxtoe35pdP5FYEP2+ATXESIqYkjfsBesfY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use new read/write zsmalloc object API. For cases when RO mapped object spans two physical pages (requires temp buffer) compression streams now carry around one extra physical page. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zcomp.c | 4 +++- drivers/block/zram/zcomp.h | 2 ++ drivers/block/zram/zram_drv.c | 28 ++++++++++------------------ 3 files changed, 15 insertions(+), 19 deletions(-) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index cfdde2e0748a..a1d627054bb1 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -45,6 +45,7 @@ static const struct zcomp_ops *backends[] = { static void zcomp_strm_free(struct zcomp *comp, struct zcomp_strm *zstrm) { comp->ops->destroy_ctx(&zstrm->ctx); + vfree(zstrm->local_copy); vfree(zstrm->buffer); zstrm->buffer = NULL; } @@ -57,12 +58,13 @@ static int zcomp_strm_init(struct zcomp *comp, struct zcomp_strm *zstrm) if (ret) return ret; + zstrm->local_copy = vzalloc(PAGE_SIZE); /* * allocate 2 pages. 1 for compressed data, plus 1 extra for the * case when compressed size is larger than the original one */ zstrm->buffer = vzalloc(2 * PAGE_SIZE); - if (!zstrm->buffer) { + if (!zstrm->buffer || !zstrm->local_copy) { zcomp_strm_free(comp, zstrm); return -ENOMEM; } diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h index 23b8236b9090..25339ed1e07e 100644 --- a/drivers/block/zram/zcomp.h +++ b/drivers/block/zram/zcomp.h @@ -34,6 +34,8 @@ struct zcomp_strm { struct mutex lock; /* compression buffer */ void *buffer; + /* local copy of handle memory */ + void *local_copy; struct zcomp_ctx ctx; }; diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 7ad89df9a304..1e2ba3590a39 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1561,11 +1561,11 @@ static int read_incompressible_page(struct zram *zram, struct page *page, void *src, *dst; handle = zram_get_handle(zram, index); - src = zs_map_object(zram->mem_pool, handle, ZS_MM_RO); + src = zs_obj_read_begin(zram->mem_pool, handle, NULL); dst = kmap_local_page(page); copy_page(dst, src); kunmap_local(dst); - zs_unmap_object(zram->mem_pool, handle); + zs_obj_read_end(zram->mem_pool, handle, src); return 0; } @@ -1583,11 +1583,11 @@ static int read_compressed_page(struct zram *zram, struct page *page, u32 index) prio = zram_get_priority(zram, index); zstrm = zcomp_stream_get(zram->comps[prio]); - src = zs_map_object(zram->mem_pool, handle, ZS_MM_RO); + src = zs_obj_read_begin(zram->mem_pool, handle, zstrm->local_copy); dst = kmap_local_page(page); ret = zcomp_decompress(zram->comps[prio], zstrm, src, size, dst); kunmap_local(dst); - zs_unmap_object(zram->mem_pool, handle); + zs_obj_read_end(zram->mem_pool, handle, src); zcomp_stream_put(zstrm); return ret; @@ -1683,7 +1683,7 @@ static int write_incompressible_page(struct zram *zram, struct page *page, u32 index) { unsigned long handle; - void *src, *dst; + void *src; /* * This function is called from preemptible context so we don't need @@ -1701,11 +1701,9 @@ static int write_incompressible_page(struct zram *zram, struct page *page, return -ENOMEM; } - dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); src = kmap_local_page(page); - memcpy(dst, src, PAGE_SIZE); + zs_obj_write(zram->mem_pool, handle, src, PAGE_SIZE); kunmap_local(src); - zs_unmap_object(zram->mem_pool, handle); zram_slot_lock(zram, index); zram_set_flag(zram, index, ZRAM_HUGE); @@ -1726,7 +1724,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) int ret = 0; unsigned long handle; unsigned int comp_len; - void *dst, *mem; + void *mem; struct zcomp_strm *zstrm; unsigned long element; bool same_filled; @@ -1773,11 +1771,8 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return -ENOMEM; } - dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); - - memcpy(dst, zstrm->buffer, comp_len); + zs_obj_write(zram->mem_pool, handle, zstrm->buffer, comp_len); zcomp_stream_put(zstrm); - zs_unmap_object(zram->mem_pool, handle); zram_slot_lock(zram, index); zram_set_handle(zram, index, handle); @@ -1882,7 +1877,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, unsigned int comp_len_new; unsigned int class_index_old; unsigned int class_index_new; - void *src, *dst; + void *src; int ret = 0; handle_old = zram_get_handle(zram, index); @@ -1993,12 +1988,9 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, return PTR_ERR((void *)handle_new); } - dst = zs_map_object(zram->mem_pool, handle_new, ZS_MM_WO); - memcpy(dst, zstrm->buffer, comp_len_new); + zs_obj_write(zram->mem_pool, handle_new, zstrm->buffer, comp_len_new); zcomp_stream_put(zstrm); - zs_unmap_object(zram->mem_pool, handle_new); - zram_free_page(zram, index); zram_set_handle(zram, index, handle_new); zram_set_obj_size(zram, index, comp_len_new); From patchwork Thu Feb 27 04:35:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993746 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B89DFC021BE for ; Thu, 27 Feb 2025 04:38:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4989A280012; Wed, 26 Feb 2025 23:38:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 448DF280005; Wed, 26 Feb 2025 23:38:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2EB73280012; Wed, 26 Feb 2025 23:38:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 0A406280005 for ; Wed, 26 Feb 2025 23:38:03 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id AA8E18022A for ; Thu, 27 Feb 2025 04:38:02 +0000 (UTC) X-FDA: 83164467204.14.F91C219 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf14.hostedemail.com (Postfix) with ESMTP id D059D100004 for ; Thu, 27 Feb 2025 04:38:00 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=KagGeR3K; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.176 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631080; a=rsa-sha256; cv=none; b=tj/U2mZ4B9A+1AQsoWCMOzKy6rtPLsw6UGnR8uuwcd8X8UqfRuLzYpNpSHZjsJxqIFzdnB rDOJYxPwLYutY9Sj6mk3NEX/2CwBBWlWxJSDKWlN5BoxPG78U7DltzgmaWAY+Jt5MJ2h06 KUU28of0JsqMPGNg15c7V1XoSAAMK6k= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=KagGeR3K; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.176 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631080; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8vuEfNhU6ZAruk6qIF6JGw0mo2a915MB6BnSqH9CQes=; b=8EMG381UDF1i5bwsdkEeTZWeezfuBfIMjo5aU/T86gtnpeQudsD5X16qXjtcCem+LFR00D +kgUWsGIfVLOXzgAiqj+STPqE3vkAP8sBF3uRKPnXDvZWsw01cJRiKy91mb65hTsBNEVtB cpcJL5yYT/mQp2UkDPxHNkDlhx6Fguk= Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-222e8d07dc6so9974135ad.1 for ; Wed, 26 Feb 2025 20:38:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631080; x=1741235880; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8vuEfNhU6ZAruk6qIF6JGw0mo2a915MB6BnSqH9CQes=; b=KagGeR3KsD5EnwC29XN6SgREIEU3v0rxIVrWBwNn5+Q39XNfbSEEj5CTXwLRnaU7Wd QgDsWj6GodMwXq6dfGKQaOvugDKo7xtlrDTi0JVF4Z8KB8mGkaLgi3sYjnq/1zKkD3C5 uNbPqcLmeH3/TuzTw3G1+biz7nVEqY/pC+6/0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631080; x=1741235880; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8vuEfNhU6ZAruk6qIF6JGw0mo2a915MB6BnSqH9CQes=; b=aoQasp/5tsG8rOkPWoTbUS5H4QwiE4j+BxfGoXbwpSMgbpMeJli3IgtZnQzrPwr0EG Hmot+eHpLO96iO2SsB6B5rfkbTynIJTlKQ73oUpMtbYj03aAgw4Sq12jH+K/F/OJpZNJ SwhW+cUv5PwQAy2J+oKwpZlGi6yhqWautyq9/2gkIucO/w1fXvHbqBuQ9BFCSlf0lz3g dyI4venYuxkhX/LQMFahIZwmDaoTLdb1NPSM7FlPN8qu0u/2TPM7yjZlGdpPW9pqq2mP 7/18GOrBOf6s2jUGaLH1Hf6ikKsEUQyS8wun+52YfebUDn1qD09u/ioaUWbHHdZ0kKJb IFCg== X-Forwarded-Encrypted: i=1; AJvYcCUiZ6XgZxOhzJtsxtvL0oVAWbKQR96+++I0lWzUTLcv+2Dui+v3WJMpjjXo7nETynHWdqnbdXqTXw==@kvack.org X-Gm-Message-State: AOJu0YxmYNRDnA1jxmkhK9PbIjSoRpNX9bp8e8+UbDSIHDatwiUWimcv /f7rK13DJ6DNwDYTTBry1Gy9sb4/8jyctqXw0Jtbo8P1O3kJCyHzueJ1FcO8Rg== X-Gm-Gg: ASbGncu8khOyXde1F+bhGqNKkjZc3KjgtDgyvggGYxFzXK++dakmeceTnY4ABXsIHKD iHj8Usl/y8nuWYrUMdvLmACxijGtLEz/k8sWE+qOIeBybGIcNWJh/2pYz9LbxUeQ7arhbSf2bEi fo/E8zBcgD5+hBw/RXt5xQv1Nasn2CwH0oer9MH4qeuZAsWgzP9Z2er0jSfykkvOelT0XCZQi2C vzYtO/FChKP5hMizZ0uDZiV+L4A6jhsD3ee0VDfZTSUVU9Q3b5RqK70A/YFlCjAEMeyBa6Xapzi Fyr6tP8X8pHdhNKnEgjZDU+JUpmN X-Google-Smtp-Source: AGHT+IEe2V/zeMcvZORXlCVMsgDJF6cKsc0Tcc+m2RnLoDmlcE2u9VnCp5uqjB7GRbUjo0EoaqQNIQ== X-Received: by 2002:a17:902:e752:b0:215:a303:24e9 with SMTP id d9443c01a7336-2234a1649d0mr25953335ad.3.1740631079813; Wed, 26 Feb 2025 20:37:59 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-22350533a10sm4886515ad.247.2025.02.26.20.37.57 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:37:59 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 16/19] zram: permit reclaim in zstd custom allocator Date: Thu, 27 Feb 2025 13:35:34 +0900 Message-ID: <20250227043618.88380-17-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: D059D100004 X-Rspamd-Server: rspam12 X-Stat-Signature: gago5qsn4hftctbgufgsnwzuyn6wujrg X-HE-Tag: 1740631080-463868 X-HE-Meta: U2FsdGVkX1+YwhGm8oFu+b0Qrs3JFKbGmWOtelc/bzEzLaRVnp6ejOPIe3X4/WVLRVD2xX+dDAc8PFDXEPyYuOhowOGlPMn2YvlkPoxn37pFjMtCleEcAb2Cei7mBDU1d5o5C31ep37BwoUXw/rbTXI6KH9L7V+w1kApIaW5B4Z3DRKn2UzmSa/TQavJ/egGSITbiflp0MYtEXxXYkysfTaLON1pDIQ39TyTW0mseIvunmzfgRL5Ka2SVdfpoSZwx4y0/ath2a3O+JjqPd5W5sbglbDhwjk1+heBfkfPFppghWtbtpiKJNOCdOMNo8Qca0H27NgR6Lk4pDAkEATt7yUPSAkBRBmo4mEFsWO60HMUlBCedIxedxVmcwW9Mmx87g2e+QrGUrA3Vsuv5Ba6qI8c+9gY7U5fLAKpEE0MrA1imFPDZXcjw6am6vJ7TbVbvPbpuAR6Uw2snVeh6BTMP2syvOZWrM6itZN2rc63hpK1dFo7UUSM0O4SoZgSPPU69JEwhcb8gJhoSkv3VJif+sDbNEjOlkSWbaRecLXkxc81kKTeSK1BGId0mbiw2JmbkxSe6DBPTUe44cTZsqNYHilfALgGxNqEPN6d9ysR+bfrj9VeDc/AC5rlVQWbL5Q2Uec6A8r+IP2/2vLZJti7gg4ySyN2hA40wxukkgV+BdzdpWqt5AsDXRcoDH0RJpMHMdaS2wu3bY0h4iDY3+Hl1g93k2ZWagxG3DkaKzFpNYKDCUfSvff62TSHCQPTwW40o3YyfboJNfEl9j/PctrL3HJVgBXiPvg2UbJIgVEDN02Sk2x2b0O/rkJJdSz8fFaD0g5Pex9oCBPbHH3553fmFginOQ7Rn45amGgc+T2Q0m6eo6vObPEveFMLMoYxoxcoKkiHBCENOYY8a4G+AaRalSJRedK8wWTIyckumPu2Nis1eM94euAZ5rgWVQAdiaUtLHYnat7VnMVQ+3Ch5hN PeTpsvwP xTBL0kwDe2XvWo5EdJjEcS+MTZsS+mQV2tvsfayfaTYaxvh1e7PyLD0Wh1al1YtoUWfDyx4ujKRSuVMLprhWHd4zmoPefXWHEqhwJNZBwORU6tPXuKIv7m3lNipHkPNBbActAzybpQpqmQ+My+ZZR1dV9hCHRjj2FpiPCT9H31+vq0cE+zz986Swc0laxpv7RF9i1MYu1VWTSjFpfZ5w9FJjkCKwAn3UIZUXiurqAJ+tO1GlEoEIwbe8I6m6o8asm7KqrUZanDkd16bWMXq6faLpzEP1VVMvfIpkbK1lCShb1ClXP5tttcBJt8g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When configured with pre-trained compression/decompression dictionary support, zstd requires custom memory allocator, which it calls internally from compression()/decompression() routines. That means allocation from atomic context (either under entry spin-lock, or per-CPU local-lock or both). Now, with non-atomic zram read()/write(), those limitations are relaxed and we can allow direct and indirect reclaim. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/backend_zstd.c | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/drivers/block/zram/backend_zstd.c b/drivers/block/zram/backend_zstd.c index 1184c0036f44..22c8067536f3 100644 --- a/drivers/block/zram/backend_zstd.c +++ b/drivers/block/zram/backend_zstd.c @@ -24,19 +24,10 @@ struct zstd_params { /* * For C/D dictionaries we need to provide zstd with zstd_custom_mem, * which zstd uses internally to allocate/free memory when needed. - * - * This means that allocator.customAlloc() can be called from zcomp_compress() - * under local-lock (per-CPU compression stream), in which case we must use - * GFP_ATOMIC. - * - * Another complication here is that we can be configured as a swap device. */ static void *zstd_custom_alloc(void *opaque, size_t size) { - if (!preemptible()) - return kvzalloc(size, GFP_ATOMIC); - - return kvzalloc(size, __GFP_KSWAPD_RECLAIM | __GFP_NOWARN); + return kvzalloc(size, GFP_NOIO | __GFP_NOWARN); } static void zstd_custom_free(void *opaque, void *address) From patchwork Thu Feb 27 04:35:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993747 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBEA9C19F32 for ; Thu, 27 Feb 2025 04:38:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7A80B280013; Wed, 26 Feb 2025 23:38:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 757AA280005; Wed, 26 Feb 2025 23:38:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A8B8280013; Wed, 26 Feb 2025 23:38:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 39EDB280005 for ; Wed, 26 Feb 2025 23:38:08 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E7B2F120287 for ; Thu, 27 Feb 2025 04:38:07 +0000 (UTC) X-FDA: 83164467414.05.3685FEF Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf03.hostedemail.com (Postfix) with ESMTP id 1133C2000C for ; Thu, 27 Feb 2025 04:38:05 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=hhJMK461; spf=pass (imf03.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.52 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631086; a=rsa-sha256; cv=none; b=hlO8ABVXOAOI9sEftIZEuSS6UpoCCPlw8k2gZBVCzr45qtOKfC70+NYFBTkIpQ3NLtc5U6 hcE8w+bZlb4UqlYwkumy1dyzeHyAesBATTTBqDOV2sZ3Wm6FtIsqcvpbMuds+R9IBk6wuR mFPOEeBwkpdT6VIWZJXFV7/4DOhhlig= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=hhJMK461; spf=pass (imf03.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.52 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631086; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5Z7L+E0AxAkJHYT9s94E/nR5ku/ie8tNmkgfIKxsvlQ=; b=iOZiw0y5983+FtOjjg9b4g2n80MQUm0j8u2p2Fq9FQzzLDRQtEmyThpVei8nXrkSNvxk4B a3Vi3mpvCU65C9BUvpC0UWR/DjijrqBluD/dKjUf2D69m0MPc3+DE28TSpVlzh2103KkQB Pk2U2dmWdUOnB+c+QRIdqFNyTIMDCck= Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-2fa8ada6662so1125413a91.1 for ; Wed, 26 Feb 2025 20:38:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631085; x=1741235885; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5Z7L+E0AxAkJHYT9s94E/nR5ku/ie8tNmkgfIKxsvlQ=; b=hhJMK461dn1uGgLdrzJUnqlCcdFlFQvvh/T1zsHdUdL+XQFndyvV7ZHBaO7oBzupH8 brrecAh5sTbJlKpvPFFpIG72l0DwGw9OC2BWM0OqUBHdmS94aPnAv0jyswxC2gVc5Cim RWu8Fj5HbaW/1PoxNt94nhnxYcwitL8Q1nCJA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631085; x=1741235885; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5Z7L+E0AxAkJHYT9s94E/nR5ku/ie8tNmkgfIKxsvlQ=; b=aji2qKsPYfwrCOfvleQKfxZzqKhi+Mr+dNYaHzpREdpJmgGEBbj7OQIwtJQ/8LvREO ePVvLL8n4VDt3fMy70nNV4VqKQ+U1i/k/hod7X7Cm94oh0d/92UlEvY/WSvQileDUnOv 9BokmeCgbH+cCQ5oFvzKNEVk3l22X80LYr0Ad1ecikvbgJw11K/nff+mCHMbTIYqhhro BLBGokYJgXf7ypsgSUS2oSy5UitpSBtrbMTfTa0Zd/Yl407DP8p1OBr8D2FYP56SJieQ 23VU8XOpRWadP2pSjRVg7cluLYDoFm0UscEr70i/4n1ch2dUbM+bDsRVzBMLp3LB+Jua yoCA== X-Forwarded-Encrypted: i=1; AJvYcCUjYXfyB74L5rtSuUvhHcnqZDp0LI7OAixW3H/btc929/6AM4QxinW6/W2CMDwutvb9smI4HXt8Ww==@kvack.org X-Gm-Message-State: AOJu0Yy7KEz2CWei4DlwZmSgIq/9Y2e5LllInMNFXEfQ8i7HH8dIGu9s fahEKKpcCbxlxb/iGifgGeEz5fdXOUTsy0QcNMvUGsraou1cTSPBNCXTbJxKqA== X-Gm-Gg: ASbGnctchwXuNTFniDrtoCMPBuxCBkfVAHJ/lftEnxAocAOMmpP6JwxIMheHXSXseEU TtJnqXqnsK66ro6Rx6EqVtwibqz6mh96i4f13k9YI7JRGXZLXc7k4PN9xjALgFRrQW0F7mvFFzx v29gUifRnh9reFmzZeCiNjWVuRYb92wJdXjiB31n7FiyLqygAACNwB7b7+s4bj0cXQg06quBwAr ukMf1jWBhGmCGS/i6D8WfLGJSFwGAhNa0eZRHtpAkonYbmsFh66KTZEgvh50F3mbM791KXi/EBD uel56FsIGF3529v4gkRUlrple10d X-Google-Smtp-Source: AGHT+IHkiQZTH+QyI09NsnNe2+VUmytHTF3AQc8esT3P6nxO5c5MPzYujs2/5+Dmg9L+ktNJZYSfyg== X-Received: by 2002:a17:90b:548f:b0:2ee:5958:828 with SMTP id 98e67ed59e1d1-2fce86adf6cmr41590472a91.9.1740631085050; Wed, 26 Feb 2025 20:38:05 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-223501d2901sm5005945ad.30.2025.02.26.20.38.02 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:38:04 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 17/19] zram: do not leak page on recompress_store error path Date: Thu, 27 Feb 2025 13:35:35 +0900 Message-ID: <20250227043618.88380-18-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Stat-Signature: zg4njcbepk1zi34s56desrqy6x5s7yw8 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 1133C2000C X-Rspam-User: X-HE-Tag: 1740631085-309785 X-HE-Meta: U2FsdGVkX18dqYRss8tltNnOPt+29E4QRBW4L0orJ7pYggtRz7N1T3d2rESmvyuEzCZMl8Nr0IlEIruPLDVGj2f3nPOALiC2aCeG2pgEU/yssRut7nWzGL/qhOi73KMKF7FKBNE5bs7vUelW6s3peMv6fr2MwHTj2ROqpncyjGLTrGrx18HnXJKK32aW+Dj2xdT14HWhXDwo8nsXwbdaj+HEMknNsKF0g0+Kee5hppSTSC9qu2CxWGGrErO/M5eODR972/FGOWSjRLaqxYlUc0hUKQG0ThpA7wkdkavubyn1usVjyBv+2nBnfiQY9+nMwquORX/mo6u85r2nk2Gars1Zu5VM3y8YdN3ORRn3KwvFz/tKuB1FacvWumeMhDGra6BShfcmwPFfHpXB5/EsJzzEc/E7eAxPoumcRZmnclKsAnebNIqRU+Z7hFuTS+4TnMB460F3uJJ7RLPcYJAZZrNKz3SKlgbVyIPtOD71CnaPpbno0jVzuWp1DbnDFUEe/Q23veEX/5qWKqVocRFqPaWfG2fYGho8X984Y3PLzF+ob7tioYKrAhdkoyfmk7qVt6ss0B7eB99lLRCb9XPwLGYzJeUfmjMFvgomU+DQYcDn0Yv1ooKrd2QRHdmMmgi4V44nD6JnUZVNi+3sVSxhlmYH7IJlKKvBsqbzrQ2vBw/5hkzlQ2u2tUuwJTZNjKWiFsceHwctorWBWb/l1TqRphMbdWSgoZ+6KKSMZ/iIzE/II0yWsx8HEBy++s6wF2WveJvLLvousjqvGLrLHv1jZvrq+o2pFwgwRyO/nIdsNukloC/2msHFFjH4OBy70Ni8jwnyuwYrQtxqjlMJ/qn866Mz+L3WpyA1AW3Q3A8JMIeyTJCyxDUGUPH/G2H5sTjM2HrZurfeBjhza9rByizoIF8w7dh77lbnWEjhySqpro/u1N3Fx11xIZ32l3Lt62b7Fz7FMKAPjgUIW2YIGUs Z5i542Eh YQ0f/GAIs6ARBHv+FMKbovmS5r+91Allkaw6DxqHBPP7C5UKG+rZqD2gp6Rgx2bW4Izyg5uvLg7/V2eg2jL85g2SWUES/kYhqydyN463ZoNmwUVtIOznLlY5F/gf6hSEQHhZECGzm8jSqXoRkjU/eRZn1tqnRXDZshrZVlxWtKIV4Me3ilLQ8gE2P3uGFxcdUZm913vrWs1zgW3E4JZhYwg8WLascQ7ZiVnUnTgarAdGNw/1PsQUslalWGBur6S0Z3Dlp12o7k76uOkPOsSUG7FgTx37qhacpgNRTLp2j6Rh+ckqTyLEtztDQ4sSauXpDwYcC3vVCE9qPtEa2Gb1XD5o37EJAC4G0cPFinCzfiWCpLwQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Ensure the page used for local object data is freed on error out path. Fixes: 3f909a60cec1 ("zram: rework recompress target selection strategy") Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 1e2ba3590a39..e8375fcdb36e 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -2013,7 +2013,7 @@ static ssize_t recompress_store(struct device *dev, struct zram_pp_slot *pps; u32 mode = 0, threshold = 0; u32 prio, prio_max; - struct page *page; + struct page *page = NULL; ssize_t ret; prio = ZRAM_SECONDARY_COMP; @@ -2157,9 +2157,9 @@ static ssize_t recompress_store(struct device *dev, cond_resched(); } - __free_page(page); - release_init_lock: + if (page) + __free_page(page); release_pp_ctl(zram, ctl); atomic_set(&zram->pp_in_progress, 0); up_read(&zram->init_lock); From patchwork Thu Feb 27 04:35:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993748 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0591C021BE for ; Thu, 27 Feb 2025 04:38:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7574D280014; Wed, 26 Feb 2025 23:38:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 70725280005; Wed, 26 Feb 2025 23:38:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 55D05280014; Wed, 26 Feb 2025 23:38:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 36B08280005 for ; Wed, 26 Feb 2025 23:38:13 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id E2B02160233 for ; Thu, 27 Feb 2025 04:38:12 +0000 (UTC) X-FDA: 83164467624.11.7BFC985 Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by imf24.hostedemail.com (Postfix) with ESMTP id 08803180008 for ; Thu, 27 Feb 2025 04:38:10 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=XV5mJFtN; spf=pass (imf24.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.45 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631091; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hD0ulf0FeWPvWMK0YywecJZZj4meHTFhM9R2+0rTTi4=; b=JbNpemhkdH+klnTV1TbjkZ5xlqgPkwh3Y45UjISmctqg7cRNpzHhACq0fn8H5EwHdaCZnV G+Ba0UNBbUqe7dX9ef1xywmCzUPDF4QnmR/opABdz0mnAl066BbwIHuxbzpv3m/eIe0Mmh RbDPY/MuH2QeWD7io904WOhgkpQZrqE= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=XV5mJFtN; spf=pass (imf24.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.45 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631091; a=rsa-sha256; cv=none; b=DCowjixXJVirnWfL1teUdk7IshOznzrPmwfJ4H7frJ4UuFxvtmDhlQhyJrHIIMh+5THeN7 yMoG/2iG5EInfeoy/14HqSNcDJQAT8Ryt0n3zTB9D4Jnwpbk7SEZZ/JPQOQAsIGhGb7g8+ ceDqgiZ0TLcN5M9u8fHr6Q8BZcXn31c= Received: by mail-pj1-f45.google.com with SMTP id 98e67ed59e1d1-2fcce9bb0ecso1025171a91.3 for ; Wed, 26 Feb 2025 20:38:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631090; x=1741235890; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hD0ulf0FeWPvWMK0YywecJZZj4meHTFhM9R2+0rTTi4=; b=XV5mJFtN/aln5ZqF3NmJVV41Esb4cTfbGKxTmfT5O6oJ8pOcxgXAnYZkH20xPANKPt pHepcLda85fGuDfwUDrz/7TVU/Fw+LHK8i0wGhEph895QrXOZoBlwQTiN702k1opKKBI ADZTPuzZtG9wRRPUki1LU3ASanrN6cBVnageI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631090; x=1741235890; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hD0ulf0FeWPvWMK0YywecJZZj4meHTFhM9R2+0rTTi4=; b=vVfy47qJ/3ewLZXt3GK0tIYwfNf0sLSvSfADDJA5flZLcrivgOvdUfTjkAiKxcAlP/ gc2Sspg79MN1FrFvBgsDaXlU2pIuwOEZmm0U5u9kxE2tKszYnQpaFPDVJHIWmV/0klT2 1yw7cFUQFI3ne6zfT0HNBuVzN4sxk7vgi5/nO9GXhhuUBDjRpCZ9ssX+qWV4rk3sBSbd I0e2JDvAb2FmYioy7Gb1eQBtsdAuKpi1SH88B/QeodXjJ9Boqb2VLOzU63+1148LmA8A jMDaqdh+3G7pGFNsohamwwq/266LXMvOvOW+loY4oLOYU0GX8dGdx9VNtppPiXBDlReL /YaQ== X-Forwarded-Encrypted: i=1; AJvYcCUWXQaBdPnG6xtvmZhGFPvLSfncPT1A+0p71gGMVAJuwgfNiiUdS4ez5X7icx8ClaOj00UZLdDtDg==@kvack.org X-Gm-Message-State: AOJu0YyjiumKKuE3KJ8wyaLywmvfn6QkoH57JsUCufJc18ncgwWsxBRb 23aVOFMA00tmfZeSOQk8xpS7IGgwN8mA0SrvHdnznGegy+IK+tsDW2Vr3AuTdw== X-Gm-Gg: ASbGncsCaL3uppaPaIuOqZo+2jhBKqyrI0gZ+8jt9J4f9zj/mY29emGiEofRvONkRmX q1Rn4gZ5l3u3PvMPxt9ICR9hQy1B+WgCQBA0+MT/DrMXUbImD3WbOkJ3Lv0A0d+GWVObdIrTIZL +KMPeypB+1B+r/BzTnQpW50cf957z8sPWpKbv6Es5EeMlTa2wnoL8F0Cc9wFOJXhcGUgRNpB1oR JN1r/mJWI/+AmQf8otsbyy/c2Oj4R8BAi3Oq/AvxKOCaKYZoUME5Njz5KgApGJF5O+xNveeA+z5 pQ+iVuH2VgDXjnw/ZJrafRj0cGXq X-Google-Smtp-Source: AGHT+IFidzq43r+0NRTs951DcUKcskfAO05VG8wu01wBEQtR1k0fOqC/zriCBDU47fEYDjlBVHYkmQ== X-Received: by 2002:a17:90b:4c8a:b0:2ee:3cc1:793e with SMTP id 98e67ed59e1d1-2fe692c7f83mr13786908a91.32.1740631090068; Wed, 26 Feb 2025 20:38:10 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-223501f97cesm5057285ad.60.2025.02.26.20.38.07 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:38:09 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 18/19] zram: do not leak page on writeback_store error path Date: Thu, 27 Feb 2025 13:35:36 +0900 Message-ID: <20250227043618.88380-19-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 08803180008 X-Stat-Signature: zjie3faki7mmwbmz65azcf7t9ma1x13t X-Rspam-User: X-HE-Tag: 1740631090-43887 X-HE-Meta: U2FsdGVkX19m3vHWIleORSKBE/0JPF+17qOreNp2+nTyaGvcJ7x3CywlKnuT7AMd4HMDNpah4IBooJQhfkgI1kJjgEzqiKb5OkdgU39hzJ4FCv2J9A9qFjk3RZvns6MAeHeRvjnVaiqIJ83JBuLtn2WvkYm/PpSRIBdZgUo1vPlgBJ2XUBvaGrPJKCjI0/ybltjs4fGwEYE119G6xXaMn3Ycrh56SLQygH4QlK08b+0NIEgR75K+PaWXsg80WEsMxQ68KrUdscpid1jSbswDXW4HJQ1jMEfrz4M20K1AL83Z2Uv++ArcmYc/TnXuFqENMi/NWtenYIZ9A0oU1BMYr1vGoMRHSEegUCLf7/gj5ucs3ljnUBKtHT1LIT6LSCg8LqWw5Yk8okaZaODxcXAWtAzssO6sBqmwtgjM3o3A5BSTHAyA0aDQvzE+JiTu25uzeabzZF4NNPNmAIloPLOO89yYsugIhsORjb/9cQRQqAwFn2Zm3WUAl7rV/JBiQnVQ56bbSvBGN7qNTwEDIS009JgsRXEz4uYoXu3gpToyFOAHDpFxjT5MofZuhU65VvgSuf2Lldb6MJyfvpB1TPORpWzz8q+BD4wQf1w8hiu5ceM+U9QVnGCsKH9as/1enq8zjaHVhtP5C0+MUXKd/nj46P+/YK3EnGFrCveOjdSSoIyvXfdAinJKWTuBSMuvMR20XIS1AZoDjFMFVQuu7Y1FfyA1miPR+RbpfXOt8VoB+rK66ar+VxU9at/Ssh89jsNJ183MkIQgDEFXO9kR7i5HnY7SfuSGSTxkEy2zXbKQ1argrBbU9d9YgbbMfPodEgyE4Tns7ccIOPlJIr6uK8cRlgio0YZTK4UOEGCkjNRI9cjuuhf/AB372SX7WZlSIz/93uMOb7ZAYPztDF6Qg4AVa6Iz5+TZ+uq7AuDv7bRhRHoG+cAJ1TBoW7YU6Qsb+130q+W/eV9gKmi91UWwVH+ kWChYbC9 JXHbJp3M2WqtGPEMj2qZBlqgrqLt20pOhJsBWB3bgmU6v4U5z0ovQwjMW5LPCRDF7oHLq+enYLpO+QImlkkYDCt/7oNZOGH7b2d41Q4F8ucD9HQl7g/x8Vbcum9P3xq8osNhtC+0g6YjY2aU8LpGEmOzBXvZw4yl8LQ2PZxoWr4EcwuElbJyoe+bZa9hH4Cla36TtZwayZ+WMPq5HS04MfgSJ9ujXpXifLnz93xCMZNAb6vwFUXTa/WuPIy8X93REPwGqWoXC53a4Wwo4chHUqtd2WKrZ81JcVili70EWAqG5tmmR7RRQZ8BXR/coXtQjqXq9fTWR/r1cDLo8G173QR0mUnMbx9YCFmGBQV/CMp9Dr0Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Ensure the page used for local object data is freed on error out path. Fixes: 330edc2bc059 (zram: rework writeback target selection strategy) Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index e8375fcdb36e..5dac525f4cd3 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -787,7 +787,7 @@ static ssize_t writeback_store(struct device *dev, unsigned long index = 0; struct bio bio; struct bio_vec bio_vec; - struct page *page; + struct page *page = NULL; ssize_t ret = len; int mode, err; unsigned long blk_idx = 0; @@ -929,8 +929,10 @@ static ssize_t writeback_store(struct device *dev, if (blk_idx) free_block_bdev(zram, blk_idx); - __free_page(page); + release_init_lock: + if (page) + __free_page(page); release_pp_ctl(zram, ctl); atomic_set(&zram->pp_in_progress, 0); up_read(&zram->init_lock); From patchwork Thu Feb 27 04:35:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51560C021BE for ; Thu, 27 Feb 2025 04:38:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0C7B6280016; Wed, 26 Feb 2025 23:38:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 078DA280015; Wed, 26 Feb 2025 23:38:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E0D90280016; Wed, 26 Feb 2025 23:38:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id BFD5F280015 for ; Wed, 26 Feb 2025 23:38:18 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 80D12160228 for ; Thu, 27 Feb 2025 04:38:18 +0000 (UTC) X-FDA: 83164467876.17.AA142AE Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by imf18.hostedemail.com (Postfix) with ESMTP id A94531C0009 for ; Thu, 27 Feb 2025 04:38:16 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=G4Ak9tOg; spf=pass (imf18.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.181 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631096; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C/ge7B6UGXcSd47ZURwKYcUmpuFGdMGIsEsj1cDYmyw=; b=5fQh+Oe2yvGlAdBYfnjnU4lrkQnMBUjWfidfuZCqgXuMx0ZthdxE1qwHFP4qILsInQuNOV EEIRGuTzt1sFstmjjlozPmie76zHkkK0sN+UUpFm+60Cgd6ttv8BOaN7AQOh0X3Lx3W911 PMnqSd7cynDPSoTDGKE0FpDja67S4qs= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=G4Ak9tOg; spf=pass (imf18.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.181 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631096; a=rsa-sha256; cv=none; b=VUZex27I27ocpbMu4dGZXosw6bcfxQfooPmit82xEYjop9Pr5JAWVKiSXPVyRgLqJ/Am9k W4vm7Ik8YD4RJnbaLO5Cp141pZCx9jctj9iIG2P7I3yGcuoX18QAHJq9ZuyYNKKjO1Yg4J 3HbuBiWxBHZWBCJfCxc/7kWvY8FtuxY= Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2211acda7f6so9991595ad.3 for ; Wed, 26 Feb 2025 20:38:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631095; x=1741235895; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=C/ge7B6UGXcSd47ZURwKYcUmpuFGdMGIsEsj1cDYmyw=; b=G4Ak9tOgc+gDJffm6MnpBhkUSHR7TooIUuAdWCD55w3WtgzkeamnccGJVzj2DHX45U H6l3oaD13feM6mcB7dmytnBqmDRh0WMi3HS/zTtGlb6mGh1yhvrV/Nv6lXDhwmy/dfWG I912CeApNCwtD7ASu2t6LEDT0QTGI+ozfyrqs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631095; x=1741235895; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C/ge7B6UGXcSd47ZURwKYcUmpuFGdMGIsEsj1cDYmyw=; b=EWqaudJ9UmstTFIz06jIwCyarjAHODEZ9NJqn0q3wq70rT2vqWdScI+2Cgfz3Sk19/ jeIqauYmbCQyxGHZF2fe/E37cJc0J24cPax+1wVRkH1FvzASjnMtszMtYRbAoR4cqcVC I82KQqo1zPE74XOCA6Jm8lQPZW8FEtPe8xzsPjLaBp+Fhsm99pDv3qVzR1QBBSPxIaxH yWICT4oO5cAXflE0ktl4BjT5To6d3YDDDqFAcIChzwQbJdY96G0iUfD0Uiil9HaUziFy KtMg6UKwv0myqf9MokD8izQwEipYsaCcWy1ToLtNNxHZG/0i6kqTkS1ahnZLPFqoCW5S OMnw== X-Forwarded-Encrypted: i=1; AJvYcCVtVVcHUCGx9/az7znZm45Z5XTIQY5sQRs2MitZxMIT4SliNF632rC6rpMvMJLLkfGe9Vl88cfdxg==@kvack.org X-Gm-Message-State: AOJu0YzTCGw7nT4SBqcZHM3hl+UabSiGezUAyZIxfDhKJeixCUSmNGw/ 7laE7C/IEqvKdFWPfZxco6cK4tGgFwdQG2cxY87trYN1qmbw/8Sw2eHhNJUnzA== X-Gm-Gg: ASbGncv+BHba3dhI27/KKj++dq3KwUpIlBdfsLxiLl39GbOi5BYJ7lNN8lJki43pAzZ WDh5E83w/bKd9cacISnswWVlAtx1Me9yIL6cMeY4t9SDk8uhAtp1LP4G+agCg7F9LZYZQ7ZVAqs yzjUHf6D2HaohMFklC9dTiS4/vwmzYsVGcmk0sN1USQL6p2wKS30pToDKB1yGRGJifl4rCf9NR+ cfAykudxtMKlTahOcAJvejPVHDXBDE5fkU88i2+Tf2iIBqVLCXOU49AaCkbw3zWrd+l9v+pXhZx 1d9DN18et6oNduKt3gnqJA/sRpSa X-Google-Smtp-Source: AGHT+IEpXBKk19SHbSOiApwPW0gT/JlWHme806+0gknSxKLl1xCL+FuacjL88fnx4daEmXHOHQZArA== X-Received: by 2002:a17:903:41c8:b0:223:47b4:aaf8 with SMTP id d9443c01a7336-22347b4ac28mr36091275ad.52.1740631095639; Wed, 26 Feb 2025 20:38:15 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-223504e2291sm4941155ad.187.2025.02.26.20.38.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:38:15 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 19/19] zram: add might_sleep to zcomp API Date: Thu, 27 Feb 2025 13:35:37 +0900 Message-ID: <20250227043618.88380-20-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: A94531C0009 X-Stat-Signature: amwx5amm5nezh5rpu7etot7qbx6f6jqz X-Rspamd-Server: rspam03 X-HE-Tag: 1740631096-725887 X-HE-Meta: U2FsdGVkX18tCRaVZMbqK62IKuDWBSXsd4qpYIMJKXScyOGzPbuBqDbnmVNdQoeiflII1GFIUdD1YrynuH0+NtnV7DARxQn4hfMAUyXrmKLjv/jJRPnWNNYnB9O9z9SdWDVOpE117tGM0Zd09YTCMLUz1DmafAbSkO0+5AB+D7WmY5gAblvsaJaxh1FFuvBXARJCMy80umfMTP/ZY6me8mEkQCrv+hqU/CCJ/lha622hRoyVXk/1RxJ4HPaEE/XpzLxTp2n9QQlUIqYXIAqtUzELejmxs3y+l8gvx2anVmJ4YPDKYiW84fCYq5iLQXMhfhqLGI7fPiVfpEib2fY0FxI1oGsybDUvM03UgBt6a9ukyvNltuNeZH10/PuYYBSESzw1l4obk5REztwAqCXn1uRJ6h73egqmFqh4/k4s63crvyOSttdsTCjQ+RjdzAx6Zxh0UN9/zV7uVvrSTtuj83ho1lUyJtaRbWpBU1V9fYGWAdIN1XmSNBPgXpRwxF5Vi/AscF2VaBDY8ophquMTeU0OF2VLF08Axn0sFKJ5/YsWLtmmMUUFwUYrt4q8NCOGo3KQTnrwxrQg0wQgNN+1SVh5qMMuzw38AuFTotn8oEEmNrRJhbzxLFHodBteqghDnLuIURon3X2J/LJAFeIcKRftB5NACrxH8mDZvPEQhQkxFAyyIjNC1J34Jw/dw+L1p47f0My8yl6IoEtgEo8Ul/AsiULwWr5zUyJJmdLHdu55hAXJVw3TYaPKfdP34y4pG2qU+K2LJH5ulN1nUy8IlGyGZNv8DeHriNeRhDSOoDRYfk4HO1RTsT/fELjDs6kS/H1CPa7i3VtwB/nQ1/0kfiWTUayyWw7n8UgTZeDmdVw4XLI//gIQV07tcogXLz1bihEffQjT/rsfoGyp7Bjq+DgHQdTlHxt3k/N8OnXfEQsRCCTwq94t1vXU7Iu2Y4Nbh7PO8D3z+77ByyGekNY XHkY2RfC /lMrVFrKfd+TsxJIZL2Lo21QJUmV19hRsvr7mLOpiE3WJmFUALp31slSzdNdBk20Tgbo/oqWNLrTMPt81AuE81x0X4h0b7oyPcSNrYdHyvQr/CJ9vp8HXpVGDiE6SL96qbo4XwS2nT53+0BSToJXQegmSdLlBPR4nuP1ZdcvJ+azRIfb3lcEWbUUXc19+smbV2VuA/5tVALzAbnZ+71FJzp44P2WmtIv0mu4Hiv88MkpMGgRA0clRdqpCoOUpNhfCMULM6qpIAYtmXfZM5p3Th2ZClqXlEtWP+7xxAkeft52CTIt02EEa7d5T3C9oUS/nDEmnA61Hi5jC63E2PSVpoLcvdKIoaD2/TgymVY43I7YVPHw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Explicitly state that zcomp compress/decompress must be called from non-atomic context. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zcomp.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index a1d627054bb1..d26a58c67e95 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -146,6 +146,7 @@ int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm, }; int ret; + might_sleep(); ret = comp->ops->compress(comp->params, &zstrm->ctx, &req); if (!ret) *dst_len = req.dst_len; @@ -162,6 +163,7 @@ int zcomp_decompress(struct zcomp *comp, struct zcomp_strm *zstrm, .dst_len = PAGE_SIZE, }; + might_sleep(); return comp->ops->decompress(comp->params, &zstrm->ctx, &req); }