From patchwork Mon Mar 3 02:03:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998051 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20399C19F32 for ; Mon, 3 Mar 2025 02:24:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A20D86B0085; Sun, 2 Mar 2025 21:24:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9B4096B0088; Sun, 2 Mar 2025 21:24:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7FC9B6B0089; Sun, 2 Mar 2025 21:24:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5A8056B0085 for ; Sun, 2 Mar 2025 21:24:44 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 130461409DC for ; Mon, 3 Mar 2025 02:24:44 +0000 (UTC) X-FDA: 83178646488.07.AAD93ED Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) by imf17.hostedemail.com (Postfix) with ESMTP id 338004000D for ; Mon, 3 Mar 2025 02:24:41 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=hTAuFOMD; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf17.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.44 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968682; a=rsa-sha256; cv=none; b=mtdkZ28xx5tB1BnTCQ837uHaB3ZD5k7r31kb/8C2shBixuX6c5hvq77WZjo9ZZNDzx6wqg 0wOlHzl8YefHNyMoQMl+mfYoONRS8LuRRCgbvZZUn44un/zwkmg+rQ8zsmeStztvgxgmRz KSeTH75NtgIZ0b1fLzF9cJXJyIQ4w1E= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=hTAuFOMD; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf17.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.44 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968682; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=24epfTRD5B0/Gn0qvdHVkfmT+fNP4MZPLg9AakHcZPk=; b=imbbkwMkcHywXht96KoGZDbckcN9BEdjSVRDcUKMD1jX9za1V07hxowwLV1ntoEhJPTkAh +86r+4v3UBJmHbwZGAe1msmkexQMJdprjRf7Sro+CER5IM1QVtu5yuXJVEtqEia7zNDKD4 w1RxbEJFKWUe5bPyXJ3foatqzQsJ1zA= Received: by mail-pj1-f44.google.com with SMTP id 98e67ed59e1d1-2fea795bafeso5767527a91.1 for ; Sun, 02 Mar 2025 18:24:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968681; x=1741573481; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=24epfTRD5B0/Gn0qvdHVkfmT+fNP4MZPLg9AakHcZPk=; b=hTAuFOMDMWWWvpmnQzAENYPzazK9+cx+oTOmlnqhy385kfWx3OHUxhzvqemWE1gDbq ly0KOGnKYYm4roJX7EWULy+/9Yd/TP8QF/UQ66rrVWjxLL1vaxzpfsj2DrTfx15zZxBI sGtuCNc17z6fPDoDaius282/P0T8HEBvHcpes= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968681; x=1741573481; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=24epfTRD5B0/Gn0qvdHVkfmT+fNP4MZPLg9AakHcZPk=; b=gpxVp/mhhv/kDWjoNaBHSeEayUFVcCC/0xsLeMOTfpdNtiSjSWK13k/QPFobTlK2/C WjbOlXKNS0Zm3woKMDLpbfh30SgJuUd2HVQJkHG2VUGgQsdF19ImdtMlNNlaFtdCbPki 1TRji2OaiN/r6MJgosezZQxDdSCyQWyYt2DgxizSEseh1jQf4KjiJiyxkKKw9qIJDkaR Qlm6+jkTkAqNUOiaW1HlzwdthETNfkwx7I/GIc9kUyig+kaZndkWHbSssos3lcF4e7ys MXXtv7ZTB+WxtqEAz8DuzSUFxR1TotAF8tuvA1DbCoc8zNZfuT+Ie1SB+3ZmlqCgYPuP O7Mg== X-Forwarded-Encrypted: i=1; AJvYcCWzF+fyk7U6dQ9PUz3jj9llH0wP6q70auda5ZI5N/sZbLpP6U7/UlDytInYNlB0SGT79xhXuiws+A==@kvack.org X-Gm-Message-State: AOJu0Yw3rsuQDzvYGe6A5VwV58aKYtvREX1rXEv4AOdAYedV2vSWUdzT JayPevte1ol6e4Gvz831tK+dFhRrPMY7mbzj/uGStG5jORVBYQq1XVp7gEkwuw== X-Gm-Gg: ASbGnctrvNghHYyCOF302NbMm3tDy8ACsxD+tFgBS0S9pw18KgybTT0J84vwUXiSRW1 66oCs4Cimq9sGD4lgmCK6YHp2JqIXWiBIMJB7YSoI9y3xtKBY8DtsMUp7kneHtZTKcUngilLOHb dAVAVFVESzzWbWw6okyYeN6Uy9vIxcsDrUlQ6ayN0xTTWVnUUtg94jSFjdwcSY/1oJTkmyFkr70 PjkFfh1oul5OPIpM6HLTG8UfarGZFtwStNZGvZyi5rMDaZeNdZohg3mPsOytMiesnjAQ+ijPoM2 Douq/xtGFzVhM+hGDGEZJoqRPcF2POecxu3xOlBAmvCd/ew= X-Google-Smtp-Source: AGHT+IEzPpFTdNA4qdxJB0dxJj4fKFXJpJ2zIhvE5aIYLQNZyq296Wz9RmH9FDhwIK3/6h7XuPIjIQ== X-Received: by 2002:a17:90b:3884:b0:2fa:2252:f436 with SMTP id 98e67ed59e1d1-2feba5ced0cmr17673056a91.3.1740968680936; Sun, 02 Mar 2025 18:24:40 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-223501d2610sm66715445ad.14.2025.03.02.18.24.38 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:24:40 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 01/19] zram: sleepable entry locking Date: Mon, 3 Mar 2025 11:03:10 +0900 Message-ID: <20250303022425.285971-2-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 338004000D X-Stat-Signature: oxe5hw7odqb49tdndgh8u33kw6cxwkoj X-Rspam-User: X-HE-Tag: 1740968681-893116 X-HE-Meta: U2FsdGVkX19S3EUfDHWZs/CeH//V76+lQmV8ag7S0F+w3cZMR3/H03S6njZY+1s1PqFxaBxlqGSS5TcnyfDjna6i5VBYESLi9tKKzLDuYUXS29HORehB89bw8s0mE7puFCJaz1WOYRra1b1UQscDmzo+mbPGGzqqLw4xn614cm1CkEk+lyE3i+XG2eqb+gkJMePMSvCKDwPAtXTMxlohTQNrCecZ/5Ya1uN7chG8ZYTzdlKlp/Cl0KMZxVfRfuvglhJ1NRw4NteRQGI1sIA9gsrmW5BZWanqEBxVnc9lcdWfIDS/MenGtXscb3W4/aP0bMiuI1woYcUfxEexZ39ZcQolKgW630NjpKfHytX46tP14Q2h54llSpM+Kh+yUryD15x6LuYwYPPB+5bm+h2Fb55mM53GnIwZVS6/PY7cFWbqGQH0kwZqc76kFkqq/2iHwUcf5IEmER8LLZLN5HkRg82gzH6M88ZmB1LLPlcB7u4Haocq+cB+L+Gjoa0tipQVqg056IPlMmLwicj3UdnKY5kiKcvf/TJRIggSRYLZofBj+KGIVvV67v2Lc+7V3RjOVU0qDYsjZGw6KG9pI1XL6KdWGpsl4thD2cEk+QqJOUe1dPJFNI3Mqc5Z1z2Y4nDnCahUTmpakO3rhlMNt3fnSBuz7nMSHaJMWp1Eg+Xswaas/U7wElmxv55HBahKCqToT2PYplDw5TFfbZbp1tgWdUr0NjHCsid+q2EhQL+27TjQHaBOd4E3qAeYg78bcVTTUCD1jpnIR8D3TDcCLQwDiUMfXECUFGdceJPQWFzu17z9I+XVa/2NLwSpIwwh8PIa+F7mgFCrq7EP636P512IpfPIzfL+TQC0aCzu1eO5RHJjk8HhWaZlD0pMOo5yEghLToD7CM7GW68zDYBPveZWcMkF4Cs6OChZB1g4HJ0xIhKhqS5GBDDWeXgQSokc/bjO2sWPGTyyxiQWG5kmY9g ZIy87d2F LTIKF9m0aok3L+qSXwKD7xtv7Jl2pQrSibqxG2emG5GZDP8fnfdsIssi9814J1d+Xb3wDt+ydQrNMH+p9ZHukQ7U/yM1dRTvEARj6h2MTggTARL9gerr6J59oMP4dtoobUs95R0t7878iic/0HxSyvZFHIkma429/Rg2Vo5ivxBBUrBFrlndc+MUhnaoJnMP+mTXCXz9kcHoCiGcusV7s5+j48RUGmeEiAhCe19oX3DURgkZCnvxhYOTjpx75vuggKF+l90XMQDOfcjdF3GuBVAGmjic1oWYXyGEHD4mDk7h58T8EQk2NFgQv5A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Concurrent modifications of meta table entries is now handled by per-entry spin-lock. This has a number of shortcomings. First, this imposes atomic requirements on compression backends. zram can call both zcomp_compress() and zcomp_decompress() under entry spin-lock, which implies that we can use only compression algorithms that don't schedule/sleep/wait during compression and decompression. This, for instance, makes it impossible to use some of the ASYNC compression algorithms (H/W compression, etc.) implementations. Second, this can potentially trigger watchdogs. For example, entry re-compression with secondary algorithms is performed under entry spin-lock. Given that we chain secondary compression algorithms and that some of them can be configured for best compression ratio (and worst compression speed) zram can stay under spin-lock for quite some time. Having a per-entry mutex (or, for instance, a rw-semaphore) significantly increases sizeof() of each entry and hence the meta table. Therefore entry locking returns back to bit locking, as before, however, this time also preempt-rt friendly, because if waits-on-bit instead of spinning-on-bit. Lock owners are also now permitted to schedule, which is a first step on the path of making zram non-atomic. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 54 ++++++++++++++++++++++++++++------- drivers/block/zram/zram_drv.h | 15 ++++++---- 2 files changed, 52 insertions(+), 17 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 9f5020b077c5..70599d41b828 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -58,19 +58,56 @@ static void zram_free_page(struct zram *zram, size_t index); static int zram_read_from_zspool(struct zram *zram, struct page *page, u32 index); -static int zram_slot_trylock(struct zram *zram, u32 index) +#define slot_dep_map(zram, index) (&(zram)->table[(index)].dep_map) + +static void zram_slot_lock_init(struct zram *zram, u32 index) +{ + static struct lock_class_key __key; + + lockdep_init_map(slot_dep_map(zram, index), "zram->table[index].lock", + &__key, 0); +} + +/* + * entry locking rules: + * + * 1) Lock is exclusive + * + * 2) lock() function can sleep waiting for the lock + * + * 3) Lock owner can sleep + * + * 4) Use TRY lock variant when in atomic context + * - must check return value and handle locking failers + */ +static __must_check bool zram_slot_trylock(struct zram *zram, u32 index) { - return spin_trylock(&zram->table[index].lock); + unsigned long *lock = &zram->table[index].flags; + + if (!test_and_set_bit_lock(ZRAM_ENTRY_LOCK, lock)) { + mutex_acquire(slot_dep_map(zram, index), 0, 1, _RET_IP_); + lock_acquired(slot_dep_map(zram, index), _RET_IP_); + return true; + } + + return false; } static void zram_slot_lock(struct zram *zram, u32 index) { - spin_lock(&zram->table[index].lock); + unsigned long *lock = &zram->table[index].flags; + + mutex_acquire(slot_dep_map(zram, index), 0, 0, _RET_IP_); + wait_on_bit_lock(lock, ZRAM_ENTRY_LOCK, TASK_UNINTERRUPTIBLE); + lock_acquired(slot_dep_map(zram, index), _RET_IP_); } static void zram_slot_unlock(struct zram *zram, u32 index) { - spin_unlock(&zram->table[index].lock); + unsigned long *lock = &zram->table[index].flags; + + mutex_release(slot_dep_map(zram, index), _RET_IP_); + clear_and_wake_up_bit(ZRAM_ENTRY_LOCK, lock); } static inline bool init_done(struct zram *zram) @@ -93,7 +130,6 @@ static void zram_set_handle(struct zram *zram, u32 index, unsigned long handle) zram->table[index].handle = handle; } -/* flag operations require table entry bit_spin_lock() being held */ static bool zram_test_flag(struct zram *zram, u32 index, enum zram_pageflags flag) { @@ -1473,15 +1509,11 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize) huge_class_size = zs_huge_class_size(zram->mem_pool); for (index = 0; index < num_pages; index++) - spin_lock_init(&zram->table[index].lock); + zram_slot_lock_init(zram, index); + return true; } -/* - * To protect concurrent access to the same index entry, - * caller should hold this table index entry's bit_spinlock to - * indicate this index entry is accessing. - */ static void zram_free_page(struct zram *zram, size_t index) { unsigned long handle; diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index db78d7c01b9a..c804f78a7fa8 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -28,7 +28,6 @@ #define ZRAM_SECTOR_PER_LOGICAL_BLOCK \ (1 << (ZRAM_LOGICAL_BLOCK_SHIFT - SECTOR_SHIFT)) - /* * ZRAM is mainly used for memory efficiency so we want to keep memory * footprint small and thus squeeze size and zram pageflags into a flags @@ -46,6 +45,7 @@ /* Flags for zram pages (table[page_no].flags) */ enum zram_pageflags { ZRAM_SAME = ZRAM_FLAG_SHIFT, /* Page consists the same element */ + ZRAM_ENTRY_LOCK, /* entry access lock bit */ ZRAM_WB, /* page is stored on backing_device */ ZRAM_PP_SLOT, /* Selected for post-processing */ ZRAM_HUGE, /* Incompressible page */ @@ -58,16 +58,19 @@ enum zram_pageflags { __NR_ZRAM_PAGEFLAGS, }; -/*-- Data structures */ - -/* Allocated for each disk page */ +/* + * Allocated for each disk page. We use bit-lock (ZRAM_ENTRY_LOCK bit + * of flags) to save memory. There can be plenty of entries and standard + * locking primitives (e.g. mutex) will significantly increase sizeof() + * of each entry and hence of the meta table. + */ struct zram_table_entry { unsigned long handle; - unsigned int flags; - spinlock_t lock; + unsigned long flags; #ifdef CONFIG_ZRAM_TRACK_ENTRY_ACTIME ktime_t ac_time; #endif + struct lockdep_map dep_map; }; struct zram_stats { From patchwork Mon Mar 3 02:03:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998052 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC655C19F32 for ; Mon, 3 Mar 2025 02:24:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 16CC6280002; Sun, 2 Mar 2025 21:24:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0F633280001; Sun, 2 Mar 2025 21:24:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E8CFF280002; Sun, 2 Mar 2025 21:24:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C5AE2280001 for ; Sun, 2 Mar 2025 21:24:48 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 7880DA30D0 for ; Mon, 3 Mar 2025 02:24:48 +0000 (UTC) X-FDA: 83178646656.30.04CD52F Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf03.hostedemail.com (Postfix) with ESMTP id 9872B20002 for ; Mon, 3 Mar 2025 02:24:46 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=OFssOlcv; spf=pass (imf03.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.171 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968686; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jDqZrH9W9+qGmX1OXU2Vyl3xsabTv9L/+SLAveE4fcw=; b=TQ/0zQeMUzmwZFXzZPVIBc5C3ixzg8qbpZ41K/5Q3eLeSmsDXz6pB9MfeyKeT4Gze/29BY oIix1cyjmISHJMJ9xkFcCfQ+IU/RrXa7+Iqznccu5egYWBNqp4La4uEE2AmOgQaHGWTopa eBxPH/LsgDRcx2nNy3bl2j1yhdc3sXE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968686; a=rsa-sha256; cv=none; b=5dt00ZBQkv422l0+1tFbqtHfzfuyJ/BsIjNrjedMps78JKYQ6ADuna3ZdUkTG3MRaX7m0s Ox1vQopHRqIb7vOZ6WpmUaE3IDbrMKMwwBRkcqgHXjx2BovKW94c5/feYlTF7iNuspzinI 12SFmQs2AZ9vFh5hja7QhNZCRn7iLT4= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=OFssOlcv; spf=pass (imf03.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.171 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-223a3c035c9so8993335ad.1 for ; Sun, 02 Mar 2025 18:24:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968685; x=1741573485; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jDqZrH9W9+qGmX1OXU2Vyl3xsabTv9L/+SLAveE4fcw=; b=OFssOlcv3N+QXMqZSJpZO5KYiQT7GQkrlPJ64RrN6IpfO94lSM8Y3p1xOqx3Q8QdzM BrNuaej5ZRdRsxHXyyx2AHzRcX8eSbxr9k3PldE7sM7vQyVX7n+GseYJx8z2MyF9Rhhe RT/Bq8Y5xzlWFOarhXPi1XqvPxS1KpfX26t1U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968685; x=1741573485; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jDqZrH9W9+qGmX1OXU2Vyl3xsabTv9L/+SLAveE4fcw=; b=Z41ctU0jLsdzsYuixvDWEeAKE2VDxaIYz1j3GRNfi8CdMiYFbK5HQLEgqBNGmBV72W KIUdXHt1VyVockhghgUtgsFa4uB3g2BBgVHxSmiNy9AHP98OxHBoVUhyCsm6anrRDXmi 7MuaWR+2pPndI2MqBpII1CMNXaX7ATqmBuZpi4zXoJdQm7d8es2fbv7TtlaO51Lcup0b oTZiWUamZ9Un9Aj6f90d8mMOfMGkhS/BciuvgKzas55/RwzVWXzDXhyl+263jZ1+4xFt EMeEuT2It5J/yRWr6QQLlu2n0BXgQggs1mHJ8EZ06Xy+koEWzVByk7ZZ2crMwSZyZ9Mi FQ/Q== X-Forwarded-Encrypted: i=1; AJvYcCVl7ZqRI/wNUO8rhfvjKO5HmlRGSe577FczZUfCMBHsFTSbVI16ehU8moctprmTX4aqAjewYslz6w==@kvack.org X-Gm-Message-State: AOJu0YxNyZxKQ3hRi4N3a4bwoIgrrBAoyqXDKRaOrWFSXuNSkGR51WEN H0f2o7SNLuM+N7odHvFW/VGAakwtD64TOeD5u+PweA4fnRp0Y6S1T3wirCs0+g== X-Gm-Gg: ASbGnctypYinK3kL1aiGf3HzYAe9VyiMvsyyuJeGGQX+zIYTDAlRCToFVMOWLp6Tmyy 54C+zNTWFS6DL7xAkwKCLsH46YSUyRJcPVuAR+7nvJNxaNDeAKIXm1pXLjr+nA/3sq+OLrPc8Zw imfSjXNHZlFxSE0Zg/SzOm5kx/GByy8sMjlr713VckUOcHHmdCQbgbpYy1M5bOYB7dzlA3gqtlC OxZZWcgxjOvvZHjFEZ0dVnJ8mOI86PSo7e+B92hOJG1hpx3DIz9z23SiEnkeFTOs3CSXyTB2UXW QMNs5eqCeMwZnsqxxEsL6KByyQ6VEXn23GTkJVGHsN9fAGo= X-Google-Smtp-Source: AGHT+IGEtduyJTi51HPkBHttjdqcJfHiW6RA1OErSt85BYtPP72gIAiy5XzU4m9bzS3f4VCOBOOEpw== X-Received: by 2002:a17:903:2986:b0:21f:6546:9adc with SMTP id d9443c01a7336-2234a38cd53mr253511975ad.13.1740968685435; Sun, 02 Mar 2025 18:24:45 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-223501d5247sm66569735ad.26.2025.03.02.18.24.43 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:24:45 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 02/19] zram: permit preemption with active compression stream Date: Mon, 3 Mar 2025 11:03:11 +0900 Message-ID: <20250303022425.285971-3-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Stat-Signature: u3se4gw5jw4j7ipngzbtoyoqfcgb5bs8 X-Rspamd-Queue-Id: 9872B20002 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1740968686-702130 X-HE-Meta: U2FsdGVkX1+nRstfqQKkMJ5JSnqo5D5S7ZVo7eohwq5sjXXUolN4cqvmI90kzKBz7aYHJuR9PL7Auv7kDuoZFgAv711uViMV657qMyN7MEwpPGLDxcTh5bTk2EnbEO4ytkA5Sqtdwy3Li+9P/TYx26P30QsLiYeKwnMGXHJYjb463ApoOnydWNSURVut+cAtbebayxMu5wGp6BEqx7aSwnpqohWlt9TknYGNRw0VavF67SO4LmJI9erRZvQ6G4+i1AqMnPGFx2LBZEvHUtc6gGLEwBnmIJlktKPrcWCr0ut4ds8NlzfftDjSHirSXgltcmIXHltXm2Q2mWi/TnCNWJ43c/720MJUS8AAhwyXfW9KHkQlM7BkgMZ4xzeZ8HfWKzcBVYROG9LSm40D0sXvdzWH/KR8H3+qkY+iciow/Z+5NkLai2Qg3ZfuIKXP02nYZAa7UBNW3aPn0DeTJFgn/bSsFXnoFnaglX4CIFbcxsBJnKCXrN0k1X2J/OOsNKM3qLI3mwzpoZnb7/vxbgG3NheHjx19alxrlg2zHfJfSRuMFiBfgVBZ6GimdGdxJS8j3SCPu7wZXm2cwHfIRhwAmfZyIc4UFxTrlF3o8fAYrxC/46HKz9bQi0gVK6CQV5YXX4mJN/fJ6Pu5K3ofUkC6m+zzJWWK5Q96hveTMA0xuaCr65deqQZidqkPb2Kko6zGBvxX9HxeTzIzbVp9EQZjQKUJsw785mtSAmZBKL7c0u8KkkQdsep616pOHixqDN/1rX5J8DTQ+Y8iPXJG9cNG29GJqvEmqUbYLDSJJgW1CXROR2FKmkc62utkadrTwY1NRR077xeXonCcdPKNAvOzb4tUr/SinWt0g83LfZC46LFNIuqQoSX53Rwt1MuhRRNMa/NA0eSDuPUEaXz5f4XJAKNgNpuqzzzvoqBowsLz2qAnNPfiRXjLvbAZO4VKjJ735FwDnddywAbWMN2Z4IL dRcLrPjf vCXuFakQKfC9cHQxdZTnC7dygiIzht3nTIA4TLzBxqaPmQAxCy7WKwVE1dPQqAyvBeuagvRq7v2pOdpkhX7tVQVIqyJWjPe+hL54y2ahbAKpbQ6LLyTuZF7we6Jg5dfYpghEZsiRRhF0r8f1ZRGBSKuMRgiQREnFxSCyjoGfQOotKa6XFyIDtBkszV9SVtn2v6g/Xa3hoKyn/TIZ6fNjyuyH1fOkUcoTYJTWE9PuKRTesNWKcDfSsYJnZRsqLk1uUo1qCwyMQSxjt6Ojea+8ymZ6gTezE/nlO32WFOabQ3ZObsOVkC4wmjCwLDg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, per-CPU stream access is done from a non-preemptible (atomic) section, which imposes the same atomicity requirements on compression backends as entry spin-lock, and makes it impossible to use algorithms that can schedule/wait/sleep during compression and decompression. Switch to preemptible per-CPU model, similar to the one used in zswap. Instead of a per-CPU local lock, each stream carries a mutex which is locked throughout entire time zram uses it for compression or decompression, so that cpu-dead event waits for zram to stop using a particular per-CPU stream and release it. Suggested-by: Yosry Ahmed Signed-off-by: Sergey Senozhatsky Reviewed-by: Yosry Ahmed --- drivers/block/zram/zcomp.c | 41 +++++++++++++++++++++++++---------- drivers/block/zram/zcomp.h | 6 ++--- drivers/block/zram/zram_drv.c | 20 ++++++++--------- 3 files changed, 42 insertions(+), 25 deletions(-) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index bb514403e305..53e4c37441be 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -6,7 +6,7 @@ #include #include #include -#include +#include #include #include @@ -109,13 +109,29 @@ ssize_t zcomp_available_show(const char *comp, char *buf) struct zcomp_strm *zcomp_stream_get(struct zcomp *comp) { - local_lock(&comp->stream->lock); - return this_cpu_ptr(comp->stream); + for (;;) { + struct zcomp_strm *zstrm = raw_cpu_ptr(comp->stream); + + /* + * Inspired by zswap + * + * stream is returned with ->mutex locked which prevents + * cpu_dead() from releasing this stream under us, however + * there is still a race window between raw_cpu_ptr() and + * mutex_lock(), during which we could have been migrated + * from a CPU that has already destroyed its stream. If + * so then unlock and re-try on the current CPU. + */ + mutex_lock(&zstrm->lock); + if (likely(zstrm->buffer)) + return zstrm; + mutex_unlock(&zstrm->lock); + } } -void zcomp_stream_put(struct zcomp *comp) +void zcomp_stream_put(struct zcomp_strm *zstrm) { - local_unlock(&comp->stream->lock); + mutex_unlock(&zstrm->lock); } int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm, @@ -151,12 +167,9 @@ int zcomp_decompress(struct zcomp *comp, struct zcomp_strm *zstrm, int zcomp_cpu_up_prepare(unsigned int cpu, struct hlist_node *node) { struct zcomp *comp = hlist_entry(node, struct zcomp, node); - struct zcomp_strm *zstrm; + struct zcomp_strm *zstrm = per_cpu_ptr(comp->stream, cpu); int ret; - zstrm = per_cpu_ptr(comp->stream, cpu); - local_lock_init(&zstrm->lock); - ret = zcomp_strm_init(comp, zstrm); if (ret) pr_err("Can't allocate a compression stream\n"); @@ -166,16 +179,17 @@ int zcomp_cpu_up_prepare(unsigned int cpu, struct hlist_node *node) int zcomp_cpu_dead(unsigned int cpu, struct hlist_node *node) { struct zcomp *comp = hlist_entry(node, struct zcomp, node); - struct zcomp_strm *zstrm; + struct zcomp_strm *zstrm = per_cpu_ptr(comp->stream, cpu); - zstrm = per_cpu_ptr(comp->stream, cpu); + mutex_lock(&zstrm->lock); zcomp_strm_free(comp, zstrm); + mutex_unlock(&zstrm->lock); return 0; } static int zcomp_init(struct zcomp *comp, struct zcomp_params *params) { - int ret; + int ret, cpu; comp->stream = alloc_percpu(struct zcomp_strm); if (!comp->stream) @@ -186,6 +200,9 @@ static int zcomp_init(struct zcomp *comp, struct zcomp_params *params) if (ret) goto cleanup; + for_each_possible_cpu(cpu) + mutex_init(&per_cpu_ptr(comp->stream, cpu)->lock); + ret = cpuhp_state_add_instance(CPUHP_ZCOMP_PREPARE, &comp->node); if (ret < 0) goto cleanup; diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h index ad5762813842..23b8236b9090 100644 --- a/drivers/block/zram/zcomp.h +++ b/drivers/block/zram/zcomp.h @@ -3,7 +3,7 @@ #ifndef _ZCOMP_H_ #define _ZCOMP_H_ -#include +#include #define ZCOMP_PARAM_NO_LEVEL INT_MIN @@ -31,7 +31,7 @@ struct zcomp_ctx { }; struct zcomp_strm { - local_lock_t lock; + struct mutex lock; /* compression buffer */ void *buffer; struct zcomp_ctx ctx; @@ -77,7 +77,7 @@ struct zcomp *zcomp_create(const char *alg, struct zcomp_params *params); void zcomp_destroy(struct zcomp *comp); struct zcomp_strm *zcomp_stream_get(struct zcomp *comp); -void zcomp_stream_put(struct zcomp *comp); +void zcomp_stream_put(struct zcomp_strm *zstrm); int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm, const void *src, unsigned int *dst_len); diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 70599d41b828..dd669d48ae6f 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1607,7 +1607,7 @@ static int read_compressed_page(struct zram *zram, struct page *page, u32 index) ret = zcomp_decompress(zram->comps[prio], zstrm, src, size, dst); kunmap_local(dst); zs_unmap_object(zram->mem_pool, handle); - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zstrm); return ret; } @@ -1768,14 +1768,14 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) kunmap_local(mem); if (unlikely(ret)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zstrm); pr_err("Compression failed! err=%d\n", ret); zs_free(zram->mem_pool, handle); return ret; } if (comp_len >= huge_class_size) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zstrm); return write_incompressible_page(zram, page, index); } @@ -1799,7 +1799,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zstrm); atomic64_inc(&zram->stats.writestall); handle = zs_malloc(zram->mem_pool, comp_len, GFP_NOIO | __GFP_HIGHMEM | @@ -1811,7 +1811,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) } if (!zram_can_store_page(zram)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zstrm); zs_free(zram->mem_pool, handle); return -ENOMEM; } @@ -1819,7 +1819,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); memcpy(dst, zstrm->buffer, comp_len); - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zstrm); zs_unmap_object(zram->mem_pool, handle); zram_slot_lock(zram, index); @@ -1978,7 +1978,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, kunmap_local(src); if (ret) { - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zstrm); return ret; } @@ -1988,7 +1988,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, /* Continue until we make progress */ if (class_index_new >= class_index_old || (threshold && comp_len_new >= threshold)) { - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zstrm); continue; } @@ -2046,13 +2046,13 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zstrm); return PTR_ERR((void *)handle_new); } dst = zs_map_object(zram->mem_pool, handle_new, ZS_MM_WO); memcpy(dst, zstrm->buffer, comp_len_new); - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zstrm); zs_unmap_object(zram->mem_pool, handle_new); From patchwork Mon Mar 3 02:03:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998053 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6A84C19F32 for ; Mon, 3 Mar 2025 02:24:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 565C6280003; Sun, 2 Mar 2025 21:24:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4EDC6280001; Sun, 2 Mar 2025 21:24:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 36799280003; Sun, 2 Mar 2025 21:24:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 11ED0280001 for ; Sun, 2 Mar 2025 21:24:53 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id B6D3D120796 for ; Mon, 3 Mar 2025 02:24:52 +0000 (UTC) X-FDA: 83178646824.19.0224142 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf07.hostedemail.com (Postfix) with ESMTP id D12B040006 for ; Mon, 3 Mar 2025 02:24:50 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=gmKN9QqU; spf=pass (imf07.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968690; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pZsh5cPaFoEDPyEJH9aQiY1L0urDRF/CUPRrOejpG5k=; b=koDpEjh1unMboHjAx0QWD+XgJhzmaE2xY9/3jZFqI3NTCsZHWE9j3yr6eaIgKq2qAN9vJP NB9FKMYWQ5Nd7K5j5sKj3YIG/uu5cUPVckjRTgItZpKgeyXis3Z1x3+Xnl/HNzGMHW9ynX +CfKezKCGtzaqB4SGcaLnWYJK4JeFrU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968690; a=rsa-sha256; cv=none; b=qhFwLZ88ppXHERY3sisR3fByDkgP6Jz+v3Ndn+BK7z0MdG4pufoqXKPyTMv4/9+kk5lBS6 TbtvWMKA9Ux883HpMzwU6YZi7bYMXUWnZcyKP05xRJftvd8E3ZnuW1OYcgUR5o65/ZzrT8 ztPUc4pDELxbDsDe6Ku1tSMgtMcKoyU= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=gmKN9QqU; spf=pass (imf07.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-223959039f4so17314835ad.3 for ; Sun, 02 Mar 2025 18:24:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968690; x=1741573490; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pZsh5cPaFoEDPyEJH9aQiY1L0urDRF/CUPRrOejpG5k=; b=gmKN9QqUyhgtqdbvsCYdtzDaKXUFNe3CrJcSBDtQeS4Lml29+5l5ArvkHlZMKHikLg FHCGJzbKXF6iMv8+PlC6f4oLcsYsEqz3k/gx6aT2PVp2sTmpMy7hzD3pl2aaXlMZNZDW +TrZzVQCNJjHhAUE/kKX59R2OHGpli/FDzY9I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968690; x=1741573490; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pZsh5cPaFoEDPyEJH9aQiY1L0urDRF/CUPRrOejpG5k=; b=sAtepp1KAsOfg57GCBE1z7XcAnZoCHYEsUjFoJoQFQMPun6RLzmBrdAqFLDEyPepWs f7n5w9TX9DOPc68wMoYpEz7bhf6v3ZcweFuKJCuety0YCtptAGu06bX/PAFRGCd0RWWb Y+Ool9rVkajfkX8EeSwZrVoIXsBSQLMbpcBZnysXoAp9MLBT8PN2bT/itkvNOj0n5u7q aiB+McNlOBy/AGyd7c546diUJZyMHfjzcwWIMkDN3C5myfnSFVIdAwrFzd131NF+XgKQ jvRnF1y+dUJBsGxtzshUR7l4Oq4201A4+/o8S0ba02xwmWWPztm6lbVTYtEYHE2kK4FL 7OrA== X-Forwarded-Encrypted: i=1; AJvYcCUEaVVXntuoN1LtoVC2FRiW90y2e9O2ynLg4+Au52h1LYOE5kYvcgnTkCexuM+h+NytNJUK4QbfXw==@kvack.org X-Gm-Message-State: AOJu0Yw6TA+h6z5lTlEWGuaPgLMkwSUQT5ph7vcoSh40uVkTlK9YFR1i w6/d2W356tCBwR1itEISsWT9TIj1LUBR+FMQtPdEOYfP9KTuEgXv7diPjcnrGQ== X-Gm-Gg: ASbGncshQDAw+MOzUXSwQxfPGmCzKg41waRL3cCyaqZ0oiCBjXhaaUlRelWXYv9h3px aVHa7wqlYtK7yNT5yU17Kbegw9E83nP48afObT0Hm76yvpTWVPcjsJnvumBxA4Io34k3jX9QB5K Shnof5fqJP93cvECo63bzkLVf1G31PapkHZoYCOlZlnGL0Z1z0i5RyDFQkOfwrl8Il31DdXsMJR nNU8vxbjjg4/di/D9NqF1GYdU25eQJO5tR09tYW9xrG1itgi6z28Xb7fDIcEc6OuU1XppGeCtYu nHJpg2CPIGu/YciN5R2T0lHNiDTr0aWWOhf1Ylt9IfddGJo= X-Google-Smtp-Source: AGHT+IFRsspYWzVXCyDoL5ZFIh1SjDZmc0euiZ+Gem/pRrrBGftzVfrbZvTQsBlZb+pBOhlTra4AuQ== X-Received: by 2002:a05:6a20:7289:b0:1ee:6ec3:e82e with SMTP id adf61e73a8af0-1f2f4e3cc66mr16110519637.29.1740968689829; Sun, 02 Mar 2025 18:24:49 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id 41be03b00d2f7-af221346042sm3142765a12.28.2025.03.02.18.24.47 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:24:49 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 03/19] zram: remove unused crypto include Date: Mon, 3 Mar 2025 11:03:12 +0900 Message-ID: <20250303022425.285971-4-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Stat-Signature: 58i8mg9ikhzy3z1hobj8yf81zysq4mbm X-Rspamd-Queue-Id: D12B040006 X-Rspam-User: X-HE-Tag: 1740968690-201910 X-HE-Meta: U2FsdGVkX1+LoFfkS3iHhHwBNjbzQQTU70/1Pm6iIzwaDqyZYh+GcszQjeYF/enUWbJ4+tuqcMwkFpDp8n6eLRihY6OzwkC103ZdhrF5EjyYGWrHPaJiGHGlPSg/hmO0SIjv+wYWEQ8Pa1GHWxHOXyhxcYfTdwp1OD9dDMxxRnnT5cV+pBvzQdDEFd7KSj8AtGIopzTKECek5l9t+EXe/NtvzIHmh/6I4ntTof/l/qK9kIK+6cDOYHQRv6myLCORXlTG7nva7XTLvsqA7MwMhUzpOnEMTSg/MUkoHLZH/FXfovZFAMdJ7nQqiH1D2e5grMGUtaDxt7kHe84/IbUSJXgw0te7k2DbgxvAfSxFy2XF/1tCzIduoo1X9EgwvUmtYhHwafE9qeJDUImtHJgBsJgxWqvW9wYzQePEcmP3ge+zSogGaLbrn5/1dL7AP5u8wDQFbCdTY5ROfVeZLGMhLsFyRCzpicytMMKcvUPVXbIKd7h0vZUXIl4FgBgfva6TaWU2jvCyl1f6P3zt4BT6FuCzH6aDcnWzsEUbndQTcEptSU+Kkx7+Un3S247zAS0ZpO4TKjfuCmUCChPsEnznlY/EV5SWaJN113gM4Rju2yf5Vix8lATzX/gMweRnzTtl4OBOTf7C0kQgpxfHCZ2K0InMy3ho6oKnJG6Ogy4anjxZKvPi2WzyvHkWTTMbEMkhImco2wpruC1eiaLkJ0eGssS8N08P3dU3++PUZx/EV7mYH9Mf0Qs2ZhNwXnKeIWrkySgJx3Jb1ahy+Uu4JRrVbECwiRMlkoAseqSgWkZ+JPOoLGIFwAxeUz+SpljPkNIT0JTQj124xeHeO0pjyOf31Qz5ELievnS3nd5t8x67CAgRv5i/ULEMsAGLvp1c21zAo662tG4YhMD0a5Fui3Le3WmSpmWJGyKsEC7riHTbvdK0BXv7X9biN4m+WzZgnDma7uZAXsbXz84hahu+yDG GQZ3hW/0 ANxuOAyR62ChwZGestckCRcUPtHAOF7qhpVjxJd5ts84RwK/ZUlme4nve6p9SUiPfsJjKkQRK7KDBFsTnzz8prK8HFn6PSLk1zi7r+/00sEIMrG/5hiMnpB0v4GGQnsLjmXGmftNKLgrc27n/09TDloC6hFukmM3j4obHpYDJRh/zQZGdR0hqm+g3sKFvLSI2+r8TcNQ2OmxOKShDIDiCH2fHpwwB8f+7Dh7qsnXWN/iMkwDGL0CDUYaX5oBu7iGygXeepbFcT/4bMfbKmjC7lytHupSUZ38RJbw17uMzOpPljO3EKj/En61oxQHeql47Yop+qQe49zPEAyuIwoDVpkorp5EP/irQ6/eA X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We stopped using crypto API (for the time being), so remove its include and replace CRYPTO_MAX_ALG_NAME with a local define. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zcomp.c | 1 - drivers/block/zram/zram_drv.c | 4 +++- drivers/block/zram/zram_drv.h | 1 - 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index 53e4c37441be..cfdde2e0748a 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -7,7 +7,6 @@ #include #include #include -#include #include #include "zcomp.h" diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index dd669d48ae6f..248dab7cc7f4 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -44,6 +44,8 @@ static DEFINE_MUTEX(zram_index_mutex); static int zram_major; static const char *default_compressor = CONFIG_ZRAM_DEF_COMP; +#define ZRAM_MAX_ALGO_NAME_SZ 128 + /* Module params (documentation at end) */ static unsigned int num_devices = 1; /* @@ -1148,7 +1150,7 @@ static int __comp_algorithm_store(struct zram *zram, u32 prio, const char *buf) size_t sz; sz = strlen(buf); - if (sz >= CRYPTO_MAX_ALG_NAME) + if (sz >= ZRAM_MAX_ALGO_NAME_SZ) return -E2BIG; compressor = kstrdup(buf, GFP_KERNEL); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index c804f78a7fa8..7c11f9dab335 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -17,7 +17,6 @@ #include #include -#include #include "zcomp.h" From patchwork Mon Mar 3 02:03:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998054 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 674C4C19F32 for ; Mon, 3 Mar 2025 02:24:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EDC30280004; Sun, 2 Mar 2025 21:24:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E62EB280001; Sun, 2 Mar 2025 21:24:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB6DE280004; Sun, 2 Mar 2025 21:24:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id AA039280001 for ; Sun, 2 Mar 2025 21:24:57 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 545121409E0 for ; Mon, 3 Mar 2025 02:24:57 +0000 (UTC) X-FDA: 83178647034.25.DA2067E Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf29.hostedemail.com (Postfix) with ESMTP id 79181120006 for ; Mon, 3 Mar 2025 02:24:55 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="nvgjww/Z"; spf=pass (imf29.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.176 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968695; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bWS/5MPcWfCH0A9JfQHKT/0nLElRz817vLkO7NmScms=; b=xKY08HG558yBY0/5u39pkqwwFN994haITqBsxWPcMvtTFBQuvTOLsu9eaXaqe/F8ynCwXc yFvvCXMH+8QeqzmBCNPQBFHd4Deew7hNbCozGt7XHejqPcmN98FbX88CxJcuowBSU11JPt rC+35MTEDR2q/flRK25Sj4DOD4Oe6HQ= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="nvgjww/Z"; spf=pass (imf29.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.176 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968695; a=rsa-sha256; cv=none; b=s4njq9zBpLuLNMqOTZn6RZcb/hoRZP8DCF8RTIFrvsXULdk4axcpHrd0hWXWVpY8YaOugI HLet1dURC76ZMTMVZdcy7ftV2xXGeF+ltIOEjnVmjK0iJHrLsjthcq9Ir7pa2f0SGZkK4X i5Kdkj+NuIY9nB9kydEpixoT2kAhlOE= Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-2239f8646f6so14277195ad.2 for ; Sun, 02 Mar 2025 18:24:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968694; x=1741573494; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bWS/5MPcWfCH0A9JfQHKT/0nLElRz817vLkO7NmScms=; b=nvgjww/Z8VguBlP9ApI71UkNAJ+nwfvGvJ5oD+twqOKsj++I8PnTCcEBifQ7qtwgDI 9hSLLdJ146WSSNPBeaBCHRJl767eQNUmt2ZwLlNqF+IVNZK9jTZ0aNnnWfdptKZPqqn5 R5FEPpiKZNyT/J7nF4BT6k/szfuvRotqJfkqI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968694; x=1741573494; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bWS/5MPcWfCH0A9JfQHKT/0nLElRz817vLkO7NmScms=; b=LKBskvYI+vhyTB1BuxSeA3HEGKHHYfNPEal1fxQbZ2nPFoJVya0KG0WnC8GFb3aogd gFv/IyWUyIcSt+8kr0Jz4zdWzKDLoefeRlu2BfKNWlxZtGqYIpDC/lbmAUcbQU1+WvFR CsJqmCONFxKx75AAWtQu7b4HxcgN8HYQ3rrejzhZdfC4N82qm8dBrSZeYJL0uhwoZ3JX BsY/8++E1eqpGGwvj4fF40iF5B2kAjK9g58Mv9YRUZyZOAmplZUuoHh+TDBlUkuuyNrX UPoUrxfhkv5nxYE1prGt8GgAL5LeQLgVo149xqtS/uDgvuI139FxUXN0dX02eEP8Q7Nt dDAg== X-Forwarded-Encrypted: i=1; AJvYcCXx6yhl2fJwcbn1nfWVBzdoKcFpjHj/m6DDmzlnAMcnYVQWEeE1ByI2JwdUjagxismfk6TqfIHkJg==@kvack.org X-Gm-Message-State: AOJu0YxU2J5e/BRjcwqyY2praoUtTW2rlVa0yNNlncr8zoLBTaMOvQiZ 1jNwoiSkMN0QXtPanTc+/MK4KUbKElmEXgiiVmKxLbjlaNfwVUGqDP6e9vzSMg== X-Gm-Gg: ASbGncvOeiyfCkiwKyuHAkRqE8E/btBlY/5PqmANPtBlRrWdLqjdWh3kTdXmPkhKnzc wAxfNOvE8JoB7bzrYE+IkhI4ACx484GnyV7qcgGsl+wxWC30qC2YNC2mguk4ceaOMOmGX07MYVY 7BcajT6AK+foUrx7USosJjVtUQiiKEz+MqaC/iDSGJSDh0aq9ZNHKI7ndmHMEG4ZMH3BtwM/Hds w5fwpcRJ1OK8JCapXVX7LzOzwBpWF8gVRkPLi3wuhN41rwO8AnnJppnvgQdxULYL0jnVf9u5dcG Sa9hodcznSm284HLySS8DKOkf43v7ocMaxhLEReICznutg8= X-Google-Smtp-Source: AGHT+IHSDPg6+GlHfcOXSQ76GatOFpUQPAH1lRliAnvxfV2K3YTVpkBHpg9ehwbe5zk/2PLiRfIsOg== X-Received: by 2002:a17:902:e807:b0:223:6657:5003 with SMTP id d9443c01a7336-223691f8f1cmr201725005ad.32.1740968694360; Sun, 02 Mar 2025 18:24:54 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-223504c5cc6sm67129725ad.112.2025.03.02.18.24.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:24:54 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 04/19] zram: remove max_comp_streams device attr Date: Mon, 3 Mar 2025 11:03:13 +0900 Message-ID: <20250303022425.285971-5-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Stat-Signature: 6h44eq8arsn987dnjwd8bybhhkx8qh3r X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 79181120006 X-Rspam-User: X-HE-Tag: 1740968695-275964 X-HE-Meta: U2FsdGVkX1+DdUCJWa/LM4KfhBgMqUBXBVqJqn4xFBvpxrrtmpdIkJ1icAiz9Ll7en2Wn+mQyUHV+q60wVwJuixw/kZMXekJiz1w5ReNhdTIrGe0L4awT3jVRrHwkwTEBt01W94LJ2MD7XJcQm/S7/W1FkUZbX+mMh3sb6UWKjBVEQVvp4LbdPLwEGKl2wiSHUYmvvd4K4+LPepzu/sAN4pdLK8fy+ckZvlSz17a/j2WuYTTL8nqummXrqdF1C6izpUY8BYczfKf2V0lG6a35FIpxpDGED9rAqVm7wevgZXm95O6jjp6LzH1LIAVCzJe4Ys+WFU3Cb5JlDAfr7sGTYST54QsI6sC9OtzWRDd1cxU9JhIZuEbapFKAXQuLQzZA/rOium2QJ3wGMzIptGmExRoyQhem9Vjqdl5zUA3KedBcdCbShXh4Xr6XVZCK+Jmj69y8Pk0/ktY52VvgcVbpk4Mul2BwSlhafyfkcJ8HpRWx/vFkVrUVCBRcIHaheSVPXBuD4Z7NSCTnBf7RtNiT30wWZr1gZDSyBbbkhBC4N+94bvQeNromKU2wBsPsomXVCWa+FdrCMENfRDNHdKnuYMSRJunCBM7E7ZkPEtuwZCkr6mPjXfPYfmWLsPhNZkLvmIHPBKartKMCbsyqldgD7aq6KCUOn3WIo6Bxc3byOArXhDPYTMqb4mNfaq6SNUziPyeQMBxAiOgKwBWF0CQzAel54wKgb5VgWeGe/ZYmmvU1larz6pv2MFEjytGrXE4ELUgPi3scpis2/NADhyhGC+Q8HWeHY29bPLGYzOtyv//N7qon6JJC0ecThu6tZ+8eiVZBTDD+WRnA7CPBA55aSqnAOdZmZmhrlQL/1x2uXVJH26Lhf8I8veooKPCd5d5X5cHdkPeW1eY6/YVMwFDYJdL2Pjcu/1fzs+kJjnivm66KXrzjmjhL1A7LuC0sqmq4zMkxpRbJAGvG+vXqLd GhU3fYCw pFQ440u5OII3nMnFgWsHtoLiRShndpvcQVNoG4ta00CBE9GuPwhpY9bFCGDlTUR5PSXS7L0aC2PT8/i2xa2JAKuaq4MVp9KS5q2h93LAZSBnEtb8wqoLrQiJOLU9ZTMe2y9CBXExhVMAAiRJt+TV1DYHTJtSZ198vs66BHUK20XphBkbRgkxBsjwP2PgN0WLKG7ue5QQ7cELWmqHe39f4OcpeR1e1ZSTXPOazXVsNFK5B2/X6NWkBn30LmFIfPh5uYJOl83Xa5cIAj/GFqYyNHaafxzc6NU3p4mUSzvUhqaU4duL5Juc4WJkOVSEs0ohCw6rnU/SzedghkJ9x0TD1Glc1BUNHNB/J99RegUFy0qSQUHBkoVpXkC3e0CyPD/b2xBXV X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: max_comp_streams device attribute has been defunct since May 2016 when zram switched to per-CPU compression streams, remove it. Signed-off-by: Sergey Senozhatsky --- Documentation/ABI/testing/sysfs-block-zram | 8 ----- Documentation/admin-guide/blockdev/zram.rst | 36 ++++++--------------- drivers/block/zram/zram_drv.c | 23 ------------- 3 files changed, 10 insertions(+), 57 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-block-zram b/Documentation/ABI/testing/sysfs-block-zram index 1ef69e0271f9..36c57de0a10a 100644 --- a/Documentation/ABI/testing/sysfs-block-zram +++ b/Documentation/ABI/testing/sysfs-block-zram @@ -22,14 +22,6 @@ Description: device. The reset operation frees all the memory associated with this device. -What: /sys/block/zram/max_comp_streams -Date: February 2014 -Contact: Sergey Senozhatsky -Description: - The max_comp_streams file is read-write and specifies the - number of backend's zcomp_strm compression streams (number of - concurrent compress operations). - What: /sys/block/zram/comp_algorithm Date: February 2014 Contact: Sergey Senozhatsky diff --git a/Documentation/admin-guide/blockdev/zram.rst b/Documentation/admin-guide/blockdev/zram.rst index 714a5171bfc0..7ad4c86f8258 100644 --- a/Documentation/admin-guide/blockdev/zram.rst +++ b/Documentation/admin-guide/blockdev/zram.rst @@ -54,7 +54,7 @@ The list of possible return codes: If you use 'echo', the returned value is set by the 'echo' utility, and, in general case, something like:: - echo 3 > /sys/block/zram0/max_comp_streams + echo foo > /sys/block/zram0/comp_algorithm if [ $? -ne 0 ]; then handle_error fi @@ -73,21 +73,7 @@ This creates 4 devices: /dev/zram{0,1,2,3} num_devices parameter is optional and tells zram how many devices should be pre-created. Default: 1. -2) Set max number of compression streams -======================================== - -Regardless of the value passed to this attribute, ZRAM will always -allocate multiple compression streams - one per online CPU - thus -allowing several concurrent compression operations. The number of -allocated compression streams goes down when some of the CPUs -become offline. There is no single-compression-stream mode anymore, -unless you are running a UP system or have only 1 CPU online. - -To find out how many streams are currently available:: - - cat /sys/block/zram0/max_comp_streams - -3) Select compression algorithm +2) Select compression algorithm =============================== Using comp_algorithm device attribute one can see available and @@ -107,7 +93,7 @@ Examples:: For the time being, the `comp_algorithm` content shows only compression algorithms that are supported by zram. -4) Set compression algorithm parameters: Optional +3) Set compression algorithm parameters: Optional ================================================= Compression algorithms may support specific parameters which can be @@ -138,7 +124,7 @@ better the compression ratio, it even can take negatives values for some algorithms), for other algorithms `level` is acceleration level (the higher the value the lower the compression ratio). -5) Set Disksize +4) Set Disksize =============== Set disk size by writing the value to sysfs node 'disksize'. @@ -158,7 +144,7 @@ There is little point creating a zram of greater than twice the size of memory since we expect a 2:1 compression ratio. Note that zram uses about 0.1% of the size of the disk when not in use so a huge zram is wasteful. -6) Set memory limit: Optional +5) Set memory limit: Optional ============================= Set memory limit by writing the value to sysfs node 'mem_limit'. @@ -177,7 +163,7 @@ Examples:: # To disable memory limit echo 0 > /sys/block/zram0/mem_limit -7) Activate +6) Activate =========== :: @@ -188,7 +174,7 @@ Examples:: mkfs.ext4 /dev/zram1 mount /dev/zram1 /tmp -8) Add/remove zram devices +7) Add/remove zram devices ========================== zram provides a control interface, which enables dynamic (on-demand) device @@ -208,7 +194,7 @@ execute:: echo X > /sys/class/zram-control/hot_remove -9) Stats +8) Stats ======== Per-device statistics are exported as various nodes under /sys/block/zram/ @@ -228,8 +214,6 @@ mem_limit WO specifies the maximum amount of memory ZRAM can writeback_limit WO specifies the maximum amount of write IO zram can write out to backing device as 4KB unit writeback_limit_enable RW show and set writeback_limit feature -max_comp_streams RW the number of possible concurrent compress - operations comp_algorithm RW show and change the compression algorithm algorithm_params WO setup compression algorithm parameters compact WO trigger memory compaction @@ -310,7 +294,7 @@ a single line of text and contains the following stats separated by whitespace: Unit: 4K bytes ============== ============================================================= -10) Deactivate +9) Deactivate ============== :: @@ -318,7 +302,7 @@ a single line of text and contains the following stats separated by whitespace: swapoff /dev/zram0 umount /dev/zram1 -11) Reset +10) Reset ========= Write any positive value to 'reset' sysfs node:: diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 248dab7cc7f4..93cedc60ac16 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1103,27 +1103,6 @@ static void zram_debugfs_register(struct zram *zram) {}; static void zram_debugfs_unregister(struct zram *zram) {}; #endif -/* - * We switched to per-cpu streams and this attr is not needed anymore. - * However, we will keep it around for some time, because: - * a) we may revert per-cpu streams in the future - * b) it's visible to user space and we need to follow our 2 years - * retirement rule; but we already have a number of 'soon to be - * altered' attrs, so max_comp_streams need to wait for the next - * layoff cycle. - */ -static ssize_t max_comp_streams_show(struct device *dev, - struct device_attribute *attr, char *buf) -{ - return scnprintf(buf, PAGE_SIZE, "%d\n", num_online_cpus()); -} - -static ssize_t max_comp_streams_store(struct device *dev, - struct device_attribute *attr, const char *buf, size_t len) -{ - return len; -} - static void comp_algorithm_set(struct zram *zram, u32 prio, const char *alg) { /* Do not free statically defined compression algorithms */ @@ -2540,7 +2519,6 @@ static DEVICE_ATTR_WO(reset); static DEVICE_ATTR_WO(mem_limit); static DEVICE_ATTR_WO(mem_used_max); static DEVICE_ATTR_WO(idle); -static DEVICE_ATTR_RW(max_comp_streams); static DEVICE_ATTR_RW(comp_algorithm); #ifdef CONFIG_ZRAM_WRITEBACK static DEVICE_ATTR_RW(backing_dev); @@ -2562,7 +2540,6 @@ static struct attribute *zram_disk_attrs[] = { &dev_attr_mem_limit.attr, &dev_attr_mem_used_max.attr, &dev_attr_idle.attr, - &dev_attr_max_comp_streams.attr, &dev_attr_comp_algorithm.attr, #ifdef CONFIG_ZRAM_WRITEBACK &dev_attr_backing_dev.attr, From patchwork Mon Mar 3 02:03:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998055 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE72EC282C6 for ; Mon, 3 Mar 2025 02:25:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4F181280005; Sun, 2 Mar 2025 21:25:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 451CF280001; Sun, 2 Mar 2025 21:25:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2CB90280005; Sun, 2 Mar 2025 21:25:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 095B3280001 for ; Sun, 2 Mar 2025 21:25:02 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id AF753C09DE for ; Mon, 3 Mar 2025 02:25:01 +0000 (UTC) X-FDA: 83178647202.05.E5792E7 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf04.hostedemail.com (Postfix) with ESMTP id C8A5E40006 for ; Mon, 3 Mar 2025 02:24:59 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=oEXI6mfH; spf=pass (imf04.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968699; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8qnIjXigJRpaB/eOXemhT39qobrTOIzhvUPCZvOaudo=; b=mm167VsncGAsjKMlINaWDeAkhDffI2MBUwsr5cSuiDQPLYxJTag34sN5BOlcf391eRiYEq 8qFyR9sQoskDEdxfhQu8zK0QsGP6uI/S4lGrPGtpau+g7/G/s5wzyLLdQ1gXMfdSscQi56 GWhwoIu1xIQVSU9Xp+Bn2OGJDlkd4ww= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968699; a=rsa-sha256; cv=none; b=DW756mg1JmtquXbCh5icM29CL95pxKzd8reKQIk8UfKlzaKJE+Egmjv+gmliwWcJ+RylSX xiX1bFj3gyVbA4AZNRstbl3veVb4la+Bx1hRQDZZ2YuwsgSybg1oksNGLjkQx4QeP84Dep VWWpLxu4Guh7+9lQL94r8O60BnAYROY= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=oEXI6mfH; spf=pass (imf04.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-2234e5347e2so76908185ad.1 for ; Sun, 02 Mar 2025 18:24:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968699; x=1741573499; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8qnIjXigJRpaB/eOXemhT39qobrTOIzhvUPCZvOaudo=; b=oEXI6mfHgUDiMWinCBU8+nNIPS14lQprhBUrQc/NmQAj4SAYdVv84aRVVBHfRrU9Ay QLPhgM+eH4bVJ9Nqc319+aE+Kjt8Zl75vJHftPlctWnbmc3vGUQw0t/AXx//GVa/cbt1 UVgVlibV99AdR7+quaUsOi0/HpPSf3VtbRZEw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968699; x=1741573499; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8qnIjXigJRpaB/eOXemhT39qobrTOIzhvUPCZvOaudo=; b=I4nKE9ax2T2QItTN+LwkwCcSsfnhSEvsRoV/6k2bpD/9TmhxF+iYwA1IC3nQBUUOWD aLR79K1Mt/LqVa9zkADTZqAqcuzGUDo5+eIG1G6YI7QgK8p4+MynNhZ1Eezq2JinawkI 5L0L1PoW2KTkMh8HK29J0alYGhRH/f2Zrbunlj7kvC8kt/qAu1BXWzYNO5KUnHk/qwT3 gPpOYWqy6O8sY9oIRU66IEos1F7uS+A8Zsvx849xjRnvB+i3aZU2o18JkwDn6s66qT/P 13J41IJbaemMqrEQrxqWVseBc93g4HKTIQR/Rz8A9krqxSqpoO1vrnAF0ejWcVhjPpl6 Iysg== X-Forwarded-Encrypted: i=1; AJvYcCV5S69zFPJQUNoydyIvsdrpuxnVr6QYo50wU4M9KJgKWrPBOn4owZanmL+BbsTQUkbkSMQNdY+s/g==@kvack.org X-Gm-Message-State: AOJu0YzF1tFSe2hnv2EBjSzBjPRqShlz5garblOV32FWTg2UMaNb3F7V V1Vx23AYLklEepjvVSavQtR19L2Lh3PnVIgzB026m/IfwoTEXsg9SWWZLYfXlnQD4okNaSm9aTI = X-Gm-Gg: ASbGncurhMYT3hQ+kmRJlbmgSKP8Oq1fc32GAouM1k4uSVLsIc6hQqGFaMvJfYV6a20 +9bqC07jBe1u0Xu66xMa6DSCzGvkhaI6GuQ5H9v1zo2vI6OwOJ8nn/4QvgjW00iHTEkiDr1x8zy b7u5aMF7MTC6y0CICUI74xp5GzjoDxJvRQSejIMwugkQMYuqwq6LmVf75hbvsox5CB/3j7LLYNM zoppCFQMY241zrxljK0x9ru3dL2a5tPTY5Mi7jtsGPzQ/h97kBy2heASjGHJQgrVTbL1t/B3e8L SPpZ1ZDnCfJ57Dfgnv6/oCXMOlN6DcNKKxS/CxO5jKexQd0= X-Google-Smtp-Source: AGHT+IH2tyXRUFNLNEQKOHaFYFPbUYX5TokW3hDEBo4R90Yp1othTxhspagFYn7ka7w60KBXBXHJeA== X-Received: by 2002:a05:6a00:1795:b0:736:3cc4:76db with SMTP id d2e1a72fcca58-7363cc4886amr7491350b3a.24.1740968698689; Sun, 02 Mar 2025 18:24:58 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-73632e76e1dsm3701620b3a.89.2025.03.02.18.24.56 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:24:58 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 05/19] zram: remove second stage of handle allocation Date: Mon, 3 Mar 2025 11:03:14 +0900 Message-ID: <20250303022425.285971-6-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Stat-Signature: fkqm1pna8iypwo3i6atpip6w4jx15yj9 X-Rspamd-Queue-Id: C8A5E40006 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1740968699-782216 X-HE-Meta: U2FsdGVkX1/aCjoTYj6UYwI0FDXNKpA6rsIAFMp8ORmQt0ea+dmJnY9SSCgWXhgcOWRblC/8HDdVPU16epayP0BPS/atWsYZSuYOXPzDQC1pjx3+N/QeS+z+Jlb6pKu3nf25ju0hnZ7+drH16zEAZEl3Tq4J9SMKCjEz5xbQrwmZNyWpM3IHHNZfxGruDK1QuFwIFO6e6JxjI+aFgo/8qvFWro1Z93Lg6Ddu92BqePzfGFDg3h2z6bZj+Rt1dS8gq0nq7ZNKpwlb8D/Ch5HDFX3K65VYXmlkgGuctIU6RjsH01Fm0A9I+j9q67yH1ei7/9e0i5rAbbyrmeP0BwRNyzuJ8WNtrY30a6lmzlJhthmxKQ1Ny2bKgRkonSwnM58nZkCpNu7VSKV+XBTDt0bDRx4ACUrG2CykL7smt0jOBMIL7tQWc75BbHdOlNnz0CzqINEEmriN9oPOJoMBLWm8BUA+W2OMhAR/hQkc/mOhhsVFb43p6elxNf8flAdWVYTKoW4ZKjjrv6Wl/+vezAXb7urOUU0dydnXmC2n27IHor14BMx9hq/3YSJ1jszxawhjTQezDhI/ZplDC7O6ZY/FhLrrCZMGkoJguxMS+/800a9gVLc2ahh6zMo0dzxq/mfK2JZrOBQQWJsvCthutJxaVD4OeRxJMvC13X8b8E01ugtkhgGNmu3lUSZy7ncLFvf8tAZLiWnkSGZM6pZanICRIi9WcCIwdKi0XvMs8bakTt9KQ8OrFOSy5LY6FzyFSMoplzPbvxCZyhcD6G4aWJsyO/uenEatycHxDi6oplmSnsQjl9Q+E3YdPEYyVF+GYqOAgFyyAb8p9UcbkJ1+I0pPZ92egLCTai3myKneKwmv/ibDmJCxYInk4nsFY1NzF4Fe7DQRhQeK2M2sCnfXhsIeioTyNybNtfkEfcBoE9upx80vHeg3JcQWFXGtwzA9viavhBn5nG1NSeGdCudIQSL X6jGHMi0 kYBwGEuE4Fje5cQU2jmRkH2Ov0p+spDB2WRLdw7F6Uqa/tZ09ATp7iRI+9yAeEXOqvEMsW3XQaf+zyoOQjhT2sHu0XPfUtZ7YSb+xuRiEWI1GeBXvxqu8dF+8wteQvrIytMHqmSSssuKrx/C7pMhXL1UkT2nNmuUzSNdgIw00wpCQoa6kCxZH4HCcDsFxRL7cliQ1297y0ioiW1+EzvLFtf/hLo1xb7Fwmnj/RQKSn2dJOjzZsIMjZHtxZtarnUX/sy6PwySwP2NZuQKFdxeigowI+BWzzXlzdWEXAP5Gu87NrXh5E9a25BHr3w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Previously zram write() was atomic which required us to pass __GFP_KSWAPD_RECLAIM to zsmalloc handle allocation on a fast path and attempt a slow path allocation (with recompression) if the fast path failed. Since we are not in atomic context anymore we can permit direct reclaim during handle allocation, and hence can have a single allocation path. There is no slow path anymore so we don't unlock per-CPU stream (and don't lose compressed data) which means that there is no need to do recompression now (which should reduce CPU and battery usage). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 39 +++++++---------------------------- 1 file changed, 7 insertions(+), 32 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 93cedc60ac16..f043f35b17a4 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1723,11 +1723,11 @@ static int write_incompressible_page(struct zram *zram, struct page *page, static int zram_write_page(struct zram *zram, struct page *page, u32 index) { int ret = 0; - unsigned long handle = -ENOMEM; - unsigned int comp_len = 0; + unsigned long handle; + unsigned int comp_len; void *dst, *mem; struct zcomp_strm *zstrm; - unsigned long element = 0; + unsigned long element; bool same_filled; /* First, free memory allocated to this slot (if any) */ @@ -1741,7 +1741,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (same_filled) return write_same_filled_page(zram, element, index); -compress_again: zstrm = zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); mem = kmap_local_page(page); ret = zcomp_compress(zram->comps[ZRAM_PRIMARY_COMP], zstrm, @@ -1751,7 +1750,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (unlikely(ret)) { zcomp_stream_put(zstrm); pr_err("Compression failed! err=%d\n", ret); - zs_free(zram->mem_pool, handle); return ret; } @@ -1760,35 +1758,12 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return write_incompressible_page(zram, page, index); } - /* - * handle allocation has 2 paths: - * a) fast path is executed with preemption disabled (for - * per-cpu streams) and has __GFP_DIRECT_RECLAIM bit clear, - * since we can't sleep; - * b) slow path enables preemption and attempts to allocate - * the page with __GFP_DIRECT_RECLAIM bit set. we have to - * put per-cpu compression stream and, thus, to re-do - * the compression once handle is allocated. - * - * if we have a 'non-null' handle here then we are coming - * from the slow path and handle has already been allocated. - */ - if (IS_ERR_VALUE(handle)) - handle = zs_malloc(zram->mem_pool, comp_len, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + handle = zs_malloc(zram->mem_pool, comp_len, + GFP_NOIO | __GFP_NOWARN | + __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) { zcomp_stream_put(zstrm); - atomic64_inc(&zram->stats.writestall); - handle = zs_malloc(zram->mem_pool, comp_len, - GFP_NOIO | __GFP_HIGHMEM | - __GFP_MOVABLE); - if (IS_ERR_VALUE(handle)) - return PTR_ERR((void *)handle); - - goto compress_again; + return PTR_ERR((void *)handle); } if (!zram_can_store_page(zram)) { From patchwork Mon Mar 3 02:03:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998056 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87A17C282C6 for ; Mon, 3 Mar 2025 02:25:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 20AE8280007; Sun, 2 Mar 2025 21:25:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 193C2280006; Sun, 2 Mar 2025 21:25:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F00B2280007; Sun, 2 Mar 2025 21:25:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D29F3280006 for ; Sun, 2 Mar 2025 21:25:06 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 978691C7F47 for ; Mon, 3 Mar 2025 02:25:06 +0000 (UTC) X-FDA: 83178647412.17.AC70CF5 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf09.hostedemail.com (Postfix) with ESMTP id A9BE6140006 for ; Mon, 3 Mar 2025 02:25:04 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=XXjUPYs5; spf=pass (imf09.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.52 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968704; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DytrayV9Mjuo6syNe/AR1TRl5JXW87zNZiaeyGLF4cM=; b=qa/TiTp2r8XlYU0SuvyJtk/A2FZ2E9qQvwF75Hf+dh//SZmrHDDsMS6rBmEkyxfyciDoon c1mXLkIMBI0gCEKlOiAvABS04aF2bDIDTvhAFEI2aIUA+Z4MsNk1yATcZlmhwYlpA93uAy wCW7fNxW30Zl3b2GyEGI09Wpw+XYtZM= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=XXjUPYs5; spf=pass (imf09.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.52 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968704; a=rsa-sha256; cv=none; b=WrZO623I/uWKLhi9Q4qGZssHMtngD4PQssL6/ge+XzXBFIDCOnigjOoSAl4IJ0nZJHFH7D UDBlHpm+Bz4z5VtTpRVuhwK/wi7QPuCxz4DyvF07+BPF7m4D2uxIfX9CMoTePKrOYXCvKf B0Ymfi9ePbqsUKEEfM3zysNh0ofTy6U= Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-2feb96064e4so5638026a91.1 for ; Sun, 02 Mar 2025 18:25:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968703; x=1741573503; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DytrayV9Mjuo6syNe/AR1TRl5JXW87zNZiaeyGLF4cM=; b=XXjUPYs5/B0pBy85Rjn55z1TfZR6qg32L5jaWMvFYoTkaFdwdjC5s2rS2iDQ+9N7cL tc4ZBV7vtDRkfMmcxXHLLYGHiSefnNBMbXTmpnjwv08Mi7EcO8e6tRg0uDD2YErHOsTi kXV5M6OPs8vLUbJ59CqOvq+yLo6kAhjf31Ke0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968703; x=1741573503; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DytrayV9Mjuo6syNe/AR1TRl5JXW87zNZiaeyGLF4cM=; b=BhhIjEVD48Sp0M4UKu62ghUlil4ESCjj1CYWXmiKDy8Pih81ikhgn/aYlAfqlM8UhQ mudRQ2nBoeVIoSJhGCic+Eg24r/h5flZyabEFk5muMbwMm4awwqN+D0h2vGUiIoNH+j/ U8TwK4uKqn1jKUIyVnXhJWED9EGE9hkmdEFjR5amll0s5cSB4km2vesHH7wi85bcXCPy G6kfycfdyHh8lIZg8lP5enS8GvZyljvr99Ax194MdTi6idWsLn3+aM2wmtfqAS1YHjz2 w+lYIzxnJ9YeOD2rE8koL3p9V9BT/5oCVncYFCXic1wFT20jOcojPvB0djAaias+zbzs TSNQ== X-Forwarded-Encrypted: i=1; AJvYcCU9dloXiAD453TeKhxT+wfJt6FSoT8yuBF0zMAsuTc5rCXCLK6x6w2lj2aJmHqcoP9Gkc54cRIGRQ==@kvack.org X-Gm-Message-State: AOJu0YxSvhGmwMVtWSc0NPG8upxjHlLJe57SUUrh0m6hZUObzGA7jLi3 hQEGEov3M5PDNBfGWviEFaTyhjBfr5/OWsei/djkD4+i1gdATQOWlW9pRp508YS7ckTQXAgz9KQ = X-Gm-Gg: ASbGncvLokCFI5Ffi9wOwq9t1BmU3AS0GQKR+QPq2k6OIdgSBXJeDBIe0n55ouoyehm QXWN3MKN7hETvT4O5AmrbkpfJ9TfhhW62ciG2JnLl4Y0fop4eXr675fm7BkbnRooD7CcXmxkRrU dfkZ70Fg+FXDVfcvq+jn9Vb9hfTM4xAe75H+OBCjbumuLdvQD9VfifPu9nt7fNbJp0H0Qj4qYl7 OSZOTgkNDbFZDq3f/V5cMmesxSUnsiE1i7SqiLw4Q7NN5qPRoJnNnUtZrL9EaWLV/dupzDS8dNj nvntmNNlxaWdnEo5pXeTGfxPktJXARQSeXPUxeJgyXGzBfY= X-Google-Smtp-Source: AGHT+IHl5Crctco8Hu+E8EG5MtD0b7+xg8Bu5mtdXcu6UNEW0JPimU6uMCyi2CRbQ02nGaq5KYpYtQ== X-Received: by 2002:a17:90b:3ec4:b0:2fa:b84:b31f with SMTP id 98e67ed59e1d1-2febabedb6amr16911551a91.25.1740968703617; Sun, 02 Mar 2025 18:25:03 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2fea676c4dcsm8609448a91.17.2025.03.02.18.25.01 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:25:03 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 06/19] zram: add GFP_NOWARN to incompressible zsmalloc handle allocation Date: Mon, 3 Mar 2025 11:03:15 +0900 Message-ID: <20250303022425.285971-7-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: A9BE6140006 X-Stat-Signature: km6ed5788466w6ar4o4obbmx93cn46ji X-HE-Tag: 1740968704-42084 X-HE-Meta: U2FsdGVkX1+4PlMU6nmAk7/t+70N7Ue5FLg1PhiWUm3zYeZec+h+2ewWgjpiQ1lWCpGnVaKxWekO5z2QNS/IncsKXDdkxfERswNpWDgCJaAMy6bknFyph6k5tpJpVasEYN3z+POw045HbuE+oycTXqVJEo1fFP9KHD1MXDYNNklsV2QypK0S8AnTozmcxlWI+Aw4Ah9YEUzmmTewQ2AtANCYyzhoXBOGvhyvVSGKwQtR/l+QwFQiFo51M6By0UCRzjJlxKaj1HP22C2Ze6OOUYgyfycBVLj+vcsab3TyfHblzrYNd4DAFZmiu5BgETzPUqe65dz5BvR4QIqFeOXj8pFdSel9+QPe+1c08UG5asiXAfaSpDQj6ygWt5gGz8MV5bYNT79t+20OFOatOyWrOZiLzAt9ffG4JMacsBQ6iGtKGslVhga3yzlX9UjUuumqWBhcQiFxlTnMsEM3ndzyC62bqh0Dtn74zDk7lg51Ckbx66yy/66PcCvRski9xf1jsRLwqourkPcn9D7Sr98ntytCM4ZdhtJObOyiQQE0123MPVEzE62LwjMR9O1o305VWcxdImZe8ftNkgWAkAClTks97h+TUkEcAGywM76j+QTxXE6Hcx5yI3TBTY+jeWjO/KFs1Vj3MOoVuKboXqecqMGeR1uZ6mcw5G2cFar9e6enrNtLqjgQol8rrVwIs/QeiZ63MXHYKOmHdQ678XReJkJMnPYXSyQAQiqK8n0JOgRMPvddyLrHnrqfPMvFsrSq8OyyE7v+1+ogn94zDHtvfd7/0JX2VcFURvWrOy4bo7B0BK7NNhqE8VmqnO4oqZa0MTov/IrST9UAvDEYFRKiK6Tj4oEutLFuHdyzAD6BbVSwdpDC81SIACpsKDbQtN65a5lKC5fXDN5W3vTw/gkhhjn59HFLx4jlsRDlpvcEMvccZnpyrtmWO7eu35REjmBiSCo7NioYR458ReGsplt ipOFf6/H mU5i3ccYjSBh1KD9sS+kRevrBJNY1p1kUmJMkeKAkO8vbp9e2C6TGIpcWF2ImaXvRGawK86OVbeB/1Pl7R6Tolts8VR4RGfZUEDA+J5H+nJGZPBtzm0tCOGyUy/gTjxnHYh3X+I7GBCEtABh30c6OS5gVfGUpAndW8kRKm9KBohmou+TMeID6EYLMrGw12l4MsTZeF5vUjLsCyjtJ7echBQ/qER49c+jvQxrTh0uADwMyuwH9ZdcHu1BHUzy2vX2skgSCwXQQ405VbJfKEGcZNavpXjk8NtcpC5cgmCUmXAvleFAv1rUihhHZRxEV8daUPJUDCq3qQP4BiWwOxjmIhyaljQOeZtngZW475nzWiMooO3I= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000126, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We normally use __GFP_NOWARN for zsmalloc handle allocations, add it to write_incompressible_page() allocation too. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index f043f35b17a4..249a936b6aac 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1691,7 +1691,8 @@ static int write_incompressible_page(struct zram *zram, struct page *page, * like we do for compressible pages. */ handle = zs_malloc(zram->mem_pool, PAGE_SIZE, - GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); + GFP_NOIO | __GFP_NOWARN | + __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) return PTR_ERR((void *)handle); From patchwork Mon Mar 3 02:03:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998057 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67681C282C6 for ; Mon, 3 Mar 2025 02:25:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E6557280008; Sun, 2 Mar 2025 21:25:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DEF75280006; Sun, 2 Mar 2025 21:25:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C6939280008; Sun, 2 Mar 2025 21:25:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A8CE2280006 for ; Sun, 2 Mar 2025 21:25:11 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 3D0861C7DF9 for ; Mon, 3 Mar 2025 02:25:11 +0000 (UTC) X-FDA: 83178647622.16.8909E5F Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by imf14.hostedemail.com (Postfix) with ESMTP id 4DD3D100007 for ; Mon, 3 Mar 2025 02:25:09 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=M+ubZukk; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.50 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968709; a=rsa-sha256; cv=none; b=YSwWYR4xwZ66e1HZp/DV48K8TD0vj/z2ACJjBPozy/g3LkiT2nEtfrvr2CfwgWMNqqnKvE 2HpvcNZoFP6PFX5FKsuwqBJ/jZnAHvwiRL4/CxecwlGXn6ukuVSXd1EEBnb9Z1WW677/vt dk7K3UZNACrensnuUmtWRUvbyu0QwAY= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=M+ubZukk; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.50 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968709; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Q9mhj8QPNzHEXvTH0OVXBitiCwr2gmDik1RNTqYshT0=; b=M5eC0+VGChLb27OD2TSel2npV9ayzvPJQj0l8L2Tn66QXna34N6eFeAyhTOGdmBDS/TFh8 phF5+w2scU61UeXEkXYL9zkYT8D93xLINYucTT5h1uTZeQ5wMwlDGhBYCTzKJI+BOzBIST /rqJEbN2sKOXkNYZlkIUE1u4B1U1/44= Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-2fe96dd93b4so7159355a91.0 for ; Sun, 02 Mar 2025 18:25:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968708; x=1741573508; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Q9mhj8QPNzHEXvTH0OVXBitiCwr2gmDik1RNTqYshT0=; b=M+ubZukkob8Nw0sB5hGxuqCkyBm5PCk7IwPWzl/Qcdd9LIq5stoSA96U0j80yq4yYH OOfTQy/zRFh+UQ2859gjBMLL8/ZwesjXnX7t24mySwdjlTM3GOkkDqwYOMa+J8IeTla4 4HwyXsWZQaid8xjxQnb5+auZj5r73Fyia8RjY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968708; x=1741573508; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Q9mhj8QPNzHEXvTH0OVXBitiCwr2gmDik1RNTqYshT0=; b=e6tHFAiuoFulINBFYwb3n4lQ/PUwm+PlYpgm7sJ9vI4r1+1RSQdBYS9Hyk1bVcDHOq NF4UiR7CwMTsDwIChixnPdSXIEafYee4WmFy9nGO5FbtH3ETgrMTw40NmoAzq13W0jS1 CyhbtxNlBNLHkEp33Yro+8sCZWyVjdeBoeMIUulw6mbLvoe2Psb1rn2OMTRHM1zk4Nkg 6icSJ/YAbU/5JdKE7h5o2IADDXg0yb1Tz8+h6zLJxM/g+vLDd91UA3XijJxSpqw4rLym SqC11eA9MB+mSldP2HAk9njNzzuwBJphty1kQjggbBTMMdyZQrjYsEPMfN1CNqe5Zfwx AMVw== X-Forwarded-Encrypted: i=1; AJvYcCUUwoZ6P0DQhtHYs9fYP3h702Kzm+LCw+UfMFfogtJddI8hPlwBCU2DUwaDnDrGslEg0ZV1grQQ/w==@kvack.org X-Gm-Message-State: AOJu0YybF3hO5/r8zSinJI1MDCs+ELG+XDlnt9xIp0Y6pIba83dAimAg Qy7SWMm4zT/9dZWzS8YkkE3tO5udfQUDIdzMDaCUXHso1BwpY3wVg3Z8wuOILA== X-Gm-Gg: ASbGncugKhggZjTt9vq5WDkXV25PyJcWdl9QUosRRniC4iHIN8iqiqVsJdvMV6Y3Wm7 Rj/PotziBk/8en3z0CpQa0oNenvuTDid2aTq+T0N8yFVts8rvlmrEtUpmDWQbU8pakK+q3dZUXD uqUgIAVynNNznSHFukHaok8vE8BWNwixt+wBjsrbEs26LkEB16z0tNVsgc6qaNSx6eT9u/lqeD6 H0b9lPi2FPBJAee5a3wKKkWBUvNMHnAPFNoNdWmt06H1H8ceM4J86KvmheStI/+/mKaYX7uTMQB U5jNpuGK3FmmIzEGsctX76NWdHwCZfTBOx6sdLv2v91Kd9w= X-Google-Smtp-Source: AGHT+IF66VEKY5wPrt/KNtaj7+PAWT785N+ktrJ+PZh+m4XyOgJ988yBEbAqjba5GwAAOb7NWMraHg== X-Received: by 2002:a17:90b:2513:b0:2f9:d0cd:3403 with SMTP id 98e67ed59e1d1-2feba95b413mr17873240a91.16.1740968708303; Sun, 02 Mar 2025 18:25:08 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-223501fae03sm66894545ad.84.2025.03.02.18.25.05 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:25:07 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 07/19] zram: remove writestall zram_stats member Date: Mon, 3 Mar 2025 11:03:16 +0900 Message-ID: <20250303022425.285971-8-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4DD3D100007 X-Stat-Signature: s9haj89knnf5n55m38b6yuzjyiy58tpf X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1740968709-161058 X-HE-Meta: U2FsdGVkX1/GtQIbW1jAR+JpyhgzPmpTv33BCnHPo5Nz8banxDomCGXmuikCSuEjyevFpnXQWE2JQeKClhaE0GXrAi0yN06Fe9atOQxI5Gdbm0ZAXbHN8gX73c3dURkegqH+lTz3EPflcj3CMvcWqRfhroJsoZp1GuAIRZvXooSxSPcK0vOchexiAZAfJ71Th4w+MiKcxODgpcVG1pOGKmF8Rxo10NKIznt7l4BQit7AJtJb/6NpNup6xpzz/xMHzTDNIVCnP0iJ5XkAb7UtF1w9/IfBsLw9ILcdbHZaT0I+MUAfoB9PYuyuNcKLz7aCo8dAuVr26ySoYRiG1dx6SxA50b43iCKiltROholQkdV5WETYQXSnvTvC73hfvbjGtmCU1Ur1GxrWjjTpbLMkyUzO+rYaFqD+TRWqNQKIFg+FhmNnRFjMQxcfGt0LWbpWhFlbkf8aKzSBA6/OAjZGZ1OfbRKNsT90gdQJdvxh5dhQ+g+TDnOBpt9LgT1arZNCVW1vgkror7GYHoKvi9XGDunIxrYTdFoK7PjXAw/OWeTew9/8CPU3eAXYlxp1qAvoXXEfyd1h0dhrTDt88NHh65z8v8ele5QkNRl3Cxb1YNm8fu891SRLELF8E0j0/viUKKdm+R5r0IccUFtr5o84PZKUWz3IEdhpjoZqSzQR54wJUlmgvn9Arzfv8uyGvx/Ou24kjtf0C79/F3LRL6pGjEZ4AahvhI6/ry7VfwSNHAfA5uPkxXNHQqqJoVt3TkPt51ZHJ6VDJ2dGWtCWTJ6xO9wZ3jkI4wAsC42MqevhCNipE3UiQzJRES5ftGrchRySiH12y62jyB60kblEDbv7irOh77Wf6e83aglf8i2+ejIGCi8eFNShCP6f/zyTHnYG1rQAlIJEoHu6g6EYMPsJh8tN5/HrhAPl0x/AdwlpxYezB1XjFfYCJCm2c4hEOeoQHq7KABzg6k1R5kQvlii yIZ7elaO umqk8XOHrPwAoPX0snr+ES9XfLZq6Ujqm9286DXKw0Sdr6p4Naoegu8Lk47hwwdkKaCX0J05Xg4iMVqqI1x87eK/sFAenXYgTAQJpEtR6fkm3Sm50jVcjK3G2rWpe1x3owB0QNto/DOKTdXSpejsoKrap170F2S5Jj7NQpCyvE2nMWBfXm9UhXSRKRQG+Gka7lmkE+dpL5vPbhEsVAJE/2fzn6Jvd17NMvIZAAD4styifGT8xBatHxY0vfXOcURHBxM/M9xngMHznWADG81Iu2rL3+oRqzOeICtcdgc7+lmEzyny5u1UZ+p8XXNrOyEkI1nFYwQ0CR+w6UiH+XTrCXEvr71fcVHzGeH2qSJJkJVAtaAc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There is no zsmalloc handle allocation slow path now and writestall is not possible any longer. Remove it from zram_stats. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 3 +-- drivers/block/zram/zram_drv.h | 1 - 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 249a936b6aac..fc9321af3ef4 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1437,9 +1437,8 @@ static ssize_t debug_stat_show(struct device *dev, down_read(&zram->init_lock); ret = scnprintf(buf, PAGE_SIZE, - "version: %d\n%8llu %8llu\n", + "version: %d\n0 %8llu\n", version, - (u64)atomic64_read(&zram->stats.writestall), (u64)atomic64_read(&zram->stats.miss_free)); up_read(&zram->init_lock); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 7c11f9dab335..6cee93f9c0d0 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -82,7 +82,6 @@ struct zram_stats { atomic64_t huge_pages_since; /* no. of huge pages since zram set up */ atomic64_t pages_stored; /* no. of pages currently stored */ atomic_long_t max_used_pages; /* no. of maximum pages stored */ - atomic64_t writestall; /* no. of write slow paths */ atomic64_t miss_free; /* no. of missed free */ #ifdef CONFIG_ZRAM_WRITEBACK atomic64_t bd_count; /* no. of pages in backing device */ From patchwork Mon Mar 3 02:03:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998058 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A203C282C6 for ; Mon, 3 Mar 2025 02:25:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EB4D8280009; Sun, 2 Mar 2025 21:25:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E1703280006; Sun, 2 Mar 2025 21:25:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C8FE7280009; Sun, 2 Mar 2025 21:25:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A8E2B280006 for ; Sun, 2 Mar 2025 21:25:16 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 394E7A30F5 for ; Mon, 3 Mar 2025 02:25:16 +0000 (UTC) X-FDA: 83178647832.06.8276D02 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf01.hostedemail.com (Postfix) with ESMTP id 5BDBF40005 for ; Mon, 3 Mar 2025 02:25:14 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=RhqU63Qv; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf01.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.182 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968714; a=rsa-sha256; cv=none; b=plCBiPI/qrPeFi9R+gYI5crDnFaVKCJrEbAG70tswbKoL/vTXk+5cRe6Impy+/W28JDZSq o4CMjx5LyCG/z8PttOmSNa/OIh0LqoxfxcsE5zIeyImr3ap+VHl4ADQfpquvyndrSyL9jt r9Fwnmzzfap31Q+Kn3PqbvMqx324jKk= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=RhqU63Qv; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf01.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.182 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968714; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eo+gLdw4ALzCaj1nm0+nyzWVeAdZV8iG/qfb0LErfSY=; b=0bdzu8lC0y68eWtxpkH1nC9u3nthuT40n0YzgKbsf0X5Q9ZPfpAUcroaJ3A/Qh+VeXu157 y76JnGNVOLyZdP3iSQk598KWMZON+UQwW6ukxOTb6Ou2iMS6qtATxXHjvN4/ABqaIVeK4Y uM1EIdMdYW2A1A3qLQFQuQZJFWuVULI= Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-22337bc9ac3so72883525ad.1 for ; Sun, 02 Mar 2025 18:25:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968713; x=1741573513; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eo+gLdw4ALzCaj1nm0+nyzWVeAdZV8iG/qfb0LErfSY=; b=RhqU63QvKFl8jtGUC8Moi58XRJUuPzPAV8889ZyzhHxXzOAotFLLLIrX+W5bMQNrkW nk9/LhRQX3gLiHu8dMqgRkRwv4F41mkOPhXT4Az4gxvDDNZgj8LCtdn+aKBfGoEAo6fy tr9OJBhfzu6o8pc+VlNfq64xHxXb+XPSc8MFc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968713; x=1741573513; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eo+gLdw4ALzCaj1nm0+nyzWVeAdZV8iG/qfb0LErfSY=; b=Huh26IokxRmooCtccy1f8S+Seo3cyyFuHfkujai+KErUn6l1/yx5VxyAkj/IYfqauC TN02SfPHh30so0xyfMA4pJI6gye86CsdKgwXA0Rl6zjtkbZz1VnA50IkDMfE9ixD7nNq YN1/gL4cUb68lXPacJ29wpaWXgeFSBEGuO6CbKLKQhlylaOKMGC9c2GSBoPnvPziLjmV hdDglM8sBVP54lR6TyfSNfnpLEuisaB5A4U6uLobCk2Lp9WV0YBAWZgu7+K3n1NNR/tA WFOpWRD62vBgX/cejffmT0+Hq9KLXE5zUsdMMARKlMmrA0HuZsQePM21FCKTod9aDgXt NPyA== X-Forwarded-Encrypted: i=1; AJvYcCWC49Dr0oGIigBsOrghg9PBvHYUyUnsyvii7xMzrzOnAd5NzWgr6rtAnJTStpoDisugPsBgjEZaYQ==@kvack.org X-Gm-Message-State: AOJu0Ywgy4NFaKirD12JeURdG9daFwiUpwRU8dT9Bda1ttVE9Dyzxg0C +9M5mXqrc7BXtI7/TQAqTwvrgtJ4pHBXesT4DaIR/kMnEozvotqdZ/nZcoYwgQ== X-Gm-Gg: ASbGnct3+Ur1kOUrnvK8oI7g9Kct9pQqy/n3475avSjIjYMpc8SnqYDV2p9EooJXdeb GZP0zDZhdtAaMY3WNauBdQbU+O4ipeg8QV9yLDqjDkhqO0ZC5bFdOvGvwXACpP5D+RWzpvbnyyj AriSCIMQ6Q2Z73HkGsqJcM60dpnA3+xiwIheMa11Dgd3Cd4ru+qfPpv8IsGvD+A5m+STDuUdlup rhwS6cPUOBvVbH0Neh5Z48gSxD9/+RRloQzrmQVO1bmuxkRBaHvgc12UMOpKSehREScJcGxQU86 x/1pHzVmk6eGi0YhsZqJj5tKliA86OkJRa2po7ycGmMiVdY= X-Google-Smtp-Source: AGHT+IFnUy2t+MQbv/ON6Qzd6lspOJ7em303a/vGQ6h5MRjg7inxZUPbfNYd5XTa2AnmAT9Ovl5HQw== X-Received: by 2002:a17:903:1983:b0:223:5ada:891e with SMTP id d9443c01a7336-22368f6a1d9mr130419345ad.7.1740968713338; Sun, 02 Mar 2025 18:25:13 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-73638e4ab93sm3434904b3a.103.2025.03.02.18.25.11 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:25:13 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 08/19] zram: limit max recompress prio to num_active_comps Date: Mon, 3 Mar 2025 11:03:17 +0900 Message-ID: <20250303022425.285971-9-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 5BDBF40005 X-Stat-Signature: r56s3kap76ib4iiqpmoa7sh5bh95kmw1 X-HE-Tag: 1740968714-130701 X-HE-Meta: U2FsdGVkX19TWhqwd8pf+PGbCAHY5sBg5Jw7/TsdaG7Z8XhgfGaEFNm4t4UO3K7MRc7yGlRQIw43qvI5MZIS6SK2IJ8LFA3a9awTsJE57tLKVGAgbbhEC1MftKm+MdLeqggM6AWvZGYV9BCgg8yuOqk0VW57bsff0Q0zuDFTu+SYF6WY7vbVQVnAkR/7HP43Q5tfs6Bzz99KjB+264onFrjwXiMFVWXzYznOnCKJa//khpkI2Xv/cXb5xxk2qMSPaw7rYuJU0m4k9oka15NapxAC5Ccu9zGtga4qoEsLHS6PmnR87m+s/KWj8m3hp8B7bHaIlK+Aq+wik9qlkWMTkHyKoQxZ8dE2te4+Yz5cHKgevlCpHEiHRTqz0GqbaJwFMF7vu2iLPcnFDb6XHb5pnJx140Izjmd2NuYGq43JDa22ajLlJTkvyOxJFhizHB2acf9T5cm3G2OdE0TuC1ugQtFsURfehi/WUBhoajN0neP0fyCExbTpHaRNmec0NE49ixSnWryj6ZFHlSX+DmuC8DtTUeiOaAJixWu8b6SNHE9pUDMW0bQb3Ct3TCQ0f736UrZAaEnUEQ92vFluu3LFDHhlR4o3jth9HvobSuV08UminPj0+nlM3EhVcHLodE7dgZYdOHJGyld1paMQ5SwrAktgxtf8MrCWNLsWnrEPjrOeMQJmJl7BelJQ3pMJZ8+MlGxf+AkVPnc8oOR/g5G3KLKzVD0sj+Uw91l3S4LW8/+/dVliijHd5Tptxeq65TitP/8m+41AJteLbvk47Az5+VCakIqsaIMpg7UPWKVBdK0O4ZAGNxFCeJljWvrkMYWYWT0+cxwPAyCzom26+dEbIZ+MMCGaepthr+C1xqETvUGouJU31+7d0yF2pqy+ZcXbFfba+9ZHhWp1YD7Im7N+qj12Qbe7gm9tX5DyC5o5moYAzqkgR1tTzVPmXDHbzzup13QPPl2/upY4vH/4gPb y6+LjVe3 feDIyRnSBWL22OuLUMOAykI6i9q3kNkA/5aQUiMjAvR6IfLzOimiH4diPTYE1tF7AgEimE6mFKUvVE47+GF1MnUs0YuR1Kdhsl9TSx00JCzccQSXi433Jor3Kqk2jmTiwWNPBPJnLVwT3TtMJnb22dz/V+qZfLa8MS7/uAcktVPWSSrhcAEMGpfEtOa/pcpAFnkymAN1Aepf3TIN47TFPUrYSn28nr8hLh6R0MK4nNcgjt+hwh+nPxrBcYa+p8/u5K5kLqHMeWqrbmCxje1omFMeBuMzUf8mUlpDAeUP+3NaapsE5gRV8rOVWNmtmQd/cFFYYLTCqnbqNSJinwuEh80R/eprHubguKQNAbUlGz9fMnYE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use the actual number of algorithms zram was configure with instead of theoretical limit of ZRAM_MAX_COMPS. Also make sure that min prio is not above max prio. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index fc9321af3ef4..776c31606eec 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -2027,16 +2027,19 @@ static ssize_t recompress_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t len) { - u32 prio = ZRAM_SECONDARY_COMP, prio_max = ZRAM_MAX_COMPS; struct zram *zram = dev_to_zram(dev); char *args, *param, *val, *algo = NULL; u64 num_recomp_pages = ULLONG_MAX; struct zram_pp_ctl *ctl = NULL; struct zram_pp_slot *pps; u32 mode = 0, threshold = 0; + u32 prio, prio_max; struct page *page; ssize_t ret; + prio = ZRAM_SECONDARY_COMP; + prio_max = zram->num_active_comps; + args = skip_spaces(buf); while (*args) { args = next_arg(args, ¶m, &val); @@ -2089,7 +2092,7 @@ static ssize_t recompress_store(struct device *dev, if (prio == ZRAM_PRIMARY_COMP) prio = ZRAM_SECONDARY_COMP; - prio_max = min(prio + 1, ZRAM_MAX_COMPS); + prio_max = prio + 1; continue; } } @@ -2117,7 +2120,7 @@ static ssize_t recompress_store(struct device *dev, continue; if (!strcmp(zram->comp_algs[prio], algo)) { - prio_max = min(prio + 1, ZRAM_MAX_COMPS); + prio_max = prio + 1; found = true; break; } @@ -2129,6 +2132,12 @@ static ssize_t recompress_store(struct device *dev, } } + prio_max = min(prio_max, (u32)zram->num_active_comps); + if (prio >= prio_max) { + ret = -EINVAL; + goto release_init_lock; + } + page = alloc_page(GFP_KERNEL); if (!page) { ret = -ENOMEM; From patchwork Mon Mar 3 02:03:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998059 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8573AC282C6 for ; Mon, 3 Mar 2025 02:25:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 16DE428000A; Sun, 2 Mar 2025 21:25:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0F672280006; Sun, 2 Mar 2025 21:25:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E8B9F28000A; Sun, 2 Mar 2025 21:25:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C2FC7280006 for ; Sun, 2 Mar 2025 21:25:21 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 804EC12078E for ; Mon, 3 Mar 2025 02:25:21 +0000 (UTC) X-FDA: 83178648042.15.90A6873 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by imf20.hostedemail.com (Postfix) with ESMTP id A27591C000E for ; Mon, 3 Mar 2025 02:25:19 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=WOxD2Dop; spf=pass (imf20.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.178 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968719; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NTrj6AIBGg3fen5fqcQqiq+dr13fIgG4uK8Ep+QAN48=; b=451Nbp3uKo0XNccXy25yAQ8uDgP8KLGTUNUIgCoGpP9idT0aVH3RSq6MMqDHK0EQWdgfYC sowkMyKOutUda0IpxnU0w7aRW9yuh8T9YOaO3QqDKyUJvt0b9FBAFQZdMzklkUkUgXQlW+ RM2HuYN+VVuGtVyY026OCxK+J1Rt8AE= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=WOxD2Dop; spf=pass (imf20.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.178 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968719; a=rsa-sha256; cv=none; b=WuiLn8s4PBNFOfnGqukta9N5cK7PDjdf0uQTz1ah/2r7SQOlm8RQPk9M2B+2ZQOOlU0Zo7 Forg6gqZt4mf3crx+P8CSVSotlc9+/VVkk5Q+39aleht0JO1r8lvVPMs8Dme0GaSBWyOtV 6/6g9pVi5ZLifI4jZkT0VWlrXIa77+E= Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-22355618fd9so63391285ad.3 for ; Sun, 02 Mar 2025 18:25:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968718; x=1741573518; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NTrj6AIBGg3fen5fqcQqiq+dr13fIgG4uK8Ep+QAN48=; b=WOxD2DopHzR+yNhyMF5cpHgb6Pwkrxk9AUhBPycSsJk09p4N9h3u+UQRfsDsCO80YM 8tXaqrtvIuLFi+djO2elwnS98u7O5NWTQpQsxeezaF2tzHhx6TYU+KprPz4kdsWj9vkm QH8EpNwpBEW3Fptn/byQd4A7p6jPB26VQvqFY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968718; x=1741573518; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NTrj6AIBGg3fen5fqcQqiq+dr13fIgG4uK8Ep+QAN48=; b=udw2o6P0WZdBcOHL9cEITiq0MetSE799V3+x2/f7maauVr8/pxmnoSK/+HCUy0hEKW WLuE3rfMl4eopu+ob2+1zZ15dBrb0VvkGR2q3MOpLaE4N+mwe748zuZsPGk6bcKHyMUT /06HuM80CjgsB2u/u6OzDKhn2bYdQSGG4DQYP1PSvT867kmjsgVMJpDsNrLKadxEqhi8 ClD5D/YO+m+oOMAm8dW9JFGXB+FcYupvKTkrntVJNBdC+Q2N0zO+hxgMu5WcFGyxL6aZ W7I430dAuEVKu+XHdQb96IhlzuXlJAeWf9tW946x1azgRgYYZs+cT1SgCEie1re4JrqG iLEA== X-Forwarded-Encrypted: i=1; AJvYcCUYgFktw8TdZZIrYC2pUPLiQQtgYIlfmihD48qeVCUaylIu81Cun3KuxSqiEGE+zEDpFK9Vj0ebKg==@kvack.org X-Gm-Message-State: AOJu0YxCLCkSgcYo7tVsxGqOb9CbLLL6vtMaP/iZ2H75QKt0bJN0GVC+ Bv5t24iQLFJrv3iwg9kGDvRWknCQFvhK7CgdQ97bU34AQ6F8TcYNVyexM8Ns4w== X-Gm-Gg: ASbGncuU1rlFSgmixwdgv0EhH9L8mglx0+pK8xlQtanpz4pzL6ozt9s/YPjBl7zv5vR 7YlwzG8y0jXJ0RTtDQQmN/5+Xn4Rn9SbMNTIEtQDK4OYIrGBWPrXhFuxOA+Zpdz4uMXg0Ff1+bk p/AaH+8HfB95kQLCCaHrq0UIUS9FXTKQuAQ0HQkdtU4wkBE/EF7Vm4wMl0aZiOj7e4McqIVuvHV 9IMwb9S49fJRg5Twbmb8jPMixur+6fq4GQvZycudsf4cXKWDYK8Vu1UG+ajlXORAQqjKB/ifUbS z6c8dkh+TImghMy/zXFYsFUv3iPjO6oX4JgdACleCl9Q7xE= X-Google-Smtp-Source: AGHT+IH8tkcTWEOl7teGtIH3prNMr9e6TjPXzSkEQyQdyulF5O353ekND1QrhJUZq9VyBdS7SHU/9A== X-Received: by 2002:a05:6a00:4f96:b0:736:51ab:7ae1 with SMTP id d2e1a72fcca58-73651ab7b97mr3452819b3a.16.1740968718515; Sun, 02 Mar 2025 18:25:18 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-7363f62117asm2509047b3a.57.2025.03.02.18.25.16 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:25:18 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 09/19] zram: filter out recomp targets based on priority Date: Mon, 3 Mar 2025 11:03:18 +0900 Message-ID: <20250303022425.285971-10-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: fg78ckr883egght1kw59iwt3cq7z4k6t X-Rspamd-Queue-Id: A27591C000E X-Rspamd-Server: rspam07 X-HE-Tag: 1740968719-716057 X-HE-Meta: U2FsdGVkX1/DkvvBwyEkQnFzIxJ2PG5GAu0aZx8MCyhQH3USfDXS0v7wALLzFTNZVYxrnf8ZSNUg7RssnbfZUWY99+lmD1sSnBZoE4MWaGL4TGPkVOZhZKSBtdfErArRoCTo2pRj+RLrHjYVln7BszAdf1U1qTRCvjOYwT0Dc0ZMacx5KgABjv9IXgzjJDKhxIz156c2Xq8bjFuBTnH/Au8ejHmuS3DOTbDY/li6FYSBHIQKwlmiqzarG+7eqImpM73dVuizRNnJ29v94XNTl/USLYzAaotkFiEIZSgoj6ck8iUIWoK/95UHZSzGu0RswRExxdJRmHYbR8U4WsDG5/nnriJr+HTMLVLI4Z+S6om7ucLH24Oasy0uwF85/4507BYcQL7tbw59eiJHkYeI37tw+jDcpAE3c9ImCDqHbbREpA9ovs4xo7EomOi98zaUerqCU5Zq3GfrKlgM+z6BZNL3WGoJSn3Sw7rSIgeauaF2lnnIVQymCOhiYkuGVj+zDfSyOaiRVAHeuQAngyK8BZ3ml8ht8SQH0sjWXLXIaZr7QnL4k4vc+ax+3mis48Pcto28ZBdPDwurJEU0Gk07ep4sfxYFN6zAm80uzEDcl718EMmv90Pe7BXpgj88F4TKmKJwp1zM7iwHxiNBhvRTU4uV0v2YREZlLfrte0oamX5fDv/HI8vbk4cInU6+5qQEVvWZcL64SGyb8gO6VfyLMqtgEfGsOf1DcFUoj48UuYI6u7I6VGQFgI0Te0iqgTuMFyjaCsmYEHKs+1LlHLYqD0omPQnAKnP7dxNeQS1MnWegeP8hBHAtaV79hX+TR96cPlQF7aVnzBAo/FaSvEMnELgIGXg7g4KW6U+r8bxqnIK4wlnW2aSiu/i3A8gNyfZAw5IvdBYQ+8LXaFAMMH6yc3amBZLsiGDe2kFKF/w1H9V7ycb814+oOrpVpwMZdym1dejypendW3J153y6rsc UfJUp8Uf fImbuJlTWAcJ5WZLQkkGsTKHrZu+sMhdmfc3TCwfFsU3CZob7u1jxWKMgee4AzcNRAVgKzh7EV39QadFq3zZrTMC7kM8BMjrcoQgg8M56MvS6aZSZcNptjQqfAhl0X9mwMKZpdfKt9gyyevUqqm3d7U01b+PDWNFkulmopQjxaS9J1DR8e0TUJAVfFaJ0EFbSi4R0+XrFYnMefXc9wJBwrQ7aZAeqbdu0ZZjsgfCIlbYgJs0HjI5zQ65yGfBQXNdUTSq0IdfGgr5FjlK6gnDYExAIs8UqxxMxJQJTWcdgi9z7SlAv6RLnebHX7g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Do no select for post processing slots that are already compressed with same or higher priority compression algorithm. This should save some memory, as previously we would still put those entries into corresponding post-processing buckets and filter them out later in recompress_slot(). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 776c31606eec..6dee885bef9b 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1823,7 +1823,7 @@ static int zram_bvec_write(struct zram *zram, struct bio_vec *bvec, #define RECOMPRESS_IDLE (1 << 0) #define RECOMPRESS_HUGE (1 << 1) -static int scan_slots_for_recompress(struct zram *zram, u32 mode, +static int scan_slots_for_recompress(struct zram *zram, u32 mode, u32 prio_max, struct zram_pp_ctl *ctl) { unsigned long nr_pages = zram->disksize >> PAGE_SHIFT; @@ -1855,6 +1855,10 @@ static int scan_slots_for_recompress(struct zram *zram, u32 mode, zram_test_flag(zram, index, ZRAM_INCOMPRESSIBLE)) goto next; + /* Already compressed with same of higher priority */ + if (zram_get_priority(zram, index) + 1 >= prio_max) + goto next; + pps->index = index; place_pp_slot(zram, ctl, pps); pps = NULL; @@ -1911,6 +1915,16 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, zram_clear_flag(zram, index, ZRAM_IDLE); class_index_old = zs_lookup_class_index(zram->mem_pool, comp_len_old); + + prio = max(prio, zram_get_priority(zram, index) + 1); + /* + * Recompression slots scan should not select slots that are + * already compressed with a higher priority algorithm, but + * just in case + */ + if (prio >= prio_max) + return 0; + /* * Iterate the secondary comp algorithms list (in order of priority) * and try to recompress the page. @@ -1919,13 +1933,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (!zram->comps[prio]) continue; - /* - * Skip if the object is already re-compressed with a higher - * priority algorithm (or same algorithm). - */ - if (prio <= zram_get_priority(zram, index)) - continue; - num_recomps++; zstrm = zcomp_stream_get(zram->comps[prio]); src = kmap_local_page(page); @@ -2150,7 +2157,7 @@ static ssize_t recompress_store(struct device *dev, goto release_init_lock; } - scan_slots_for_recompress(zram, mode, ctl); + scan_slots_for_recompress(zram, mode, prio_max, ctl); ret = len; while ((pps = select_pp_slot(ctl))) { From patchwork Mon Mar 3 02:03:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998060 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E77AC19F32 for ; Mon, 3 Mar 2025 02:25:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9362828000B; Sun, 2 Mar 2025 21:25:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8C022280006; Sun, 2 Mar 2025 21:25:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 739FB28000B; Sun, 2 Mar 2025 21:25:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4EE59280006 for ; Sun, 2 Mar 2025 21:25:26 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id F08491207BB for ; Mon, 3 Mar 2025 02:25:25 +0000 (UTC) X-FDA: 83178648210.25.E1CFA6D Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf14.hostedemail.com (Postfix) with ESMTP id 13A73100004 for ; Mon, 3 Mar 2025 02:25:23 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=JoMJcUKE; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.169 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968724; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HsfvfF6qJrjX8Yvxt2fgmxWzD/A8epAgFC3OUlbDf8w=; b=aOdcOGRhWQkgkYl5mFPzblGCaPvfSq9LfCnOb2UA4A8hpE8I76fJVyY4cpCWT/7ZphxbHL WY8xUmBSANNB7azbHYpEUuGMcg4z8iNmaJUnU+5Bl+8UM2LXzCN1UONzxFOmwavSf6/qVu WsAbxrVK4mleR5us+ZxQ4pTByath7vk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968724; a=rsa-sha256; cv=none; b=dbqnQmeLl4m85QpRJjsspXUkE125FRfwPlxNvqsOOvR8vjXtIcth/OalgCVqNhtqP/dx1r ZjTxEG4wdKGrDrZESBUYnzW01aehBDt+DppYlBHpwPLI4qOL0UNN8kiezwIWYc6cZhRY1p 9T+6q9vIieS2wvjBp08vOuy6qsBBcxE= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=JoMJcUKE; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.169 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-22382657540so21842965ad.2 for ; Sun, 02 Mar 2025 18:25:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968723; x=1741573523; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HsfvfF6qJrjX8Yvxt2fgmxWzD/A8epAgFC3OUlbDf8w=; b=JoMJcUKEQgYy7KxlTU0GNEsXL7bVa2v5FtJmpMYeEVisLQi2uZXWxiUVfOPG8RTFRx uOwAKme3ho2tvyIedfdBuWU8vKk/Z4mpTBWjcoCXM2CpcoMFB2f9oxv9Fhzav3099TiK 0W4xKglnpy7mR/euBdJ6Gxfakwfk+eAfxuy5E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968723; x=1741573523; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HsfvfF6qJrjX8Yvxt2fgmxWzD/A8epAgFC3OUlbDf8w=; b=rp3lOBV7sQPyAV2x5BGdamhzFu8/iP7Fc+BpEtcXgRSBXg3P+PFkBbFp8z1k4KaGbs HfEsd3LXFrgIZ3i2fEKdwJO9hpKAHfUWaoAOpJfGYwginkXOQXghptGZF/UIB9AYG98K U4+km5/hrp7/FQHD87MMx2Nj5Gg7PoZlsLqeMZ54Qi6+JTKltXCCTW94ZMQW8R4koyO6 2eDMRV33EP1EaNw53e5trww1cs4O0z0hAFx7XxryjFzNa+EC5tImfm5nt32/6MerinpX /tqn+q1HWRojb4MLFbDGehisaUKIwyoay8JRwLbtEU3wtzegqoc/QBSnZXfJfxL7LUIS uOPw== X-Forwarded-Encrypted: i=1; AJvYcCXF+ZPgT5ozBSntv3yxhfzu1x1EqO8aPFAlPLM6ApSAhOUY1pVUXqp5Q+sjGF4WSWZbdm6hu8/NOg==@kvack.org X-Gm-Message-State: AOJu0YwGwpon35PRXfDJuX4fI0eQik/ppPRtEK8JGbZzeItBjW4+CfXD p2fAtn3b5/oZFmWkf1lq/IkEuV+rBLa+EFV0yKf5XzLuRyPyBd1Z1NNn/06/WQ== X-Gm-Gg: ASbGncs5rlz6AWdvTLqOt5qgeyVKR7aFmMCBbGpkRr2Oobz30kt4J/YPpzDMgGsmZFF 65N1hb1GhFnLy1xbsBaV63WWfKWzQpb7SWkddFiSpfqXK3yVyRtz7c2gpeI+FM+VoePTfyMcJSn 4BA0Ht3wWif3YOVWPsR1sV61sOJkg+m6Tcm4Pe7czEK7IAjKh4/7+4uBjgC/mu98VfdELVbPKbJ Yj1satMgaNLGPChXvv6g/u5fQVtlTIuStNbWFuma2ipG8Q5VGK8eROMXorE58iCSmmQIh9e6/eN sZLwZEmONLQVsx+k57780vhEcEzbIRORF3fX20fn5nYya4A= X-Google-Smtp-Source: AGHT+IESsySExdYFB+0EQ5FWR9mq2/h7S2JmgVfs+L+05mw3XKfkerymt3bltrTq9TH/mZJI9SEzSA== X-Received: by 2002:a05:6a00:a11:b0:730:927c:d451 with SMTP id d2e1a72fcca58-734ac42ce9fmr14388946b3a.20.1740968722924; Sun, 02 Mar 2025 18:25:22 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-7364ba1371asm1572928b3a.5.2025.03.02.18.25.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:25:22 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 10/19] zram: rework recompression loop Date: Mon, 3 Mar 2025 11:03:19 +0900 Message-ID: <20250303022425.285971-11-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Stat-Signature: 1hjaytnocn178sdyjmif4enh89iy4i15 X-Rspamd-Queue-Id: 13A73100004 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1740968723-662105 X-HE-Meta: U2FsdGVkX18Nj35vzqeyo2JG6yuFrq+U80fEZWpERg/zgkhUHcPdVfuEGnWC33GKdlk+gkUGnXl7EjDrd16zEi9CTVMUkzelvfwgKIZ+/TtbWTnGMQZvky/OwPETtSRg76tc7bj+AVUYrJD7J2qTa0vfqwfFmyg8ippjD0ehfHfd5Jr0ppoUUWlx/Jn7I7eRy3woKaDiso6922Xs7tdlmQL0ESz3GRcELPzIoV2hneVgY+CzIpHcFqHuReFBknQnN5zFIe4SFusryxF+DmSGhjv4HhCibiKm3Ut2TNsHuoRk7aldgaairweqNmlmxj6wVMDYDHt/uwRePPrWByOfDRleDLioFXVdFfiQwoJXlvJqeWCelnVhEFND/G5O3DV69V7tL6+BKF2mA7zaylnLiFqv/IhIn/HmTWQjJUefjpq816vPDiARnhBdjxUBb4HNu1H3aR5q5qM12jwDH9vaKH1LW+oW7b2CPehN0Z1ZzxYjzDsTIzLp/owJRv+mgqVk5zl1xTVwVKFGbngtPZlD9H2KietpNpOBJOvb0ffqpsX+4u4eHF6k1aIJwSCm6KeGcwuTfAdT67r3pdfiBlfOT2EOo7AzNHqhB0/HN3AWUuvopbpvSm8ixRMz9xr+v+HeiuCgbU2U1MdN8rXgnug6WatDwv/EGofBww/B0RBaIykIp5RSrapnPI6/oCrhTxXERzlpv3h75A6NEZ9ZdjIH6gVfAXb8kmdSpvWD2Su7XedR03WUnu0ddt7ZCMRrtea7kG+A+6HiHrxLKwWVKAXcII9qp/ltamEIYHrl9aREV014d2K4N6t3u4gTbBFKSGrSVrgsMzelVFZHxH8LuLVwaM6rlFpEDIciPAZyYmNpMPAjlaP8mSdoTAb9GCnQYYmkm/BrcxsmGxaS0EBEAmwgi6Oo1e00koZL3Y1+nToz2vOs4PvU0lbsL/4LTuxXi9M0pkDCyvlVTE3U/C5GepP D1U2vMfL Cr3sIf8Hol7aXIIgiRT/0xmVboM1oNiwcdTV6xDUILOhX53UF0txahhW9cj6KVLlskwX9ljQd+TXqKetYFrxR1ifJECEXor6q2V6yqnnUPl4Mes4Tcig0RAliTuza0fl8TEjhyKuyA5MXSg2YWC7Zac7BJxTCuU+ppqors9+RDMYjqVyKahn8sPuTxGpt+K3yUif+370ZGB/lf6pUFS2HRVy7Sd/mYUzGGYgKCFwNzuVs9Hhy4KbbHgZGCyE0xN6yUhf+89/0BZnyXzqNi/vIPNcop9lp+aSi6wMkmpj3brr0XuZYzvOz3M/vSQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This reworks recompression loop handling: - set a rule that stream-put NULLs the stream pointer If the loop returns with a non-NULL stream then it's a successful recompression, otherwise the stream should always be NULL. - do not count the number of recompressions Mark object as incompressible as soon as the algorithm with the highest priority failed to compress that object. - count compression errors as resource usage Even if compression has failed, we still need to bump num_recomp_pages counter. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 54 +++++++++++++---------------------- 1 file changed, 20 insertions(+), 34 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 6dee885bef9b..bb88b63d193b 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1888,9 +1888,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, unsigned int comp_len_new; unsigned int class_index_old; unsigned int class_index_new; - u32 num_recomps = 0; void *src, *dst; - int ret; + int ret = 0; handle_old = zram_get_handle(zram, index); if (!handle_old) @@ -1933,7 +1932,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (!zram->comps[prio]) continue; - num_recomps++; zstrm = zcomp_stream_get(zram->comps[prio]); src = kmap_local_page(page); ret = zcomp_compress(zram->comps[prio], zstrm, @@ -1942,7 +1940,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (ret) { zcomp_stream_put(zstrm); - return ret; + zstrm = NULL; + break; } class_index_new = zs_lookup_class_index(zram->mem_pool, @@ -1952,6 +1951,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (class_index_new >= class_index_old || (threshold && comp_len_new >= threshold)) { zcomp_stream_put(zstrm); + zstrm = NULL; continue; } @@ -1959,14 +1959,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, break; } - /* - * We did not try to recompress, e.g. when we have only one - * secondary algorithm and the page is already recompressed - * using that algorithm - */ - if (!zstrm) - return 0; - /* * Decrement the limit (if set) on pages we can recompress, even * when current recompression was unsuccessful or did not compress @@ -1976,38 +1968,32 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (*num_recomp_pages) *num_recomp_pages -= 1; - if (class_index_new >= class_index_old) { + /* Compression error */ + if (ret) + return ret; + + if (!zstrm) { /* * Secondary algorithms failed to re-compress the page - * in a way that would save memory, mark the object as - * incompressible so that we will not try to compress - * it again. + * in a way that would save memory. * - * We need to make sure that all secondary algorithms have - * failed, so we test if the number of recompressions matches - * the number of active secondary algorithms. + * Mark the object incompressible if the max-priority + * algorithm couldn't re-compress it. */ - if (num_recomps == zram->num_active_comps - 1) - zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); + if (prio < zram->num_active_comps) + return 0; + zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); return 0; } - /* Successful recompression but above threshold */ - if (threshold && comp_len_new >= threshold) - return 0; - /* - * No direct reclaim (slow path) for handle allocation and no - * re-compression attempt (unlike in zram_write_bvec()) since - * we already have stored that object in zsmalloc. If we cannot - * alloc memory for recompressed object then we bail out and - * simply keep the old (existing) object in zsmalloc. + * We are holding per-CPU stream mutex and entry lock so better + * avoid direct reclaim. Allocation error is not fatal since + * we still have the old object in the mem_pool. */ handle_new = zs_malloc(zram->mem_pool, comp_len_new, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + GFP_NOIO | __GFP_NOWARN | + __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { zcomp_stream_put(zstrm); return PTR_ERR((void *)handle_new); From patchwork Mon Mar 3 02:03:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998061 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F30F2C19F32 for ; Mon, 3 Mar 2025 02:25:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6C5A128000C; Sun, 2 Mar 2025 21:25:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 64E10280006; Sun, 2 Mar 2025 21:25:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4C86628000C; Sun, 2 Mar 2025 21:25:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 27B60280006 for ; Sun, 2 Mar 2025 21:25:31 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E59AC1207BA for ; Mon, 3 Mar 2025 02:25:30 +0000 (UTC) X-FDA: 83178648420.08.718653C Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf16.hostedemail.com (Postfix) with ESMTP id 03AD918000E for ; Mon, 3 Mar 2025 02:25:28 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=jd9Qo2zB; spf=pass (imf16.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.177 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968729; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/kGJl06ulFbB/829XkPkygYoBTQLBZ6+y+IJrZY3g6o=; b=iQfvKkVdk6u9tpxyjWnymRap8liLf3Po1cLIerr82NPw3zBCKA/B/3Wx+egnYfMl78a0KD q56nAfFiud8X8awOkppYolug5TQG4/nUNGdyXekHiCHckxbTFAgkao2lyRvfFnKyuV7Vwj c8/V0VS29DiQ72TJc52BlC4IAvGH6Hw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968729; a=rsa-sha256; cv=none; b=lXRo3ruWX0QnMsgqnWyGugCSCrBWhrJvllhaAh/NgBV7YGyZrDxWHEn9IJiA8XgAZx5wDJ BX6fvYZqOx45udILhHmjQaPTu2EsnrbSSghuVdEBhdWzk7yAzWHp6swh5OIhZEFyI7irzQ TdkCOe4sHckq9UOed0KO9/60CG7N0XE= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=jd9Qo2zB; spf=pass (imf16.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.177 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-2232aead377so74994095ad.0 for ; Sun, 02 Mar 2025 18:25:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968728; x=1741573528; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/kGJl06ulFbB/829XkPkygYoBTQLBZ6+y+IJrZY3g6o=; b=jd9Qo2zBwJWbMgBLKgDO/dCkIweHyaCFUmzeuNHDEMkXkklbo3WvyqoNZTl6+OSUUL yw+JOJnWmVcfWjYPz+tSpppq4276Ln9ac0MxhxKXhSZag4EnA5ouWH3Itj6pOSBOVrU1 O+258FFbK0R52cwSm5nLcbTHDc8JnN5sr9I+o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968728; x=1741573528; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/kGJl06ulFbB/829XkPkygYoBTQLBZ6+y+IJrZY3g6o=; b=B+K1/8A+L1JWWm3EZHHtzXAAmth3J6U3b1PfJwTHHkXDjbv4nMdNXmn7mww8xqsFaP R4W96hPiAj62Q7OyDq5js3toGxl0CZAagXEJE4S6uUyrC/ik5SCrpQWWaf7v2xreIrJU rG4M5qQgL2oFJhHrd/KpqCdh9qrIgrxdiqeh4M5I/wgNsbUN9rPXdCro9xv3Xvu6/CjU GaCd2LEt8SM6DFzN74maRbZ2vZdB1pAKMhya1vERsm5TU16dTkrUgagnMU7lHOLDpAp/ l2NzsNyLjRtZSBWhFIDJcvGYIfxYUllNcL4kPY+EWAxzWTPJUKUDbkehk9JcChblGiUu 28HQ== X-Forwarded-Encrypted: i=1; AJvYcCVZ79CZkkh9O/Rbyiw7cfxDpeihtW0pmfIH7F5doWKVBtlC+AuN4NapA+kq3/Y5ryeC/k6zJBaHrw==@kvack.org X-Gm-Message-State: AOJu0YyEyD45gkEMVykkZa+1ly6yC4FUMS73BAp4Lh3oFPnvkCjVTz+o ercUfIr59pRnNSDFk+dZgkc7Od6hXQD0lR5HWBFTfDn7poF2XktdQ/sPbDIQ/Q== X-Gm-Gg: ASbGncuoQpG1AK4x+Js31y6Z88FvJ3CB6d6uc0noGczkpLhiVHj0HIRTVd02aWzNIBR hreMfzpo00Q4ml24gBZeLz5yx16qMxWQrnpMO6fMoCV1Wpt3oW6FYOGv/PZXNO96j6oNsW4piIs l+mWDa3UnPB4IThPiK2C6yYyGB9Fj1CGSF9UQXgqzNQsCGcdZ5AVi5n0wSTXdssZYbAgqsyo91j 3Zi/WSzgQr0xuGG1mVlSTKr/FWQRTutZGVJgfCLXt0UOZ0wVDfdUZ1LQ9TwjrsQKtQ01NfGjzoO zRur41kDEhTE+lcq0cug7BZjqrbwRebG58u8mAtOOjm5Pbg= X-Google-Smtp-Source: AGHT+IEugHIYcHEDUOXp58GXQB8o5prMjl8/tvIa3ET7IFiNu4ViKFsWk4lTQ15ZE884fzJ8YzuupA== X-Received: by 2002:a17:903:198e:b0:220:d272:534d with SMTP id d9443c01a7336-22368f92a4dmr206219935ad.22.1740968727919; Sun, 02 Mar 2025 18:25:27 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-223504dc489sm66538975ad.159.2025.03.02.18.25.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:25:27 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 11/19] zram: move post-processing target allocation Date: Mon, 3 Mar 2025 11:03:20 +0900 Message-ID: <20250303022425.285971-12-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Stat-Signature: yndg5mqa7cgnaktz8kkhn4fa1odmdgkt X-Rspamd-Queue-Id: 03AD918000E X-Rspam-User: X-HE-Tag: 1740968728-575 X-HE-Meta: U2FsdGVkX1/wP8w1X9gwb9gg5CHB145xVSLNMcF/QPNfIAiJ0aUwdi0M0f+xD41WDLSc8I2wzh6H8J5sqcvsCh8NbVb30Niajh3Xr9+eB7XQ2yTpT2CZ3Xo0apqzZ6n/HqPUM+Ym5Tgh5kJxpYmD+kqHuBYXMbY2noRYc7oqmX8hh0jp18HlY4gDUwabmogVcP403OtA7SobEh5e1EpTUIbeXn6rNk9a2Yu10a7UVZ7g+tHBoCWh07t/mh2Cyf3f/2mmdmWByZR3hG9Y6Eoqe8OEaOZdKWU58g4jaWLm7AR+lmUCE8rfVaYgeWstRy9brs+79PZf5IbopsyjXHkJNr2ol1jWVSfEa9qiGbxnHJoHuyFBlfiMr+Sx0kLsARlOYFr8Nnfht9W6P4MBa3/ZlmtNGo5abOZ8YvA3/YYddduKjjbdCJFipHp+0xESnW4BVS8ISMUgAyoh1QsrkdxFfb3yloGvbtFo5OI61wXdrHwittWZffedjG2E0jqb5ilaKx3yE3jnayJifVDnZ1BY4yrMCx6Rl9pmxSxpnDktkqkzBFnhJmqAGmlU89C5VqkWQVa1nZ9MuDOjS5q3rtBUwYpKR2xNtuAK60m48NfXt6Oq64JgAVwfDh6mP0rb+PjoVUJBLRGXZBHz+bHJM0+HL/0cSq7gjvUkqdg6tEL7ohHIzmmIrvMYeoPcq07dDARIHwpQN4IZvDUQYAN3sb7IfvA8a9uJQIdDb9x1DtEsjIbBIIDWJMNZBrh4jaR89n8gcfXlePhC1OZKGIH2kEI6wRLnZwTnxyR2rWYR3rcTG9RXkQKEQuGSMdUF0r/dM9C2cXybPi+/VUk9dkPxAR+uHvkg1MUOsnJcxJCb1H08So5p2F/uc2Zm/yRS2o0XmFBgGhho1+giny/4ElTb9g+vvKil089d2efqRooJShj46FWPFDYYZ/aTwbk5Eyryd+ZpYRsTlmNWGKMUQ6OUkhC aw/XcA9W mA3jyxH6wM1uCa2z29BFd1T9cgG5fbWpU2W0+ckizYTL+ec4qk4daFBgQeftMBEqYVlWwegxB1UJO2vJozbxdYb2bwoupYh1fBmi2LFDP1zqtaLHHylOGX1RmYsjLd0wpDKYzYZW1EY+MJ/KJNJ4IIKFvmVq1sdabmsjLEuoFgjBHtEtqbSkx8e0jGxaaE2WMFw6AcoifnII4y1B8wcl9GS+Md9cogmdDJ8gzyz7BW1q+A7kl3z2cBPfMmx8GRleefApyQUiwIZ5BwSVS87Xk4aFMc69J2xYeYDdSuwELErUG4gm9Fvl8Etyrnu6u8zs/iV0xgvLBXVcQyA+WmBKmP0ua7+h77/XK2Fmk X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Allocate post-processing target in place_pp_slot(). This simplifies scan_slots_for_writeback() and scan_slots_for_recompress() loops because we don't need to track pps pointer state anymore. Previously we have to explicitly NULL the point if it has been added to a post-processing bucket or re-use previously allocated pointer otherwise and make sure we don't leak the memory in the end. We are also fine doing GFP_NOIO allocation, as post-processing can be called under memory pressure so we better pick as many slots as we can as soon as we can and start post-processing them, possibly saving the memory. Allocation failure there is not fatal, we will post-process whatever we put into the buckets on previous iterations. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 50 +++++++++++++++-------------------- 1 file changed, 22 insertions(+), 28 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index bb88b63d193b..f6e887f94b71 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -295,15 +295,24 @@ static void release_pp_ctl(struct zram *zram, struct zram_pp_ctl *ctl) kfree(ctl); } -static void place_pp_slot(struct zram *zram, struct zram_pp_ctl *ctl, - struct zram_pp_slot *pps) +static bool place_pp_slot(struct zram *zram, struct zram_pp_ctl *ctl, + u32 index) { - u32 idx; + struct zram_pp_slot *pps; + u32 bid; + + pps = kmalloc(sizeof(*pps), GFP_NOIO | __GFP_NOWARN); + if (!pps) + return false; - idx = zram_get_obj_size(zram, pps->index) / PP_BUCKET_SIZE_RANGE; - list_add(&pps->entry, &ctl->pp_buckets[idx]); + INIT_LIST_HEAD(&pps->entry); + pps->index = index; + + bid = zram_get_obj_size(zram, pps->index) / PP_BUCKET_SIZE_RANGE; + list_add(&pps->entry, &ctl->pp_buckets[bid]); zram_set_flag(zram, pps->index, ZRAM_PP_SLOT); + return true; } static struct zram_pp_slot *select_pp_slot(struct zram_pp_ctl *ctl) @@ -737,15 +746,8 @@ static int scan_slots_for_writeback(struct zram *zram, u32 mode, unsigned long index, struct zram_pp_ctl *ctl) { - struct zram_pp_slot *pps = NULL; - for (; nr_pages != 0; index++, nr_pages--) { - if (!pps) - pps = kmalloc(sizeof(*pps), GFP_KERNEL); - if (!pps) - return -ENOMEM; - - INIT_LIST_HEAD(&pps->entry); + bool ok = true; zram_slot_lock(zram, index); if (!zram_allocated(zram, index)) @@ -765,14 +767,13 @@ static int scan_slots_for_writeback(struct zram *zram, u32 mode, !zram_test_flag(zram, index, ZRAM_INCOMPRESSIBLE)) goto next; - pps->index = index; - place_pp_slot(zram, ctl, pps); - pps = NULL; + ok = place_pp_slot(zram, ctl, index); next: zram_slot_unlock(zram, index); + if (!ok) + break; } - kfree(pps); return 0; } @@ -1827,16 +1828,10 @@ static int scan_slots_for_recompress(struct zram *zram, u32 mode, u32 prio_max, struct zram_pp_ctl *ctl) { unsigned long nr_pages = zram->disksize >> PAGE_SHIFT; - struct zram_pp_slot *pps = NULL; unsigned long index; for (index = 0; index < nr_pages; index++) { - if (!pps) - pps = kmalloc(sizeof(*pps), GFP_KERNEL); - if (!pps) - return -ENOMEM; - - INIT_LIST_HEAD(&pps->entry); + bool ok = true; zram_slot_lock(zram, index); if (!zram_allocated(zram, index)) @@ -1859,14 +1854,13 @@ static int scan_slots_for_recompress(struct zram *zram, u32 mode, u32 prio_max, if (zram_get_priority(zram, index) + 1 >= prio_max) goto next; - pps->index = index; - place_pp_slot(zram, ctl, pps); - pps = NULL; + ok = place_pp_slot(zram, ctl, index); next: zram_slot_unlock(zram, index); + if (!ok) + break; } - kfree(pps); return 0; } From patchwork Mon Mar 3 02:03:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998062 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 828D8C282C6 for ; Mon, 3 Mar 2025 02:25:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1219B28000D; Sun, 2 Mar 2025 21:25:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0A99A280006; Sun, 2 Mar 2025 21:25:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E169B28000D; Sun, 2 Mar 2025 21:25:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id C016C280006 for ; Sun, 2 Mar 2025 21:25:35 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 74A6B160A20 for ; Mon, 3 Mar 2025 02:25:35 +0000 (UTC) X-FDA: 83178648630.29.6E5E5C6 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by imf15.hostedemail.com (Postfix) with ESMTP id 9082CA000A for ; Mon, 3 Mar 2025 02:25:33 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=DWTYNiFh; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf15.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.181 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968733; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CYVJH5MaLfbxaKGZoDOncyDyqc70bzD7eQWV2O8vsj8=; b=gkibOwPXSA+KkLy2ScbPgz2hiXppwnfTg5RVAoKbadhaJV778u+tIesfOwNWNWj1X/2kvs n6VNP0KX1QVMaNYCc4xeqYUWcG/n1CbngBg3ugY7Zd4p38tAKHzBAyLOk58FzhRf21vkLH hZRA9N7Q/RkTkN1i0qkwpBZbxnBbs7s= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968733; a=rsa-sha256; cv=none; b=P5lX5TVH+QPFqZoU2YjbdFLOT0YfJR+1+ATyE6FYz2aYaZx+UtKy2ny9ui159eSV1MHzWn Fu+BzlvfXt8DFthkj8EpyR5kgfkHJy9P8UnMRxH3IycH2nc2VEr1HgsooIlw4iP52+IjmR 6WnoYLnlIsOb03FtpVvDT2OI+Uiwcis= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=DWTYNiFh; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf15.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.181 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2234e4b079cso69136855ad.1 for ; Sun, 02 Mar 2025 18:25:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968732; x=1741573532; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CYVJH5MaLfbxaKGZoDOncyDyqc70bzD7eQWV2O8vsj8=; b=DWTYNiFh0N5RlayShO2PGdKurtwhTLZWWO5b4XJxtbz3TnegW9h6SMxNFJDXc+jC+m 2GHzfw4uMB7A7bo8DrxcC2DosQ7h86pc7oSl/RKysJ5suFEG61pUrzrtexTtG9lVDYFy uwGk0LQjAbJi8dgbUyxfOXhgIT3cWyumdfLdk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968732; x=1741573532; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CYVJH5MaLfbxaKGZoDOncyDyqc70bzD7eQWV2O8vsj8=; b=hewDagChFrWNSc1vtdJoxgD/KHUmSdOl4xlS2U3n/l3WjckOGSe37pGE6gHu1c8eMH TmCnPLAcv3fq1GJFd0/7f7EvfR1gvfxfc7SuwxWqQyW2QP7ZA1eBqm07am5kIDfwKbnP llx7vIFgeERbB0cCInEmtFFpI/OIB81cuSmrP4XjZv2eQv10gwIjI9n+d9yanrIFRHl8 GCPlLrO8eOwFQIBSZVZSgl5HEosUrk5Fyw/tH0Y15RbzIhiMmxg9CSv0mNLUx6Zlje0n pBaBGtxsUBCXvJ7J+adAGkTV4oNeKtEJdS8cVDJw2LwSysfk0wUrXUTUq9nPO2DvkD4U A5/w== X-Forwarded-Encrypted: i=1; AJvYcCWOXE6hcRzidGlrruisTW3NRjQ14pqpVrCqr9rcCQ+8Qf/9FB9hJASV/xhiCmaCbOO/6RAPWTnozg==@kvack.org X-Gm-Message-State: AOJu0Yyi29/4/PvQG3p0EA2V3/LXtv1bvDteD6/iiC0XcIJ7t1xfGQAH MMLbRF+oT/tzkrEIEYmtNndE1KJ/qeUYKEYHorlj3h5BSP6165mvUxgvzOBO2A== X-Gm-Gg: ASbGncuKfdE1wQJRYsfOB08s9RvpNpq4Wl8WVuMUNFirFZ6emtJNTzufASe5CXFhSY9 dhiB1fnn5gMXbbZlUvE3+Mg+SLPs+o6G68rt8L+qITLfHM2ta3CNltqLq/oUkj6vXqUiDAAinVl EXF029f7785PbyZpjkZB4WbBDMKiASwgXlfbjIc9GJtOzkYtSp7e2hZN5PMvl8tCBGBC7y0YZef H8EgivjtkVY+GZsDUw5OEjerDgOZ028iX+Cz+EcGiNpelZnQO/vVzcOJSrmaPbFv4GZAVVexNAg UtNhk4UNcijQ3MdgKxPQRMewl5uShfmKk0VTzehtdj+CAo8= X-Google-Smtp-Source: AGHT+IEgYxZUnegN30z6tSMQEsHJ1L6y7oPydeFduqm2ElX6isqJ4TV2BparuhraX3uF1fCfP0eGTw== X-Received: by 2002:a17:902:d2d0:b0:21f:6cb2:e949 with SMTP id d9443c01a7336-22369244fb0mr142389775ad.52.1740968732436; Sun, 02 Mar 2025 18:25:32 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-223501fb004sm66940215ad.64.2025.03.02.18.25.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:25:32 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 12/19] zsmalloc: rename pool lock Date: Mon, 3 Mar 2025 11:03:21 +0900 Message-ID: <20250303022425.285971-13-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 9082CA000A X-Stat-Signature: epj1g9p1ticn8p434t41635jixiz5k65 X-HE-Tag: 1740968733-650585 X-HE-Meta: U2FsdGVkX1+1zhaotmqvqcT1MxJUR5N8D3IO9p9eM/q5SKwr7pKKLqd0b2jO9ig/cz+jLn7l3+XQcBlv+n6tR/DjMiiQsGL/F4kl6GiMEH3CSCe8qxHWskbVdAnZQe8HhDcPWKSwpn6WCB+KHqiK4I2GjPSL1wgR2XeSyfU6Rtas8uE5j4VMSrYMPy5rXBAbyrIn38NTMka2Ll2NnJt1Eejr8Fx0sZ7R1aAABxUrPmZcqV+8wY+la1QeH8jHclS/xQ//o2NO9jQylChpAvpk9p48Lfg/CAkMxyEbLWjbDrF0AOzuMBF5zWBxU2u0H5gXnqiyFpZaIgzyj/DAe1ZFZgFwwCLRUs23D55+JPyXrcOJpDBuvtt5SbrpsgrurZ9wf5/3hQfBySDSeeCjTiegjR6Jc8y5Qj3CrZIi4VU7wHKQWP2SFJwRA6bUibtoOwdsLVE/ta2PJ6CT1Sqxuf3fYvSqOUlFywg3LsETu9nrpMh9n3SEScMwzTOemEsR3cJpPFylUQfKNEirMqtKRel431hP8xaZ+ed1uqs8oFFbuHGjb73lNn+SVmsMCg6RkUXCHXIDKCUCkhSreMGR3afEqkWkDZAngESqnLIqAICYYTMGnhQwJS7+Vz4q0rz4F0nC/4wnYRrHc8oa+7pc6ZOqo58Y4sKt3lBOcrbC1iXoZX9blnhOLnHnZtqF/QUgUd3tar+gw/If0mxlz+GZfbKF0fI+Ot5IPG3Q18jeUMrqzNKaDSwyxuuwNQFVou/BSjsRQ/h6lBMG6mPDg9DVoHoFIr6bKGEIW2qAxbgKhmCzNW6SKsh4YP/gil9DppiKRTvH6CRlPe9voYVDCoRyF9EXvmpxFGQrbEYo31VrGW0i/sFX7CZIoEeoICUlTQwbj1hlTdpJeimkIMzY54YIE2vF/x/mtGJRR9EX/9J6/GB3lhe+9EyR2/jxhRo5M9/6fkN3D1RbYXSnBzBXTvkrjP2 /RnVaiOn GxAAgP0+9NEajJNtyCyriuT2YhI5Na/WZLDjiTH16NBAxFMQ8RNQRlk7IwMN1fHpK1xLdJT9SLA2pbUISDiOaBPy8sI/fNDuiwv4HsoYg+v6KggMhHiL35vSUkP9kGeLiHysY4eo/nHQwV3d6K8kuQC3BLcOAPsMAKoyaph5EvggnJZyogPHz6E0TBEqsHI10eUPC34Cog4rhi5DEJ8WZN7Awj2mz+D7krLb7ENycw2lu723AV1PmlVITqrfv00ECcw/SjA2LVnpSOPDQAO7O5HHnSYPlvz51ETqPbbvG+BeqL7xrMxtIQjp7rg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The old name comes from the times when the pool did not have compaction (defragmentation). Rename it to ->lock because these days it synchronizes not only migration. Reviewed-by: Yosry Ahmed Signed-off-by: Sergey Senozhatsky --- mm/zsmalloc.c | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 817626a351f8..1424ee73cbb5 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -18,7 +18,7 @@ /* * lock ordering: * page_lock - * pool->migrate_lock + * pool->lock * class->lock * zspage->lock */ @@ -223,8 +223,8 @@ struct zs_pool { #ifdef CONFIG_COMPACTION struct work_struct free_work; #endif - /* protect page/zspage migration */ - rwlock_t migrate_lock; + /* protect zspage migration/compaction */ + rwlock_t lock; atomic_t compaction_in_progress; }; @@ -1206,7 +1206,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, BUG_ON(in_interrupt()); /* It guarantees it can get zspage from handle safely */ - read_lock(&pool->migrate_lock); + read_lock(&pool->lock); obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); zspage = get_zspage(zpdesc); @@ -1218,7 +1218,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, * which is smaller granularity. */ migrate_read_lock(zspage); - read_unlock(&pool->migrate_lock); + read_unlock(&pool->lock); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1450,16 +1450,16 @@ void zs_free(struct zs_pool *pool, unsigned long handle) return; /* - * The pool->migrate_lock protects the race with zpage's migration + * The pool->lock protects the race with zpage's migration * so it's safe to get the page from handle. */ - read_lock(&pool->migrate_lock); + read_lock(&pool->lock); obj = handle_to_obj(handle); obj_to_zpdesc(obj, &f_zpdesc); zspage = get_zspage(f_zpdesc); class = zspage_class(pool, zspage); spin_lock(&class->lock); - read_unlock(&pool->migrate_lock); + read_unlock(&pool->lock); class_stat_sub(class, ZS_OBJS_INUSE, 1); obj_free(class->size, obj); @@ -1796,7 +1796,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, * The pool migrate_lock protects the race between zpage migration * and zs_free. */ - write_lock(&pool->migrate_lock); + write_lock(&pool->lock); class = zspage_class(pool, zspage); /* @@ -1833,7 +1833,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. */ - write_unlock(&pool->migrate_lock); + write_unlock(&pool->lock); spin_unlock(&class->lock); migrate_write_unlock(zspage); @@ -1956,7 +1956,7 @@ static unsigned long __zs_compact(struct zs_pool *pool, * protect the race between zpage migration and zs_free * as well as zpage allocation/free */ - write_lock(&pool->migrate_lock); + write_lock(&pool->lock); spin_lock(&class->lock); while (zs_can_compact(class)) { int fg; @@ -1983,14 +1983,14 @@ static unsigned long __zs_compact(struct zs_pool *pool, src_zspage = NULL; if (get_fullness_group(class, dst_zspage) == ZS_INUSE_RATIO_100 - || rwlock_is_contended(&pool->migrate_lock)) { + || rwlock_is_contended(&pool->lock)) { putback_zspage(class, dst_zspage); dst_zspage = NULL; spin_unlock(&class->lock); - write_unlock(&pool->migrate_lock); + write_unlock(&pool->lock); cond_resched(); - write_lock(&pool->migrate_lock); + write_lock(&pool->lock); spin_lock(&class->lock); } } @@ -2002,7 +2002,7 @@ static unsigned long __zs_compact(struct zs_pool *pool, putback_zspage(class, dst_zspage); spin_unlock(&class->lock); - write_unlock(&pool->migrate_lock); + write_unlock(&pool->lock); return pages_freed; } @@ -2014,10 +2014,10 @@ unsigned long zs_compact(struct zs_pool *pool) unsigned long pages_freed = 0; /* - * Pool compaction is performed under pool->migrate_lock so it is basically + * Pool compaction is performed under pool->lock so it is basically * single-threaded. Having more than one thread in __zs_compact() - * will increase pool->migrate_lock contention, which will impact other - * zsmalloc operations that need pool->migrate_lock. + * will increase pool->lock contention, which will impact other + * zsmalloc operations that need pool->lock. */ if (atomic_xchg(&pool->compaction_in_progress, 1)) return 0; @@ -2139,7 +2139,7 @@ struct zs_pool *zs_create_pool(const char *name) return NULL; init_deferred_free(pool); - rwlock_init(&pool->migrate_lock); + rwlock_init(&pool->lock); atomic_set(&pool->compaction_in_progress, 0); pool->name = kstrdup(name, GFP_KERNEL); From patchwork Mon Mar 3 02:03:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF314C282C6 for ; Mon, 3 Mar 2025 02:25:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 42DE328000E; Sun, 2 Mar 2025 21:25:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3B5E2280006; Sun, 2 Mar 2025 21:25:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E11F28000E; Sun, 2 Mar 2025 21:25:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id ED7FC280006 for ; Sun, 2 Mar 2025 21:25:40 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A0E261A0A08 for ; Mon, 3 Mar 2025 02:25:40 +0000 (UTC) X-FDA: 83178648840.25.975D380 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf16.hostedemail.com (Postfix) with ESMTP id BD66718000D for ; Mon, 3 Mar 2025 02:25:38 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=CbM0reeF; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf16.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.169 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968738; a=rsa-sha256; cv=none; b=VQ72ABu5N8HuEtudPUXiCOAzttHwTYSuH2dmDPIB1LsbLTPt0H8GBGmcvMXBhXFnoeY7Xl KDEyn9oGLaip3bVDtyovPA+8y4Z1bTcAPtQfZgydA41oIlY1e0tKZgo/CTxsqbs2WxSSIS RG0MWIzMRGcNr+RGFXRiqGrf8piHIIs= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=CbM0reeF; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf16.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.169 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968738; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ua9rnaxl59hYuly6c376LLOYIZDnLCeuJzWW228A97g=; b=5MG9r1Ii6xe73OSfSXiQGDTMCyMmSTnktkZnGPYPmG7KEwMmHZamsRCD7OB4nXALS7Tfzp ggPgq0gmvSu+y0ZxBwYch9TwAcFDId1UznmBvJ27C4UWt+1EzyP1WgOPtVnCvVSE3NkMXb v9MYBFEY/wNmLCRbkbLhjk8V7AU/AfE= Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-219f8263ae0so70244355ad.0 for ; Sun, 02 Mar 2025 18:25:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968738; x=1741573538; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ua9rnaxl59hYuly6c376LLOYIZDnLCeuJzWW228A97g=; b=CbM0reeFPPV9+iq0CuuM588QQT4rIr1SbkWh4T87YhO5ikZEgzqlR1j5z9F5KK3PR5 ET+7r8O2Qs+YiOexQ5vI6A0+jCEyx7AA84oIFw6Av8CT6EmhOoJrI/9ZByWbsdvZ+rkQ 6YibUApuC2gHFBCJ1paiviZDnkghJSNK+MBEU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968738; x=1741573538; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ua9rnaxl59hYuly6c376LLOYIZDnLCeuJzWW228A97g=; b=rdfE+nd8gMjBnkkxbJnXjoFYaDWoG7VhvUWfxalXBIKkxhWbI2owmWc85kuAxfIGr+ I1glXIWc9X1n/6l19nkQY1JLnhm2jprTChCcxj1i7g0ZLNa+Gql3WdWD0sV6C1iEbaDr LuxnzyGa3li935BzA3p/VYcni+OlRhXPla+LpiVUgwY/Btjs75NiWwXF3V9QIYZCCxMT u4/QpylCuK9HR8fnzHeOdB9wyAiWRjkR81URJSAUMZGrjLctaOFO3nSE74wHCjuMstxH hby5E+beE+mehdkrXcFihyqOpat/RHyiE/M5LPij3qZAJj6sq2liIE+f5rA99RbwD5+L TcIg== X-Forwarded-Encrypted: i=1; AJvYcCV4cMMUZIsgNbU38+COh/4nGqTE1vfUNzDziMcpzDyhHJwPYHXup8hp6pGPia3ue9Uw3zopW9HwlQ==@kvack.org X-Gm-Message-State: AOJu0YwQU0sWs0SFi5r1t4LUIwicLo5hC9gUbZaAS/oCpyM8zl+DNAVx 1GGO3bAm34rvYo0Fcy3IySFCSKuDyTPIypLwpW7ReY4z6qIuWAlzKCwNRjzkwg== X-Gm-Gg: ASbGncsCA8tMQ9qxhBK3GpxZFNpor5xDjZVY7kwMRcUw5traJzuDCIw4g32wW44Jzq7 CbIyC3vW5SX/sn2KKbqRNahiaPRmmRQFU0+Ty8U5zWmm1Y0iQIOLNp/DIPyU+5IVEAHIwRsswma OrRkjUt6/uPtysPDxISP5YJjfTlbWt99uYiaoNhZV3lNhht6cZ7VkcPw5hFkALzGnVY+XnWJo4i 9x4YtElU8GiBqPbJBUJ8/wSfDnSgfaD74yIMk2GNtkZG9lvDqckTOogQqjHL5wyQiXtPzhXLzYa SvBoyYiJMwUGoCaCOzDRutU9CTuBV+qa2NzlJWs779g+AIY= X-Google-Smtp-Source: AGHT+IHtZ5c68bsAtTNZLqPzKISnIUv2uwTEuuqfc15VikOP6YUn0971YwA/MQ1+Ofo0Wu55CvXwCA== X-Received: by 2002:a05:6a00:a91:b0:730:99cb:7c32 with SMTP id d2e1a72fcca58-734ac350410mr17588314b3a.7.1740968737528; Sun, 02 Mar 2025 18:25:37 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-7356173aedfsm5377318b3a.95.2025.03.02.18.25.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:25:37 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 13/19] zsmalloc: sleepable zspage reader-lock Date: Mon, 3 Mar 2025 11:03:22 +0900 Message-ID: <20250303022425.285971-14-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: BD66718000D X-Stat-Signature: q8pqyfn8ms3hjbj6oaanhp1nkmiopg61 X-HE-Tag: 1740968738-367279 X-HE-Meta: U2FsdGVkX19yXnhzOMMBS2jx7J34FyQ7i2vqxtHvj0tOm++NXemLl6ECKfngPp9bffbBpA4TqPFsayRZVRXQoemY/eIbQ2OV6veZiYet17hRg+kVObkIXsoer061TwYDJAc+Rqw0VmOBJIorT/cQI0xI/SNjYTa4pKTqZOx5w3BjRa6YgYfd9uXh5QIK47CBwssfl7B5gNl2/Zew6X+UyLR8f40n916Zaj18DgMyck5NH+CC/jbqwFnSl9Nc7hfjjWhw2PSJEBKqc9v4bnzWxYgabAmWru6Q7SJ+GXtT0zOte5Dfyqbz/U9x42O8rMdBy91Hlb4Y8qdXNK2IDH0EH9fgYlG3wxnK+iSrvNg1V9WsL2YkciFp5eiBycIqkyU1T105rpz6gCK3SphVGxByUEpMkZ7lVjau8Ns8FxIJlxe6L10DmYHz3CzKYBI7tjH87d6mrbEmfE1Fb+6RA0cE3wYTdVjfIrxRaz3MKrHZ0pGayR4tu4i3/RkwBzcPpSp2iuxn/aaF18iTjQdGTSfjeEKvwb0O85Z8cIISwiSI5z3HrIzyocSaAwqdyIdd8+AOWOFjeq9kNG7GOqNpEBiDN2bxzvpM3J/kipNqIn+5hWpiNatDm3+bpdztGDq/03ohC2j/hjL6i3YX9YvMEyzwGpYrWWO66eDVflvQJFUtiBE6/unAwP4pwdbA94E1d71TZaGjbALzAuWjinTtNHhBo7aKGvooMw6Fud+3GxCxFhjlOwmOBYfd6vFcQoYFHnpwDEWO0nDHx7HbA8kaUgGt058fzWwpDT6sYEGIHixWpSEiUaOBjrRfSQ2HVmZMIcEkbLkQNhKQjurKNDXpO6ppMviF9nCqSwK65iK/G0vFkSgOB9zEMyOrpMi4JcFt+sn8RZ5g9UFM0X0C4pNjBfG/iI6hGfsTmUm9oaHsJGmVdnGB3xKa7yCBiwI15u2NPdb9rAvhCAaX8+cbY49s/F8 Z6muuNbD BiLeY01kKb0marVlxEdpqKZxfiuBiUgbQFN6r7XqXXlkON00p7sv9iIIu0PutVEVmiBiiTc0hJqg5B0h1c/1Eo85I5AGL8pNrCaSAc76BpLOOsQrqQzIAOyEGERsl22acL93PaXxzYLvPzQ6sH578rmJV2dz9bFpbVGaw4OZbbc3dFZEW01LMjnr2Og2cAZotMB9v4pzqDWKDo3dwke7jf7JEKbrnniJ65V20+YXa9aOMj0IiLHDmXhn254iFH8jsucqqrRV8cwiC0tZQ8I4axpZbd9EeG3zjj/llJsiUeHNfPAndJutDxBcpIdhJWshJOsCnrcTu4a84UsU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In order to implement preemptible object mapping we need a zspage lock that satisfies several preconditions: - it should be reader-write type of a lock - it should be possible to hold it from any context, but also being preemptible if the context allows it - we never sleep while acquiring but can sleep while holding in read mode An rwsemaphore doesn't suffice, due to atomicity requirements, rwlock doesn't satisfy due to reader-preemptability requirement. It's also worth to mention, that per-zspage rwsem is a little too memory heavy (we can easily have double digits megabytes used only on rwsemaphores). Switch over from rwlock_t to a atomic_t-based implementation of a reader-writer semaphore that satisfies all of the preconditions. The spin-lock based zspage_lock is suggested by Hillf Danton. Suggested-by: Hillf Danton Signed-off-by: Sergey Senozhatsky --- mm/zsmalloc.c | 166 ++++++++++++++++++++++++++++++++++---------------- 1 file changed, 114 insertions(+), 52 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 1424ee73cbb5..afbd72363731 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -257,6 +257,15 @@ static inline void free_zpdesc(struct zpdesc *zpdesc) __free_page(page); } +#define ZS_PAGE_UNLOCKED 0 +#define ZS_PAGE_WRLOCKED -1 + +struct zspage_lock { + spinlock_t lock; + int cnt; + struct lockdep_map dep_map; +}; + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -269,7 +278,7 @@ struct zspage { struct zpdesc *first_zpdesc; struct list_head list; /* fullness list */ struct zs_pool *pool; - rwlock_t lock; + struct zspage_lock zsl; }; struct mapping_area { @@ -279,6 +288,84 @@ struct mapping_area { enum zs_mapmode vm_mm; /* mapping mode */ }; +static void zspage_lock_init(struct zspage *zspage) +{ + static struct lock_class_key __key; + struct zspage_lock *zsl = &zspage->zsl; + + lockdep_init_map(&zsl->dep_map, "zspage->lock", &__key, 0); + spin_lock_init(&zsl->lock); + zsl->cnt = ZS_PAGE_UNLOCKED; +} + +/* + * The zspage lock can be held from atomic contexts, but it needs to remain + * preemptible when held for reading because it remains held outside of those + * atomic contexts, otherwise we unnecessarily lose preemptibility. + * + * To achieve this, the following rules are enforced on readers and writers: + * + * - Writers are blocked by both writers and readers, while readers are only + * blocked by writers (i.e. normal rwlock semantics). + * + * - Writers are always atomic (to allow readers to spin waiting for them). + * + * - Writers always use trylock (as the lock may be held be sleeping readers). + * + * - Readers may spin on the lock (as they can only wait for atomic writers). + * + * - Readers may sleep while holding the lock (as writes only use trylock). + */ +static void zspage_read_lock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + rwsem_acquire_read(&zsl->dep_map, 0, 0, _RET_IP_); + + spin_lock(&zsl->lock); + zsl->cnt++; + spin_unlock(&zsl->lock); + + lock_acquired(&zsl->dep_map, _RET_IP_); +} + +static void zspage_read_unlock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + rwsem_release(&zsl->dep_map, _RET_IP_); + + spin_lock(&zsl->lock); + zsl->cnt--; + spin_unlock(&zsl->lock); +} + +static __must_check bool zspage_write_trylock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + spin_lock(&zsl->lock); + if (zsl->cnt == ZS_PAGE_UNLOCKED) { + zsl->cnt = ZS_PAGE_WRLOCKED; + rwsem_acquire(&zsl->dep_map, 0, 1, _RET_IP_); + lock_acquired(&zsl->dep_map, _RET_IP_); + return true; + } + + spin_unlock(&zsl->lock); + return false; +} + +static void zspage_write_unlock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + rwsem_release(&zsl->dep_map, _RET_IP_); + + zsl->cnt = ZS_PAGE_UNLOCKED; + spin_unlock(&zsl->lock); +} + /* huge object: pages_per_zspage == 1 && maxobj_per_zspage == 1 */ static void SetZsHugePage(struct zspage *zspage) { @@ -290,12 +377,6 @@ static bool ZsHugePage(struct zspage *zspage) return zspage->huge; } -static void migrate_lock_init(struct zspage *zspage); -static void migrate_read_lock(struct zspage *zspage); -static void migrate_read_unlock(struct zspage *zspage); -static void migrate_write_lock(struct zspage *zspage); -static void migrate_write_unlock(struct zspage *zspage); - #ifdef CONFIG_COMPACTION static void kick_deferred_free(struct zs_pool *pool); static void init_deferred_free(struct zs_pool *pool); @@ -992,7 +1073,9 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, return NULL; zspage->magic = ZSPAGE_MAGIC; - migrate_lock_init(zspage); + zspage->pool = pool; + zspage->class = class->index; + zspage_lock_init(zspage); for (i = 0; i < class->pages_per_zspage; i++) { struct zpdesc *zpdesc; @@ -1015,8 +1098,6 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, create_page_chain(class, zspage, zpdescs); init_zspage(class, zspage); - zspage->pool = pool; - zspage->class = class->index; return zspage; } @@ -1217,7 +1298,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, * zs_unmap_object API so delegate the locking from class to zspage * which is smaller granularity. */ - migrate_read_lock(zspage); + zspage_read_lock(zspage); read_unlock(&pool->lock); class = zspage_class(pool, zspage); @@ -1277,7 +1358,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) } local_unlock(&zs_map_area.lock); - migrate_read_unlock(zspage); + zspage_read_unlock(zspage); } EXPORT_SYMBOL_GPL(zs_unmap_object); @@ -1671,18 +1752,18 @@ static void lock_zspage(struct zspage *zspage) /* * Pages we haven't locked yet can be migrated off the list while we're * trying to lock them, so we need to be careful and only attempt to - * lock each page under migrate_read_lock(). Otherwise, the page we lock + * lock each page under zspage_read_lock(). Otherwise, the page we lock * may no longer belong to the zspage. This means that we may wait for * the wrong page to unlock, so we must take a reference to the page - * prior to waiting for it to unlock outside migrate_read_lock(). + * prior to waiting for it to unlock outside zspage_read_lock(). */ while (1) { - migrate_read_lock(zspage); + zspage_read_lock(zspage); zpdesc = get_first_zpdesc(zspage); if (zpdesc_trylock(zpdesc)) break; zpdesc_get(zpdesc); - migrate_read_unlock(zspage); + zspage_read_unlock(zspage); zpdesc_wait_locked(zpdesc); zpdesc_put(zpdesc); } @@ -1693,41 +1774,16 @@ static void lock_zspage(struct zspage *zspage) curr_zpdesc = zpdesc; } else { zpdesc_get(zpdesc); - migrate_read_unlock(zspage); + zspage_read_unlock(zspage); zpdesc_wait_locked(zpdesc); zpdesc_put(zpdesc); - migrate_read_lock(zspage); + zspage_read_lock(zspage); } } - migrate_read_unlock(zspage); + zspage_read_unlock(zspage); } #endif /* CONFIG_COMPACTION */ -static void migrate_lock_init(struct zspage *zspage) -{ - rwlock_init(&zspage->lock); -} - -static void migrate_read_lock(struct zspage *zspage) __acquires(&zspage->lock) -{ - read_lock(&zspage->lock); -} - -static void migrate_read_unlock(struct zspage *zspage) __releases(&zspage->lock) -{ - read_unlock(&zspage->lock); -} - -static void migrate_write_lock(struct zspage *zspage) -{ - write_lock(&zspage->lock); -} - -static void migrate_write_unlock(struct zspage *zspage) -{ - write_unlock(&zspage->lock); -} - #ifdef CONFIG_COMPACTION static const struct movable_operations zsmalloc_mops; @@ -1785,9 +1841,6 @@ static int zs_page_migrate(struct page *newpage, struct page *page, VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); - /* We're committed, tell the world that this is a Zsmalloc page. */ - __zpdesc_set_zsmalloc(newzpdesc); - /* The page is locked, so this pointer must remain valid */ zspage = get_zspage(zpdesc); pool = zspage->pool; @@ -1803,8 +1856,15 @@ static int zs_page_migrate(struct page *newpage, struct page *page, * the class lock protects zpage alloc/free in the zspage. */ spin_lock(&class->lock); - /* the migrate_write_lock protects zpage access via zs_map_object */ - migrate_write_lock(zspage); + /* the zspage write_lock protects zpage access via zs_map_object */ + if (!zspage_write_trylock(zspage)) { + spin_unlock(&class->lock); + write_unlock(&pool->lock); + return -EINVAL; + } + + /* We're committed, tell the world that this is a Zsmalloc page. */ + __zpdesc_set_zsmalloc(newzpdesc); offset = get_first_obj_offset(zpdesc); s_addr = kmap_local_zpdesc(zpdesc); @@ -1835,7 +1895,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, */ write_unlock(&pool->lock); spin_unlock(&class->lock); - migrate_write_unlock(zspage); + zspage_write_unlock(zspage); zpdesc_get(newzpdesc); if (zpdesc_zone(newzpdesc) != zpdesc_zone(zpdesc)) { @@ -1971,9 +2031,11 @@ static unsigned long __zs_compact(struct zs_pool *pool, if (!src_zspage) break; - migrate_write_lock(src_zspage); + if (!zspage_write_trylock(src_zspage)) + break; + migrate_zspage(pool, src_zspage, dst_zspage); - migrate_write_unlock(src_zspage); + zspage_write_unlock(src_zspage); fg = putback_zspage(class, src_zspage); if (fg == ZS_INUSE_RATIO_0) { From patchwork Mon Mar 3 02:03:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998064 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0076C282C6 for ; Mon, 3 Mar 2025 02:25:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4F0DA280010; Sun, 2 Mar 2025 21:25:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4794B28000F; Sun, 2 Mar 2025 21:25:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2A54F280010; Sun, 2 Mar 2025 21:25:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0710328000F for ; Sun, 2 Mar 2025 21:25:46 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id B478D1409EE for ; Mon, 3 Mar 2025 02:25:45 +0000 (UTC) X-FDA: 83178649050.20.BCAD2D7 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by imf27.hostedemail.com (Postfix) with ESMTP id C808940007 for ; Mon, 3 Mar 2025 02:25:43 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=MSeJf3Eo; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf27.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.174 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968743; a=rsa-sha256; cv=none; b=pWV48PDXw6wOJHMcSgFWqNWd9wGrqywYn5ww3LCdVTvI/CVH3JAyIvJ7YZSxf2YReTZX7L zMggxD0cI61sbjhjdyfYVor7GXZQgZFHeVvYb2U6uWRvN8tjq9wnVgZoi7OR3hWKtbXt2e HrCVolX9gcCsw8Zl6uveRBLdyghfgL8= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=MSeJf3Eo; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf27.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.174 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968743; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LmmGIMp09Xa9+ajjYrlkAeQFbd0l0inskGKz/hBM/Wo=; b=bf8RzrBjeQAZY3V9qgkEd+RX4NZZzSC2T5MheKaGbPx1now4kyEoSShFBWhFCTFnhAD/Si rSSCM8gCtxk6B1NKUA/y2345tRhj68Yv9h7ua49loVNm4rdwRjJ7wBWSGGYPL/FkX18dhM i6CKY1U0io6lR9PZdP3k4L7n8nYHGLc= Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-223a3c035c9so9001005ad.1 for ; Sun, 02 Mar 2025 18:25:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968743; x=1741573543; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LmmGIMp09Xa9+ajjYrlkAeQFbd0l0inskGKz/hBM/Wo=; b=MSeJf3EosmX3wWwpCDC/RrtsVvYJmDGiZ46AKJhsTZks3AphVXCcmIgtgD+VWBb4Jb TUkAw1mexIK0m3uX/SjSOA1N88lQ77f+C3gaPBNEKdV7BPcRDQnBpBcsNYNaQ/dXW6b/ 19wMQLlzPi8XDtTwmb5ctzqXvh3wCBNKVy+5w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968743; x=1741573543; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LmmGIMp09Xa9+ajjYrlkAeQFbd0l0inskGKz/hBM/Wo=; b=bj9lf75Ze2gvAdtDbK9bHxdkdZ2lKUOIYqgA3MHe5F96uWE/uWM6vXeQK3RtuCVi6D Z3C3acHqrCoC8E6jaeFTotouE2MsfKYmEV3WsLwROmQX6qxMeXfYOdoef2WULy0iCc6U e+LXaNH6RTBmIN7f4zZvS7KUcGk49R3jTRltAAAVBvUMWR+Xf1cAqjISA4+VoVPUbdkc qVopTR2G6Q9wYuZSIALF5zvBEequDVd5SdyKfNO4kga+nvG/K2OZsZElyBP2uZEpqBjH +xkjw4tHwIsjwrylfmghE3j1lPNrmrVoL5r6eDqoZW6MYWa5Ux/NSzewAURHYuiJ7NvB aVlQ== X-Forwarded-Encrypted: i=1; AJvYcCWdTZPaJLZZ26frFYevYGjDyykdcUdjaIj/YlTknqLr6aMGgezKXmaOXP2F9nTu6nWORGfrq4hC4A==@kvack.org X-Gm-Message-State: AOJu0YwITKaIgxJdXs4YdkX0XY36mzBQQJbS9jK+A3OEgRBz/N0cgkhc C+Ou/Lc8WZA4n1jGwvYPahCz3UZpCOn6GrRUCdclgTKU/Djyp9+pe1FjgC25yw== X-Gm-Gg: ASbGncsRK5Rth+rj6zxAGZjzT8ei6um8UMRuyl0CcpCmrYfOL+1xvzqOwiHEU8JhAgf LV7cZsWU/iCHGc93hw3K9JMe6sQiCPinmyB6MlLiaGgVyJB8Z2wrjT9/t2YCS1TNXai4ftgS4HT oBNrDBS767ldqJ5GyF2TqCLOnLV3GrQjHi8nZeCxr4+qunPEcPMTNDCJQf/GDMpLKVfWOIzPNkZ dcXekA8rwgKJJyHYPRrAeJARhDWzN9W7ZPHT/OdZvPsZajVzuf2EAUHpiEowVA0ImNpeJVffY1O zAhR7nDY1+miL7q7luvTEi9QOQv5SuTsLsKgGAw2amz5dCo= X-Google-Smtp-Source: AGHT+IE+JByFLuBtDUrc5bcjQDL8kXzJ75Xakggo0iV8YwzjqWoQMMXMd+ywhB7PLbJgopN3ZGVCPQ== X-Received: by 2002:a17:902:e84e:b0:21f:6f33:f96 with SMTP id d9443c01a7336-2234a188d50mr267570275ad.6.1740968742708; Sun, 02 Mar 2025 18:25:42 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-22350533a10sm67008045ad.247.2025.03.02.18.25.40 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:25:42 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 14/19] zsmalloc: introduce new object mapping API Date: Mon, 3 Mar 2025 11:03:23 +0900 Message-ID: <20250303022425.285971-15-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: C808940007 X-Stat-Signature: s34twc5bndcbhwc5q57uqxxd98w5ckjh X-HE-Tag: 1740968743-160016 X-HE-Meta: U2FsdGVkX19eM+6zyYbuv3sdnLmx3hCsb7UL13rBgUU9845sb1XJPY7ay16cExIu1AhrFzmKc1g2MdyOrQTg6wYJmQMEnzLME0rz9nOhxM59GYuQaGZiys3jat3c+Fz6yXyRKK+0U+VkNPh/AEN+m3FG1mCDBWHm1BYnFy6D/y9eIZQNMkIhE0um2GuiwTxa5X8aKkCZJIyKKYlM47M3+Nz6x0F/SUBFNvTeZhAjD94nQn5oMEjoVXzoHnvlKuvpjlRI5If7YwL8hwIJsQdKPYhlRnRiCJcBi5ohHKceCbyBJPVUh8CFuyLu3TKRl+llqEIK0Qes8WASnaOTBPUqrdCLNWRgy1i+nMhnWgklsT/Ps7s14fV8R3y7gjlemTyiNQ/Y/jhOhe1zRWhaS1UuMHqEKdmZGKeNvxqjIfH/Ix/fSZSDoq4KDlji7uXCxjIi46LbBf84RyZMg2qV3e4OXPGlIriopmjprBNCAS3wiNMYQIwwVGPwSLbUBWn0rRNx4puy5pCVqFr+nCpXpLLzhJIY1UWf2FSC5pMCcAymeXJXbNQ3GBCaeDKvjjKnfM+3g7lFJtVE92D7dPm+lQMnh+qJssYj1MYcYoD7OMUIhKOpUF6Ix6EzQtkd/oLkYUBP0DQRzpQbxHdNcG48NjSQBf/7IxXVZ95uhMV/Gd73Jw2yYr8Z7PbPwpfVyC8ssoBhpwKZVKAFNeJR6G+avHgwxGoBrMtMzRzjPZnxOIontMi7u34cSKVk90B0/t1R/V5ilhPm4yB/BolBaPBVatIE91TfytsuHcnh7GEMYSpRGKOyIwbYDpY4ITktVo1Xwbf/YrrK6h38+xEkDBqG9iu02cMMXw6qz8pvYxH5kfOWn1R5g6okoImysoZqbxKvduB2MljLMEZMtOUGASSUBvbRRD+o9OjTJog76S+W1qdFPwDDraxao5rZH2QRbENoHmWjhUeXcUN6a6CmtIKCxz+ L7cNoMDn 5bfc2ftYbH5oE7bvCCZs9yLeAYD3S0OMJ2B4kgwfwkGsuHu7rOIK1PfjaEBzLFyu/gmKRh5Khk/RSZDFlI1agNbYdeIkbatMGcC9MryLs6qyDKt3YIVEQD8Zf1Jftutx0RjiDzGrhSKGJ+M6KOtmNk3cUKJ7f9RN528xAyRwGbH9dq/NmdE9yreYN8IkW911Fa9v4N4MOEfR7i2JjPJRw2L2HE00HOi7KxpKcavWjcsE6jWTH+O+i4W/B7gzALUChNZqBn7FCDQysFXgeFzOTbcVO18N4hGPfbroGf7nrHmWxctW0WWWW5D83Sg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Current object mapping API is a little cumbersome. First, it's inconsistent, sometimes it returns with page-faults disabled and sometimes with page-faults enabled. Second, and most importantly, it enforces atomicity restrictions on its users. zs_map_object() has to return a liner object address which is not always possible because some objects span multiple physical (non-contiguous) pages. For such objects zsmalloc uses a per-CPU buffer to which object's data is copied before a pointer to that per-CPU buffer is returned back to the caller. This leads to another, final, issue - extra memcpy(). Since the caller gets a pointer to per-CPU buffer it can memcpy() data only to that buffer, and during zs_unmap_object() zsmalloc will memcpy() from that per-CPU buffer to physical pages that object in question spans across. New API splits functions by access mode: - zs_obj_read_begin(handle, local_copy) Returns a pointer to handle memory. For objects that span two physical pages a local_copy buffer is used to store object's data before the address is returned to the caller. Otherwise the object's page is kmap_local mapped directly. - zs_obj_read_end(handle, buf) Unmaps the page if it was kmap_local mapped by zs_obj_read_begin(). - zs_obj_write(handle, buf, len) Copies len-bytes from compression buffer to handle memory (takes care of objects that span two pages). This does not need any additional (e.g. per-CPU) buffers and writes the data directly to zsmalloc pool pages. In terms of performance, on a synthetic and completely reproducible test that allocates fixed number of objects of fixed sizes and iterates over those objects, first mapping in RO then in RW mode: OLD API ======= 3 first results out of 10 369,205,778 instructions # 0.80 insn per cycle 40,467,926 branches # 113.732 M/sec 369,002,122 instructions # 0.62 insn per cycle 40,426,145 branches # 189.361 M/sec 369,036,706 instructions # 0.63 insn per cycle 40,430,860 branches # 204.105 M/sec [..] NEW API ======= 3 first results out of 10 265,799,293 instructions # 0.51 insn per cycle 29,834,567 branches # 170.281 M/sec 265,765,970 instructions # 0.55 insn per cycle 29,829,019 branches # 161.602 M/sec 265,764,702 instructions # 0.51 insn per cycle 29,828,015 branches # 189.677 M/sec [..] T-test on all 10 runs ===================== Difference at 95.0% confidence -1.03219e+08 +/- 55308.7 -27.9705% +/- 0.0149878% (Student's t, pooled s = 58864.4) The old API will stay around until the remaining users switch to the new one. After that we'll also remove zsmalloc per-CPU buffer and CPU hotplug handling. The split of map(RO) and map(WO) into read_{begin/end}/write is suggested by Yosry Ahmed. Suggested-by: Yosry Ahmed Signed-off-by: Sergey Senozhatsky Reviewed-by: Yosry Ahmed --- include/linux/zsmalloc.h | 8 +++ mm/zsmalloc.c | 125 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 133 insertions(+) diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h index a48cd0ffe57d..7d70983cf398 100644 --- a/include/linux/zsmalloc.h +++ b/include/linux/zsmalloc.h @@ -58,4 +58,12 @@ unsigned long zs_compact(struct zs_pool *pool); unsigned int zs_lookup_class_index(struct zs_pool *pool, unsigned int size); void zs_pool_stats(struct zs_pool *pool, struct zs_pool_stats *stats); + +void *zs_obj_read_begin(struct zs_pool *pool, unsigned long handle, + void *local_copy); +void zs_obj_read_end(struct zs_pool *pool, unsigned long handle, + void *handle_mem); +void zs_obj_write(struct zs_pool *pool, unsigned long handle, + void *handle_mem, size_t mem_len); + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index afbd72363731..7566070729ee 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1362,6 +1362,131 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) } EXPORT_SYMBOL_GPL(zs_unmap_object); +void *zs_obj_read_begin(struct zs_pool *pool, unsigned long handle, + void *local_copy) +{ + struct zspage *zspage; + struct zpdesc *zpdesc; + unsigned long obj, off; + unsigned int obj_idx; + struct size_class *class; + void *addr; + + /* Guarantee we can get zspage from handle safely */ + read_lock(&pool->lock); + obj = handle_to_obj(handle); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc); + + /* Make sure migration doesn't move any pages in this zspage */ + zspage_read_lock(zspage); + read_unlock(&pool->lock); + + class = zspage_class(pool, zspage); + off = offset_in_page(class->size * obj_idx); + + if (off + class->size <= PAGE_SIZE) { + /* this object is contained entirely within a page */ + addr = kmap_local_zpdesc(zpdesc); + addr += off; + } else { + size_t sizes[2]; + + /* this object spans two pages */ + sizes[0] = PAGE_SIZE - off; + sizes[1] = class->size - sizes[0]; + addr = local_copy; + + memcpy_from_page(addr, zpdesc_page(zpdesc), + off, sizes[0]); + zpdesc = get_next_zpdesc(zpdesc); + memcpy_from_page(addr + sizes[0], + zpdesc_page(zpdesc), + 0, sizes[1]); + } + + if (!ZsHugePage(zspage)) + addr += ZS_HANDLE_SIZE; + + return addr; +} +EXPORT_SYMBOL_GPL(zs_obj_read_begin); + +void zs_obj_read_end(struct zs_pool *pool, unsigned long handle, + void *handle_mem) +{ + struct zspage *zspage; + struct zpdesc *zpdesc; + unsigned long obj, off; + unsigned int obj_idx; + struct size_class *class; + + obj = handle_to_obj(handle); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc); + class = zspage_class(pool, zspage); + off = offset_in_page(class->size * obj_idx); + + if (off + class->size <= PAGE_SIZE) { + if (!ZsHugePage(zspage)) + off += ZS_HANDLE_SIZE; + handle_mem -= off; + kunmap_local(handle_mem); + } + + zspage_read_unlock(zspage); +} +EXPORT_SYMBOL_GPL(zs_obj_read_end); + +void zs_obj_write(struct zs_pool *pool, unsigned long handle, + void *handle_mem, size_t mem_len) +{ + struct zspage *zspage; + struct zpdesc *zpdesc; + unsigned long obj, off; + unsigned int obj_idx; + struct size_class *class; + + /* Guarantee we can get zspage from handle safely */ + read_lock(&pool->lock); + obj = handle_to_obj(handle); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc); + + /* Make sure migration doesn't move any pages in this zspage */ + zspage_read_lock(zspage); + read_unlock(&pool->lock); + + class = zspage_class(pool, zspage); + off = offset_in_page(class->size * obj_idx); + + if (off + class->size <= PAGE_SIZE) { + /* this object is contained entirely within a page */ + void *dst = kmap_local_zpdesc(zpdesc); + + if (!ZsHugePage(zspage)) + off += ZS_HANDLE_SIZE; + memcpy(dst + off, handle_mem, mem_len); + kunmap_local(dst); + } else { + /* this object spans two pages */ + size_t sizes[2]; + + off += ZS_HANDLE_SIZE; + sizes[0] = PAGE_SIZE - off; + sizes[1] = mem_len - sizes[0]; + + memcpy_to_page(zpdesc_page(zpdesc), off, + handle_mem, sizes[0]); + zpdesc = get_next_zpdesc(zpdesc); + memcpy_to_page(zpdesc_page(zpdesc), 0, + handle_mem + sizes[0], sizes[1]); + } + + zspage_read_unlock(zspage); +} +EXPORT_SYMBOL_GPL(zs_obj_write); + /** * zs_huge_class_size() - Returns the size (in bytes) of the first huge * zsmalloc &size_class. From patchwork Mon Mar 3 02:03:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68259C282C6 for ; Mon, 3 Mar 2025 02:25:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EB578280012; Sun, 2 Mar 2025 21:25:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E3D72280011; Sun, 2 Mar 2025 21:25:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB7D8280012; Sun, 2 Mar 2025 21:25:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A79A4280011 for ; Sun, 2 Mar 2025 21:25:51 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 6B99E1409D4 for ; Mon, 3 Mar 2025 02:25:51 +0000 (UTC) X-FDA: 83178649302.15.384A72D Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) by imf07.hostedemail.com (Postfix) with ESMTP id 7C6DE4000A for ; Mon, 3 Mar 2025 02:25:49 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=gIqNrMao; spf=pass (imf07.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.41 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968749; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=T67dKHYswiGrddgSZiZdCiBHKq1q/ZI+uUwWC6eNCTs=; b=8qiBVp9bXaApguKzk9jqXVoZy3HmrHfig5nAcRn1KNunaea/kduLFZ8tGjmvOI6rILcvf8 EIipPIVQCghGHXGAFDNSGr8+GaOie1iQHcOR8AZMWC2e9Ya5bx/3JwQCdTFQSawL9rqhNP YvgmRVq9HWfyaezwTxXZCcbRxakP1cA= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=gIqNrMao; spf=pass (imf07.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.41 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968749; a=rsa-sha256; cv=none; b=CL9TSnm7vznWU+aKwGvlsBf3eiP8+epLstKFgXaJQSgaFHKJuvXq5r1mdvG0c6RBTgZA6V dDEH23Hv/ddwVWE70U/iY30VVmifA00mGfTJHk10NxiOCbk+T793CkAvT5MMeXsyhE2ob4 YhUftusVX0bestuIAauHaAEQsTJf5UI= Received: by mail-pj1-f41.google.com with SMTP id 98e67ed59e1d1-2fe96dd93b4so7159851a91.0 for ; Sun, 02 Mar 2025 18:25:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968748; x=1741573548; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=T67dKHYswiGrddgSZiZdCiBHKq1q/ZI+uUwWC6eNCTs=; b=gIqNrMaobZUnSkIsqx813SeU6tKI8OeCsc/hVaZ1/34lIJPVZ2AIwCZrScYtQmKHmF 23XMaWxC6gfEM+fbAtIbh8cfWsMMhPnlZVgPcuF1yp0SHIg3TQOQzd//7nsBwos7mKEu VrnnFyWYfPmdkmcQ74eQjj37HxQqIGPWExgdo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968748; x=1741573548; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=T67dKHYswiGrddgSZiZdCiBHKq1q/ZI+uUwWC6eNCTs=; b=SwlvP1IMXC/jVv0oDOzkmkjUQ1zlL3yjuRgcNSZ6G8v2GVLmitoXM33n7umlnfvhRF LHXON/wKcGoD7o2qlqRW/8okVvzPhMsZRfHPvY4/PtQ9BDIQ7B53VUasnvX7H39Y9ygQ 0/VcZZW+ju3/pn5x/wGA2xqCjAYIxKrrXRF+V8B3jdPrsj/NM0wo8xC5ki1/ln5JcJxx NTHHT4xpm/2qPooNyf8pD7eJ+tw+/fQ+DDk4E+86eHfnlpgpsmxts1o6qDhepcE6sZL+ 5ubX4SDtHHVsPspqWUJAveVBAFVORpl0/PDq1DJmuhq0IEVRogNelcrjeQ8sPwvyxIpR gF0w== X-Forwarded-Encrypted: i=1; AJvYcCVP3SLna04Boj3NzAQH0iK7DeKYeoumPqZXfZsFoFxrLZ5bEmV6PcgmQhgK88DQzF0RhhmQjo6/KA==@kvack.org X-Gm-Message-State: AOJu0YzY0Uebmv6o/D1LY+JLY+KgfM3Hypb8NAg8jgVQHyw5V7Clr3hq Npu8lbcmSHO70FSXZ1zm7bU99s3mrnYLJ1uFrt5mlo/isEDYCveGkwlv1U3Ol9Ea8+5e1Gof2nc = X-Gm-Gg: ASbGncvcGtvPFRPP/Tz3PBofzn/Pw+GEZpcMsqW3RjWTowYuW1/1/bZue1PwxCLZ0Mi cKU7VCQ3roOdPxgW+StKWKQJA4aVZ0f1ldFNdnR+k9wiSKjbNQBv1ki6bjr2yrKMinVJ+L9BdOv D0LyQ1F4GM07xGJ4y1wzSOfl82RtnedA21VwjRE8q4SxjZtZJ2bWTaM27D31LGbbdflYK+O5AZp gxDBvkKdRTGTpSEiGSoVx187rmmDkcfj+A/5QejbDrGNK049gKqGFplbdfwh4QzB/OzRradk4Eb 5yrPPa1fxyXH2wLpSd0DmvZZarpFCJJoPxTVi8ce0d/Mw/g= X-Google-Smtp-Source: AGHT+IH55gKbbFLH/4p1SdtWLi5VFnXEFbXHma9n71ec/lgw7BNRJwBiDWwI7fYS8F4jPBgKdgrzew== X-Received: by 2002:a05:6a21:6e03:b0:1f0:e2e6:f41e with SMTP id adf61e73a8af0-1f2f3cee5b8mr17138562637.8.1740968748420; Sun, 02 Mar 2025 18:25:48 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-7364002f247sm2421551b3a.90.2025.03.02.18.25.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:25:48 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 15/19] zram: switch to new zsmalloc object mapping API Date: Mon, 3 Mar 2025 11:03:24 +0900 Message-ID: <20250303022425.285971-16-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Stat-Signature: i9jtjaddbac4i53a34sf1dmsapjsdx98 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 7C6DE4000A X-Rspam-User: X-HE-Tag: 1740968749-21691 X-HE-Meta: U2FsdGVkX192jgeHapRJo5QWWxEyG7KUY42UN9WAztk3apeyB9KytcYQNByCX+/JgYf3xe4JBMdliAm7DaxO3P+bR6sv3+YWnHpnUm08nSuRtYVbVdjxmjP/6FTuPqD1slSaITaXef6OqvIH49SPeN8YnLQtGqeCrywLXfKPpU5qTzWEHEC4KoP8A6Ldp9omALcE3mdMMzUKxwcRd5/HQ0SoKJXYm67Tj3HxAMmQ9LxMEFzZArnAMF3hbZG3nJ2Z2yRyFEb5VPwWVapb2BNJkCPlNy84RPv2TSOpa2B3CDyYdK7ds8mLltYMbx46rDFL6CE5q4EBEFgvYZjm8d3J2xwgv/ZT8/2h1CjVwEPdrwqkyij8E2hIAlURLL+GW9RZOfX6n0ambh6iOnqPb4DjQA0GCJndSCoSOe8/97dZRfxUULXRSlIf7B8+NLRkACUjcVmabFtWe/pjsxU02KVdYwYjSIo8UNTTqt84UnAW2CHJW78tg/ThZgEMJMMGYTtfW5aNuF6p1f2iSulm/+R3v4XvH+5sciWG/A35iL09ENAtLZU+iiSvuCts8bLIX8n+yYkM64ALKm1iq8gIEMLZJMajojylkee7xkx5n6DM2f2Ef382rP1DzcZW4v5Lu2TQtdLshoPqo5YQK4Wke2ZqkoI4zzdgoEq3vZ95jECQ8SV1BzS5aXujQ5jrHoryHXPS39EvAB8J0Kn7Fj1vXehIpaFqf8TUtm8N2LpHxCr9u84nKx+g9iLaizy8GGJk4KkQJ5kYaRLiVQeFNgO7jGRU0NusDaM/BSOaJQ9myuRAAsLptZYZnM1SDPX8JHoINsPNvKpJb2i7Bh1cMJdZO0Hmxe94/l1gKX+gj4f6fNWbHuZTgr7HeUV5ERifI1vE9842xk4/vZB0uoqI7o4jOCgA21CkevbuBdft7KSv4CkQ7cXfXT63jnAxKy2GyOtL/Bvk9lpTW7/NrQoE2kImMu6 rUMLL0Wp whITiFJev4EPUmsdF1AvVoIWdYiCeXrzZCfPx8IeMzE7yiBzJJdSoPhikgrD5KdiULR+NRjWGZJBZAdfkGZwEol9N0GN9W1hCYjbnKH0SWmR62EmHFCSWpfHfpQRt9+wzSAjcyb4w/J3s4l6+FF1dLsKVQXn8KlCrh5OQoyW9z5uULUZ7Xdb41fQ0e++4+B7ItaLfQw9mnvcTkSA8Vj6IprDyoISPGGr70UAGtqAFxV9jsWugwn3flBf6ox3t+Rt8WehgYNDkBCHg1vFA+XydVftNPU9RZMUbcXbwcYIabh4n0ya6kT26hjTV7qxRjQJHfi+R X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use new read/write zsmalloc object API. For cases when RO mapped object spans two physical pages (requires temp buffer) compression streams now carry around one extra physical page. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zcomp.c | 4 +++- drivers/block/zram/zcomp.h | 2 ++ drivers/block/zram/zram_drv.c | 28 ++++++++++------------------ 3 files changed, 15 insertions(+), 19 deletions(-) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index cfdde2e0748a..a1d627054bb1 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -45,6 +45,7 @@ static const struct zcomp_ops *backends[] = { static void zcomp_strm_free(struct zcomp *comp, struct zcomp_strm *zstrm) { comp->ops->destroy_ctx(&zstrm->ctx); + vfree(zstrm->local_copy); vfree(zstrm->buffer); zstrm->buffer = NULL; } @@ -57,12 +58,13 @@ static int zcomp_strm_init(struct zcomp *comp, struct zcomp_strm *zstrm) if (ret) return ret; + zstrm->local_copy = vzalloc(PAGE_SIZE); /* * allocate 2 pages. 1 for compressed data, plus 1 extra for the * case when compressed size is larger than the original one */ zstrm->buffer = vzalloc(2 * PAGE_SIZE); - if (!zstrm->buffer) { + if (!zstrm->buffer || !zstrm->local_copy) { zcomp_strm_free(comp, zstrm); return -ENOMEM; } diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h index 23b8236b9090..25339ed1e07e 100644 --- a/drivers/block/zram/zcomp.h +++ b/drivers/block/zram/zcomp.h @@ -34,6 +34,8 @@ struct zcomp_strm { struct mutex lock; /* compression buffer */ void *buffer; + /* local copy of handle memory */ + void *local_copy; struct zcomp_ctx ctx; }; diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index f6e887f94b71..62aef12417a4 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1561,11 +1561,11 @@ static int read_incompressible_page(struct zram *zram, struct page *page, void *src, *dst; handle = zram_get_handle(zram, index); - src = zs_map_object(zram->mem_pool, handle, ZS_MM_RO); + src = zs_obj_read_begin(zram->mem_pool, handle, NULL); dst = kmap_local_page(page); copy_page(dst, src); kunmap_local(dst); - zs_unmap_object(zram->mem_pool, handle); + zs_obj_read_end(zram->mem_pool, handle, src); return 0; } @@ -1583,11 +1583,11 @@ static int read_compressed_page(struct zram *zram, struct page *page, u32 index) prio = zram_get_priority(zram, index); zstrm = zcomp_stream_get(zram->comps[prio]); - src = zs_map_object(zram->mem_pool, handle, ZS_MM_RO); + src = zs_obj_read_begin(zram->mem_pool, handle, zstrm->local_copy); dst = kmap_local_page(page); ret = zcomp_decompress(zram->comps[prio], zstrm, src, size, dst); kunmap_local(dst); - zs_unmap_object(zram->mem_pool, handle); + zs_obj_read_end(zram->mem_pool, handle, src); zcomp_stream_put(zstrm); return ret; @@ -1683,7 +1683,7 @@ static int write_incompressible_page(struct zram *zram, struct page *page, u32 index) { unsigned long handle; - void *src, *dst; + void *src; /* * This function is called from preemptible context so we don't need @@ -1701,11 +1701,9 @@ static int write_incompressible_page(struct zram *zram, struct page *page, return -ENOMEM; } - dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); src = kmap_local_page(page); - memcpy(dst, src, PAGE_SIZE); + zs_obj_write(zram->mem_pool, handle, src, PAGE_SIZE); kunmap_local(src); - zs_unmap_object(zram->mem_pool, handle); zram_slot_lock(zram, index); zram_set_flag(zram, index, ZRAM_HUGE); @@ -1726,7 +1724,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) int ret = 0; unsigned long handle; unsigned int comp_len; - void *dst, *mem; + void *mem; struct zcomp_strm *zstrm; unsigned long element; bool same_filled; @@ -1773,11 +1771,8 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return -ENOMEM; } - dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); - - memcpy(dst, zstrm->buffer, comp_len); + zs_obj_write(zram->mem_pool, handle, zstrm->buffer, comp_len); zcomp_stream_put(zstrm); - zs_unmap_object(zram->mem_pool, handle); zram_slot_lock(zram, index); zram_set_handle(zram, index, handle); @@ -1882,7 +1877,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, unsigned int comp_len_new; unsigned int class_index_old; unsigned int class_index_new; - void *src, *dst; + void *src; int ret = 0; handle_old = zram_get_handle(zram, index); @@ -1993,12 +1988,9 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, return PTR_ERR((void *)handle_new); } - dst = zs_map_object(zram->mem_pool, handle_new, ZS_MM_WO); - memcpy(dst, zstrm->buffer, comp_len_new); + zs_obj_write(zram->mem_pool, handle_new, zstrm->buffer, comp_len_new); zcomp_stream_put(zstrm); - zs_unmap_object(zram->mem_pool, handle_new); - zram_free_page(zram, index); zram_set_handle(zram, index, handle_new); zram_set_obj_size(zram, index, comp_len_new); From patchwork Mon Mar 3 02:03:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D8CAC282C6 for ; Mon, 3 Mar 2025 02:25:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C1C54280013; Sun, 2 Mar 2025 21:25:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BA501280011; Sun, 2 Mar 2025 21:25:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9D14D280013; Sun, 2 Mar 2025 21:25:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 7C542280011 for ; Sun, 2 Mar 2025 21:25:56 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 3D7D81409EE for ; Mon, 3 Mar 2025 02:25:56 +0000 (UTC) X-FDA: 83178649512.14.5B8544A Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf14.hostedemail.com (Postfix) with ESMTP id 5BED410000A for ; Mon, 3 Mar 2025 02:25:54 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=MQIpBaxs; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.182 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968754; a=rsa-sha256; cv=none; b=y0oP86ELn/lQ2Gm6Ma0Pl44XiJa0tyYON2Ey2sps14gI/f4HUDebiTcBerQIk5ZvkT+JAH +u61G5paBpblAuhDDF1XS3v+IVL4uwgkBRoRt7q9XbW69T3s+zo/r1Dp7udJ1ySt6+ou5X t0dBqmhwrTpVABBy+1CuO54jFGknWco= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=MQIpBaxs; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.182 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968754; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=m9V04z24we0PKbaMB+rapzgCCE9wJRDyCG7EP1iiRhA=; b=U5B6nI48jJI7ZVzBCbpYck+R71kQbMoCILJJsD0660IKWNBYZBemXg/jYlGpKgqD6rOsCp 1yAjquqg06w7EUyTYx3mDBmWSKPCN8MZdv+yZrJczdHiqjsOPuEjaIkuryMZ1zqYGCX6Et h0ugeZGQa7H3KQg9YRL2XYvBNw4B/yY= Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-22355618fd9so63397715ad.3 for ; Sun, 02 Mar 2025 18:25:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968753; x=1741573553; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=m9V04z24we0PKbaMB+rapzgCCE9wJRDyCG7EP1iiRhA=; b=MQIpBaxs1cwiR65pwpZNBVUVG+tU5jUdKVdxUZZZ8dvdoursQbw8Pxsm9NO+XoGghO t0joh5ByaFaXT26Zijf82g39RhAA8r/G5YFXwV6+hZb01wzs9VfwCijz7dUBIhV1T3G4 mGix5LXgEMv1+28Ay4TKdgH2hPpiJkXQP9Kss= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968753; x=1741573553; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=m9V04z24we0PKbaMB+rapzgCCE9wJRDyCG7EP1iiRhA=; b=qR+CGRlW4/2hW5VnX5ZmPmaxIwblsFJo+fnFFlZDHdW45wT418QOSt4/EfgMvZLpdx 9JI1aV1lgl/D6zm+VJ8BN4AckGJVul40W1BQRh/sqvDE5UPwEzC2dplDUF8HzY3b4dOB W+djl4dEFpn3flthuPhO765UxSKXpnptBFKW+Cdmhjce4DrDw/esmgFRtFWZ1SYB9fAy 125u/Y3/1wsNaX6jJz7Ihcqj02uzV6XUMSlAckF6IIK7T4oBHBg4q+8btMbRkrGp+brD syybVQQnlX3xliRASdQGhSp+FYr+4d2Sg7jpflQlUyPvz/jifItXX6XBRAR+SEu7709R QBmg== X-Forwarded-Encrypted: i=1; AJvYcCXUzZQu0JalM2dSF1fScczRK5KIIbzqZWKBjFs2GSMoACAxhj1TFbTinq7PU+IpkJsw4V3ygfZ2WA==@kvack.org X-Gm-Message-State: AOJu0YyfiDNgNXsNVg/zUMarBy9IOwi+Fye7Cx0AQA1N23lS42UhicQv qtIDh3iPgPSDDGjoFNV94jxiIUkcoHU+mVcnIub3uLXE1LV7ewfx1k038UIPiw== X-Gm-Gg: ASbGncujVxt2nyhX4wYUcWHcUkz/wuMl7n3cOcgTXZDH9kFNYFvYzmbABnfcqPeZkuC 37nLhUdLFjnQJaWXRsnhDKqYak+1vUQptA+udhFvg13ORRPX7GfxBvJu6Kyzsh/o08+7+xuXvAi ZVRaUSI5H++jeFOkpvL4mucIwSM182Qzn1ZdSBM9A8H99isYQhIi82s6YNHF2VkAwQwGgxad9Bz lT2pOjEzbU02uqBNK+eeBnR8jNvhbsZDWcTF2ARC88ZyDWWIdkySOSDVNDo8DcO9R6QsmrjvWJi lzI2ZsXdEq+/VyBCubK+X8m9asI2CN1KESk1Y1i+ZHaSLnM= X-Google-Smtp-Source: AGHT+IFRGkg4HAVHTSk7P/5WiDLiMFVD4ws7CbIAd4dpZ2NPws7biEyrBHhEpztlue9ApQq6yVEBhg== X-Received: by 2002:a17:903:32c8:b0:223:245a:11e2 with SMTP id d9443c01a7336-22368f71ed9mr148206025ad.6.1740968753298; Sun, 02 Mar 2025 18:25:53 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-223504c7f12sm66879285ad.122.2025.03.02.18.25.51 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:25:52 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 16/19] zram: permit reclaim in zstd custom allocator Date: Mon, 3 Mar 2025 11:03:25 +0900 Message-ID: <20250303022425.285971-17-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5BED410000A X-Stat-Signature: giwhxyncicnfdy18ahp7eta8faij4hmo X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1740968754-747250 X-HE-Meta: U2FsdGVkX1/Y6eqaAiJibkXjUxcOvIw3+WZ2MgIxI5DxxZZinmDTvc7kmy/uT4UehXYWNKBAaG/GvykwW/Jrf4/RoDqhlgRRifRfMRt92VlnbZNiwEsq6gaymEau90ud5Sshu6OUbPO9akt+iy2dLqT542FMz2ie3pHHbje3rkkpM0yi8Tr7aPMtlxnBprSZGkJ04Pb/rNoipcLjLS3CGu4c1Qb1Xnb+WA8fgFVhgJbh4ibCxeA8qebeZZELV/0Mm3a1ruydFVIF0EsbRDH2mEtaQJImZdlIP2rHUq/U0G9U80o0O2mfiIsZ0kSymscqx3XmxMZYuOro+d5OUBknsqDgyFGUlDR7cfkmFRhCgLJdBEZUJSmLyGnao30bDV3UbK90FMnBaNPcuUvyRBdoHxjHU0Mgmjp5rF4E6IazXN3qDni8yZb6OuiGmzpUwMrL3aVqkSv9yi7O6XcCRxhevwJbq5MgxpBw8s1pIeiBGViMHI5aSvPR0CTlV32IYGlPMOQ4/zZkc6nrHibvB5/mRAzxkwtkEFrYoKrg5t8Y/K4migp9hfPoKah7+djrK8DrVqh+XLrKcuLuvu9UeVw4T2ihYzzuy6JgrN1wyZZZrMqxshT555bnoOF/QymR5xE1VGjPSazjQ4duv8FkDK1ONgP5gb8ZG/YAb1NjXbz6XqKpzLK4dkUcj/HHAqWtgRAxi7eewM+IPyGJDcYWfci/sSizastkA7IrXmHU8Htlpa4NO3DwffOenfXOvMAokFXswPZM2PMUJfKY/s+fUBq1zLjVgG4Cpu+b5Pr0fMHKUw8XTQyXVFQOBbhQg0NXjePwhZlQ4j9sAWcaymyVaqCm5nvUb58UFl4yu+9nd1LpBrijwoLUiQWITROOwVA9pqTTtYzgtN5ok2N+M0q9oWGwyPwBcJn6G76/H/JQz0vs1HAoo+JQWpNm8H/4Eh6wWp3gQncNUNlz/ges7kWky5k WmE1wYSq 5xaOVyXyGky4AptYd8/LQKV4txybMMisu/lOl0TMGkk2fYZqgmEyB4PDkE5m/1twvuesHSDAoXPHewjmD0klky0q53LoiUl2VqxzPX7jRc9f+sWRP3tS6eXvkH30df8F+ufc1ZyUJ5TxCgAk0MD0KuHhlQuza/gBvBa774ftxDkjOg8Af/wKEQM0M+khXFpa0s9t/D/HDeCD0vpNPxKI/xBOwYGXTf2qJFwmsCUHKrmcS0e3BIsBVoULoME2wMzCjCrOMQouGddoseOUO/blWkDuN7pNM2RhuxZH8JCGt7X344iW/rd2SQMLbEQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When configured with pre-trained compression/decompression dictionary support, zstd requires custom memory allocator, which it calls internally from compression()/decompression() routines. That means allocation from atomic context (either under entry spin-lock, or per-CPU local-lock or both). Now, with non-atomic zram read()/write(), those limitations are relaxed and we can allow direct and indirect reclaim. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/backend_zstd.c | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/drivers/block/zram/backend_zstd.c b/drivers/block/zram/backend_zstd.c index 1184c0036f44..22c8067536f3 100644 --- a/drivers/block/zram/backend_zstd.c +++ b/drivers/block/zram/backend_zstd.c @@ -24,19 +24,10 @@ struct zstd_params { /* * For C/D dictionaries we need to provide zstd with zstd_custom_mem, * which zstd uses internally to allocate/free memory when needed. - * - * This means that allocator.customAlloc() can be called from zcomp_compress() - * under local-lock (per-CPU compression stream), in which case we must use - * GFP_ATOMIC. - * - * Another complication here is that we can be configured as a swap device. */ static void *zstd_custom_alloc(void *opaque, size_t size) { - if (!preemptible()) - return kvzalloc(size, GFP_ATOMIC); - - return kvzalloc(size, __GFP_KSWAPD_RECLAIM | __GFP_NOWARN); + return kvzalloc(size, GFP_NOIO | __GFP_NOWARN); } static void zstd_custom_free(void *opaque, void *address) From patchwork Mon Mar 3 02:03:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75857C282C6 for ; Mon, 3 Mar 2025 02:26:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 00F54280014; Sun, 2 Mar 2025 21:26:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EDAB9280011; Sun, 2 Mar 2025 21:26:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D5315280014; Sun, 2 Mar 2025 21:26:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B1C68280011 for ; Sun, 2 Mar 2025 21:26:00 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 6BAC11409E8 for ; Mon, 3 Mar 2025 02:26:00 +0000 (UTC) X-FDA: 83178649680.12.E9BDD06 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf24.hostedemail.com (Postfix) with ESMTP id 8E38C18000D for ; Mon, 3 Mar 2025 02:25:58 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=IreV1wlZ; spf=pass (imf24.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.47 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968758; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iyo5ByX4ms7OSRp4VomEFa7attH8xkBy9eJ/3JfBMpA=; b=CfXx5WW9+Szv3AYxbvtBxiuJPncBdTZN5zpaz72YtOWvzY7YVdEBSypCKWdmqAiT7+KLqO 0boGud+y7Ww3Hc+3s7FKxnv5bu5mCbTABqZe22YuIKRpfbzJKouh66xGmKaOzbyyGcXZIC RvcuUhdrvaxKdZuvgi30LX+Z2x8QBzM= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=IreV1wlZ; spf=pass (imf24.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.47 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968758; a=rsa-sha256; cv=none; b=goFqxIA97ufzjl2aEFNMYjVV5MgGSY+m8zZDbQ1aUgW7qQrMhuKRWcVkFgFL7PQIseXx1B e5BKDVzziE8MctlUG/KgvrEuLX0rbTjnH7FqZJEUPOZcP2kWqxeBGXmFsscRU/k0GlnfEw LkXaW5Aa/DSEWlQ++XPZStLeWGABI8I= Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-2fea47bcb51so7779494a91.2 for ; Sun, 02 Mar 2025 18:25:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968757; x=1741573557; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iyo5ByX4ms7OSRp4VomEFa7attH8xkBy9eJ/3JfBMpA=; b=IreV1wlZo14eZOrAQnN/vKkC4vIMfEc5Y20voe9J6r/3nwrmN9vVhqTK6wq4Qse73h UZqGknO2obvP5Jzx/68nRxhBDAbXqfYkyS+8DpBFaJ25xiH0HycIhXbfOEUTpS0FhK6h GqWrIY1Q7wlKCJf2ZY4tcTokjcYiNp5GBdGX8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968757; x=1741573557; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iyo5ByX4ms7OSRp4VomEFa7attH8xkBy9eJ/3JfBMpA=; b=DwtizE77ebqFl3sTxLJ75O9mGA6NUEqpz6+4NRaF9fAKlN2BkGMblxPeImANlesSIF HT0Se3G0KSyS6YvfCAK2dBdoii8WYTt5H8GVruxklWAy7u3Cte2Cv7Aa+ow2t/5dkvFn qfgfV9YGAnvb9b0Rh3xyCOjZoZOGlGTyAGiH/WjHVF8g/oIFkcjxyoG305MXSrpfAjfE 2It9FoEjx3rIiVppxY710svWw0vIwTbEa6FR5Oaa9IPr687g7+DRivRTxYwkBK8N8imr KnoXe8ZoYFsvOm52/1yyZB/zbdwIWb7azfj4+40kCYP16KUvfQIeHk/oPLLXjce0KFv2 rLvw== X-Forwarded-Encrypted: i=1; AJvYcCXSm/+mdy7lWWXh6hUnuU2/ZwUuhDmOQQemg8Dpofy9SSCiuSi3zdLVCz602xatlvxiJjKIF9Lxmw==@kvack.org X-Gm-Message-State: AOJu0YwW//T3ThDTj/7yIddzhUxKh3Jm8UYR5GelbugALI4gwWURYNsj 2pBY17NiosZmvknkxiTe/iPQhBepxK6uEvnJ189ChbrX3NHjN533/iAdhlfjvw== X-Gm-Gg: ASbGncsdkNif0Q6lGnChs21FpUqJmgDwDDM03xvq2p8uTdCkTn98hRP0kP3K7zuzjet iFX85ExLX0QDgfcLOrQI4r7OOrA6Mgxuvc2gPwF5qqBz1rgpfPmtiMGaf2brHw+Cv/vQ+AMP/su xRa72T5LH8nO9kY+VGsfKfBfKPJoGhWO3aLxkEgwWeovbJsq9gMPH3fPsfks0pmOC14jMU7MMb3 qYrfttCSs1CkqShAhKeRGOhJYpeZVg6zWUanVTIt+e/heIatPUPOIOeVkH70RwPi9dEMCiOlyW3 PeLc5ye49clpD/h3SnbIdtPpfr4v+NF0wlV3HvgrjXqkAd4= X-Google-Smtp-Source: AGHT+IFJotMZFWyExSutVlC701wQGCarIM/cdTnnq47i9KiZ5lx5Nlm+Rg/03s7HRjl5+WkOajCTxg== X-Received: by 2002:a17:90b:288e:b0:2fc:c262:ef4b with SMTP id 98e67ed59e1d1-2febab787f0mr20378028a91.18.1740968757557; Sun, 02 Mar 2025 18:25:57 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2fea698f3b2sm7671186a91.41.2025.03.02.18.25.55 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:25:57 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 17/19] zram: do not leak page on recompress_store error path Date: Mon, 3 Mar 2025 11:03:26 +0900 Message-ID: <20250303022425.285971-18-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 8E38C18000D X-Rspamd-Server: rspam09 X-Stat-Signature: bepogyn9sjbuisuopqha9rq5ys8zr8ow X-HE-Tag: 1740968758-608420 X-HE-Meta: U2FsdGVkX18NmDlVkBXmE3J5YTeuzQ5tSpdkzp4DmDE1pgaDju/xA9AuQDFZzZM+xklLpRV4Wl6HN6vXY13cvsesSCDCudMqlHgR9MyXWxyFM5533QUG9WrxalvoKDkYirguRtlPm4v7RLGg7rZhqzIE0hW42L4dimcFl/zj81b2zvpZilZI4BVSg3UuBk8qHHeWvX02p4EkRaPz3TCUBGbZt/wvPvp7rYFhewip8M+lpduf+rA3Q2bV0/HQok+NnNux+ICkghhPuoUBqgSjU91qQOvMjIJZcf9vLYo5RZ2FU4sY+mv1eD8O9+rn5k+fg+1BkveUeYHSGgwp+ektxogdkADjgiIY+Y9hcyIYHNE+Z3OiWxLCJAMDOE5aiBdJY1qOtHw5h37Hd+FrE4MlqZc4o9V2DTc6Uc9TPBJzCotOJHMdrfulSh+gaVZAzTPFL/9OXlPeaQVomSY9aOPxtjIFyLL03dB2tuiK+e2x51SKfkOsKJCF968vZGW4sh4Pe6J18y+KcgdYz0VaSFDxlBkwT7EcWrJGi15QRIktyJJ2gxdjetfpP6zE53tNa/JZkBQBZpVCJiGO/Bx2T/WO83EM465k3seWb9YccLxTo23xbk4Akn53AJygOEimA0mtqQrpVg1ntGlca0KQ8NwcGxNnZ1cGgiTMW5PyRgipNofe7VMJvL/2H4Ck9q4DL63H+lb4nYnDrfZb6bM87gFwALr1oR64YwSjfJuT0I8zFLWedWvgADJy4Evx5XJa2n47HK9YZ8IXDA89Juvq1Dul2ZAhscjMwpYAGbYZ++xnUgsw7s6cpEHOOTypkpkI8MYGvGmZZb6n54zDrFvhqzHfwUMXZIisbyWl+71MqQYahn2PwyAQ4BiEciSVwRhL/K8jaKd6O19UdT5LiG14sVOLHfN0RuojOh1gW8OntPfDqBR5ZQ/sIW7B54pmniOc0vUDV8pmOC6PcdgrzzqrmSR B3D/o1ZK 3ljOJIh1bnJfTfPVN3RlRxBCfPjNXTuCgWb1tHaLgjNZEHwRRAyEIdNsnIIl6anc/kwH/xpHAhGCNZtxeEQN9PGcb4cQ6GvqDFcRmvteeVkqkTzH9oq8JXHVyBW0dXpqpJbmn6GAOCviC1YGIi5+9BCjpyjUliT9jRrSruni2HAzNRyC4W54eXfM5XESuUyvBrqP0FyZ8sdZscteZan9X//OETjNhQVjPu52lAdP8ZxoXNmoUHT2NNa7IF9gllXrBROUyxApXerOFvSrvdoXB6l3XbnclYWZZ6lUPd5ah8K5zQM7KhEUZa3YgMIkXHJwLo/I8hTBkL9M9+Jp80oSx0t4sNsWLHb+jCLn8MmQ3joXWvQ0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Ensure the page used for local object data is freed on error out path. Fixes: 3f909a60cec1 ("zram: rework recompress target selection strategy") Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 62aef12417a4..e50a5a216974 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -2013,7 +2013,7 @@ static ssize_t recompress_store(struct device *dev, struct zram_pp_slot *pps; u32 mode = 0, threshold = 0; u32 prio, prio_max; - struct page *page; + struct page *page = NULL; ssize_t ret; prio = ZRAM_SECONDARY_COMP; @@ -2157,9 +2157,9 @@ static ssize_t recompress_store(struct device *dev, cond_resched(); } - __free_page(page); - release_init_lock: + if (page) + __free_page(page); release_pp_ctl(zram, ctl); atomic_set(&zram->pp_in_progress, 0); up_read(&zram->init_lock); From patchwork Mon Mar 3 02:03:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998068 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFCC9C19F32 for ; Mon, 3 Mar 2025 02:26:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 73ABC280015; Sun, 2 Mar 2025 21:26:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6C380280011; Sun, 2 Mar 2025 21:26:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5178D280015; Sun, 2 Mar 2025 21:26:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2BC79280011 for ; Sun, 2 Mar 2025 21:26:05 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E0B7E1207A5 for ; Mon, 3 Mar 2025 02:26:04 +0000 (UTC) X-FDA: 83178649848.01.8BBBC81 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by imf28.hostedemail.com (Postfix) with ESMTP id 0DA8BC0009 for ; Mon, 3 Mar 2025 02:26:02 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Mk9jRG33; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf28.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.46 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968763; a=rsa-sha256; cv=none; b=YadOwU0DQ4AOwyQzOSMfYSNKVpRl8GhW+DdYB9RAb9Vi1YArfKE5OWZ6DvHiTeohVeM+Th WBfwMmypt6V9sBM+MZ5Z8LHUj/ofJlQn2+aivwR34Q3c7p6sdY9LFXKJDHzMKPkdRgD8vt hpJbQIOiuEet8q2oesvwhcRdLEk7UZQ= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Mk9jRG33; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf28.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.46 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968763; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7MsZXmzA7gZttOoABIScoWzRHdPrz6HqbSztfy5OTfM=; b=4MM6X0vyvEHrkT2ypqDPhCxfbjPjWGfQPB2XkSGeCHFPF5//EXNLWFaZk7g32pWLXRbVZy lc9xq2T2bF5NUOajHY+gMYt9wUw93IwIEN/0sYBb2G+4AqTdmMT+1lJjJhRQtiAu/OZtQg sIZSDDoDqCj4nSMQ53FgrN65dAsyObQ= Received: by mail-pj1-f46.google.com with SMTP id 98e67ed59e1d1-2feaca4e99cso5323977a91.0 for ; Sun, 02 Mar 2025 18:26:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968762; x=1741573562; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7MsZXmzA7gZttOoABIScoWzRHdPrz6HqbSztfy5OTfM=; b=Mk9jRG33OiwbDtEWFj7NtdE4rIvGQHEXlJq69VdyIb7h2cSr1vkQ6o6u4POTk+oIQl Rp9nk97l4eW0nwWCAR+0Z7XwQUKbznEiSYq95KPTFSUO1/6/nGF/nZdWdM17/Go4bDI+ gCaWYxgZXAImSYWgo5ezSbp9ThkkyMFRDB4CE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968762; x=1741573562; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7MsZXmzA7gZttOoABIScoWzRHdPrz6HqbSztfy5OTfM=; b=wd4pZFbHZzDyrtOhFX16z4TuPmhfOViRABUqdvHztrsLbrfabI8l8Sc7IezEl6/70r wQbZfHZgSMAutTaeiVE/6x2C6FEWaoRuNLbTquj0H/ynOwlHxg0xfMXQMpwCYNRsytuz fck2Wt54apiMK02nSn92+/hlt5vjbdAft9pmJxCKkzmIWomjElmVXJeiV35A9ap4c/ah 1ULEc4yZ1YqRGaLEOeSzKl5NxSytBD1KgFrxUFCwdtnEKgUNdQ7g4s3C0qD7mvabJ37d +J0idFPxzYA/D3XIbMpgosqoXfL69oD2KntvjctaQsHN54QCUC3TnwpuX7iWwYTyhxUV gUsw== X-Forwarded-Encrypted: i=1; AJvYcCVtcu90pf+U6ZHs637anosf7APCR9tG57cqvZ+dHzT9M0YuWe9W7woeo6to1uW6wSB5DNPdpNEjiQ==@kvack.org X-Gm-Message-State: AOJu0YynGwRkvJClVPT3+q3BVLSgX1QFyxIWdPW5M2zvyZytdg5YAfQO OotPeP6fGDAxVgElNUfTXFR9kfG5CHRJwgeNEJVKgfdkUCv2WX6pHDIQEahz4A== X-Gm-Gg: ASbGnctJE9iX3QFAKHSiwp0rCtH4KlkFfRgnsvHx/wEGKeYJSPgJh/eEl5bC89OxFlk hMxKsIoR9sogYXSCjwoQLx6IxnWdl+x+WOstCQxQ5pF6eBVdSZASQZDoAFv7cbj1PEfnqPjW+Mb 1LtoL33v512Cq+6662qQsDCOHLPrbxzeLBKfHj/xHG9uphzWR5p6E1j52cNQ7TePeWfHOeVwsR9 g4l8gxrN/aeYBFYH7MBu5i/cy4GsPxOSgW0VrxhtDWyWYdylaZ+NaxcYnsjodU5evb9oFX+yAkN w7II7f6Z3jxQcdcCCGrT0LMNaIz9KcBgvxttfLU4uy20rb0= X-Google-Smtp-Source: AGHT+IEwLUQ8+oZJN0pj6P2S75Tk9SYQBRSYoeVDDP2x+EgMUD0KKR9qXN1nP4rIBG836Z6i3jCjHQ== X-Received: by 2002:a17:90b:1ccd:b0:2ee:c918:cd60 with SMTP id 98e67ed59e1d1-2febab78da2mr17797763a91.20.1740968761993; Sun, 02 Mar 2025 18:26:01 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2fea699090dsm7711363a91.48.2025.03.02.18.25.59 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:26:01 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 18/19] zram: do not leak page on writeback_store error path Date: Mon, 3 Mar 2025 11:03:27 +0900 Message-ID: <20250303022425.285971-19-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 0DA8BC0009 X-Stat-Signature: hrjaruto8pg5p7nz9mkjibj3i7dr9fqn X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1740968762-651181 X-HE-Meta: U2FsdGVkX1/Rxfu3ztNlgsTm6Sc1IneY3E//CZDPGEuEVX8pnU++OtK56CROHizbCfXDXgP/Jbn+oNnvZ3xwd6hTrngcyt3uDZH4O5xJ7Hk5SeekV+hFsES+tbUzKjaHzWEjDGGosrLRZgBQ7aaOq/mvsL7BFHVL5l+M+1fxYqJOTe1w3zHZzu1vkY0keogAUpjiVIcL9WAY1NtS3wjXqBgN9vsT+lI2fzLwONEgE9a57MyKy0NJfYLB4hnJAPl+pILD778bYDT7zPS1dpcwxdpPeUTE+1YmdsPGSRM/kQhzR3URQAkj4bbqW++rir6tl0VS3ng09Ur8ovW0pyn0gp3fvP5Jx/alOzSOvydk5eWxfVy40XlKP0UdVFqR/7IzFrCtxcKp7yW3UAmGgVha72Tk/knX1aml7a4wiLdcpyFIUPw+mGy/n7D/Kc1pwcy8t8LX8DfzlRTQF76YgEn8kBL+pRrjOfnM+UnB2LLTXRqWOJZXVlSSN2hPPXsb54rveIUnzsNgJSEtyuGxNepxS8RVB+e1Q2Qguu+e22BB23VmiXJ33WiAy5IDKsR5wDR8sKcwG9+jz1aQtkQ1G8t3ZJoxf1qGFcUEWwCJbS8Q4/2RCBA3C56AfxoQ98dZtftfOLAn9ftxzsTOD2j5VqC8hy1aADXunBRlNe0rsACjYiuuxyIUoFMn8ksHEzwBnN3zZl6qOOeTl58xBCVkjrvdg8lDGCB60l6Q5ap8GfIoRIUYaXr3nPAwhI2JQAFsF+5GmLANqqEWTDPif7f9huWt47UASYN/RJlCgGIS6eKSesxUk8kh5MiINrceQQBDgqXonSjdZfXGWYS0h2yjaBASOgh5fUmMEUoCYBqYg70QuK2y9x9Kmv6Q65hF2pw/mvy1IO08zwvyX1tx28vZApK/9AbiArUNF/qu4KdaqVhOz5H0kA2eSDfod9aJwatYaxnkWWSxoDfy48ys5Z/suLJ bZ+8u2hu vTVZo6B1L2PJJ2EAWLO3MkGkFowMKPaB1ZBGHsl5rydKxCIgUrGXyJsfkwPy+9tJpqK7rcBYUiisjS6f3sh12ZEim5gauS35DpP4XXthbec30/iIJ2EKufKzZcJiGDGlfUWpolXxVQ1PkkDeExq7mJ60PMVA52aE0lifJuNDPGRZeBYJo5xvHu71XRQbZaPwvlu+GKRTq89SmoFg53UG7AxiK63jDTh53bJSmQ0sRUrGrVmne9UWIhrhFOni8fpO3ykaOo2RuA8Xx4lSMRPcNjoHWuYmBSY8QgcpsjCjpzA/kIwfdl3Od7exRGELJQUUtUYdf3cGVrphSI4R28NtXRz3iprqkJzgIge0YqckrqscmoA4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Ensure the page used for local object data is freed on error out path. Fixes: 330edc2bc059 (zram: rework writeback target selection strategy) Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index e50a5a216974..fda7d8624889 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -787,7 +787,7 @@ static ssize_t writeback_store(struct device *dev, unsigned long index = 0; struct bio bio; struct bio_vec bio_vec; - struct page *page; + struct page *page = NULL; ssize_t ret = len; int mode, err; unsigned long blk_idx = 0; @@ -929,8 +929,10 @@ static ssize_t writeback_store(struct device *dev, if (blk_idx) free_block_bdev(zram, blk_idx); - __free_page(page); + release_init_lock: + if (page) + __free_page(page); release_pp_ctl(zram, ctl); atomic_set(&zram->pp_in_progress, 0); up_read(&zram->init_lock); From patchwork Mon Mar 3 02:03:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998069 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51A77C19F32 for ; Mon, 3 Mar 2025 02:26:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D2EEC280016; Sun, 2 Mar 2025 21:26:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CB72A280011; Sun, 2 Mar 2025 21:26:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B2FF5280016; Sun, 2 Mar 2025 21:26:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9009A280011 for ; Sun, 2 Mar 2025 21:26:10 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5D079160A37 for ; Mon, 3 Mar 2025 02:26:10 +0000 (UTC) X-FDA: 83178650100.20.29A5D48 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by imf10.hostedemail.com (Postfix) with ESMTP id 7F6AFC0005 for ; Mon, 3 Mar 2025 02:26:08 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=kOBPI4vw; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf10.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.175 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968768; a=rsa-sha256; cv=none; b=qhJxnH/f5AiIJSD68CsTXXQ7tt8QZz0sCipkgewcRa4k9mEyRyzq/hVaKPJ52XQh+duV8V R1wHLZjX5N3QZarhcUnVO3nG5ewC2kunRkuichcAL0Cj/PmeEe/ElfR+rDQsPGg/9yKx1f G4KKxYDZnnj7fVYqNbRB2sGaw8Wxs10= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=kOBPI4vw; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf10.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.175 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968768; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uqfsMunD7m+H43cqKfQWJpYb2LZblnvSJ7I+sWg0jpM=; b=v3Xy2InzDS+RCNEyWkWvgacsMR5kZxtavtMEkmz8c+dZZcKBEv/TBE705W563lamc7Wkzl DRd/SWiZ7SmLSzfiofZCbhOwbsQTsV/iTDeW2ZwiLlWX604nxpD6HMDK/bbCdv6eZQh2GH i+Buo2QcAntasdcBTicprAmAD0sR3kI= Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-22374f56453so53535975ad.0 for ; Sun, 02 Mar 2025 18:26:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968767; x=1741573567; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uqfsMunD7m+H43cqKfQWJpYb2LZblnvSJ7I+sWg0jpM=; b=kOBPI4vwnixZ3NawiTwWVAEX63fI4AOB1WF2oBj2G0fu5Q+bHCYo60XHqIO22Pzr8v eYR+d54lAGc5roQmU52xnaF2WO+R+Wn6tWsJ/1I5ThkP74/wm9t5LUBdh7xtK1E8y0p/ UMX5JJCEq5a+hIAPtlGW1jsegKJBJgnzQQXTY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968767; x=1741573567; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uqfsMunD7m+H43cqKfQWJpYb2LZblnvSJ7I+sWg0jpM=; b=AOWu4Xi0kBu37wXL6B6YnIbQzlamlG1nVynH/BL739TyMj68tOoc1SbowxureEx+Kt sjAlve16JL9f4/3ysQyn9YDfZqVHw7y1fwhtG2Wr2N642vMXjxY20+kwNKb6rW8/16FM iTXmNVQusFRW1BzcBo3dxzZ9EwV9HUUB9ZXPxMdcsnRZpmwHr/cMOFskXFj2baZMQofN 461Qtgc2LOwz/OgJ5YQ0t8pIQq0VmJqvBUf7r0rF/CdpwBVUOfQZYg8/oVmULZc2pKIG jTax1Cteu69JO8545LGnV7dFHoNeGPXn3Ms7j15XzdcPTfKL68ZHV7UYMcCUDF8nKlR4 eMYg== X-Forwarded-Encrypted: i=1; AJvYcCUxxK+2MIuLFnzOGMGHXdEba6wP/hOIb8y2lrdUXYw7wopIOMXoPDOfgQFb11uS+F6t4Z9lDPpv5w==@kvack.org X-Gm-Message-State: AOJu0YzqOEhB+Pm5cDBqZYLD65bJ1QWrb+67y1bBP62R+HpZi6curiky p/eGkymbwAm9RebhO5EIJzUuYjCwhrInQtHRiKUUh4M/SrOZ/lDZC7HUiRdCkg== X-Gm-Gg: ASbGnctzdXjOWAiovV5gOey0kZ0nssrnSLHCA8kOIcyGi4FVPeEuu4FArwNPQFfmxIG MB1o7U9Tg2FbNJtEjBs/nM7Lcd0OHrQMuJoo7MMzD4gRi65dL5FB6/pR3AVPxYc3NsfmXTxe9hQ NFDDP+nHASOq2ccSnKTp5FsQhv39RwpW+MICRVW7VB2P8bCBbIUWwXGNoWTU9qccC/QhPD3eiLn 17G4jw08vPJrVIEq7KZyUnmrER6jvKPgwbWuasOmhOz8mYjpXI086thlmcKPXrBtqauNEo49cBO dbwrzgrbv0hXUQgtctPibfg6Bkm5MIkkilhAc+3QNm9LoJE= X-Google-Smtp-Source: AGHT+IFee2geL4KKk2MpUs+7Jc0KaKLEfwBOxn21cLTIdP8Q7RgxcXKrybsQAiLxlVLTASPWxF9odg== X-Received: by 2002:a05:6a00:b86:b0:730:8a0a:9ef9 with SMTP id d2e1a72fcca58-734ac3fffa7mr16371017b3a.22.1740968767388; Sun, 02 Mar 2025 18:26:07 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-736556c8dccsm787620b3a.100.2025.03.02.18.26.05 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:26:07 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 19/19] zram: add might_sleep to zcomp API Date: Mon, 3 Mar 2025 11:03:28 +0900 Message-ID: <20250303022425.285971-20-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 7F6AFC0005 X-Stat-Signature: iuy8w7e8j6ukz3wyc7hzwiig5ijwqny5 X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1740968768-701707 X-HE-Meta: U2FsdGVkX1/qGwHaBTHjINLxCAGxDRbILAts9vtODHqWdnoMXV0Rb6deoGI4yngmIH4tD/CmQkomMKf0AHgmhZO6dHalGWyeiBgyGnl7dMSy8GgDuGkvgp5klZy147eEsofVyMzFFacH6drwZ1NnHcgqnmCvnPYG4wTcBBu6gqoqWd9Mgtb8K9ZawHSyB/YL5aLj1VrKcFHl16D/gDzUFQjDbWC2+9Fcf1jGUpZKb3GRgNlZzeKKnAX/grVY8ZvuoqYD+27Wb/k/WBCS3RLje0MoPLw9Qw3poZiAzb/dIFS+CaA93eztKVIGfYZcY78uUlxlpFQff6MJvkrr3sO3k+7ZmIGXv5lQG70a2ayXJ+kz7rt/Jilqst+kWHPkIfSn0yPc3SZQj0181EfouxOG0NXIFbOjG+TWumdJqvys0RvMF79ojiSnIVMCwfPnqgkkxdwvjEEr7B7LDUBypQ0r+d8nEm7909cm9AID0DHOCDoUbHVo9sz8Nvm0+rPL+2dVAPc0qS1KRYOkAKu2dY+/Xk/Us6wmUnIGFvq5sMWEm4TpcoYA9R5EGb+wIB84cQYubffPqyMKEd/uH5MSjTP3BKQ6TJFsF9AfVo6YRc7SMDO5Lt0+z2MSfFhYkP+koT/9AJlfdDAHWTqJILXqdWgx1fvLx3Hwemp7L4N3qNQD34OAIW5FFr2p8z1EPz1ZIezAkm5Lean/DqDFPr8dGrsaOHb/ZdrNC6JRiO4QPRMPx5V2W1iqw4shrYggJaFr3peG/y9eSvFyFw1CHc8e+L8bS0zBwPE/caI2zxnqVHwGYTtUsVZ4fEM/saWSiBvx2zbhiu3cpav6lj8UflwZ+W+p21PE8CerKPGV8RkyU8iMbZtw66umi90ebgm4l0FdHok+WpH4dUUcErawE6DmGm40gNTQGNBpU89T6KD00YaGFyUkF2JH7G7XJ+KptMzptJRlVcTPAJPMCEU1PpDKwPN b3vJ3Bpf X3PgLz+lr7+luvEMxEA9TZULcgi4JLaPKFX0n1T9BbfFPCOLLXXpQ/E0kleKAjSR6sEURRZUK/K5nu6XyLKY6qvzoqQIBHvbFcZ61jsq6quFgk6jSeekbTR2ui8kgKd9kFuX4C/I1qPspfOHM7L3dfg1MBQaN+mHPH3K/qUjbK7BKPsOtwbG+u3srJ+WoHJjAyyKAQOpUqwfBzPLJN392SyV/rWln9LktyPzsT05dw3Z58oD9wIewXsEjKEnPPZs2MAaIqkHzxxwF8riLfurn19xJnyYGX43YkCo25eUE/yI+V1CTgTfz6tFFzeiMQF5pdf0/KtsHIhOdYWnYS0MIxtcrcJ8AefHRR99jhlU+SHZbBxc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000006, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Explicitly state that zcomp compress/decompress must be called from non-atomic context. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zcomp.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index a1d627054bb1..d26a58c67e95 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -146,6 +146,7 @@ int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm, }; int ret; + might_sleep(); ret = comp->ops->compress(comp->params, &zstrm->ctx, &req); if (!ret) *dst_len = req.dst_len; @@ -162,6 +163,7 @@ int zcomp_decompress(struct zcomp *comp, struct zcomp_strm *zstrm, .dst_len = PAGE_SIZE, }; + might_sleep(); return comp->ops->decompress(comp->params, &zstrm->ctx, &req); }