From patchwork Fri Feb 21 09:37:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13985081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E9F6C021AA for ; Fri, 21 Feb 2025 09:39:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0B8BB280003; Fri, 21 Feb 2025 04:39:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0681A6B00AC; Fri, 21 Feb 2025 04:39:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E4BEE280003; Fri, 21 Feb 2025 04:39:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C89136B00AB for ; Fri, 21 Feb 2025 04:39:14 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 714B5140C1C for ; Fri, 21 Feb 2025 09:39:14 +0000 (UTC) X-FDA: 83143453428.22.CAA60A8 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf16.hostedemail.com (Postfix) with ESMTP id 8CEA5180002 for ; Fri, 21 Feb 2025 09:39:12 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=RlbcXKgk; spf=pass (imf16.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740130752; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=owpeQWEBy2AJNnuQaGP0HmEMVbn5ZGw0EqTWg3icB+A=; b=RfmJf6R3ZSHhvnGVewPqIjrm3KLgf8PsfwmntWnuUV8hjx+XOAhoaZjWrYI4elTlNy5OqZ WxTzTy94r2vc1y+kRgC4StJbBMmQXqzlfNPIlZUxLMejth9NLRRo1mL6ZBEhQTlnmVKs+w bDZhq9GVsB34KAU34sEr2ygvWGSgEzw= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=RlbcXKgk; spf=pass (imf16.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740130752; a=rsa-sha256; cv=none; b=4mKRbHP+MqWuwUbrs/+ObwZLWwjj+fdLghxkbp5fAZCI7iYSHtAY8uH4jVbX0f1N7N+2aC sz3P3u7imZ2EVVeJXKtumCA7zFGJIkqyivs/sDhF775g4e1qUGEwBewzwerEa4DU/ac68F DtXtkf6Zx8B3NOFObvmc3zQ0mzzsbxs= Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-221057b6ac4so34809875ad.2 for ; Fri, 21 Feb 2025 01:39:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740130751; x=1740735551; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=owpeQWEBy2AJNnuQaGP0HmEMVbn5ZGw0EqTWg3icB+A=; b=RlbcXKgkmaQanrwnWfxhLKuRB3zMVSU8TdlflS51mGiCYqJdPfN0WmWQhQtcnzfEKz 6gcs5tb4ELMS9InVNNwEtE/7oIx4c0GELb8dVDDaWOHT8bdidBow05gKL9gG9UQhU9R3 SJUjvgoAaOuLrLhU8PUXzKmQ+40oIS5fyNnCw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740130751; x=1740735551; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=owpeQWEBy2AJNnuQaGP0HmEMVbn5ZGw0EqTWg3icB+A=; b=tdQEesXj4VkdNn+GbeJxtAv1tbOfVXYc7BrV//wcqoHzy3NtnANDOTT4DUiyJ/ah3b E17yTN56Q5/xc6n7Wwj/i4Dc4mHQYurZ20FE5HrzFV4YWwUlJn8bH0Y3TCE7LgAjrF7R HCQLf+XBucgrQWaav5gXhN04viu1G/xqGsb1UVsqs1oJTYhKuYHJjBPPQF236nlcvsfR W0URzkHKIzpRIAQmA2BF3KwNBaYKcFWEQ75zgxdB6VDnsrZOJ/9AbIcSQPiKOl5eGSQQ LoXNlBClZizaflrQMiGOpmTGG27gX21khbbFkAvkbuRUXTJJqfYMAVerwZUXxL6h+nUn NxVA== X-Forwarded-Encrypted: i=1; AJvYcCXkUke76FHd5sQFSYPXVxzQxFoD95B5OY23J3UlfJzKG73M9xNubQifT2pLYKsUO01pJNhnAUE6Rg==@kvack.org X-Gm-Message-State: AOJu0YzdGUlvh/Kt33qA315h1qpdZU/LCgoa3/vOWEGC1J40nCHyFnH9 w2Jy5MNxeAqVHZcnHu/5zTtPcKXCggi0Kxzo5KlXRINZtSoyjvUckU42+b1SUw== X-Gm-Gg: ASbGnctdEkp1uIiqSBtgSZKsvduJNnibuW6FZ3feoD/phZnUZyF5+c6TORi58y9B928 NGOOF5JWwaTPHEND/WcWvVkL03jRl1hdQriTRJMiXj/ncgxD7FkJLR63S9YXefRfDEbT3w1+1Eo V26QyxIWiYMgkPdhQtZ8Cq+QF5B9p2kZzawYUPNhGgn34hUE0nrsyv8x364LSyISaWFlx8zxCdo hUQ4J0z7KXQ4/ra0W7FG86M4zs+iHXC12wi4wmYnpxSMbe7US65jXf1JtNvFB4fzZbgn4zxVaeN 8flf3U4gngmqaVXzJedLiGKS0VA= X-Google-Smtp-Source: AGHT+IEr9op50ypLXHrxjr4hZ7IvQcDqM446lcCXIuVBN0RImmgqBI3rJCeE1hqij4hU017n9SwtSg== X-Received: by 2002:a17:903:2309:b0:220:eaaf:f6ec with SMTP id d9443c01a7336-221a0ec9501mr37135415ad.5.1740130751439; Fri, 21 Feb 2025 01:39:11 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:f987:e1e:3dbb:2191]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-220d5367e97sm132666595ad.61.2025.02.21.01.39.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 21 Feb 2025 01:39:11 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v7 05/17] zram: remove two-staged handle allocation Date: Fri, 21 Feb 2025 18:37:58 +0900 Message-ID: <20250221093832.1949691-6-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250221093832.1949691-1-senozhatsky@chromium.org> References: <20250221093832.1949691-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8CEA5180002 X-Stat-Signature: d1g83e7ucnqq9awwce9h943txd55n31h X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1740130752-523252 X-HE-Meta: U2FsdGVkX19Oqzy/Xby1olNktYNU3OSCeP1aTDBh57ilbcm1LcrwYE4ChhFqgEW/O0XgbVfIR2o67Bd3tFa6dfy1Gb2rAsuyQqxZJWAuHCf0Mr9hBQ784MezRsmDTGG+Omq4fkb21510pne5hQVqkwOClzfLjMiQ4FefYB+YKHKnzVHCwVzSnJT0HJnbxAfqJXSfeUssHr+1DH1PITk34SnzleC9o7tosZMTJvgmxbrX2dqkHRgOWNF65HsUjdovthTPGRH7QYHgVXWNsxeCNv3gssXDnJi4qC1UDglPN1TMuFYweEUXywpUgZOGgznzkstaCHLnZrMaaY5/xWbDAt/hgOGyfuDxpyEjamrQ9nsR16+3+IZYJojielKKP1HOzRqOuD8imz6a3bfQqSQQz1JFaVIhQ0I8XdEtSMGHXSAJq88os8AEPXZo/j3K9PRvvDJGo9PAJS4RgoPSdDqGIKYUNO3BXg3EkO+rZypVQgWFXJFOezFTkTlx2w2UersP5vdVf4rqifAqsTQHSBKmkNMLLjK+MchPuwVNybsdjnedLNqWGgqijc8T9sXEAoYcNo/K7NGqYdr/XZAwSvmEAD9WJPdpXtbv1vND1Xf2GlWPRsq64qgQd4ehe+Rgv5aw6XyrMCi+dKlVKvRmvXsmG28vh8fpHW+NTI64mRFLjFrKST6e4dFjCOY7I3Ye97k9Qw8CAiFOALFTNfRxk/BK6Z0gr3u2AUYWrDK3HpLpeARC+MvrnF5YrgwNRTl/C0SmZVetqLdDNQwrhhIdxckWTK/Q9xwFjaNdHouWCUwWkk1wLYef1i0uSrESg+TRdYQiH/fo1wlOsrXSu3FApQta5tgc0W4qClEyt2PZyTUmlXWZwrWmvzx9GsML8enjmVZvUE9BnIDcshxPQNwhtMCAScdBVRt5Bv/O5CaNVukcwxJvEfmDvDZATQmW9utiP/F2tjNUz8pGgQ5CFL+Wf5Q kebMqlmq hNtrT7YGZiqYoL1T3fUhswMjJIUf53kra/w+qfmXoJHgooS1q5iYYyKjGZWxR+OBCXJFt9oVQpk+MpiVU19rz3XNGSIgU761nV3rygiLXz2kj4tyDVnEHvgRJ0RggoFajoBtobHgG1WEjQcc//rKL2tkIj6+1PQ7rJC/IydPQNepoC7h5UYGtMrjLpNBcjWTWfYsZqZO/3bMD4gvNwAgAQfjZwqDvD4PFj/BluN88TFCWT7dNxuwlushe6dblRiW6RqZ4SoSnYUZv2k6ShLyYgxK6JbxDZ3TC44cehC5UUD5yY6ReDyLMj54N2A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Previously zram write() was atomic which required us to pass __GFP_KSWAPD_RECLAIM to zsmalloc handle allocation on a fast path and attempt a slow path allocation (with recompression) when the fast path failed. Since it's not atomic anymore we can permit direct reclaim during allocation, and remove fast allocation path and, also, drop the recompression path (which should reduce CPU/battery usage). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 38 ++++++----------------------------- 1 file changed, 6 insertions(+), 32 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index d3b70aae2660..b805a3661101 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1729,11 +1729,11 @@ static int write_incompressible_page(struct zram *zram, struct page *page, static int zram_write_page(struct zram *zram, struct page *page, u32 index) { int ret = 0; - unsigned long handle = -ENOMEM; - unsigned int comp_len = 0; + unsigned long handle; + unsigned int comp_len; void *dst, *mem; struct zcomp_strm *zstrm; - unsigned long element = 0; + unsigned long element; bool same_filled; /* First, free memory allocated to this slot (if any) */ @@ -1747,7 +1747,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (same_filled) return write_same_filled_page(zram, element, index); -compress_again: zstrm = zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); mem = kmap_local_page(page); ret = zcomp_compress(zram->comps[ZRAM_PRIMARY_COMP], zstrm, @@ -1757,7 +1756,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (unlikely(ret)) { zcomp_stream_put(zstrm); pr_err("Compression failed! err=%d\n", ret); - zs_free(zram->mem_pool, handle); return ret; } @@ -1766,35 +1764,11 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return write_incompressible_page(zram, page, index); } - /* - * handle allocation has 2 paths: - * a) fast path is executed with preemption disabled (for - * per-cpu streams) and has __GFP_DIRECT_RECLAIM bit clear, - * since we can't sleep; - * b) slow path enables preemption and attempts to allocate - * the page with __GFP_DIRECT_RECLAIM bit set. we have to - * put per-cpu compression stream and, thus, to re-do - * the compression once handle is allocated. - * - * if we have a 'non-null' handle here then we are coming - * from the slow path and handle has already been allocated. - */ - if (IS_ERR_VALUE(handle)) - handle = zs_malloc(zram->mem_pool, comp_len, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + handle = zs_malloc(zram->mem_pool, comp_len, + GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) { zcomp_stream_put(zstrm); - atomic64_inc(&zram->stats.writestall); - handle = zs_malloc(zram->mem_pool, comp_len, - GFP_NOIO | __GFP_HIGHMEM | - __GFP_MOVABLE); - if (IS_ERR_VALUE(handle)) - return PTR_ERR((void *)handle); - - goto compress_again; + return PTR_ERR((void *)handle); } if (!zram_can_store_page(zram)) {