From patchwork Fri Feb 21 22:25:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13986378 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0978C021B6 for ; Fri, 21 Feb 2025 22:30:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 736F1280005; Fri, 21 Feb 2025 17:30:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6E6E0280004; Fri, 21 Feb 2025 17:30:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5878A280005; Fri, 21 Feb 2025 17:30:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3A1C6280004 for ; Fri, 21 Feb 2025 17:30:38 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id CFF0A1A08B7 for ; Fri, 21 Feb 2025 22:30:37 +0000 (UTC) X-FDA: 83145397314.17.003BDC1 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf12.hostedemail.com (Postfix) with ESMTP id D8CE44000D for ; Fri, 21 Feb 2025 22:30:35 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=LRn75+y9; spf=pass (imf12.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.176 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740177035; a=rsa-sha256; cv=none; b=U7Z/OYP6tqgGmopR623V1op0BtEqfRUpoRhHxr/V3js67D6sot7+ixjsSTiv2QdgsIN3mE dYExF2aGdm8mxNuRRDIfKTcxBahUpAFCy+vN3a88X1DBJ14dIVOZDBSe370tsHIDilvhbm G1cRc7z1gwmrltdXVc+IVX1JVBIVP24= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=LRn75+y9; spf=pass (imf12.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.176 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740177035; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZGMz8miVpZuxydZjLRlixJe565Cxqs6XaHLZ0PLT/Cw=; b=K33I6GJDSopO58f2gExe1Dm1XCgAFTNv6rRo71A9qN/Z8jlh8zeupb29eAmrjv1rBjusvf 50iOaNIYuwc32GwnhIldVC/oYlCi1eVshZ8W3ih/RF5U6JeWvgbPBFz60g8Ryhau3Iom0V e+yqw9vdQDhjKqd2DMNpcb06QJOArz4= Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-220d39a5627so41848325ad.1 for ; Fri, 21 Feb 2025 14:30:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740177035; x=1740781835; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZGMz8miVpZuxydZjLRlixJe565Cxqs6XaHLZ0PLT/Cw=; b=LRn75+y9Pr5O6fohUWxdU9aMibOl59jVyBofwmmoKRAvgf/5P5NnMCOfLWljC9P8iP VCiWxzFz0qxVoBudTigEcfcm4MjGVMe0fr1I+FIEZbAxj/cU2W1RlfKasn7NRPhvvKXF 7QLleqT1HIXiU6zAepohR/yh44bfpoJylWqfk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740177035; x=1740781835; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZGMz8miVpZuxydZjLRlixJe565Cxqs6XaHLZ0PLT/Cw=; b=emmfkaE1S1Zc0OyiWmGPxjP0Uj7P3l6+bVrvZP0vt1wDEwX3A4j5BUrtAhF7HpzekV G9yJ9ezyANcT0uiIl2PBasc27hAm1acEF4ZT+CmfX8fNmiW2qxzE0pcJBMI1y2Z4ceUI TzDlenWgksEpE5OJeWA8kj1VQTPX7XKP177IISG6Mza7vb8a9yrIiHy2AKg0wAYuXJkL 5OJuegDQ/I51e+5GkyJWb3OQQyAo0FyDb8DinDKDJ2aQHM+Kw2pb7LYDAfwcNxVE/Hr3 epS9fyzGTpgLHe7ggO/6rOuLsAQ1/yuMu5v2KzG3DtwD869nd8dg+roXz6YEByBbGNt0 A/Gg== X-Forwarded-Encrypted: i=1; AJvYcCW885lCCVsWleFOmtiWUVLlcChUXbU8ThlrOXtyawijJVgKF7q+NLlJm1DJStVp4jEX+BFl/6k4KA==@kvack.org X-Gm-Message-State: AOJu0YypbEBvy1ukaPBaSzgT2WnQtdlRGhfGwoi0nm10XD9MpRlJv97Q K8eVQmx37bGF5ETZcAnHSNXGg1cqKhABq5924aBmndMBm8/iBDy+WrK/yyZK+A== X-Gm-Gg: ASbGncuORyx86+5cBLqmqZYQhAWHfKWSW9nm2RHcONHWc+hX5JG58zCc0PYbfTc4Cds 1aVZcTsopLASzWb5KYaiCHb5VIf9rFyF1F+io6y/DOFjAaeyJug8rTXOois4OBeP5D4DNHaygbd wuz84Xe4nJHHhybaiMuf85y3FiI2TBOmaXFghK+s6dVMnCx+bjXZ5O6uPysihb8Hgj1Hi44bMCW 5OZGYrj0i4Bw3mqhTSX4RfJbjIrVmtOZAOKZRUgbnVQsLuNdi3wvfa1YBQJuszJnYyRwHgPzjq3 OMgqaJI0stlBzVRP/q/S9sg+hpc= X-Google-Smtp-Source: AGHT+IHL8w233VXHSZOeZO8XETvndpBr3OMmgSLQskKaw+oSbhIFchdvDqKDGu9+zeJWRzKLmmc+Yw== X-Received: by 2002:a17:903:2288:b0:221:331:1d46 with SMTP id d9443c01a7336-2219ff8278fmr73678355ad.2.1740177034842; Fri, 21 Feb 2025 14:30:34 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:f987:e1e:3dbb:2191]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-220d556fc57sm140834015ad.194.2025.02.21.14.30.32 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 21 Feb 2025 14:30:34 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v8 05/17] zram: remove second stage of handle allocation Date: Sat, 22 Feb 2025 07:25:36 +0900 Message-ID: <20250221222958.2225035-6-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250221222958.2225035-1-senozhatsky@chromium.org> References: <20250221222958.2225035-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Stat-Signature: 97xc5us7o4diaeh6f3fcdukferf1p9gr X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: D8CE44000D X-Rspam-User: X-HE-Tag: 1740177035-776752 X-HE-Meta: U2FsdGVkX19ZzS+pCaI9I2OctWi9qWvysg0F2OIpXPRBXASxb2Im2IW8xIp/MJ2wq1ymeQ5Pvknl5ntrnbF3Rm6v97IMd2y0YQ0UwrvF9wKyrhrMW0H0+2dZ4q9IGO909QEKOYfrsE7w4z90QWDodHY77Oi1rtTFqzc6gvoCkiXHp2Rn/RNI2uStcRW/7cT7XM3FejqCqToiaQan2u0V0GYrFxWilKbKdOl1Ki/m5j36RXWJx+PsDSkcjWSCRmlGCK4dURGFfBzE2F8aoyXE4Twrq+3q+82UZAwCXwOXVfaHb46CM03ez5GBToKaHYJQZ4NQ95a4JlrGdF9+ABTw6Gs/JHnBSa0fsQQkbgguq0cZ5ZbbNkDIf2VjNL396VEuUISrGzvJmakkw3K1Mr390mo1YWw7eBUhCh6ApIZVlD4ilreq3XTAqCBDm5j0UnKiEkjTgJsfD4zn0Q8hxoT3f+8zbfPoBmXD6edVm13JO5ZXyim7QIbJdE1qnzwWUiMct4Z5vJ4uDrGpkPDBSXFqfc5+kiSlrhuzkpqCn3dvfEahPzyJi2+ZXEUIVE+dhUnX9ywyvKPpPOm0jyF9HXvPzKIWBMRqf06JL61geek+zFkaGqjv9gIUikfHxoUNG18CubbtN3acFrizG3jvsTUc6nDzIPztWpkcjmfQtg5cGlRjSoKD6nVVDOh4ERdzwrlTr9zB5uagAptvZWHnbw+ZCpg6wpyfq0f2yy/UXOyASZMAJhzKeqSXUcEW6Wk9S5EHiiPvWW4dumGc/4FNNfB/DTUT8RTFDGbSlrIQyZ0dwQospNSwDYKFevU99Bx3OD8oJfBKHc/W1zeh0iJofPujTysQwA2F6UH3Y53p28Ci6tofFqgmzrLn8kWZ8dEqmkOoKKwyccg6hd/hcYzYdsYNmrlMHTRNkrrkvBZHJMu8zHQFKNyW8mGJGOHO/0MUfTOg1sKjBdhWcrbuQkYvH6o FWWTaCRb ROcwmIHv7tY5J5qVu03iehrfmN+NkST0wtW1QO+YeCuaMYKD+FquSqPfF5X7pf9Cofvn92LUxO5G5iZIrPLvX7gDDfcBWmJFklJe+n/pEV0iEZ7KqXoMEE/6ellQygQ/97umNpCFpUu48j4/BOSnifwZeS6npyWSLg19aeJKWD6bPSCt2hgnCMRY9BN5ZEkAhVDRNzkOyE6fmLWrcSVjc3uyZrtZpAAt3ux6xLHdiHRJYIKXkIbpqdtqg17716HYiSx9WgnYW60qB7nNZpPEYAF7PMLCbxJeoBgq57lSk6V7t7qyuModKUTT2jA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Previously zram write() was atomic which required us to pass __GFP_KSWAPD_RECLAIM to zsmalloc handle allocation on a fast path and attempt a slow path allocation (with recompression) if the fast path failed. Since we are not in atomic context anymore we can permit direct reclaim during handle allocation, and hence can have a single allocation path. There is no slow path anymore so we don't unlock per-CPU stream (and don't lose compressed data) which means that there is no need to do recompression now (which should reduce CPU and battery usage). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 38 ++++++----------------------------- 1 file changed, 6 insertions(+), 32 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index c7bc0c9f3f2f..4ccc1a1a8f20 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1729,11 +1729,11 @@ static int write_incompressible_page(struct zram *zram, struct page *page, static int zram_write_page(struct zram *zram, struct page *page, u32 index) { int ret = 0; - unsigned long handle = -ENOMEM; - unsigned int comp_len = 0; + unsigned long handle; + unsigned int comp_len; void *dst, *mem; struct zcomp_strm *zstrm; - unsigned long element = 0; + unsigned long element; bool same_filled; /* First, free memory allocated to this slot (if any) */ @@ -1747,7 +1747,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (same_filled) return write_same_filled_page(zram, element, index); -compress_again: zstrm = zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); mem = kmap_local_page(page); ret = zcomp_compress(zram->comps[ZRAM_PRIMARY_COMP], zstrm, @@ -1757,7 +1756,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (unlikely(ret)) { zcomp_stream_put(zstrm); pr_err("Compression failed! err=%d\n", ret); - zs_free(zram->mem_pool, handle); return ret; } @@ -1766,35 +1764,11 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return write_incompressible_page(zram, page, index); } - /* - * handle allocation has 2 paths: - * a) fast path is executed with preemption disabled (for - * per-cpu streams) and has __GFP_DIRECT_RECLAIM bit clear, - * since we can't sleep; - * b) slow path enables preemption and attempts to allocate - * the page with __GFP_DIRECT_RECLAIM bit set. we have to - * put per-cpu compression stream and, thus, to re-do - * the compression once handle is allocated. - * - * if we have a 'non-null' handle here then we are coming - * from the slow path and handle has already been allocated. - */ - if (IS_ERR_VALUE(handle)) - handle = zs_malloc(zram->mem_pool, comp_len, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + handle = zs_malloc(zram->mem_pool, comp_len, + GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) { zcomp_stream_put(zstrm); - atomic64_inc(&zram->stats.writestall); - handle = zs_malloc(zram->mem_pool, comp_len, - GFP_NOIO | __GFP_HIGHMEM | - __GFP_MOVABLE); - if (IS_ERR_VALUE(handle)) - return PTR_ERR((void *)handle); - - goto compress_again; + return PTR_ERR((void *)handle); } if (!zram_can_store_page(zram)) {