From patchwork Fri Feb 21 22:25:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13986386 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E41CC021B5 for ; Fri, 21 Feb 2025 22:31:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F015C280009; Fri, 21 Feb 2025 17:31:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EAF8D280004; Fri, 21 Feb 2025 17:31:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D295A280009; Fri, 21 Feb 2025 17:31:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B5247280004 for ; Fri, 21 Feb 2025 17:31:00 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 746771408F9 for ; Fri, 21 Feb 2025 22:31:00 +0000 (UTC) X-FDA: 83145398280.21.919FDE4 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf17.hostedemail.com (Postfix) with ESMTP id 5DF544000F for ; Fri, 21 Feb 2025 22:30:58 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=RzR9ZIoL; spf=pass (imf17.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.171 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740177058; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yMiFPpRmjEuGJFug3yngS8shb+rlTOrw+KkttJGol4I=; b=1SEMFEZFMol5eN4S5uxjcfrWMQSHwPVMjlC2CQ1+zqSxwu0KADaxbFe/e8EjuabPpL22zW pcsHgpTI5w5X0fbWXlJyHreMohRBAKBlkHuAz2tq5h+apFoe32cCDt/ycTxG/VguZldLrR AsiSP4yjoQdGcAhZCsYLP4B18XTl+M8= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=RzR9ZIoL; spf=pass (imf17.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.171 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740177058; a=rsa-sha256; cv=none; b=UyZdnvgvMXPC+EgZ/5TzqA3dHMiKDrcfqbfiQ0ddYaAUQY3eIlsF/IYeB/X6hAfvRX00bq 1HTQRLZrDNrkRa19SqVC1zObEJBGibKgjV+7hl40mD7aWv3TGtgXMoYNCd9aYMY1yO14a9 OuIJecHxP2+4BStEwPGTMkxsQKzZeOg= Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-220bfdfb3f4so59879605ad.2 for ; Fri, 21 Feb 2025 14:30:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740177057; x=1740781857; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yMiFPpRmjEuGJFug3yngS8shb+rlTOrw+KkttJGol4I=; b=RzR9ZIoLOs/UPBR/jXNmPBH3NvaILnVCOrFzVjzUMK/NvoRVn3s3VKeP+vE0YMdl6X eMoIIQ2Yji+saX/YgJNHKY/UHDya2+vBMTSxphBnJWBFMJd+BSe6VJ6lwpFNVaRJ2QQg IUYfFejj16QLkVLLq8uxnzlYG87Q9BDDRXqlk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740177057; x=1740781857; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yMiFPpRmjEuGJFug3yngS8shb+rlTOrw+KkttJGol4I=; b=QYY3QISqPdpPRZ95Ymh0Wqs/ygiQUFtkqQKmynKLJ2yUTn2CFWQcEwj+x78RrZpHeQ l11aL3X35fLK5IpRbs7d//QaK+enB181eS7FoNRem2xRL7mgO7n1qZ+9nFzKMUvpXcIL 3zCyVGyISlU8guZ56Zq9wccqmuCnSPh3/4cJQ8DouTDYO00DcI/E1sJeDUbLpQVk/L5m rSnu3wkyicxSJgZ34YpCMdJ3r6FqMW76Fjteba3WO7SlXDfsAIgRiqcdlY8ucEmdbXJ8 dauz6qUbwn0aii6Et3TjUlTQ5ANFjOUI8KRouuKLt/J+7GYWRNTZm/kVjVdJVeFShYLv CTJg== X-Forwarded-Encrypted: i=1; AJvYcCWLYXNcX/lmnsdIWP/G7ggqLQUzG/to/EjGOtpdIonoWL2nF/FjbOUP+9gJWSOfoGKAEJOGMimudg==@kvack.org X-Gm-Message-State: AOJu0YwW4xRj65z/B0R8zraqVTGomWK7Vbuoh4EQmGd00rT3NEqjI4MO o6b6C2UUHJoZaQ6VCJVGLxjD/EJ9058OuX/Ujd+ROb7pCBxf2gGURwDwhpdEBw== X-Gm-Gg: ASbGncv1w+UqHChwsh2IiqGI9+pxEmBxMCC7vnopGOpYOAH3e/KyntLfrxiSFWRzhmt BkEcREvzLKODqnl9MOXOUyE4g25Zg//rjf4jpAub3aINwcynG87BBWKKaBiWAXQNW5IedwNIB20 4tEEgAZNzCMLQgdGk2xcO8U5IxtCoqEwUkNZARtsyBbugibTtDSkgy3gBQdG6FKZLOx1ZZx+nbh GHoDmbRC+s90lehU+7VfFzKMOoiaIjrTCaO+0tTWx+Y8o724hiaGvRDKvJHk63X+HRSJbyQmcjn h12qKE0GKDb2ycXRa0CLLFgmdr8= X-Google-Smtp-Source: AGHT+IEQrQ7GyEMKY0xC/QjOZMueeZal8jjrcZZyRbZ+PqQnVupxiIMR8/upUlsyn1jWc83rlKp/Qw== X-Received: by 2002:a17:903:32c5:b0:21f:1096:7cb with SMTP id d9443c01a7336-2219ff50eecmr69781335ad.20.1740177057264; Fri, 21 Feb 2025 14:30:57 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:f987:e1e:3dbb:2191]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-220d5349056sm142860655ad.22.2025.02.21.14.30.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 21 Feb 2025 14:30:56 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v8 09/17] zram: rework recompression loop Date: Sat, 22 Feb 2025 07:25:40 +0900 Message-ID: <20250221222958.2225035-10-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250221222958.2225035-1-senozhatsky@chromium.org> References: <20250221222958.2225035-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5DF544000F X-Stat-Signature: djk3eny3kkz9b9xa9nxt7rgqruxadq7z X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1740177058-719546 X-HE-Meta: U2FsdGVkX1+TkYfd2C3jq/f8FHdDVcF/d35DsTDJirN6tivCRzUBlpjxeh24udwJp8kSnH53QlJho177cWW61zaXp1zNgxx0ob/pptHCKJixeptZh/jA57c9oIEV83qpEtvMQQBtfHk5dN3UAppxjCId71Wg3Qd7QEnOkKhrk1EFOpx1gX8Xe16wy6dyNW6CMsDJAGjGGAjs857gqk2KIyZGKxKv1Zu87x6Acan341HDucJm5TpWLV2DloLx+hM5jF3mpWssRlu1vkbPWvdqxxX7+AV8TT2Tl5jOqjLgDW/mJ1IYaXxM68ysTS1tb11EJDOwVX6D3F3cgf1oIEC1hErGzSQVMxxaYHbJTjmuGkuEpdr+hSF58+V/kCjI9spcZCt/Bnh2Tdzwlp3cdTvlSEpMwAAFz5Ad3UxaZZi5mrm2WBbE83cIgZLOLa6/s2dMaj77NFmupYQZ8T+/PWqXdIfy5UIvXEvz1qg/19GNxvQKCK5rzyhkMlgvgtmQYNPIi9y1j6JHNiaSgQPSumnLHXGzaTrcDTPNpO+wM8pFOgytjNEX46Azaips7+n9h2KgHHcqv3z0YnjsCLc+x3lcnZDBT12cm8NZOr9sV42L9n7iy/CvK8jnL9oUEPh+8MqKdrPS8vANcx/FJXCo9kKhC6btPw+P30HINUdLLxULS7RdslRsY/1gxoXt1p7usn9lG1LK/OVVieU+9XPqCJMVB4mugN3uBFgFYiov+S7/tGiDppIaPWGDiqZUEqkcBV5nFXU7FzJrMzhhvFNJkbEARB9QeaN8roj/FOPoQIytG00KcVmwS7b7jiySoXoSWdtFL+vsTOAh64r+M/Ij81G0Qi7ERnuT6uhX0lzTbb79hYfdfUuULgEAiKV94Jhxd9J8/l2+q90AIIeqSbt7WwY8q0X1hcLFnR8/4+oa3so7K6JXzSaVF6M2NhSV7Qak3UJwlpvUHuSD1drnSnqTsOl mpzo/WBa VFUgfgmbJt3/ggv0F5nEzLD2wX358nvgyDooxZxlCqJhU3iGHODkJoXme4GCdJuJmQmN7uU4YV8XNJz+DKJVWiqnTX5+WZn5oWKXvTKWX1i253iaaCmgF126+W72Shumtg++jG6ZCloG/FXmGROsZf1YrYdFAV/EwpNIZoj/ho0YjQjniwnPzdTQVAflSQ9URabe2Rie62acHTMUBptB+VjXQbIVH6wraWCBQfofv43sjjI3E1GfyJlqeDfChrtqpwSxf7lvmSb9MkJatDGvgR9fEyyjCW+tW+R4j4vRZmIEaDWflzbF9fNx0sw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This reworks recompression loop handling: - set a rule that stream-put NULLs the stream pointer If the loop returns with a non-NULL stream then it's a successfull recompression, otherwise the stream should always be NULL. - do not count the number of recompressions Mark object as incompressible as soon as the algorithm with the highest priority failed to compress that object. - count compression errors as resource usage Even if compression has failed, we still need to bump num_recomp_pages counter. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 53 +++++++++++++---------------------- 1 file changed, 19 insertions(+), 34 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 92908495c904..b96be8576cbc 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1892,9 +1892,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, unsigned int comp_len_new; unsigned int class_index_old; unsigned int class_index_new; - u32 num_recomps = 0; void *src, *dst; - int ret; + int ret = 0; handle_old = zram_get_handle(zram, index); if (!handle_old) @@ -1937,7 +1936,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (!zram->comps[prio]) continue; - num_recomps++; zstrm = zcomp_stream_get(zram->comps[prio]); src = kmap_local_page(page); ret = zcomp_compress(zram->comps[prio], zstrm, @@ -1946,7 +1944,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (ret) { zcomp_stream_put(zstrm); - return ret; + zstrm = NULL; + break; } class_index_new = zs_lookup_class_index(zram->mem_pool, @@ -1956,6 +1955,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (class_index_new >= class_index_old || (threshold && comp_len_new >= threshold)) { zcomp_stream_put(zstrm); + zstrm = NULL; continue; } @@ -1963,14 +1963,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, break; } - /* - * We did not try to recompress, e.g. when we have only one - * secondary algorithm and the page is already recompressed - * using that algorithm - */ - if (!zstrm) - return 0; - /* * Decrement the limit (if set) on pages we can recompress, even * when current recompression was unsuccessful or did not compress @@ -1980,38 +1972,31 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (*num_recomp_pages) *num_recomp_pages -= 1; - if (class_index_new >= class_index_old) { + /* Compression error */ + if (ret) + return ret; + + if (!zstrm) { /* * Secondary algorithms failed to re-compress the page - * in a way that would save memory, mark the object as - * incompressible so that we will not try to compress - * it again. + * in a way that would save memory. * - * We need to make sure that all secondary algorithms have - * failed, so we test if the number of recompressions matches - * the number of active secondary algorithms. + * Mark the object incompressible if the max-priority + * algorithm couldn't re-compress it. */ - if (num_recomps == zram->num_active_comps - 1) - zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); + if (prio < zram->num_active_comps) + return 0; + zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); return 0; } - /* Successful recompression but above threshold */ - if (threshold && comp_len_new >= threshold) - return 0; - /* - * No direct reclaim (slow path) for handle allocation and no - * re-compression attempt (unlike in zram_write_bvec()) since - * we already have stored that object in zsmalloc. If we cannot - * alloc memory for recompressed object then we bail out and - * simply keep the old (existing) object in zsmalloc. + * We are holding per-CPU stream mutex and entry lock so better + * avoid direct reclaim. Allocation error is not fatal since + * we still have the old object in the mem_pool. */ handle_new = zs_malloc(zram->mem_pool, comp_len_new, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + GFP_NOWAIT | __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { zcomp_stream_put(zstrm); return PTR_ERR((void *)handle_new);