From patchwork Mon Mar 3 02:03:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13998060 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E77AC19F32 for ; Mon, 3 Mar 2025 02:25:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9362828000B; Sun, 2 Mar 2025 21:25:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8C022280006; Sun, 2 Mar 2025 21:25:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 739FB28000B; Sun, 2 Mar 2025 21:25:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4EE59280006 for ; Sun, 2 Mar 2025 21:25:26 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id F08491207BB for ; Mon, 3 Mar 2025 02:25:25 +0000 (UTC) X-FDA: 83178648210.25.E1CFA6D Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf14.hostedemail.com (Postfix) with ESMTP id 13A73100004 for ; Mon, 3 Mar 2025 02:25:23 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=JoMJcUKE; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.169 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740968724; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HsfvfF6qJrjX8Yvxt2fgmxWzD/A8epAgFC3OUlbDf8w=; b=aOdcOGRhWQkgkYl5mFPzblGCaPvfSq9LfCnOb2UA4A8hpE8I76fJVyY4cpCWT/7ZphxbHL WY8xUmBSANNB7azbHYpEUuGMcg4z8iNmaJUnU+5Bl+8UM2LXzCN1UONzxFOmwavSf6/qVu WsAbxrVK4mleR5us+ZxQ4pTByath7vk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740968724; a=rsa-sha256; cv=none; b=dbqnQmeLl4m85QpRJjsspXUkE125FRfwPlxNvqsOOvR8vjXtIcth/OalgCVqNhtqP/dx1r ZjTxEG4wdKGrDrZESBUYnzW01aehBDt+DppYlBHpwPLI4qOL0UNN8kiezwIWYc6cZhRY1p 9T+6q9vIieS2wvjBp08vOuy6qsBBcxE= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=JoMJcUKE; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.169 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-22382657540so21842965ad.2 for ; Sun, 02 Mar 2025 18:25:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740968723; x=1741573523; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HsfvfF6qJrjX8Yvxt2fgmxWzD/A8epAgFC3OUlbDf8w=; b=JoMJcUKEQgYy7KxlTU0GNEsXL7bVa2v5FtJmpMYeEVisLQi2uZXWxiUVfOPG8RTFRx uOwAKme3ho2tvyIedfdBuWU8vKk/Z4mpTBWjcoCXM2CpcoMFB2f9oxv9Fhzav3099TiK 0W4xKglnpy7mR/euBdJ6Gxfakwfk+eAfxuy5E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740968723; x=1741573523; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HsfvfF6qJrjX8Yvxt2fgmxWzD/A8epAgFC3OUlbDf8w=; b=rp3lOBV7sQPyAV2x5BGdamhzFu8/iP7Fc+BpEtcXgRSBXg3P+PFkBbFp8z1k4KaGbs HfEsd3LXFrgIZ3i2fEKdwJO9hpKAHfUWaoAOpJfGYwginkXOQXghptGZF/UIB9AYG98K U4+km5/hrp7/FQHD87MMx2Nj5Gg7PoZlsLqeMZ54Qi6+JTKltXCCTW94ZMQW8R4koyO6 2eDMRV33EP1EaNw53e5trww1cs4O0z0hAFx7XxryjFzNa+EC5tImfm5nt32/6MerinpX /tqn+q1HWRojb4MLFbDGehisaUKIwyoay8JRwLbtEU3wtzegqoc/QBSnZXfJfxL7LUIS uOPw== X-Forwarded-Encrypted: i=1; AJvYcCXF+ZPgT5ozBSntv3yxhfzu1x1EqO8aPFAlPLM6ApSAhOUY1pVUXqp5Q+sjGF4WSWZbdm6hu8/NOg==@kvack.org X-Gm-Message-State: AOJu0YwGwpon35PRXfDJuX4fI0eQik/ppPRtEK8JGbZzeItBjW4+CfXD p2fAtn3b5/oZFmWkf1lq/IkEuV+rBLa+EFV0yKf5XzLuRyPyBd1Z1NNn/06/WQ== X-Gm-Gg: ASbGncs5rlz6AWdvTLqOt5qgeyVKR7aFmMCBbGpkRr2Oobz30kt4J/YPpzDMgGsmZFF 65N1hb1GhFnLy1xbsBaV63WWfKWzQpb7SWkddFiSpfqXK3yVyRtz7c2gpeI+FM+VoePTfyMcJSn 4BA0Ht3wWif3YOVWPsR1sV61sOJkg+m6Tcm4Pe7czEK7IAjKh4/7+4uBjgC/mu98VfdELVbPKbJ Yj1satMgaNLGPChXvv6g/u5fQVtlTIuStNbWFuma2ipG8Q5VGK8eROMXorE58iCSmmQIh9e6/eN sZLwZEmONLQVsx+k57780vhEcEzbIRORF3fX20fn5nYya4A= X-Google-Smtp-Source: AGHT+IESsySExdYFB+0EQ5FWR9mq2/h7S2JmgVfs+L+05mw3XKfkerymt3bltrTq9TH/mZJI9SEzSA== X-Received: by 2002:a05:6a00:a11:b0:730:927c:d451 with SMTP id d2e1a72fcca58-734ac42ce9fmr14388946b3a.20.1740968722924; Sun, 02 Mar 2025 18:25:22 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:1513:4f61:a4d3:b418]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-7364ba1371asm1572928b3a.5.2025.03.02.18.25.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 18:25:22 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v10 10/19] zram: rework recompression loop Date: Mon, 3 Mar 2025 11:03:19 +0900 Message-ID: <20250303022425.285971-11-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog In-Reply-To: <20250303022425.285971-1-senozhatsky@chromium.org> References: <20250303022425.285971-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Stat-Signature: 1hjaytnocn178sdyjmif4enh89iy4i15 X-Rspamd-Queue-Id: 13A73100004 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1740968723-662105 X-HE-Meta: U2FsdGVkX18Nj35vzqeyo2JG6yuFrq+U80fEZWpERg/zgkhUHcPdVfuEGnWC33GKdlk+gkUGnXl7EjDrd16zEi9CTVMUkzelvfwgKIZ+/TtbWTnGMQZvky/OwPETtSRg76tc7bj+AVUYrJD7J2qTa0vfqwfFmyg8ippjD0ehfHfd5Jr0ppoUUWlx/Jn7I7eRy3woKaDiso6922Xs7tdlmQL0ESz3GRcELPzIoV2hneVgY+CzIpHcFqHuReFBknQnN5zFIe4SFusryxF+DmSGhjv4HhCibiKm3Ut2TNsHuoRk7aldgaairweqNmlmxj6wVMDYDHt/uwRePPrWByOfDRleDLioFXVdFfiQwoJXlvJqeWCelnVhEFND/G5O3DV69V7tL6+BKF2mA7zaylnLiFqv/IhIn/HmTWQjJUefjpq816vPDiARnhBdjxUBb4HNu1H3aR5q5qM12jwDH9vaKH1LW+oW7b2CPehN0Z1ZzxYjzDsTIzLp/owJRv+mgqVk5zl1xTVwVKFGbngtPZlD9H2KietpNpOBJOvb0ffqpsX+4u4eHF6k1aIJwSCm6KeGcwuTfAdT67r3pdfiBlfOT2EOo7AzNHqhB0/HN3AWUuvopbpvSm8ixRMz9xr+v+HeiuCgbU2U1MdN8rXgnug6WatDwv/EGofBww/B0RBaIykIp5RSrapnPI6/oCrhTxXERzlpv3h75A6NEZ9ZdjIH6gVfAXb8kmdSpvWD2Su7XedR03WUnu0ddt7ZCMRrtea7kG+A+6HiHrxLKwWVKAXcII9qp/ltamEIYHrl9aREV014d2K4N6t3u4gTbBFKSGrSVrgsMzelVFZHxH8LuLVwaM6rlFpEDIciPAZyYmNpMPAjlaP8mSdoTAb9GCnQYYmkm/BrcxsmGxaS0EBEAmwgi6Oo1e00koZL3Y1+nToz2vOs4PvU0lbsL/4LTuxXi9M0pkDCyvlVTE3U/C5GepP D1U2vMfL Cr3sIf8Hol7aXIIgiRT/0xmVboM1oNiwcdTV6xDUILOhX53UF0txahhW9cj6KVLlskwX9ljQd+TXqKetYFrxR1ifJECEXor6q2V6yqnnUPl4Mes4Tcig0RAliTuza0fl8TEjhyKuyA5MXSg2YWC7Zac7BJxTCuU+ppqors9+RDMYjqVyKahn8sPuTxGpt+K3yUif+370ZGB/lf6pUFS2HRVy7Sd/mYUzGGYgKCFwNzuVs9Hhy4KbbHgZGCyE0xN6yUhf+89/0BZnyXzqNi/vIPNcop9lp+aSi6wMkmpj3brr0XuZYzvOz3M/vSQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This reworks recompression loop handling: - set a rule that stream-put NULLs the stream pointer If the loop returns with a non-NULL stream then it's a successful recompression, otherwise the stream should always be NULL. - do not count the number of recompressions Mark object as incompressible as soon as the algorithm with the highest priority failed to compress that object. - count compression errors as resource usage Even if compression has failed, we still need to bump num_recomp_pages counter. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 54 +++++++++++++---------------------- 1 file changed, 20 insertions(+), 34 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 6dee885bef9b..bb88b63d193b 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1888,9 +1888,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, unsigned int comp_len_new; unsigned int class_index_old; unsigned int class_index_new; - u32 num_recomps = 0; void *src, *dst; - int ret; + int ret = 0; handle_old = zram_get_handle(zram, index); if (!handle_old) @@ -1933,7 +1932,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (!zram->comps[prio]) continue; - num_recomps++; zstrm = zcomp_stream_get(zram->comps[prio]); src = kmap_local_page(page); ret = zcomp_compress(zram->comps[prio], zstrm, @@ -1942,7 +1940,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (ret) { zcomp_stream_put(zstrm); - return ret; + zstrm = NULL; + break; } class_index_new = zs_lookup_class_index(zram->mem_pool, @@ -1952,6 +1951,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (class_index_new >= class_index_old || (threshold && comp_len_new >= threshold)) { zcomp_stream_put(zstrm); + zstrm = NULL; continue; } @@ -1959,14 +1959,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, break; } - /* - * We did not try to recompress, e.g. when we have only one - * secondary algorithm and the page is already recompressed - * using that algorithm - */ - if (!zstrm) - return 0; - /* * Decrement the limit (if set) on pages we can recompress, even * when current recompression was unsuccessful or did not compress @@ -1976,38 +1968,32 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (*num_recomp_pages) *num_recomp_pages -= 1; - if (class_index_new >= class_index_old) { + /* Compression error */ + if (ret) + return ret; + + if (!zstrm) { /* * Secondary algorithms failed to re-compress the page - * in a way that would save memory, mark the object as - * incompressible so that we will not try to compress - * it again. + * in a way that would save memory. * - * We need to make sure that all secondary algorithms have - * failed, so we test if the number of recompressions matches - * the number of active secondary algorithms. + * Mark the object incompressible if the max-priority + * algorithm couldn't re-compress it. */ - if (num_recomps == zram->num_active_comps - 1) - zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); + if (prio < zram->num_active_comps) + return 0; + zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); return 0; } - /* Successful recompression but above threshold */ - if (threshold && comp_len_new >= threshold) - return 0; - /* - * No direct reclaim (slow path) for handle allocation and no - * re-compression attempt (unlike in zram_write_bvec()) since - * we already have stored that object in zsmalloc. If we cannot - * alloc memory for recompressed object then we bail out and - * simply keep the old (existing) object in zsmalloc. + * We are holding per-CPU stream mutex and entry lock so better + * avoid direct reclaim. Allocation error is not fatal since + * we still have the old object in the mem_pool. */ handle_new = zs_malloc(zram->mem_pool, comp_len_new, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + GFP_NOIO | __GFP_NOWARN | + __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { zcomp_stream_put(zstrm); return PTR_ERR((void *)handle_new);