From patchwork Thu Feb 27 04:35:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1334C197BF for ; Thu, 27 Feb 2025 04:37:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 53559280007; Wed, 26 Feb 2025 23:37:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E479280005; Wed, 26 Feb 2025 23:37:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3389B280007; Wed, 26 Feb 2025 23:37:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 11D3B280005 for ; Wed, 26 Feb 2025 23:37:02 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id B84DB1C7A08 for ; Thu, 27 Feb 2025 04:37:01 +0000 (UTC) X-FDA: 83164464642.08.F1AFE5E Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf11.hostedemail.com (Postfix) with ESMTP id C667A4000D for ; Thu, 27 Feb 2025 04:36:59 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=od01zQUK; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf11.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.177 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631019; a=rsa-sha256; cv=none; b=B1vjs1jJth/NoKo/bbb36n0IaveXxJPuZEYqB1MBXffSH80jfyBhkPy9zvlCbpBaTqqUzR TTCmUJgRR6GVMQAkr3EjlNBUhbX5IApSzZAutsEYU+VP+GD0K9whgQqEV39iD0ZY5aLSv7 r93K6RDhRrv7Ctg7BQ4ADy9sCGCXVmw= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=od01zQUK; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf11.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.177 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631019; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Be7BmKV5Rl+tSCSL+PenBDpRFLFJDOVjENq1jnQl0Z8=; b=ckiCYJusXb9/N8tW8BWKK6J979RPtG2h/92xE5f5G37xMDIGCzAuCC7cqAZ54PosY59Xte ZrBZQDB6NABqwpmSrwGGeSLsiIizTuJ31ZmDhRAt1QTbzT01XloiSf7KvGa9OhCoK2Sluo VBxGQ4WBM/SEbk5zkqAJzdjb4r48NW8= Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-22355618fd9so1975165ad.3 for ; Wed, 26 Feb 2025 20:36:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631019; x=1741235819; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Be7BmKV5Rl+tSCSL+PenBDpRFLFJDOVjENq1jnQl0Z8=; b=od01zQUKJws+q19B5kLMmBGtp2RjseAe8MyIIXt+4OKN42DLDBA3+oStU0lw+mfHtH wGVT/7E/+WplA8km9L/ssG9nqtwoLLqNj1QV6akgehWQWFYkZ2VJF5h3IS2iUl1mEPwn uZOXJFhjJiAttYlazLSnqmireKSyzuUCgy6oM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631019; x=1741235819; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Be7BmKV5Rl+tSCSL+PenBDpRFLFJDOVjENq1jnQl0Z8=; b=u+MQ5n1dahwRxSBO+nFUviCbfnnnl12RrJCekFTsR0NpoXjmr6WbSWy6RbU7RM+1qD wUDJ7YscnlIj7wnE8riIs9qZiN8Frl5+37rwXTeQeS9sCF/1TC5XKy5F8TPX/7/lnxX5 5I9+1+6z3bZ4Iv23BvcxEDSkknzUMG6ngN+lwEKrZvIbw9xriq2BgCEHttZKPK3/gWyX S7EUfOIkzg2LzUMtk6+KnHVVbVvX/EhM7c9z0Spu9bWtKOXSF/79QxscjNtxISMd7hzA h53C8OxwKWEVQHY1Q3c+nmxxyQilwkb5yZEp1v1VxNtH/iMzWQW8JpJ0eoQStjodLsmr 6w+A== X-Forwarded-Encrypted: i=1; AJvYcCVwz/CfBFKJEY8I1J+dWsI/CgjvjHFDEJMg4BpGq2Y/P8KlT/rGaCF+OJGqB/+zG8UbRMI6mciIYA==@kvack.org X-Gm-Message-State: AOJu0YyxCPjo5HtDvKpkNzNnPsX9YZnGLFxavELUabIc0y/fvtGzGk5l fy5XQ3XCBXDwhwjT/BmLYhBQz6MiH3CjngKQPeutUk6sNiOINg2tmcBd3MiKng== X-Gm-Gg: ASbGncsYhW8EYj1jgqW3PKWW3CokWzTWwRstVga8hfZWbFq4Yky7cbIEMIfsJvPrFDZ 5YBZQT8pSYfv21UgI80AG/CqTsuJf/yoWm7WLXuGOUssYOhQuIKXyuI+fYH3/a58I9IwV21T7wc OfnHDmyF2Qpd/Sidh1ieK9/mQypjHd6TemBt3N8tWJurKZPleBQFBK37gBN5QULtSLDWNwoAelN lUlcJM6LSMiQlQ0w07V0oIVkgPA9tC0iUqQcbbQSix1Prc2TD5lGSorDkWyYLE34cWm6WoJRL2j jVw1a2aW4JQlbSOOyu7caTzbGT8U X-Google-Smtp-Source: AGHT+IFJgaRFQVylaLj6I4IPqbcJ3f5L/ef1xGtQVqtC08AgMT8m0YWQWIDG6Ie7Da0oM3f1AlVfQQ== X-Received: by 2002:a05:6a20:72a1:b0:1e1:ca27:89f0 with SMTP id adf61e73a8af0-1eef3daad8bmr44973568637.37.1740631018819; Wed, 26 Feb 2025 20:36:58 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-734a003eb70sm485399b3a.124.2025.02.26.20.36.56 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:36:58 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 05/19] zram: remove second stage of handle allocation Date: Thu, 27 Feb 2025 13:35:23 +0900 Message-ID: <20250227043618.88380-6-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: C667A4000D X-Rspamd-Server: rspam12 X-Stat-Signature: 54h8fytspnd41kmcei4439ub6bjy3ef9 X-HE-Tag: 1740631019-759641 X-HE-Meta: U2FsdGVkX1/0WZcOqZkpeRscgbacla94E7S3OUHgAYgdmIeQuGHy86mRFuz2UlhiEARo/KlsZRgXqNtZC6yLEk+jgpjRMfs2nnYTZ7UvwuYj+Ca/IYk0+GQkb6x/LIE+AaxejO8VCTZdBPft5+5nswS+Yw5u1XQm7rGU4iep9m4C0vSazHoZtiYWiWKYSYDVmq3agFnEa+352YqRR1zYsRTfMTT2biXwwhxfQHW5cjwFZAjaHdSnbNW+zK4ASmszrwfBqxeROsWL8diCQF2GhakZ5+oHwwifM6TeTXSJoCxZkPxbV2D3F4oNMBbxISeZGR54th3PeBM/ypLoFhvCAWJ1LpwvG/MJasS44smwxNqYfjCCQ092c93hOyrMLIUXEs60TB3L+l+Eh1hFfzouog6g/dZPGcuyvPg7BBKw4acrL/7heeHhnHIcpAw3f+gVTcBrwTdhCIrr2/4ocQDBHEL2t1M9vG0/RxccNjxDxFaYxoIfrXKGeUFkxUCK1Ua8vax7nPlJhVjk2OPYnX+l3D9HiHp53Abx6Xpo1cD7d8cA6QufRJELdIqOcE9JSCMA/xIVwNoWXtkUWSs4T71NvmeXelTLgC3/gSD8NavarkhUoHRKQqSXKod8ttUIfDCcG8Aueyzxok6/VrIRuJ01keyw6CilLwtC0GnbcODGsLkmlRc7DtONRIIpAPtp+wwW/ejDQLDZ8UPeMNZ+3v/fqMufcqFJdWfQOibEKUSRNgY0Xb8NW1r8fCCc/4hRYdXtUZdn4QiVHVnPa7ffeVlJ8nVhk1xxhYDE6pI+GOsOAzmWxGyKHC5sXehpTDDOztDEjRM1SerAiaKzI/s06u76Rc1gytMv8Tayt1spmICpI2q6Z5/jg5XuivpEK8jueZ7QLuvk5u1V+hYeUyEVD2VxM2KFw9LE5UozSePjxrSySiBMxYBf8i8hciLmtQcz6UaX3VrPY6QGcrLciWvsH7/ +roNunNx XA6Y01EU6JPWNz7nDcOEdM6Bpwin8RyY5S7LKghiwuRrBW3/ddvi/bAvBt00O0t3NC/THMhov7JC127EGM65k/DtjxZ5EtAnLQabHiBe7bSR2ThFa4UruOagQKXxo8DGtskaPzcJQ0GdHlTApv5W4jjdDsD0QhjmnADyT66vji6fTrK9sc7Jyy0hGF0bsTqm9+JUzAfDIvyT6C0SIZu8CXwZPVUVUASPsGLVnyR0Wb9HS1bBqoqVp3qLShvfxC6ewU+Rlk82R5Khcu9TrVcqa/kvne9YfmoaxSCLUrfcKL9d9sQxt5PkUFvwZ2A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Previously zram write() was atomic which required us to pass __GFP_KSWAPD_RECLAIM to zsmalloc handle allocation on a fast path and attempt a slow path allocation (with recompression) if the fast path failed. Since we are not in atomic context anymore we can permit direct reclaim during handle allocation, and hence can have a single allocation path. There is no slow path anymore so we don't unlock per-CPU stream (and don't lose compressed data) which means that there is no need to do recompression now (which should reduce CPU and battery usage). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 39 +++++++---------------------------- 1 file changed, 7 insertions(+), 32 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 5f2c08337ba9..28aa1c437d83 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1723,11 +1723,11 @@ static int write_incompressible_page(struct zram *zram, struct page *page, static int zram_write_page(struct zram *zram, struct page *page, u32 index) { int ret = 0; - unsigned long handle = -ENOMEM; - unsigned int comp_len = 0; + unsigned long handle; + unsigned int comp_len; void *dst, *mem; struct zcomp_strm *zstrm; - unsigned long element = 0; + unsigned long element; bool same_filled; /* First, free memory allocated to this slot (if any) */ @@ -1741,7 +1741,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (same_filled) return write_same_filled_page(zram, element, index); -compress_again: zstrm = zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); mem = kmap_local_page(page); ret = zcomp_compress(zram->comps[ZRAM_PRIMARY_COMP], zstrm, @@ -1751,7 +1750,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (unlikely(ret)) { zcomp_stream_put(zstrm); pr_err("Compression failed! err=%d\n", ret); - zs_free(zram->mem_pool, handle); return ret; } @@ -1760,35 +1758,12 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return write_incompressible_page(zram, page, index); } - /* - * handle allocation has 2 paths: - * a) fast path is executed with preemption disabled (for - * per-cpu streams) and has __GFP_DIRECT_RECLAIM bit clear, - * since we can't sleep; - * b) slow path enables preemption and attempts to allocate - * the page with __GFP_DIRECT_RECLAIM bit set. we have to - * put per-cpu compression stream and, thus, to re-do - * the compression once handle is allocated. - * - * if we have a 'non-null' handle here then we are coming - * from the slow path and handle has already been allocated. - */ - if (IS_ERR_VALUE(handle)) - handle = zs_malloc(zram->mem_pool, comp_len, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + handle = zs_malloc(zram->mem_pool, comp_len, + GFP_NOIO | __GFP_NOWARN | + __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) { zcomp_stream_put(zstrm); - atomic64_inc(&zram->stats.writestall); - handle = zs_malloc(zram->mem_pool, comp_len, - GFP_NOIO | __GFP_HIGHMEM | - __GFP_MOVABLE); - if (IS_ERR_VALUE(handle)) - return PTR_ERR((void *)handle); - - goto compress_again; + return PTR_ERR((void *)handle); } if (!zram_can_store_page(zram)) {