From patchwork Wed Dec 13 04:17:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengming Zhou X-Patchwork-Id: 13490331 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2366FC4167B for ; Wed, 13 Dec 2023 04:18:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B798A6B043D; Tue, 12 Dec 2023 23:18:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B2A3F6B043F; Tue, 12 Dec 2023 23:18:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A18CE6B0440; Tue, 12 Dec 2023 23:18:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 91D466B043D for ; Tue, 12 Dec 2023 23:18:26 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 5168CC0ACD for ; Wed, 13 Dec 2023 04:18:26 +0000 (UTC) X-FDA: 81560488212.12.1389EF2 Received: from out-181.mta0.migadu.com (out-181.mta0.migadu.com [91.218.175.181]) by imf22.hostedemail.com (Postfix) with ESMTP id 53D27C000A for ; Wed, 13 Dec 2023 04:18:24 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=bytedance.com (policy=quarantine); spf=pass (imf22.hostedemail.com: domain of chengming.zhou@linux.dev designates 91.218.175.181 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702441104; a=rsa-sha256; cv=none; b=IJrxh6LYMvtcgVesyh886TVmDKVn4gLLeU4mFPrIpK2VLSZBEliT5KHpkErzgroo7dtxEj 5/nrGa//ewQcGSYQsL0VfZRWsUMlcOtYjDrWuqOQHiETmnulMFwIvBd/0VgrCE5/4BOUjA zpzV8Nv+vvxqUP2jIoFAguzV/c9Ru5U= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=bytedance.com (policy=quarantine); spf=pass (imf22.hostedemail.com: domain of chengming.zhou@linux.dev designates 91.218.175.181 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702441104; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=P5EA/+QLKKJRAwRxL5HzdYaYuPEdViyrUrcmPc+WPWw=; b=Y5Sc9twZk+/85QMqF225S+XZSCCFCSO6NGM6P4GhKbp4DmXhXag9IMFjxLdw2KFgng56W3 /ffS2+RD2Rg+1GcTF2n3De/uOvKCd6JqeU7hB0XRr5HV6T/zX0cYKw/s9UqqXUAS75/lNu rUolkgQVPWBMADp0U9RVY5rqKYklJhI= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Chengming Zhou Date: Wed, 13 Dec 2023 04:17:58 +0000 Subject: [PATCH 1/5] mm/zswap: reuse dstmem when decompress MIME-Version: 1.0 Message-Id: <20231213-zswap-dstmem-v1-1-896763369d04@bytedance.com> References: <20231213-zswap-dstmem-v1-0-896763369d04@bytedance.com> In-Reply-To: <20231213-zswap-dstmem-v1-0-896763369d04@bytedance.com> To: Andrew Morton , Nhat Pham , Chris Li , Johannes Weiner , Seth Jennings , Dan Streetman , Vitaly Wool , Yosry Ahmed Cc: Nhat Pham , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chengming Zhou X-Developer-Signature: v=1; a=ed25519-sha256; t=1702441093; l=2420; i=zhouchengming@bytedance.com; s=20231204; h=from:subject:message-id; bh=4hmeraw6zxa25WbpK1ECDI7EvpvobxeRhEkruBDznLg=; b=mbsWJQiE+cXMa66hJjNiMeb6vb0c2Wd9Q8gByBTeYUg30xknG8Mp4sD3keTJZ+y7+jzNC8l+X ujPVN2QVCJ9BZB6v/vneplbZq2p6q14W2exDJ4xuhzyTJtIF+0z++Bz X-Developer-Key: i=zhouchengming@bytedance.com; a=ed25519; pk=xFTmRtMG3vELGJBUiml7OYNdM393WOMv0iWWeQEVVdA= X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 53D27C000A X-Stat-Signature: ec59gndgpnucwyr4hk41uxh8pa561b58 X-Rspamd-Pre-Result: action=add header; module=dmarc; Action set by DMARC X-Rspam-User: X-Rspam: Yes X-HE-Tag: 1702441104-723952 X-HE-Meta: U2FsdGVkX18h1RNpprfwwNnMIjs5rzE6j0ZyQuZYdOU3FwjoS/KUQ/Izss+XZmxFqTheMtawuy2i59MF2OgWeLvyuCA1RSfFUZYC25xXqY0vGLCpMFKrZPwZ0EIJp4wtPysSACbgYYac7g4ZntfgmP+hJXGPzVFmrR4O+JF6Snwudws+L3imqWwygWEV7zRsFxJdzb9gqENloyUQhde4aWYsoq7XzSjl0U1P2Uf141AMfiyrQsEkak0lbV5nU+QLsM0yW8wIa646MTOElfr03/5B34BddeK0ig/dXY5fZNe6yeb7xJej5xpOZa85QGjMEsg2l1oKtuAC4sW8k+oyBVtx3wHYwkstn+L1933AIbtw/XYSAwB9ZodxVTkebGbfc/BSTTL0ziizhEwTjSbBWmqdrGtyqy3ElRDN7MCng4LvhfNakhcMjmytaJki1xF49NH2L1YgKzBN7W9WRGy9MFXrQ2bZV8MtGEHiiHz0lSumvOKfO4CwEe6Flxw/bAlNnPTMv4CRR4EFuH9jCMb4HuxDuyjqn5Ky/HhoUf9HLwetjU4bycGCra4bcFKiecsWe6lwE29+XczY2XpQt0daWv4+Zmb2QOLaZ+qKj7M2TwaoB3FX5mVpAad/BsisFjEVI8eT0XRGDm6enVzPd8Qeb7Kd5CMInikVhiUEtsHKRBz7CUzkCKTE84R1G4HgoYJPWWqFbFxfoiy5Pa7YCbdQXXOqTB6SGzyLS/m+x/rHOoBoF1L6bDn4sYCKEYTqIGi1SMWlOI2xDR+5uuiCWlrghvUtekb30dGWCLWv5eRrkMqvHiSlX2e6E3odJzYcZP/Ip2EL3T+yZOIY3wfPMIf1nody8zsHQkQODtJi93/rK8NE/KUfkogWB+dLhJLZ6t4pqcDQS+BcQaI+YkXOoUjjHHbxftKFNObcujHwUj8+mTOfmFwQ2B+anJkhMaE2Ak8E8zntztsvpmU5wd+hf4g d7LQdlYq M48gRn78trMIinEVilgWBjgRJzh3qPeI0n6zfSHcwD1na3aXF+SnhSjhUWqXIiaxOksBVG6bU+jTaYHkmuwrRW/P35xq41KG78h5ysR6zJIGm2RNxsCBRTzCGN2zUdN45E0iy9bDz4Shi6xIDJ9jcfx3bj9ElaTc2G9H+ht+cZa3D8M+pumYRASwrfE0a+3nw4BnVTFDSbre0WR0hNYemUVQqB65sAguGX/giNH/quZMny6GYUz/zoVYMjra//t4wHxEakGbSbIf9qWqGsi9yRXWR9NeXSJDzvas7D4GcZ5SAZE6+KtpfEqth9iXOg3KSnl+xE/YKHZtrgfVFgPT38fPpw8SyEJLHwS2J X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In the !zpool_can_sleep_mapped() case such as zsmalloc, we need to first copy the entry->handle memory to a temporary memory, which is allocated using kmalloc. Obviously we can reuse the per-compressor dstmem to avoid allocating every time, since it's percpu-compressor and protected in mutex. Signed-off-by: Chengming Zhou Reviewed-by: Nhat Pham Acked-by: Chris Li --- mm/zswap.c | 29 +++++++++-------------------- 1 file changed, 9 insertions(+), 20 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 7ee54a3d8281..edb8b45ed5a1 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1772,9 +1772,9 @@ bool zswap_load(struct folio *folio) struct zswap_entry *entry; struct scatterlist input, output; struct crypto_acomp_ctx *acomp_ctx; - u8 *src, *dst, *tmp; + unsigned int dlen = PAGE_SIZE; + u8 *src, *dst; struct zpool *zpool; - unsigned int dlen; bool ret; VM_WARN_ON_ONCE(!folio_test_locked(folio)); @@ -1796,27 +1796,18 @@ bool zswap_load(struct folio *folio) goto stats; } - zpool = zswap_find_zpool(entry); - if (!zpool_can_sleep_mapped(zpool)) { - tmp = kmalloc(entry->length, GFP_KERNEL); - if (!tmp) { - ret = false; - goto freeentry; - } - } - /* decompress */ - dlen = PAGE_SIZE; - src = zpool_map_handle(zpool, entry->handle, ZPOOL_MM_RO); + acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); + mutex_lock(acomp_ctx->mutex); + zpool = zswap_find_zpool(entry); + src = zpool_map_handle(zpool, entry->handle, ZPOOL_MM_RO); if (!zpool_can_sleep_mapped(zpool)) { - memcpy(tmp, src, entry->length); - src = tmp; + memcpy(acomp_ctx->dstmem, src, entry->length); + src = acomp_ctx->dstmem; zpool_unmap_handle(zpool, entry->handle); } - acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); - mutex_lock(acomp_ctx->mutex); sg_init_one(&input, src, entry->length); sg_init_table(&output, 1); sg_set_page(&output, page, PAGE_SIZE, 0); @@ -1827,15 +1818,13 @@ bool zswap_load(struct folio *folio) if (zpool_can_sleep_mapped(zpool)) zpool_unmap_handle(zpool, entry->handle); - else - kfree(tmp); ret = true; stats: count_vm_event(ZSWPIN); if (entry->objcg) count_objcg_event(entry->objcg, ZSWPIN); -freeentry: + spin_lock(&tree->lock); if (ret && zswap_exclusive_loads_enabled) { zswap_invalidate_entry(tree, entry);