From patchwork Wed Dec 13 04:18:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengming Zhou X-Patchwork-Id: 13490333 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FF4EC4332F for ; Wed, 13 Dec 2023 04:18:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 38D368D0014; Tue, 12 Dec 2023 23:18:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 13FDE8D0009; Tue, 12 Dec 2023 23:18:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D74688D0014; Tue, 12 Dec 2023 23:18:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B408E8D0009 for ; Tue, 12 Dec 2023 23:18:35 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 89DAA1A0ACC for ; Wed, 13 Dec 2023 04:18:35 +0000 (UTC) X-FDA: 81560488590.27.076E675 Received: from out-181.mta0.migadu.com (out-181.mta0.migadu.com [91.218.175.181]) by imf11.hostedemail.com (Postfix) with ESMTP id 9009940003 for ; Wed, 13 Dec 2023 04:18:33 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=bytedance.com (policy=quarantine); spf=pass (imf11.hostedemail.com: domain of chengming.zhou@linux.dev designates 91.218.175.181 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702441113; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eWTcvUoAbhOuj/P4MajoZT7YM/4SfCwmWZFBpVxrlRw=; b=w/uOBLkTg/wkGAgmUE0TePIpxzVRfzBTSQ2j4ahGCsCBTSJotZh69MSwigJMuBiJhiRGPH 1k0rGdoQyMNUBY0U/u5Z3LewVSKRMNyuqQyY/IR9vZLODXcB9bQbAEEGJMSILqDhDEg0Lh +O2H5tq9g/02brcR3iGvCE9MZQSXhms= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=bytedance.com (policy=quarantine); spf=pass (imf11.hostedemail.com: domain of chengming.zhou@linux.dev designates 91.218.175.181 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702441113; a=rsa-sha256; cv=none; b=dsJShULm03X13OmFlzksdY/HTRPTd6Y5RHUWPknC/SkJ43n6reMU4THFZyOd5JTq8QS/Q1 MRRQx/WHw7u3ODly64urL6lGP07K/sb2OmaG7oJsTxbTJ8FuC09J4OrwjBFfRFsgyYhw1v WfBJsPzy+d2eHixo9d0ZuO/2wcV1VPM= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Chengming Zhou Date: Wed, 13 Dec 2023 04:18:00 +0000 Subject: [PATCH 3/5] mm/zswap: refactor out __zswap_load() MIME-Version: 1.0 Message-Id: <20231213-zswap-dstmem-v1-3-896763369d04@bytedance.com> References: <20231213-zswap-dstmem-v1-0-896763369d04@bytedance.com> In-Reply-To: <20231213-zswap-dstmem-v1-0-896763369d04@bytedance.com> To: Andrew Morton , Nhat Pham , Chris Li , Johannes Weiner , Seth Jennings , Dan Streetman , Vitaly Wool , Yosry Ahmed Cc: Nhat Pham , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chengming Zhou X-Developer-Signature: v=1; a=ed25519-sha256; t=1702441093; l=5311; i=zhouchengming@bytedance.com; s=20231204; h=from:subject:message-id; bh=LVsWNjlGxO7eWZ2spAo/8MIeJF4v6Hli6085n3PpW7s=; b=ijQ9/feCc3OfwRSCbwZHiaWMvVlGwOYR72XX1nEbxryfbyqXRUY9fmMy48+fedmJ3WU1+lzPx h1PB4sHPNCpCvJRMoxnF1fNULuTW9QjfqI+tpyflPewkKZ77sQTkEtj X-Developer-Key: i=zhouchengming@bytedance.com; a=ed25519; pk=xFTmRtMG3vELGJBUiml7OYNdM393WOMv0iWWeQEVVdA= X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 9009940003 X-Stat-Signature: 7ywwccqxjrig5whyj56wd4itddp3abjp X-Rspamd-Pre-Result: action=add header; module=dmarc; Action set by DMARC X-Rspam-User: X-Rspam: Yes X-HE-Tag: 1702441113-864698 X-HE-Meta: U2FsdGVkX1+FXzs3NDha7DGPhoIE+/djIETsrCGfUyGAWZmWrnE5dNMYsGBwxEGxYva0HE9jLZkIs8pLFnFcAZMygfEIjeVnh13fv1CoQF+7sGT9r1W2OwsgMmugl60PYQ+NjP+co+Is8A7RYvIcWGZvGPgA01g/sblNyXEr1lpEImOHqjfHC+uDDSwV9qRp9/ibnh6Fi/ELsAuyvOXegkKKweyL/7PzJsJaoWeH1074vTjL+gQiyCeFKhlUXokis9k7UbYUuN8YjP6n58E8P49WL2jHiapOuuUMDydcDAmFl1G9qaoJAtWX+ii2HPlTg9FCfoBDbC90oLTCQjNY0QsFe5uYnw9m23BJbDMNdXuq03sKcX4QJT5A2YylJmOhhTaKKpX3KdauTk5rviaOvvFzaqVlWOFh5Ut/i3Ll1MvSWouYVmqQDpIMjgMjw8r1gNbDuPQLFEsK982g4l0QKAdwIpZf9JOrBYiB2QLwIH0m1BCO4ZxbhI/xz7TR4BHRYAH4vR/lncWKNW9Fv4SxdzhBBXju3oI+q5FCNg9vYTndXn41n2vu1oKFyjhSvVj7/cC/W345MgDRICKYMT2zpm3U0ndXVzthG1fImz1v5EICOoTmQpsbjx/g3BiAdJycihTUKruc+5llsPDar5I66PW1mDFWJ+NEydtkY+YS2oZU4NVI2umRbOWIVXXwiHYieTsO8uWuESb5GVexmWkWCtz+AZKr2+J9i5nIGeCA6pbzavg4z5EdPD1mVjYPZNfnk3I+NspRv2cur9QpDWqQz8nmbsto0XA8zQTOdGQvi6D+KQc+rYMeys+MLvg1WT34YQRxQd4MajrPV+bFjNB52Db6OwaT3NPnDNxALsbm+0jXUOS8OExarcuDaMq9BHD9FVpYHAtqrtleo0S6lnwZA6hd/m4aQdOc5ulW8iVwcsYUT8NQzgCE2YdCsEXvRvfEt8zq1Tb/UFz8jNGZxN2 ALoPxmKi gNA+ds5wUJreNbqaYkVhPRIO6lQSPXgtlGqAZwf1p7PyPt9e4VZuBZokq1K45D7nDk8sxk/hceG2NPf/QCdZylegJeSXy+SuNNB5hTn6bi3UKPf+nl0qW7rqUQqB9NcE/vZumYkG5mJzzxPv8GLi+ly3Xbio2LnOF3sUxkDhytpcvRwwmrG5X2v/xBe1bSZN2gUgwfTAeKa9TPJDb24vkPQTmuCFQFFc2DXBSqVryD4LsHuM9XLRIQ5GdjD8TFwRQXzNFi4nfAlmi0TyLOruDIVRzpkMitR3E3Z/zcHwZo8oRaA0kAHumryQ0ihPMEi9nHJP4fHPA0eRvssvRKf9F7cYwmUsvOrIX5SwVFOfYMZsUxUF+Ah7abrsfr0eJ/lFdSmthS3axDmobJxA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The zswap_load() and zswap_writeback_entry() have the same part that decompress the data from zswap_entry to page, so refactor out the common part as __zswap_load(entry, page). Signed-off-by: Chengming Zhou Reviewed-by: Nhat Pham Reviewed-by: Yosry Ahmed --- mm/zswap.c | 107 ++++++++++++++++++++++--------------------------------------- 1 file changed, 38 insertions(+), 69 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index fa186945010d..2f095c919a5c 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1392,6 +1392,41 @@ static int zswap_enabled_param_set(const char *val, return ret; } +static void __zswap_load(struct zswap_entry *entry, struct page *page) +{ + struct scatterlist input, output; + unsigned int dlen = PAGE_SIZE; + struct crypto_acomp_ctx *acomp_ctx; + struct zpool *zpool; + u8 *src; + int ret; + + acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); + mutex_lock(acomp_ctx->mutex); + + zpool = zswap_find_zpool(entry); + src = zpool_map_handle(zpool, entry->handle, ZPOOL_MM_RO); + if (!zpool_can_sleep_mapped(zpool)) { + memcpy(acomp_ctx->dstmem, src, entry->length); + src = acomp_ctx->dstmem; + zpool_unmap_handle(zpool, entry->handle); + } + + sg_init_one(&input, src, entry->length); + sg_init_table(&output, 1); + sg_set_page(&output, page, PAGE_SIZE, 0); + acomp_request_set_params(acomp_ctx->req, &input, &output, entry->length, dlen); + ret = crypto_wait_req(crypto_acomp_decompress(acomp_ctx->req), &acomp_ctx->wait); + dlen = acomp_ctx->req->dlen; + mutex_unlock(acomp_ctx->mutex); + + if (zpool_can_sleep_mapped(zpool)) + zpool_unmap_handle(zpool, entry->handle); + + BUG_ON(ret); + BUG_ON(dlen != PAGE_SIZE); +} + /********************************* * writeback code **********************************/ @@ -1413,23 +1448,12 @@ static int zswap_writeback_entry(struct zswap_entry *entry, swp_entry_t swpentry = entry->swpentry; struct page *page; struct mempolicy *mpol; - struct scatterlist input, output; - struct crypto_acomp_ctx *acomp_ctx; - struct zpool *pool = zswap_find_zpool(entry); bool page_was_allocated; - u8 *src, *tmp = NULL; - unsigned int dlen; int ret; struct writeback_control wbc = { .sync_mode = WB_SYNC_NONE, }; - if (!zpool_can_sleep_mapped(pool)) { - tmp = kmalloc(PAGE_SIZE, GFP_KERNEL); - if (!tmp) - return -ENOMEM; - } - /* try to allocate swap cache page */ mpol = get_task_policy(current); page = __read_swap_cache_async(swpentry, GFP_KERNEL, mpol, @@ -1462,33 +1486,7 @@ static int zswap_writeback_entry(struct zswap_entry *entry, } spin_unlock(&tree->lock); - /* decompress */ - acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); - dlen = PAGE_SIZE; - - src = zpool_map_handle(pool, entry->handle, ZPOOL_MM_RO); - if (!zpool_can_sleep_mapped(pool)) { - memcpy(tmp, src, entry->length); - src = tmp; - zpool_unmap_handle(pool, entry->handle); - } - - mutex_lock(acomp_ctx->mutex); - sg_init_one(&input, src, entry->length); - sg_init_table(&output, 1); - sg_set_page(&output, page, PAGE_SIZE, 0); - acomp_request_set_params(acomp_ctx->req, &input, &output, entry->length, dlen); - ret = crypto_wait_req(crypto_acomp_decompress(acomp_ctx->req), &acomp_ctx->wait); - dlen = acomp_ctx->req->dlen; - mutex_unlock(acomp_ctx->mutex); - - if (!zpool_can_sleep_mapped(pool)) - kfree(tmp); - else - zpool_unmap_handle(pool, entry->handle); - - BUG_ON(ret); - BUG_ON(dlen != PAGE_SIZE); + __zswap_load(entry, page); /* page is up to date */ SetPageUptodate(page); @@ -1508,9 +1506,6 @@ static int zswap_writeback_entry(struct zswap_entry *entry, return ret; fail: - if (!zpool_can_sleep_mapped(pool)) - kfree(tmp); - /* * If we get here because the page is already in swapcache, a * load may be happening concurrently. It is safe and okay to @@ -1769,11 +1764,7 @@ bool zswap_load(struct folio *folio) struct page *page = &folio->page; struct zswap_tree *tree = zswap_trees[type]; struct zswap_entry *entry; - struct scatterlist input, output; - struct crypto_acomp_ctx *acomp_ctx; - unsigned int dlen = PAGE_SIZE; - u8 *src, *dst; - struct zpool *zpool; + u8 *dst; bool ret; VM_WARN_ON_ONCE(!folio_test_locked(folio)); @@ -1795,29 +1786,7 @@ bool zswap_load(struct folio *folio) goto stats; } - /* decompress */ - acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); - mutex_lock(acomp_ctx->mutex); - - zpool = zswap_find_zpool(entry); - src = zpool_map_handle(zpool, entry->handle, ZPOOL_MM_RO); - if (!zpool_can_sleep_mapped(zpool)) { - memcpy(acomp_ctx->dstmem, src, entry->length); - src = acomp_ctx->dstmem; - zpool_unmap_handle(zpool, entry->handle); - } - - sg_init_one(&input, src, entry->length); - sg_init_table(&output, 1); - sg_set_page(&output, page, PAGE_SIZE, 0); - acomp_request_set_params(acomp_ctx->req, &input, &output, entry->length, dlen); - if (crypto_wait_req(crypto_acomp_decompress(acomp_ctx->req), &acomp_ctx->wait)) - WARN_ON(1); - mutex_unlock(acomp_ctx->mutex); - - if (zpool_can_sleep_mapped(zpool)) - zpool_unmap_handle(zpool, entry->handle); - + __zswap_load(entry, page); ret = true; stats: count_vm_event(ZSWPIN);