From patchwork Wed Jul 3 04:05:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721071 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CD28C31D97 for ; Wed, 3 Jul 2024 04:01:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C4CCE6B0082; Wed, 3 Jul 2024 00:01:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B8B266B0088; Wed, 3 Jul 2024 00:01:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 82AAB6B0083; Wed, 3 Jul 2024 00:01:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 5F0896B0082 for ; Wed, 3 Jul 2024 00:01:15 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id F2840160236 for ; Wed, 3 Jul 2024 04:01:14 +0000 (UTC) X-FDA: 82297091268.12.922CA05 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf20.hostedemail.com (Postfix) with ESMTP id 439CD1C000F for ; Wed, 3 Jul 2024 04:01:13 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=NCzHmjUT; spf=pass (imf20.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979249; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MgcEaFuqdLlFITvoex0fb/kXsa9hxfM0Ia4Shb0TEvA=; b=LcbPAEG9eEkq3rCC06gqYP+CpzHciSM38YjrqIVIMAPuamXY+NcWFRUGa7UTpfxvGOnY6t /s+BKxV7XMNkoYqT6H0F8PX3eg3pVqknRn6hK5QUD4jS7O3QMxd/QmYHcPqxemXtCpDRjz 2T7ZgnpzdQfye3zbyHseGaT8OqbT7TQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979249; a=rsa-sha256; cv=none; b=2ZXa/EX5mp41vLdL5WmH2ssTAWabgS+HdJO8kUWdw+OnBB9dPaF/dHmo6Y0y6WbdBWrxVd Y5Wn9CbLx+RU5LxBceNrzqIIH5YLJgCn5lYN92WSbNcWCcsqQ0vj6fIhHzYxXcE01GzfMA OUTJbqIxvyMdodszr9OhqNIDD+vFQfc= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=NCzHmjUT; spf=pass (imf20.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 26E34620D4; Wed, 3 Jul 2024 04:01:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8B957C4AF0E; Wed, 3 Jul 2024 04:01:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979271; bh=81TBE+QYmpKV96rZHKgj6AZ1fYwMgRmX/nAJKqOjEgA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NCzHmjUT1evH4SkSRoxBPGps9eH/wF60nNhj3DQNG4ErQlV8wVwU82WmzAWXzA5y5 tNO3t6uq+kDRXkCvQEm3UgnevwfPAZd+aDrqyXavCl02m57ofEk1Dt1hcPUzJtOjop gd0kTeggPQkI95uQvc6hqhxPWTNb4CN4a9PVkgKsf31qyp1EigX1Qoo7g/J6tCb+wo 8OVHJ4Vl1xXn6zVz8lgE4djN+vj7ZPBLDcQ0yFanitjDZl4YfGv8IK6ywhb2l0jFAU rApsfucEskaLl1ikSSmJ2Luh8OjWtfIst0jlXYgDc+lifp0qZ7yUhImyk/Ye/0C+br CLWA3EZfbooCA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 01/20] mm/zsmalloc: add zpdesc memory descriptor for zswap.zpool Date: Wed, 3 Jul 2024 12:05:51 +0800 Message-ID: <20240703040613.681396-2-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: 3rnqfyhzxq6cr68fmd1cnxzq7yrd7cof X-Rspamd-Queue-Id: 439CD1C000F X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1719979273-297789 X-HE-Meta: U2FsdGVkX19x115CO+KWc+S6cpgTthc1mK0HgJuikBULLi5x4vYmsXy4s7c8fJxCQkT6H8aQ+oW4E4iixcPp4syBDe9VoPyp30mbBE1VZHH7aZ6yglICsHBWBSoQHzmxHywGa+onwV7j99r8OblUQpghdYTp4RcAuMmI4umjH/lIpRAy553iHjIl1x4DRjrHALTxfAV9p4LvGjT82RbQ3Ks8Tu5r+ftrIdm9hSz61ZkPQU3WEbi/Gj0p7D1GIPV07psaIMjiUGFh35pMUzSorVOWn+CextNWvP9ceMNqWJ/S0a9CMJ7psvx/qKgC7uypX1XXseu+iCCpigpWXF/S4jF3reBct/ABI86bO0RfYww2Z9I00SQWhOaj4lT3ThR1XlKf6e4Tl2PjL/RxGpgTjWFoT69KPeFoVVoBn3q7zg2Tme6HiVkxsj5Ba70xFKz1Wz0w606iEZ4NMEmk4MlmGWi1DQrDPNwT6gtD0RhMxzHw3HiBcgNBPXRiVSLFpjZxYyK6xIuVtrgIKF1WxCuDujzjm3jd6bLYSmnRlRuZhhXs4W6einK3yoxPcO+XbQaODjc+jltAHnFa04b08x/1JKHfPqReFmcZg+KBYqmn3wsW3CdYSICrZJrhujy7Ln7E4fFGyPeSiAOutrkYrbriS2xfRiKra1IWf8rugnNdGhMGFGRnP55xjCDvMPoaLuQv1ZDJhOIUdeVtlOwtchhb5Qg1+dBHrpU8zEZEZMVeRb1u+rrUAEG6n/B1NTvAe6cLFOtc9nYCE2Xgbe7Zd2q1Uh134GfVnEfsADjcsb9+8KCGsL/gm4JmNIySvCQePVpudgxTl6ZJXE5VD+tRvjKYk5xleitO+yGr1tFUkRh39PaxbEZ8+ycXwYVFNcD/KyXFB4RGUQGBasED4r3bt6jisYP5h/le6gH/xLofj7oxUYRYzeGyZrh+I46u4wEx/ZarBaJXW/xqQ/S3YcPDP2a eIuPrGJE E3/X7NQ7DS6PVBILovqem5cvtiUfq6kk7Oe51SXgAzbqyphP1u1ZdUv70jOPi6Tm9ly7TWbZmy4ui9AFyvT06OS2VdNhGE7bqD10qbCwj+Dlyfov7VPH1GUWKnBDnS75DRLuopd/JdAdPfaNKZomYyDP9a9yzYhC3gbAfKm0kWaWCSPB0ihRHv+MjANge6nmrY1PyShn59yoCieAectU0yBHlIUIyb3AlBdv3OGNwiS0MQehArysdidiD320fAty5GUGiIEkyAC6iC07TpvL1SCJbr0dOpiJ0LjxavO+4EEKDxaNT8PIRf/jSh70gpRyLrHWtoPNQDn5A0Bc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi The 1st patch introduces new memory decriptor zpdesc and rename zspage.first_page to zspage.first_zpdesc, no functional change. We removed PG_owner_priv_1 since it was moved to zspage after commit a41ec880aa7b ("zsmalloc: move huge compressed obj from page to zspage"). And keep the memcg_data member, since as Yosry pointed out: "When the pages are freed, put_page() -> folio_put() -> __folio_put() will call mem_cgroup_uncharge(). The latter will call folio_memcg() (which reads folio->memcg_data) to figure out if uncharging needs to be done. There are also other similar code paths that will check folio->memcg_data. It is currently expected to be present for all folios. So until we have custom code paths per-folio type for allocation/freeing/etc, we need to keep folio->memcg_data present and properly initialized." Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++ mm/zsmalloc.c | 21 ++++++++--------- 2 files changed, 74 insertions(+), 11 deletions(-) create mode 100644 mm/zpdesc.h diff --git a/mm/zpdesc.h b/mm/zpdesc.h new file mode 100644 index 000000000000..4455a663ee84 --- /dev/null +++ b/mm/zpdesc.h @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* zpdesc.h: zswap.zpool memory descriptor + * + * Written by Alex Shi + * Hyeonggon Yoo <42.hyeyoo@gmail.com> + */ +#ifndef __MM_ZPDESC_H__ +#define __MM_ZPDESC_H__ + +/* + * struct zpdesc - Memory descriptor for zpool memory, now is for zsmalloc + * @flags: Page flags, PG_private: identifies the first component page + * @lru: Indirectly used by page migration + * @mops: Used by page migration + * @next: Next zpdesc in a zspage in zsmalloc zpool + * @handle: For huge zspage in zsmalloc zpool + * @zspage: Pointer to zspage in zsmalloc + * @memcg_data: Memory Control Group data. + * + * This struct overlays struct page for now. Do not modify without a good + * understanding of the issues. + */ +struct zpdesc { + unsigned long flags; + struct list_head lru; + struct movable_operations *mops; + union { + /* Next zpdescs in a zspage in zsmalloc zpool */ + struct zpdesc *next; + /* For huge zspage in zsmalloc zpool */ + unsigned long handle; + }; + struct zspage *zspage; + unsigned long _zp_pad_1; +#ifdef CONFIG_SLAB_OBJ_EXT + unsigned long memcg_data; +#endif +}; +#define ZPDESC_MATCH(pg, zp) \ + static_assert(offsetof(struct page, pg) == offsetof(struct zpdesc, zp)) + +ZPDESC_MATCH(flags, flags); +ZPDESC_MATCH(lru, lru); +ZPDESC_MATCH(mapping, mops); +ZPDESC_MATCH(index, next); +ZPDESC_MATCH(index, handle); +ZPDESC_MATCH(private, zspage); +ZPDESC_MATCH(memcg_data, memcg_data); +#undef ZPDESC_MATCH +static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); + +#define zpdesc_page(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct page *)(zp), \ + struct zpdesc *: (struct page *)(zp))) + +#define zpdesc_folio(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct folio *)(zp), \ + struct zpdesc *: (struct folio *)(zp))) + +#define page_zpdesc(p) (_Generic((p), \ + const struct page *: (const struct zpdesc *)(p), \ + struct page *: (struct zpdesc *)(p))) + +#endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5d6581ab7c07..a532851025f9 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -13,20 +13,18 @@ /* * Following is how we use various fields and flags of underlying - * struct page(s) to form a zspage. + * struct zpdesc(page) to form a zspage. * - * Usage of struct page fields: - * page->private: points to zspage - * page->index: links together all component pages of a zspage + * Usage of struct zpdesc fields: + * zpdesc->zspage: points to zspage + * zpdesc->next: links together all component pages of a zspage * For the huge page, this is always 0, so we use this field * to store handle. * page->page_type: PG_zsmalloc, lower 16 bit locate the first object * offset in a subpage of a zspage * - * Usage of struct page flags: + * Usage of struct zpdesc(page) flags: * PG_private: identifies the first component page - * PG_owner_priv_1: identifies the huge component page - * */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt @@ -64,6 +62,7 @@ #include #include #include +#include "zpdesc.h" #define ZSPAGE_MAGIC 0x58 @@ -253,7 +252,7 @@ struct zspage { }; unsigned int inuse; unsigned int freeobj; - struct page *first_page; + struct zpdesc *first_zpdesc; struct list_head list; /* fullness list */ struct zs_pool *pool; rwlock_t lock; @@ -448,7 +447,7 @@ static inline void mod_zspage_inuse(struct zspage *zspage, int val) static inline struct page *get_first_page(struct zspage *zspage) { - struct page *first_page = zspage->first_page; + struct page *first_page = zpdesc_page(zspage->first_zpdesc); VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); return first_page; @@ -948,7 +947,7 @@ static void create_page_chain(struct size_class *class, struct zspage *zspage, set_page_private(page, (unsigned long)zspage); page->index = 0; if (i == 0) { - zspage->first_page = page; + zspage->first_zpdesc = page_zpdesc(page); SetPagePrivate(page); if (unlikely(class->objs_per_zspage == 1 && class->pages_per_zspage == 1)) @@ -1324,7 +1323,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, link->handle = handle | OBJ_ALLOCATED_TAG; else /* record handle to page->index */ - zspage->first_page->index = handle | OBJ_ALLOCATED_TAG; + zspage->first_zpdesc->handle = handle | OBJ_ALLOCATED_TAG; kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); From patchwork Wed Jul 3 04:05:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721072 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E588AC30653 for ; Wed, 3 Jul 2024 04:01:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6CC786B0088; Wed, 3 Jul 2024 00:01:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 656366B0089; Wed, 3 Jul 2024 00:01:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4CFB96B008A; Wed, 3 Jul 2024 00:01:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 27A126B0088 for ; Wed, 3 Jul 2024 00:01:22 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 898E7120725 for ; Wed, 3 Jul 2024 04:01:21 +0000 (UTC) X-FDA: 82297091562.12.9835862 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf02.hostedemail.com (Postfix) with ESMTP id 51D0E8001C for ; Wed, 3 Jul 2024 04:01:18 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=M3fvr9YU; spf=pass (imf02.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979257; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=55UKQZVXLllTbhZrfzXYatMV5+KqGDMpDwKffSYOCMg=; b=bh7m5xHvXiKSfAU2jMKshnkbvwS7ZKJd2KNUxUkth9+fs89X4z5LyVSYAmE0KGp7b8w88R FR3jP8NUC4hFNcDbt80r6jKp2bvIhyziDGtz3p+sY31gVtXtdyCMJVRRV09DtaXQJUwZMV oqYhUMPgiqo03UkCAiLu1ypp1YzqVV8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979257; a=rsa-sha256; cv=none; b=QE3UnUKmamTN6HzoapWHoB4ZGdgpHTXn19mZU1opPa+++6EzRZ2DXVsBZNiRkT+CPFQpkg +ty/XnVfrs93r5IL8pC8R73ew7n5sfL/hVXqa/zmNvjrHHHgLi4xkqT7nzCzzR/l47UbKA Sj4jZ9UnM5P81EodXkcBhZuUYeIzAn4= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=M3fvr9YU; spf=pass (imf02.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 7B865CE28A0; Wed, 3 Jul 2024 04:01:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 81E71C4AF0C; Wed, 3 Jul 2024 04:01:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979275; bh=mgpKO63FhRCf6bCfWvth1dWCysiouyCKf660VxcU2X8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=M3fvr9YU3RoqDYqi7LepVhhyobK2kETlXMib4NHfNfY6Xg3gkhcZVMOUv3/ph7BuW FYbmXaLXttnr8mKSiGOtqWQCsfx4B9jfmUcCX3oooPG6ckNgLhLvFFhGujDl7UYG/m paouB5VQgKocBJwp7NyPedhmvhxN6Tz9jY3PwV7x56sVrli8XW0YcF4tZh7Tnh1SFl 6NVukRxT9usEljzgLjDhoVPOfEYQIRDXJesuIfvKKRdbh+vB59OBL8c4p2Du1xGNhv XAMnJJPn1Ygq4VTLpVP9TL5nRFRoczdwlPPIdyHOOy1gZx+qIsuZ3u4IVD4BP5/3gv OXB0uZFAiuBsg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 02/20] mm/zsmalloc: use zpdesc in trylock_zspage/lock_zspage Date: Wed, 3 Jul 2024 12:05:52 +0800 Message-ID: <20240703040613.681396-3-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 51D0E8001C X-Stat-Signature: 8ixfsnrggbabtcxbx7qdjuoft77k5pqf X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1719979278-991786 X-HE-Meta: U2FsdGVkX1+/vYPbwwS5t2D0ozYOeJ3De05Ht2TjMCGAgjymW00qBVZDA9F2R2dam1RCC9tWl/UkpXxYli0o8qfMCX31rVl+y5ExwyqkGT7Ha5f61gR2KwRAuIqSLPEsVDWnPJorV64ZrnJ2LVRlPGnCHxE6JwWWqsfPQ6wzjO6pB27EOXL9v17JVi3BBfi8eRGCW84ar4daX0rCRD6VTfGCpOJReHgknL+rcm+SdgNWUCc8UmERBEm+lkEJssSQagwxYoaBu0+CXICVCOoov/8P0zLoALf2wKEsSSiWIIK0PZSNy318BWo5JtQibkgbYhjGq5yNMWhjTaktEstC1yPUz138tgcvfUWeNbaSypPhWTbGrzE2z2Y4t0nm3kiH3zwN19MWbZW2XHnuD9K3wPgarlOqAesj62w/HiA5TcPk+NhQgFzIHWHalNGIQ1wVG4EBTkO0zDT/n3n773gz76GZLP3UUFNLtxiFyX4Fld5HoHjpmAWxDqOgLT0RX4nSgMAH49lVv/OoOjReCuOwFftBu/bw2hPttSya/sMS8U0ENXSFdApezcGbBiejAfTDe486csR4flROi9I1iXGbO6BSsFdrUrOVes7wKn30M8t1xpnXdZDTxupBL4UMna+b04hY8KyOsmiHSfpoX9msbaSe3R1cAooE06IMPp5/+MGvsd0vlwF4Qpj6uDlbyzZrEmwWHGGB3WWllfdnwBqhCZ8CHdjujw/B+WhHvNQjifxl6MqGzBvGy9w6EnxAyroh0RA/abdBLibireLLLQiCBZLNV98kwpA2z7hiwNN6/Fwp8PDEFWn0CcdTlUPMbtpEFxreDv3S0CvDR6jNnQ/oLX3okfHKChCQJLxgtwYk8p/73FMmrgrYgyK3g3V+HOw3Msej+ULZj2W7DTrsmudPVLiSp2xZGSmpVsPfvUMVlVoKzw3VmbTw4TtgwMVeEcFXxBl7hCJZpSG7hVEGKMo eiYzNdkK VaMyoEGmabIz2Gp2B/gF+eltRw+3vTBp4RpR9NJwzj7lCns8EsArhIFlry+MEehLMZKZnstPrJ+XPedle63Awp6SrXsjTE1FN3/mwGj+7oYiMbOqQPEkNXXkkZuz7fq+jh/1wCZEn8S0ENAQOIm2Nmo9TPzSI/Ph1ll2VtWWd5Htv0PuKR7mUF1Kmc9PrYHq+rr6blZbJ45r0SqvcRxPf7lE9yybC+9Z47tZjDwagZz9FdrPYU42Uu6LdY2zRDtOYaCFjVasvQfb3VmFOOpRqZNqa4N1H8pnXlq9/74UgMhiJFsqFRGjztAWnBvIsQ/XW3AUU1LajXt+E2iWLyyyn7JR5djk9ooBADCl+Ur8MP+dOqX+QuimI1VvWoA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi To use zpdesc in trylock_zspage/lock_zspage funcs, we add couple of helpers: zpdesc_lock/zpdesc_unlock/zpdesc_trylock/zpdesc_wait_locked and zpdesc_get/zpdesc_put for this purpose. Here we use the folio series func in guts for 2 reasons, one zswap.zpool only get single page, and use folio could save some compound_head checking; two, folio_put could bypass devmap checking that we don't need. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 30 ++++++++++++++++++++++++ mm/zsmalloc.c | 64 ++++++++++++++++++++++++++++++++++----------------- 2 files changed, 73 insertions(+), 21 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 4455a663ee84..8a359d228507 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -61,4 +61,34 @@ static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); const struct page *: (const struct zpdesc *)(p), \ struct page *: (struct zpdesc *)(p))) +static inline void zpdesc_lock(struct zpdesc *zpdesc) +{ + folio_lock(zpdesc_folio(zpdesc)); +} + +static inline bool zpdesc_trylock(struct zpdesc *zpdesc) +{ + return folio_trylock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_unlock(struct zpdesc *zpdesc) +{ + folio_unlock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_wait_locked(struct zpdesc *zpdesc) +{ + folio_wait_locked(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_get(struct zpdesc *zpdesc) +{ + folio_get(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_put(struct zpdesc *zpdesc) +{ + folio_put(zpdesc_folio(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index a532851025f9..243677a9c6d2 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -433,13 +433,17 @@ static __maybe_unused int is_first_page(struct page *page) return PagePrivate(page); } +static int is_first_zpdesc(struct zpdesc *zpdesc) +{ + return PagePrivate(zpdesc_page(zpdesc)); +} + /* Protected by class->lock */ static inline int get_zspage_inuse(struct zspage *zspage) { return zspage->inuse; } - static inline void mod_zspage_inuse(struct zspage *zspage, int val) { zspage->inuse += val; @@ -453,6 +457,14 @@ static inline struct page *get_first_page(struct zspage *zspage) return first_page; } +static struct zpdesc *get_first_zpdesc(struct zspage *zspage) +{ + struct zpdesc *first_zpdesc = zspage->first_zpdesc; + + VM_BUG_ON_PAGE(!is_first_zpdesc(first_zpdesc), zpdesc_page(first_zpdesc)); + return first_zpdesc; +} + #define FIRST_OBJ_PAGE_TYPE_MASK 0xffff static inline void reset_first_obj_offset(struct page *page) @@ -745,6 +757,16 @@ static struct page *get_next_page(struct page *page) return (struct page *)page->index; } +static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) +{ + struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + + if (unlikely(ZsHugePage(zspage))) + return NULL; + + return zpdesc->next; +} + /** * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value @@ -815,11 +837,11 @@ static void reset_page(struct page *page) static int trylock_zspage(struct zspage *zspage) { - struct page *cursor, *fail; + struct zpdesc *cursor, *fail; - for (cursor = get_first_page(zspage); cursor != NULL; cursor = - get_next_page(cursor)) { - if (!trylock_page(cursor)) { + for (cursor = get_first_zpdesc(zspage); cursor != NULL; cursor = + get_next_zpdesc(cursor)) { + if (!zpdesc_trylock(cursor)) { fail = cursor; goto unlock; } @@ -827,9 +849,9 @@ static int trylock_zspage(struct zspage *zspage) return 1; unlock: - for (cursor = get_first_page(zspage); cursor != fail; cursor = - get_next_page(cursor)) - unlock_page(cursor); + for (cursor = get_first_zpdesc(zspage); cursor != fail; cursor = + get_next_zpdesc(cursor)) + zpdesc_unlock(cursor); return 0; } @@ -1658,7 +1680,7 @@ static int putback_zspage(struct size_class *class, struct zspage *zspage) */ static void lock_zspage(struct zspage *zspage) { - struct page *curr_page, *page; + struct zpdesc *curr_zpdesc, *zpdesc; /* * Pages we haven't locked yet can be migrated off the list while we're @@ -1670,24 +1692,24 @@ static void lock_zspage(struct zspage *zspage) */ while (1) { migrate_read_lock(zspage); - page = get_first_page(zspage); - if (trylock_page(page)) + zpdesc = get_first_zpdesc(zspage); + if (zpdesc_trylock(zpdesc)) break; - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); } - curr_page = page; - while ((page = get_next_page(curr_page))) { - if (trylock_page(page)) { - curr_page = page; + curr_zpdesc = zpdesc; + while ((zpdesc = get_next_zpdesc(curr_zpdesc))) { + if (zpdesc_trylock(zpdesc)) { + curr_zpdesc = zpdesc; } else { - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); migrate_read_lock(zspage); } } From patchwork Wed Jul 3 04:05:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721073 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0C70C2BD09 for ; Wed, 3 Jul 2024 04:01:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6595A6B0089; Wed, 3 Jul 2024 00:01:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5B9BD6B008A; Wed, 3 Jul 2024 00:01:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E5DB6B008C; Wed, 3 Jul 2024 00:01:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 1C2D36B0089 for ; Wed, 3 Jul 2024 00:01:26 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 8D1D4C054F for ; Wed, 3 Jul 2024 04:01:25 +0000 (UTC) X-FDA: 82297091730.27.C3F2CB5 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf21.hostedemail.com (Postfix) with ESMTP id 512401C0013 for ; Wed, 3 Jul 2024 04:01:22 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=TwygWuGt; spf=pass (imf21.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979252; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=61ZUT7gO5pix+Syfo3H6V1v5+PP3SiTNV2KMQCafkgk=; b=2nfn6/RilQiBtf9lvxy4CLHc7Lb6+XbOBHBhqIoFcGGBplkneTO40qS7ucxDFg815nxfjJ /pszpszQRpqBgLq58Fjd2hqUOsz8f70RGwjDTkncFFJCv1/cb75mQs8mZVLaGSYZsABzqF daeiKdAr+ma6wNRfNJ3DUWE6MG5sME4= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=TwygWuGt; spf=pass (imf21.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979252; a=rsa-sha256; cv=none; b=CknPTuGjs3h/j3jdaVpImKjgzhnqQJPq0CpYlZfJ0XahtVMKa7kF53g5hjgBS4fzRN5/uX DwqVrLoq5tya2VYZhm+l932q+rhRaCHmNfsBJDDWoNI+jqn5B2eB2Z1+pt/vNLeAXyHOw4 FGvAc/tUdSWauTqP2g8I0HZotMNwRBc= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 71BF9CE28A2; Wed, 3 Jul 2024 04:01:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7A3EBC4AF0B; Wed, 3 Jul 2024 04:01:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979279; bh=ezRZsoVNxvO9p/Es1oKbiCO8zRXU9uaLAm4pSSvGkg4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TwygWuGtgKvxgwB10GWVsnz/3isUS/MQMC9Xy3wCXH3LbpIlnFGgsLTVo3vhpbFji FiO84usSj+i0Ez8oI8g3tH5PlCJBj++GLnlkjKGdY3AV/8Dp+4wCx9PohogcDNjlzP b4ybo55igdo98aSZFekjv6ZYFaH4LyyZwRORJ+/EQx+nW04btD9BvbObiSkwN2RLn0 3YT1WOS/2mpQWRedGerzZOzYsLU5SXoRzJkQZ3Oqj52OdO3t8zXCsVNdp2h+YlWxZk NZvdf+VxMWPVrpM043z+zyIpla47SpmiLILg43uwi2uQvkL9w9ZffuwQu96pQye/yF uOlDeyIXvkq3Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 03/20] mm/zsmalloc: convert __zs_map_object/__zs_unmap_object to use zpdesc Date: Wed, 3 Jul 2024 12:05:53 +0800 Message-ID: <20240703040613.681396-4-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 512401C0013 X-Stat-Signature: pctakwjqiczigm5xx747iuk8w4z5t7ek X-Rspam-User: X-HE-Tag: 1719979282-293229 X-HE-Meta: U2FsdGVkX1+W3hs2hFua337Qato2wj6Mk22I1EgvHgqlTTGHAk3gJaJbJi2NC+j4bkBaCE4uDDttw21Slnl8rXn7xY5EsIPMn00HdClpAdDHxxSbnkWMTiKp11+cfJLUaZrFLZSplgWQ1P52YjJntBP5zAYgpzOi4DLGfb9V6qtwRUTJE0o9N4tV5FFvUDQCHMoQQelD6uN18RGyMjdBOBNlrVKYegnw1ZhhFZYsa0tKYkDG3zjYyQcSsjXdYpvdC1st3DOZVYHNR0a736go8GXu5dlhY6lRszOEdcUutWtDMP+G1nATGSLP6TE1mKBuTeYlleg42ibjX+u6fKUaDMKyOEcJywBsp0h6ITj7P5QVahaByEjt4kAtCpao4haf9m7/1PL6g4JaQPKtD5taMnRs1l8DJaXCs/d5SpEc3DBExXYwh0eUVhQhXFnQlnW2cE3L/syZ58363LI0RXSw5lMh6M/NxVI+RlE1LBqDqASDTj1d79edTXT2dW+OIdQ57IQ4IGBXr3cD21Rat3fbTCAcl297/p3foIiDnhF8VthLEfo4WZwUTOSjTgw9434qZHeYpZ5+/8fEBwHdOE6x3Oe1QF/DtXtiSh/YexMqzhKOQy1i1Fx6dssy26f/hZk1DwY9yMMrCoa7wSP4YxJJ/3JDUboOc2IoWdhsYWgq12GWm0BBwwxVIuxQUNbWFPswV9Z+o2n/fkP5k8JcN/Ax4u9mGpXOpN4BY88b8INyJ0T4jfo76+mUc4O5U+TB1FtTHdjLTo9jVDC/S6uOhjKNJEs1LxOA18ksnGrv4usBFlfwoAAcXppAhHyNhSz16iSzbh+ZGIPvl4D/B/u4175zA/zLJbZ/XXcBpVHI85mqVSTHNMpJRfkFLomVZyJCMK0ifhTFoEf0kZ5ec4xi0qujYruqEwsD8KlXkHzQ8NC9m5r69dQjROVtly1khCxVr9RJFwXRRcVHSARa67ISlmd SVAw9OHc 23GJR3xyKANkDY0kR7w6cTpp5Yd5lsM172NyeGMwT6T4s+p80OSf3eujgpIEAd15w+5jjQ0xN3UgTfL0wisyQL4I96vdQFKoVUi7E1kkB9zSdOkNLjK/0ydlULpU0gIFZQbcbrEIz3jVJoQ9psK8kDVPZMak1o2MuRI9DxOt95lXvlVdDpI9DrFo19THSrBjR4w4dqLsv/KTfReC9tX8yqm49NRnEOXZHEpCoQKS951TYfSqMtUoYq56LnvFU9axXaxTdURnLkgOk67+toe2CCrFzEC4nbzv9SvwrQ623rrp5mP4iQK1mgfISiGzdv84Ny4GaodcaNyMIdX58ec4TO/XwQX3ILofPecNZzUdUwcmBUlwflmzy+lM8JQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> These two functions take pointer to an array of struct page. Introduce zpdesc_kmap_atomic() and make __zs_{map,unmap}_object() take pointer to an array of zpdesc instead of page. Add silly type casting when calling them. Casting will be removed late. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 243677a9c6d2..68a39c233d34 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -243,6 +243,11 @@ struct zs_pool { atomic_t compaction_in_progress; }; +static inline void *zpdesc_kmap_atomic(struct zpdesc *zpdesc) +{ + return kmap_atomic(zpdesc_page(zpdesc)); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -1061,7 +1066,7 @@ static inline void __zs_cpu_down(struct mapping_area *area) } static void *__zs_map_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1078,10 +1083,10 @@ static void *__zs_map_object(struct mapping_area *area, sizes[1] = size - sizes[0]; /* copy object to per-cpu buffer */ - addr = kmap_atomic(pages[0]); + addr = zpdesc_kmap_atomic(zpdescs[0]); memcpy(buf, addr + off, sizes[0]); kunmap_atomic(addr); - addr = kmap_atomic(pages[1]); + addr = zpdesc_kmap_atomic(zpdescs[1]); memcpy(buf + sizes[0], addr, sizes[1]); kunmap_atomic(addr); out: @@ -1089,7 +1094,7 @@ static void *__zs_map_object(struct mapping_area *area, } static void __zs_unmap_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1108,10 +1113,10 @@ static void __zs_unmap_object(struct mapping_area *area, sizes[1] = size - sizes[0]; /* copy per-cpu buffer to object */ - addr = kmap_atomic(pages[0]); + addr = zpdesc_kmap_atomic(zpdescs[0]); memcpy(addr + off, buf, sizes[0]); kunmap_atomic(addr); - addr = kmap_atomic(pages[1]); + addr = zpdesc_kmap_atomic(zpdescs[1]); memcpy(addr, buf + sizes[0], sizes[1]); kunmap_atomic(addr); @@ -1252,7 +1257,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, pages[1] = get_next_page(page); BUG_ON(!pages[1]); - ret = __zs_map_object(area, pages, off, class->size); + ret = __zs_map_object(area, (struct zpdesc **)pages, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret += ZS_HANDLE_SIZE; @@ -1287,7 +1292,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) pages[1] = get_next_page(page); BUG_ON(!pages[1]); - __zs_unmap_object(area, pages, off, class->size); + __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); } local_unlock(&zs_map_area.lock); From patchwork Wed Jul 3 04:05:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721074 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8259FC31D97 for ; Wed, 3 Jul 2024 04:01:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0B5226B0092; Wed, 3 Jul 2024 00:01:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 03C046B0095; Wed, 3 Jul 2024 00:01:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD0066B0093; Wed, 3 Jul 2024 00:01:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B61DB6B008A for ; Wed, 3 Jul 2024 00:01:29 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 71581A072E for ; Wed, 3 Jul 2024 04:01:29 +0000 (UTC) X-FDA: 82297091898.22.F74ADF7 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf07.hostedemail.com (Postfix) with ESMTP id 3AEF740014 for ; Wed, 3 Jul 2024 04:01:26 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=QNbHJxK4; spf=pass (imf07.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979256; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dc9AQUlRtwCbY9FgFRAy2KqIgbNBk/ZlKyjpYriuOkA=; b=7FwKxvA3OODQ1jP8buxOmKMpITwpSocdI2tJL1r7sPNy1ROWYgeIGVE731lvAXetw+Ttl0 7zMdY9SsRZA9cePs2gwQtbxDsMM1Bm3pT4Ytbzcj+eqbll1Ef8vyWr3OaJPH3FYlW04tJs EUv0WT/9iG9ZR6JIr4oNCsU8yNyMR8U= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=QNbHJxK4; spf=pass (imf07.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979256; a=rsa-sha256; cv=none; b=ls2SsslTWYg/8yV2lVfQApwYqXkmrtnEiKExAqpJv7nDUdFNZygTKVTTgkvsD3K4NXPhE/ 7eRndbnNUVxDCWIOCtegZ5W82VwN7IbiMJS0qy7kg9qmb4wCbxSQnR8PemH9yQttVIA72l l8Bk2UKyqmHyLCIA1pfrkR0h6KJmH4U= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 6B3D4CE0E1E; Wed, 3 Jul 2024 04:01:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7079CC4AF0A; Wed, 3 Jul 2024 04:01:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979283; bh=1lHtFt8ZtGmaai/qGPIEUZAmHxJr0sOHHO8xZgO8IhM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QNbHJxK4D6ImP2YzB0FOoQm3eGnBS+mZMAHPctPm0Ee21F3ECLriusqxrK3WCcuBx j1uedLH/jQcFqFam1Pi73yNj+PUAvGE5uysX/xNPmPiDCguFqG4F7NL8MXbZMhehtH ujUyVDOrH7NOM/L8nNAhdmsn4JET++rVfKxdTdPe0uwEQewgTlPAJIgdESwevnP6iC nAukgNgU+uUI//YtUBEgMtJH4A2ICHYlwmj2s2uRNYdAJyBl1BfYuU/6imKmtohjpR w4zdLzppZBMQCbgWGjAKg+VLvML+I0IpPi7TS9qicMsbP+vF6q4waqCBsyR5CfSrtO xtz9bmUDuPS2A== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 04/20] mm/zsmalloc: add and use pfn/zpdesc seeking funcs Date: Wed, 3 Jul 2024 12:05:54 +0800 Message-ID: <20240703040613.681396-5-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 3AEF740014 X-Stat-Signature: 8g4btc1djiwtyae6ii5q1ct5sjzs7ft7 X-Rspam-User: X-HE-Tag: 1719979286-142155 X-HE-Meta: U2FsdGVkX18JPNO2TYSQOq2bQJpIEEycqciVrqIIoCwCto23E+Is/dkIzFC8FWadoEc5jaMfxHztrkzZOutszGGSyv77aFmwM1ffquOpdUByU5YTpBfyjnz6QWEcQSep6C52CFF2eYx92Q2KiXPZbcf8aUDhHRTmVxhXcCSdCOdjElrzid0UVisK5d/k7mGVueJlJO5/D3s3Sx/CGP8pJT4lqD0Xh9KMzcCBevpdZEM4MUv04YhzLCXfRSrPrzg+dlgFrvUeEmgiBUfGMokJrpOz1yT8MHmOmFB0mUjkbUjO1RvWQ2s4JrcBve1rEk2LpGQl7my0UclYWI8f60DVs//ag20yejRV0jKovZ2O7y2F67ZnNUfPnc3HdIFZFgnPe63N7dLiK/MmaHOSt2hk5OeN/omt2N1nlPnpC2LV1X9NulHnamnuSSdMCu8Nwi1ypat2bdPmvZQtPNcP8TknzhqBOT2H7DfhIQFIQcrp2H2w15ROepMKd9+QkZ2Av3yvG+vkAMzhGP9OZWDgSttoINT+og63Qe8aXvHgfSdk45Jbd9VdFFu0VbYKLf99KFzWQog9cGL4FU1VIjaDQeqX3HPvLp6CmUwo4zz47RSl/jgLzOhQoZiVcoYbe+rGYJkAu2o1yqANaW2dWIP5PUIjAlevRvScc0j1kQMJcj/pIi1DxhYTT1YVUxLWNHYQY2h7cChW2iS/DnyLYFm3XceQBoweNkhpnkaUTpoyL1XBBTlpSmljeEI3YUxr5jZN9UVn1akRoGyVflrfdpmhJE9OK8CQm2QB+rrRjWQhby3KGIUgPVH9bGq3rdow4OkfA+Amu5XDuaMCN2RwA40WIUQdt/BYf4JPU1+h5rJv7LPQSLO2lW5qq7Yy9bm7laLGLicvqGhNoEIxTsrcddt4gNiTxaAGMslORpsrd3o+QyetlZXE5BbECNJ8RQJXT+SnyGP35d3aNEgQKv80E586rR4 BNtqBPmi TiNx2cFvkeinMRA7/HNgHR542HxNRmo+ED8bu7JnVGVHW/ja71ETYFz5529nzN8QJESNkKGWTcKLI7GdEP3M6iFlbHgp0FJgbbqtykZZT5rfiv+SFGBAOL9VnGlKSomuye1yJI652smTy7MZcijPNrVDJ8aTtUOcgUcZj7uL/XU8ubWT8uEjhBd9WObYR6sGK+yU3S6qwYPm7JHAoKoLMtt8WU9rNR+HE9hEUIMlAUb5wxrFI+DqKdvhlqNVtENzYgSqlPdX3YOCwgdgvOmzL+lQzXrTA/j8bs1kMd9eucIBacWx0gX2MYXh7fTGLdCaN+53cAOZClWylkjk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Add pfn_zpdesc conversion, convert obj_to_location() to take zpdesc and also convert its users to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 9 +++++++ mm/zsmalloc.c | 75 ++++++++++++++++++++++++++------------------------- 2 files changed, 47 insertions(+), 37 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 8a359d228507..96b9ce80ea49 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -91,4 +91,13 @@ static inline void zpdesc_put(struct zpdesc *zpdesc) folio_put(zpdesc_folio(zpdesc)); } +static inline unsigned long zpdesc_pfn(struct zpdesc *zpdesc) +{ + return page_to_pfn(zpdesc_page(zpdesc)); +} + +static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) +{ + return page_zpdesc(pfn_to_page(pfn)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 68a39c233d34..149fe2b332cb 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -773,15 +773,15 @@ static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) } /** - * obj_to_location - get (, ) from encoded object value + * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value - * @page: page object resides in zspage + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static void obj_to_location(unsigned long obj, struct page **page, +static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, unsigned int *obj_idx) { - *page = pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc = pfn_zpdesc(obj >> OBJ_INDEX_BITS); *obj_idx = (obj & OBJ_INDEX_MASK); } @@ -1208,13 +1208,13 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, enum zs_mapmode mm) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; struct size_class *class; struct mapping_area *area; - struct page *pages[2]; + struct zpdesc *zpdescs[2]; void *ret; /* @@ -1227,8 +1227,8 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, /* It guarantees it can get zspage from handle safely */ read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage = get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc_page(zpdesc)); /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1247,17 +1247,17 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, area->vm_mm = mm; if (off + class->size <= PAGE_SIZE) { /* this object is contained entirely within a page */ - area->vm_addr = kmap_atomic(page); + area->vm_addr = zpdesc_kmap_atomic(zpdesc); ret = area->vm_addr + off; goto out; } /* this object spans two pages */ - pages[0] = page; - pages[1] = get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] = zpdesc; + zpdescs[1] = get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); - ret = __zs_map_object(area, (struct zpdesc **)pages, off, class->size); + ret = __zs_map_object(area, zpdescs, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret += ZS_HANDLE_SIZE; @@ -1269,7 +1269,7 @@ EXPORT_SYMBOL_GPL(zs_map_object); void zs_unmap_object(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; @@ -1277,8 +1277,8 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) struct mapping_area *area; obj = handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage = get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc_page(zpdesc)); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1286,13 +1286,13 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) if (off + class->size <= PAGE_SIZE) kunmap_atomic(area->vm_addr); else { - struct page *pages[2]; + struct zpdesc *zpdescs[2]; - pages[0] = page; - pages[1] = get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] = zpdesc; + zpdescs[1] = get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); - __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); + __zs_unmap_object(area, zpdescs, off, class->size); } local_unlock(&zs_map_area.lock); @@ -1434,23 +1434,24 @@ static void obj_free(int class_size, unsigned long obj) { struct link_free *link; struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long f_offset; unsigned int f_objidx; void *vaddr; - obj_to_location(obj, &f_page, &f_objidx); + + obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); - zspage = get_zspage(f_page); + zspage = get_zspage(zpdesc_page(f_zpdesc)); - vaddr = kmap_atomic(f_page); + vaddr = zpdesc_kmap_atomic(f_zpdesc); link = (struct link_free *)(vaddr + f_offset); /* Insert this object in containing zspage's freelist */ if (likely(!ZsHugePage(zspage))) link->next = get_freeobj(zspage) << OBJ_TAG_BITS; else - f_page->index = 0; + f_zpdesc->next = NULL; set_freeobj(zspage, f_objidx); kunmap_atomic(vaddr); @@ -1495,7 +1496,7 @@ EXPORT_SYMBOL_GPL(zs_free); static void zs_object_copy(struct size_class *class, unsigned long dst, unsigned long src) { - struct page *s_page, *d_page; + struct zpdesc *s_zpdesc, *d_zpdesc; unsigned int s_objidx, d_objidx; unsigned long s_off, d_off; void *s_addr, *d_addr; @@ -1504,8 +1505,8 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, s_size = d_size = class->size; - obj_to_location(src, &s_page, &s_objidx); - obj_to_location(dst, &d_page, &d_objidx); + obj_to_location(src, &s_zpdesc, &s_objidx); + obj_to_location(dst, &d_zpdesc, &d_objidx); s_off = offset_in_page(class->size * s_objidx); d_off = offset_in_page(class->size * d_objidx); @@ -1516,8 +1517,8 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, if (d_off + class->size > PAGE_SIZE) d_size = PAGE_SIZE - d_off; - s_addr = kmap_atomic(s_page); - d_addr = kmap_atomic(d_page); + s_addr = zpdesc_kmap_atomic(s_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); while (1) { size = min(s_size, d_size); @@ -1542,17 +1543,17 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, if (s_off >= PAGE_SIZE) { kunmap_atomic(d_addr); kunmap_atomic(s_addr); - s_page = get_next_page(s_page); - s_addr = kmap_atomic(s_page); - d_addr = kmap_atomic(d_page); + s_zpdesc = get_next_zpdesc(s_zpdesc); + s_addr = zpdesc_kmap_atomic(s_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); s_size = class->size - written; s_off = 0; } if (d_off >= PAGE_SIZE) { kunmap_atomic(d_addr); - d_page = get_next_page(d_page); - d_addr = kmap_atomic(d_page); + d_zpdesc = get_next_zpdesc(d_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); d_size = class->size - written; d_off = 0; } @@ -1791,7 +1792,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct zs_pool *pool; struct size_class *class; struct zspage *zspage; - struct page *dummy; + struct zpdesc *dummy; void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; From patchwork Wed Jul 3 04:05:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA550C2BD09 for ; Wed, 3 Jul 2024 04:01:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 212BF6B0093; Wed, 3 Jul 2024 00:01:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 177FB6B0095; Wed, 3 Jul 2024 00:01:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6B866B0096; Wed, 3 Jul 2024 00:01:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id BD3E66B0093 for ; Wed, 3 Jul 2024 00:01:30 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 7599CA06EE for ; Wed, 3 Jul 2024 04:01:30 +0000 (UTC) X-FDA: 82297091940.08.00AFA6C Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf30.hostedemail.com (Postfix) with ESMTP id D4DD680028 for ; Wed, 3 Jul 2024 04:01:28 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Ygl1r0eG; spf=pass (imf30.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979270; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VMCQhHDdGJOKLu/KVq7ZZe30LH202MU5dyPStXWIX7Y=; b=xpcDja9NnVUrb+WrO4mTEasRSOMPbGK2UNJ/8Yr0GkUJFdk9yRrjxtHTalefzdG1ksqH4d pxj/+QBiqH+Zq9E51UM0jVw1axU1Ae6Vq5Z6rbh3/p5KxzlZZahYM5AVoXi/ENUy+80am7 9oTKgRzzwTyIgdI83L5ygarvEYZadFk= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Ygl1r0eG; spf=pass (imf30.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979270; a=rsa-sha256; cv=none; b=X8A7af8hwjlrjBihL19fT+Q005vxRjZQRfJTtqXN4lKxYP5G3k8PGjQ3kiS5zEgc5sCc/G kqendtytvldBJ9EdKg2gfyNHLX5vGJyZJt0KSfkMmMiyQNUpY+dO4FZV7T0vGl2i26qDlq 337brRpo/3ryA4oURjyUeCpkZctCZ4A= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id CFE23620A2; Wed, 3 Jul 2024 04:01:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 69877C4AF0E; Wed, 3 Jul 2024 04:01:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979287; bh=OUw0L1+EEkxStexhrY6FwqktPPZ5HwiknqjqfzU6PXk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ygl1r0eGWf9A8ASCCyxxwaRcR/wNniNavWp/6Cp1Fb7i1Ie3OgUnLu+kb/x83qU4R Rtf1OTjVxaWrDtnC9EgWP60cL+fwY6UmOlhSy0g4md3TDCs0I58Xpeo2fYMQQIC7oa mWGA7aDYGYwcXPiw7U3+CqO0HQFqXZbuFcR1HrgtLk0mCgqKSqgQeVqgpEC7iJDFyx Q4W+t234bRMiEvLqUICPWSBBwtKPv/pZtcYWjGAGrHvheKJd7nLq5kdFh/6V+jbw9b EBrXUKpPCHP3lHr26GXnoTiIoogZTVS7XgIzHuJjYkhr/Qv66r8sBvtyhi/jVwGPTl 5r1Wts9gy5n2w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 05/20] mm/zsmalloc: convert obj_malloc() to use zpdesc Date: Wed, 3 Jul 2024 12:05:55 +0800 Message-ID: <20240703040613.681396-6-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: cyiyq83umbfqwr9a19rcjskscc9o9djk X-Rspam-User: X-Rspamd-Queue-Id: D4DD680028 X-Rspamd-Server: rspam02 X-HE-Tag: 1719979288-387117 X-HE-Meta: U2FsdGVkX19DOaoVaNwvRGEW4CiQWa0OANqq83OpFmIQ+1DsQvcvAzpQOVsParJXkljqeCwPdj8jvmvgvy0infRvCZn7UV+1JD2raZlHWeBB1d6Mp1R7wdQeuPjaBJtn2wPnAf0qEAq58yn0WAEgoNBTa48S1raugjJu1pDCoGEbn1rOyZtLlnz+eXqMl+P5PE3oGpWu+Le96sG4qOnLd+9tYviXZ7qVRFVMyCCg98lXa/nk3qrt6p+zyb/pDTus0aHIkN3ks+4NZIRXmuDBNrX8XyJyHnDcueg0ofwqLlOC0jhzL3TqApWaTFQEbDRigs5Ai4piqim+otywwOEEuqvn4zCLkz8hFBwOTVd1/GmrxahdI3/+nSod9iIIomy381aLAI3DiLAWwUf/9i/YsRhw4sM25fm2ITh8dMF+EKM5WneLYosbSN7P4f5TPdXm3nDV4rHHtwTYjU8qB+qaKKc4y4Kre6tzKQu/Mvo/KPAIQgkSspwrJgJ2aXag51kShz2kFMsF7NYGEQU8TFYSR9aI/luOhrkHlM7wH5f99Ux408rlzpcwTUXQZO6YgNfTBIhtIjd7wK7bscKkUnEWUmwPwA7Cj/5R5Pf+2Dp99Q6TGm6oNVMqoFeeVxuctV3dUOmrVhSL95bA3TfSSeppyumlO362MJgIb0gxv9s+Q6yUK6PqgzTTXprHUjfqjVfbu5P7TlMACO7C/L6pX1EqMCL/j1PeDtcY7N0V+sj5kaaF43Mpw7PIlK/lTayVHO1S4HQiUyVdqs4Ah5SamYJV8NovIWSSX/SKnGs7AsXBhr7GqetpcX4wOuZVDflfiGPYwmSsmxaGvDx0ogo15aSQn6ID95H2/CJwdLML+qG1c4alt+TIvzHoT8IiQHIJzysPae4ruAh+XETDRVPhbFK1gq6E+Lizw11c7Sk7tdrxjz/BURu13yq9fH1b6imdWI1uuRLw8Ae2tKN4DQOP2g9 8zibMgWL OzCr9CIBM2TtjJKR9+m2CyyFkeOqsV1Fjk20gqLwU9U2I2LNlIRooFqq9dPFAkPKzCUoVY9fff/YtDWfA3o3Rn35lUEm7qS867OO/6x09ys1p+vz2X7MJSlM9J104liOtYMmHeE98nkKm++hcihKA0UXbo2KCxphvBiq+SN8KM9ItPqKQ8AWBdny2dA7/7J1A84qEvHphvaJ8iP0/QJ4DopuccdZ4z/zrq4o4J++XaQvGQdNMwjpCWsAVzxsBorocrsdjiji96wawn9cwIfSz6S+wU7EaOhqHIxriykgu+SYopZdTm+u6uf91E+RUvEiPgsbjrJVnDzvkxO19Oc9llJOMIV9fd+vlDIl5W2/7tXooYFAXx4h5gdK1VDmUbPOxl5Vr X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first_page/get_next_page. no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 149fe2b332cb..bbc165cb587d 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1322,12 +1322,12 @@ EXPORT_SYMBOL_GPL(zs_huge_class_size); static unsigned long obj_malloc(struct zs_pool *pool, struct zspage *zspage, unsigned long handle) { - int i, nr_page, offset; + int i, nr_zpdesc, offset; unsigned long obj; struct link_free *link; struct size_class *class; - struct page *m_page; + struct zpdesc *m_zpdesc; unsigned long m_offset; void *vaddr; @@ -1335,14 +1335,14 @@ static unsigned long obj_malloc(struct zs_pool *pool, obj = get_freeobj(zspage); offset = obj * class->size; - nr_page = offset >> PAGE_SHIFT; + nr_zpdesc = offset >> PAGE_SHIFT; m_offset = offset_in_page(offset); - m_page = get_first_page(zspage); + m_zpdesc = get_first_zpdesc(zspage); - for (i = 0; i < nr_page; i++) - m_page = get_next_page(m_page); + for (i = 0; i < nr_zpdesc; i++) + m_zpdesc = get_next_zpdesc(m_zpdesc); - vaddr = kmap_atomic(m_page); + vaddr = zpdesc_kmap_atomic(m_zpdesc); link = (struct link_free *)vaddr + m_offset / sizeof(*link); set_freeobj(zspage, link->next >> OBJ_TAG_BITS); if (likely(!ZsHugePage(zspage))) @@ -1355,7 +1355,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); - obj = location_to_obj(m_page, obj); + obj = location_to_obj(zpdesc_page(m_zpdesc), obj); record_obj(handle, obj); return obj; From patchwork Wed Jul 3 04:05:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721076 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD63EC2BD09 for ; Wed, 3 Jul 2024 04:01:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E20976B0096; Wed, 3 Jul 2024 00:01:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D83736B0098; Wed, 3 Jul 2024 00:01:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B60EC6B0099; Wed, 3 Jul 2024 00:01:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 8F8036B0096 for ; Wed, 3 Jul 2024 00:01:34 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 52FF0120721 for ; Wed, 3 Jul 2024 04:01:34 +0000 (UTC) X-FDA: 82297092108.08.15C958C Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf20.hostedemail.com (Postfix) with ESMTP id B1C7D1C000F for ; Wed, 3 Jul 2024 04:01:32 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=bQm+Zd9Y; spf=pass (imf20.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979280; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eXV+ekf1LWeDPFA3tZsyPnYFsQnE5JjnSGGA6NuHJJI=; b=b7KR0vGnO3NcTRZQ7UM6ZBaOhIxs9hM+yJMdDta6DN7oObr2CadH3qpnvi2SxW5cTu2mA9 EG+hyN6u25ruGHy/Cx0H2KOkbxs4TtJGzsnetsxUZBYIdn2667d51Lu6zVxe/nT8znwPEq GrzrK1+sGzVsjxkdCgCrYwRRZQLFgC8= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=bQm+Zd9Y; spf=pass (imf20.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979280; a=rsa-sha256; cv=none; b=52ae4IZfrdMtCecVvSRcQ3xFC6qiDeYKc+NyM6WPqiHXdr+jCuAj5ljBW2C690WK9g3Vjf aPPCuI6Y++MY1f2I8pAGB4jz3wD4bpWtDDbXYSX/FMhD+SG458I2vmdqOgznFbnmWwfKBL XIaUCpQN5vkV6om6bagO615nr18NREY= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id C579C620BD; Wed, 3 Jul 2024 04:01:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 36A9CC32781; Wed, 3 Jul 2024 04:01:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979291; bh=m9lqRC5WWSzScStlN5X+B3diMObK1z5QGj5Hed/vEuU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bQm+Zd9YsHA7+waQJ9X7qV3qEMjqCA8PuU6zGGx7EbaQLrn0Z+sWBiSbppZR2r5bD LZDiDUol0LFVFAj9f2gYKmMmmtQRa2wpvDhYnj8wo/VVVTm2pDoxJqY5AXqvRybYfp /kEvQmIraoSve7GzxVmDHK86iW15DsKP6dxmqedX/nC1gyopT0TfYbPlSnVRpLij98 zK7Vk450yQjq6BWnE+jvYCgwVw2NEV5p927PMkRf7B6bi+F3gXNc8S8tuywT89ogWu 8D3yW0XPH1bmSAbY+NsLVDPdVHF9X2oldOzo+EYZKFWL37gWVhFwFhsZYgwT5wkgvX aH2nAR6XEr3oA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 06/20] mm/zsmalloc: convert create_page_chain() and its users to use zpdesc Date: Wed, 3 Jul 2024 12:05:56 +0800 Message-ID: <20240703040613.681396-7-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: B1C7D1C000F X-Stat-Signature: pxet5scnwbpbrb4of5dcn6y8qjsarx5j X-Rspam-User: X-HE-Tag: 1719979292-881847 X-HE-Meta: U2FsdGVkX1+mtNgFV5PhSuov/gBJKfYBC/5C8/YoovkfAJiQINLg4Hihjk8u50vt4q2MO+6BaNeTsdmpd4al7psDnZXOBYsB9RvbW6EcnOSHG/TlF/uwQx746yuNbZXMpOa6wjFf7/cRFae4lnyZuFDYwMA2crwIv1qAzeu/Lj98JPrnnwrUm0qfra5wVVLL9yN9tdtPPgEsqUqywndyzqsDMWP0s9xlHZPNymMC4PtdxEUbBV2XISViWTph4b8sRM3r+bRNxoShbZoOVlktaiHa49kQ0uv0HoRUDlzwJo5l5jWwKYLEnOYdPfGVaDYkpPyDJiab9He6rJ0yD9kYhqzVDjSCQ2lPWIzZcTmn19k9kln7ezVpbyyB3LMeIlZpizpaY3x1sJTyFUPYJ7oNdJFI6/P+CekHdNrivEH+YJol8mhJZVUgdR641d07IJ0kd2cTUQnpyOtQTxIYKusD1Uf3ztUKbF1i58uQRPcx6qzyMTIxWLS1A84R2hx7zkEiyUxLZURoUlXdrlxj8Rh7FJhMlOZITOQcxefckWEBHOhJ1tyaoOTQf7G2qhBxIM6ygMCfC4W3F0/fTBywmB88uuckvmWwhpuGjVOqLjN+7GtD7nyydKJuBhaodNXO51rtaKdNhL0cXaBGy59eVwlRDabZDejn6u96yBDlmicPenAaQbFy4c0qp+afZYdQ+0aD8En0sHpGGb+gKIRffMUEUKySao93Dc8Vw8ARodhLqiYLahShkQCm9l5laeFsg8fbZMEazrArmTFFDCKnOMs+un523ZH28dxjVkUEwpBPDPcuLl4HHTj+1cJQkh/n454jWF8lcgb2cpiyzkCI+VdxCrzSDjtzfIa5bIFF+MddPiJuVRjaDA2c5vNtJ89Aaie7ZicjUiWb/62K3Ap8k1n9tEf/FIy2h8DROPr8Utt/+XjMhgmrXvbKYwth8wB8VpeT3JGOyl1JoFkVLKsjfbs Fc3MptXF hYlGNp+8sCIf34I7oYvUdRBnByONHT2dx6pKJihuZJcAJwo6CAiJRscHg376DfYBOv9Ml+hN4JD/xhnPnclN1cdr5S8D7sgPtreP4l7ZrqDhEBSq4ip1NUbsyaJP9ctegwAdszRRB9B7pWDhbE2UZj/8IJLqMrx1gKM4kymxqzhEYYAZ7WKq+yD04OsTPB/GUPCrXF9zL4siOv2lUp5eN54Qt/PVlWlBcFk39bVHAwbg1QN1AZJLSjyIVeQQuK0V0fLLKDuqSNdq09L0mqW7uF/cI/zkcKR8UWZR21jckjKDf+fPe+ZfQ80GftR7307dm+c4U59A9PAidVRdkT8z56KJQX2VhjoHWwXx0gGHb6sMEHIzV8ZKPKWmcfrFfdOEsqDtz X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Introduce a few helper functions for conversion to convert create_page_chain() to use zpdesc, then use zpdesc in replace_sub_page() too. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 6 +++ mm/zsmalloc.c | 115 +++++++++++++++++++++++++++++++++----------------- 2 files changed, 82 insertions(+), 39 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 96b9ce80ea49..a2e320f24dd4 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -100,4 +100,10 @@ static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) { return page_zpdesc(pfn_to_page(pfn)); } + +static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, + const struct movable_operations *mops) +{ + __SetPageMovable(zpdesc_page(zpdesc), mops); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index bbc165cb587d..a8f390beeab8 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -248,6 +248,41 @@ static inline void *zpdesc_kmap_atomic(struct zpdesc *zpdesc) return kmap_atomic(zpdesc_page(zpdesc)); } +static inline void zpdesc_set_zspage(struct zpdesc *zpdesc, + struct zspage *zspage) +{ + zpdesc->zspage = zspage; +} + +static inline void zpdesc_set_first(struct zpdesc *zpdesc) +{ + SetPagePrivate(zpdesc_page(zpdesc)); +} + +static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) +{ + inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline void zpdesc_dec_zone_page_state(struct zpdesc *zpdesc) +{ + dec_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline struct zpdesc *alloc_zpdesc(gfp_t gfp) +{ + struct page *page = alloc_page(gfp); + + return page_zpdesc(page); +} + +static inline void free_zpdesc(struct zpdesc *zpdesc) +{ + struct page *page = zpdesc_page(zpdesc); + + __free_page(page); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -954,35 +989,35 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) } static void create_page_chain(struct size_class *class, struct zspage *zspage, - struct page *pages[]) + struct zpdesc *zpdescs[]) { int i; - struct page *page; - struct page *prev_page = NULL; - int nr_pages = class->pages_per_zspage; + struct zpdesc *zpdesc; + struct zpdesc *prev_zpdesc = NULL; + int nr_zpdescs = class->pages_per_zspage; /* * Allocate individual pages and link them together as: - * 1. all pages are linked together using page->index - * 2. each sub-page point to zspage using page->private + * 1. all pages are linked together using zpdesc->next + * 2. each sub-page point to zspage using zpdesc->zspage * - * we set PG_private to identify the first page (i.e. no other sub-page + * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). */ - for (i = 0; i < nr_pages; i++) { - page = pages[i]; - set_page_private(page, (unsigned long)zspage); - page->index = 0; + for (i = 0; i < nr_zpdescs; i++) { + zpdesc = zpdescs[i]; + zpdesc_set_zspage(zpdesc, zspage); + zpdesc->next = NULL; if (i == 0) { - zspage->first_zpdesc = page_zpdesc(page); - SetPagePrivate(page); + zspage->first_zpdesc = zpdesc; + zpdesc_set_first(zpdesc); if (unlikely(class->objs_per_zspage == 1 && class->pages_per_zspage == 1)) SetZsHugePage(zspage); } else { - prev_page->index = (unsigned long)page; + prev_zpdesc->next = zpdesc; } - prev_page = page; + prev_zpdesc = zpdesc; } } @@ -994,7 +1029,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, gfp_t gfp) { int i; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE]; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE]; struct zspage *zspage = cache_alloc_zspage(pool, gfp); if (!zspage) @@ -1004,25 +1039,25 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, migrate_lock_init(zspage); for (i = 0; i < class->pages_per_zspage; i++) { - struct page *page; + struct zpdesc *zpdesc; - page = alloc_page(gfp); - if (!page) { + zpdesc = alloc_zpdesc(gfp); + if (!zpdesc) { while (--i >= 0) { - dec_zone_page_state(pages[i], NR_ZSPAGES); - __ClearPageZsmalloc(pages[i]); - __free_page(pages[i]); + zpdesc_dec_zone_page_state(zpdescs[i]); + __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(page); + __SetPageZsmalloc(zpdesc_page(zpdesc)); - inc_zone_page_state(page, NR_ZSPAGES); - pages[i] = page; + zpdesc_inc_zone_page_state(zpdesc); + zpdescs[i] = zpdesc; } - create_page_chain(class, zspage, pages); + create_page_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool = pool; zspage->class = class->index; @@ -1753,26 +1788,28 @@ static void migrate_write_unlock(struct zspage *zspage) static const struct movable_operations zsmalloc_mops; static void replace_sub_page(struct size_class *class, struct zspage *zspage, - struct page *newpage, struct page *oldpage) + struct zpdesc *newzpdesc, struct zpdesc *oldzpdesc) { - struct page *page; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE] = {NULL, }; + struct zpdesc *zpdesc; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE] = {NULL, }; + unsigned int first_obj_offset; int idx = 0; - page = get_first_page(zspage); + zpdesc = get_first_zpdesc(zspage); do { - if (page == oldpage) - pages[idx] = newpage; + if (zpdesc == oldzpdesc) + zpdescs[idx] = newzpdesc; else - pages[idx] = page; + zpdescs[idx] = zpdesc; idx++; - } while ((page = get_next_page(page)) != NULL); + } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); - create_page_chain(class, zspage, pages); - set_first_obj_offset(newpage, get_first_obj_offset(oldpage)); + create_page_chain(class, zspage, zpdescs); + first_obj_offset = get_first_obj_offset(zpdesc_page(oldzpdesc)); + set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); if (unlikely(ZsHugePage(zspage))) - newpage->index = oldpage->index; - __SetPageMovable(newpage, &zsmalloc_mops); + newzpdesc->handle = oldzpdesc->handle; + __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); } static bool zs_page_isolate(struct page *page, isolate_mode_t mode) @@ -1845,7 +1882,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, } kunmap_atomic(s_addr); - replace_sub_page(class, zspage, newpage, page); + replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. From patchwork Wed Jul 3 04:05:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721077 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 283B7C30653 for ; Wed, 3 Jul 2024 04:01:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AC3F46B009A; Wed, 3 Jul 2024 00:01:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A7DE96B009B; Wed, 3 Jul 2024 00:01:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8CC3C6B009C; Wed, 3 Jul 2024 00:01:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 653B96B009A for ; Wed, 3 Jul 2024 00:01:41 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 20C84140730 for ; Wed, 3 Jul 2024 04:01:41 +0000 (UTC) X-FDA: 82297092402.03.BA264B4 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf24.hostedemail.com (Postfix) with ESMTP id 36EF218002B for ; Wed, 3 Jul 2024 04:01:37 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=PE3Mp3iK; spf=pass (imf24.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979268; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0SrS60HvLY/N6/E75A0QfavAHW7YWNFLHpW+OVguC9k=; b=I1bec+66tsi5g4auEkOqG2xbl0ffQdifDzC4WQFNNpKKrSQLb5kTAxXkHkUKsRcf817Ubd b2ct9k00c1a/C2rnRjja8qgKFwz6gG1zdgtIDZ9y8ehjdp333REIHd507j8cYWyX1IigIJ 4wruFBW5dQDwjghMcQMdoEQbMJ90778= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=PE3Mp3iK; spf=pass (imf24.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979268; a=rsa-sha256; cv=none; b=bE7j8OtnH/U4iw1chLZ/guWTSWxZZwxXeSqvudUNrN2CtK1a55jMZFYAxy8PQx+R94fnw+ tp8pSn+6BpBWd2kBguuLaB4/r97EQ9EmqJcdFiiaUAAsurrIyZ84D7HGfKCRyOcS2cXchS OYWlRN3VsJQ7q82GGX6Y8XvEBnZSHU8= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 9F90CCE28A2; Wed, 3 Jul 2024 04:01:35 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2D144C32781; Wed, 3 Jul 2024 04:01:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979295; bh=82zz9HxUIo/V08TGVJ1E1ZkMfjqggCMIVYH80TRsdU8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PE3Mp3iKrGg+MQDziwiQtLqamfGDcd2BwzU0p66UpdNRzzHqrtP5zJJ9n9wogEL/9 P7K8fhmXl4L80J6DnfajNfBWAagd+7MGFej4mxZtsPMEvifMWkC8+EZmgbWyOy0Cwr LLmeNC4OLQaUHi4I0If9evvI/v/cZiITs3kNJWrM8rqSVbiAYS+paY30MeeHGZieo1 teIsVLI7MRzFlZhYW9wQutQkrZ+zhLQHFHUYWEcVOqBnkcAUUmd+yvFVqr51dTSUZD bsX/vwbYMqctqwgH4SyV5nxPb1Mib2Lyir1b/Wxfm+yvAeRhSx1GO1BEpiQtU0UJDc oF+pN+0ejOKJg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 07/20] mm/zsmalloc: convert obj_allocated() and related helpers to use zpdesc Date: Wed, 3 Jul 2024 12:05:57 +0800 Message-ID: <20240703040613.681396-8-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 36EF218002B X-Stat-Signature: saispbm6dfk5phy1y3hsu4dadq4ky9gb X-Rspam-User: X-HE-Tag: 1719979297-731297 X-HE-Meta: U2FsdGVkX19AmBO8v7UrGpUmRbnqfd6vd3cHA7i2eyyW+Q/+UhK2my29eIBK8uZanDC/5Mu4MytKNYfvfkAJckanzVdmjx9gD/0/gmffj7z9nhXG3KNWfC7DxkZlfqiPGTX2U2vornQ8vBtSzQBxsuoKjx8+0EkVEpWiHsj6wf5StCVblqj8fqrefoVVt8vDHFpTRM3qtFGNvwDrtgPSIuPWHMG4UqCievZODiPh/eoZuOxMub+3hMIyrwB8eP4TcwoK8632S+k0cEozqp3HiIqNDCTBvEsurmzFMOWWiM6qi/DEVdgSL73nJ5FzsnIBvaWigq9wZP7byBngQWQ5mfc76LKvxL3totm9YPf0ch4cN7i/FCQ8ecDxEf4wFY8Kg2usRt/ibBC26dh/gankRLGSl3+r+9sG9dF0bcaT9ZIlPvcLyJAw0B+cACjtYTO0pPMZVd1NnCYWjuRq2I5QmEKWrzrnKp5EtubNLV2WbygFLtPWHd8KWXInN5HtOCLqPgM1WAwe80iHxYQuBvofzWn1+TwpZEF348Plm7gVqARgCFloT1JE837+qzLO/SmOan1m6/oDJO0V9knW+1bSUwTi1V/oOW/k+JydtK/2Qs7TWR5NIPs74RgHelZsxsQe5ujmm7UYm6hoZw4PJ9DqvGSjnrA4NVXBQ0LAxTGhaoDAvb3Qtc5UO3558K+aTZlE/EXAwdZd3MqLCVnBU1n/POtoNXolkqB/pcpfdXRQAV862ug4WSw6h4BUCMPgttHSud1WizdZyjDlDuN4e+IpDvVIQRk4RE/kJ5jWCv+KBfYgHjef+kFYPF66PSRrKJXmU4b73lWs9A+jBJC8HhKnXvZNuEZcdgYVWxSf9EiXwhn7R+9/4fK8f5HFkf7SQ5hSXg0vIIir/p/FxmBprJVYHzBQM7gy6Ykmg03Yg0x4NmU+wcsgZwv/40NEJhiDZgJjDzyHMbJP/bTNgimObpr LglwDXn9 GSrBr1OhLAF/iNyNZco/gH/a/IXbE8M+K6h/uoyPWDunZ2aPkNCoX23lQM9aAP3KYGoZrb0cvtEOj1iYR3EiRqxjxZbC0aRbYpocJeA6kiVjgcFMYE3SY8fFKQ1++Zc3ZqvcQGu7q+cEXmsgJPEXXtVtQARCoFrLSZs7dyU5sPX+GBkrPgPbC1GvSrqBYtHBl2ph3YqZwuX9dPasq04pQg9odiwh5LwCiKIgIym64ByYZf/raua8igL/xzUZj2itUWGiobbpyX5U/btxVNEIQAtq2JcjLmuiFVdYown6Zw88aty4X34H8lctI1A6CH7kC9HMqYvEku1JXIUOxicPdUdFkY8Byc1ThuCplAqvwAcCzYSCJwcti6NeCAQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Convert obj_allocated(), and related helpers to take zpdesc. Also make its callers to cast (struct page *) to (struct zpdesc *) when calling them. The users will be converted gradually as there are many. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index a8f390beeab8..29b9fa5baa46 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -845,15 +845,15 @@ static unsigned long handle_to_obj(unsigned long handle) return *(unsigned long *)handle; } -static inline bool obj_allocated(struct page *page, void *obj, +static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage = get_zspage(page); + struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); if (unlikely(ZsHugePage(zspage))) { - VM_BUG_ON_PAGE(!is_first_page(page), page); - handle = page->index; + VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); + handle = zpdesc->handle; } else handle = *(unsigned long *)obj; @@ -1603,18 +1603,18 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, * return handle. */ static unsigned long find_alloced_obj(struct size_class *class, - struct page *page, int *obj_idx) + struct zpdesc *zpdesc, int *obj_idx) { unsigned int offset; int index = *obj_idx; unsigned long handle = 0; - void *addr = kmap_atomic(page); + void *addr = zpdesc_kmap_atomic(zpdesc); - offset = get_first_obj_offset(page); + offset = get_first_obj_offset(zpdesc_page(zpdesc)); offset += class->size * index; while (offset < PAGE_SIZE) { - if (obj_allocated(page, addr + offset, &handle)) + if (obj_allocated(zpdesc, addr + offset, &handle)) break; offset += class->size; @@ -1638,7 +1638,7 @@ static void migrate_zspage(struct zs_pool *pool, struct zspage *src_zspage, struct size_class *class = pool->size_class[src_zspage->class]; while (1) { - handle = find_alloced_obj(class, s_page, &obj_idx); + handle = find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); if (!handle) { s_page = get_next_page(s_page); if (!s_page) @@ -1871,7 +1871,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, for (addr = s_addr + offset; addr < s_addr + PAGE_SIZE; addr += class->size) { - if (obj_allocated(page, addr, &handle)) { + if (obj_allocated(page_zpdesc(page), addr, &handle)) { old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); From patchwork Wed Jul 3 04:05:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721078 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26189C2BD09 for ; Wed, 3 Jul 2024 04:01:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 98E1B6B009B; Wed, 3 Jul 2024 00:01:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 917A96B009C; Wed, 3 Jul 2024 00:01:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 744606B009D; Wed, 3 Jul 2024 00:01:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 4B7FC6B009B for ; Wed, 3 Jul 2024 00:01:42 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0B76714073C for ; Wed, 3 Jul 2024 04:01:42 +0000 (UTC) X-FDA: 82297092444.26.266C0E4 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf06.hostedemail.com (Postfix) with ESMTP id 702E2180015 for ; Wed, 3 Jul 2024 04:01:40 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=ssU8oOZS; spf=pass (imf06.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979277; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7aSS3PfeGPzIHYEgpKcCy3AJpke6a4GKIbhcdDELY+E=; b=JA3NzRegR9upk/w/Ovslve3LwgqOkqdXnxqsojo4HMDi1RI2yyWXcOhkS20z/dwmRPDPiN XCopddG/12IBTaHBDfm171bct5L1zwV/jOftxKANF1YSI10ZBP7r43MzkrW/xGV5gKhGuV 73Z7WWIyJNthM3iRmRuazC+uvihf1Bs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979277; a=rsa-sha256; cv=none; b=JyXl6TW1wlRZyfs4ah3qnkNT5q443xUR9tnIEe0yB/eePZTyx6DWdC7LRVtS2v/gbFiH6y 0oqpI4l3rdciDsHthlcfnVA7xKpPmxRwXrEaZvstpaTyJNmGaj/VyrdE9burHxWzAsoNgs A30wOCg+NdBJfqhmC7XhLwzW+70YLBE= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=ssU8oOZS; spf=pass (imf06.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id AF7D6620EE; Wed, 3 Jul 2024 04:01:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ED24BC32781; Wed, 3 Jul 2024 04:01:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979299; bh=hCA+Wp2koQFPErK04B2puEI5lWDR1Rh4V60AMISo3bA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ssU8oOZS8VnZ5kjtvdEP4lDcKyV/8A6M1yM0c3F1KIS8eUI6cNiQvdY87Bb6YlG1V 9b2Qoy4jCAjauQk9G45OXT/p5dW+Jpq9PeP5AI1GVkm+G5B19918JQ/pLqfAPCXsWg c93b4mZRWvKhOEAtTHlXnN8MLEBdsiyswP0TRu5NP202rpZDdQ7/eOVWygU8xreES0 uL4HDJpE4IYOajppi6Nr4yiyTp5ZzDwOjGh6ADzC54ITody68TXzrudEUTH1gWDv3i UwlI4ku7/0UD8QVHuSqhSnWNpJFCsL4RazQOmUsYJCjQWblxy1h9oW+ERZ86wiRePZ AzhbrwkhjIkLg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 08/20] mm/zsmalloc: convert init_zspage() to use zpdesc Date: Wed, 3 Jul 2024 12:05:58 +0800 Message-ID: <20240703040613.681396-9-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 702E2180015 X-Stat-Signature: i95bm31c9wmcehywpnyypu3bzbn38fyr X-HE-Tag: 1719979300-861184 X-HE-Meta: U2FsdGVkX1/xQNE/QEJJLc+cj7xWrF2EZW2e5EliA3OW3qspe6znaGa7vTMc1IEmQHThhhBYou0+M9B4N41skmYPGQ6FNS9xnXgVVYDd3/pp15U1+lYIWVCyVNhuklqRZ6Oke134JeUXpoACN/Isy4BJxmqYVtV67PCFVkDgRMYcRJXm/xFc6IX7PnED2zHBJYTMFrOU+WALsMY2TOEMZ1nkti6OQIXANutmbKOKG/Cmnd20XoHOY8OIsYQhbwpW4SEr1DAmVMZtjDpxcdQAi/bPqqEr2umiOm3by286ExDxbYZ1Z5n1PXTjlSEmpArDhLyePXrj41Lcn04634UHsP/p2nLcQnfMofy0iSEfz+SIoIeBQ2mjsGxbfSvUewYKK0q4mezCvu5XPNCPq8g6/f7LBYXKXi7OGLfWnNPrFXotTnIYDG5Ht7dB5DcgI4MF4U3IEy3fYUUq1gPjyB1Rr8zevYkMo2tgz9vZJdlZjFvtS30X1L2PBPFbkf8jHhjGtmfr5W2XALmqJsAtQ3vH57na44mSf6R/mTTAYz6LyNCbb5+fAeq+0t9BHW+5SQYwDMLBx6XZRUf+WaKRDiPWuxGVlELfsUaznS7PbspwLxwKH/rI/HlO4r9YzZEJNMqADXA51OsNLsdRiS7xRkjekuKKxH6UFbAJa4Pc3MZfGJO3QduaD/N/v9gjrxdEiLTZn0hrTW/OC9ssqgll80+BZ73OpAKfR2idvWrToni8Ol7nYUO0N3CL40nJo0kHZklWZ7HREiBlZKNRbFj4ootuHY8igoXn+UoKcbgMgB+ZPRhU4CQ4iMGW9y9aGC/0ifM9FMe5Ku9JbCo1O/k4UtuVNZ6gDs6ING58vOCQOIMsWjTJsSIyHut1fVK18E5DAwUGX6sFGn1711fIbmK0CLtk9OjYsV9LZmVXq9x3F6sEk7qMpUiKhnTXKyrSqB8gqlA9fNvCXV6VOTgPU1ir2u3 cJz8WuSm LsRgt56UK0W/jmFOOGH9OlnoCAJNR0h1TpKyafFHOocxlcHXbbhwL5YB8Q6BwZTLIoAAO6/sKE+jZzeOCHqap95gHUfG+WIZX1vqew8V7gUkwsBpHychdi5Z6uysLmOuGfsZC5grwSJorkzvpl6h5h7rXDVAK2G4mTYqIHVKYMD+nix8bHxH1zb9wL8ELLUrYTdhqLeVMy9cRqtnUdJdpQ8rAvtd1kEY3+p+wWjSPs7svQ0XL+lluF56/PWpXgkS9YelNUkeyuU/gWr784KIYhpKlwnGQpVfQ13F8LSpJMPxhkoAEddl06tFxe7GsV7b/MTmB0hgn+edOXYZY97XTFI9ufbtxSnFXVXDkGLpFP80w/tHbJPcrENLxWkFDb1TIMXoG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Replace get_first/next_page func series and kmap_atomic to new helper, no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 29b9fa5baa46..d3558f3f8bc3 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -948,16 +948,16 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) { unsigned int freeobj = 1; unsigned long off = 0; - struct page *page = get_first_page(zspage); + struct zpdesc *zpdesc = get_first_zpdesc(zspage); - while (page) { - struct page *next_page; + while (zpdesc) { + struct zpdesc *next_zpdesc; struct link_free *link; void *vaddr; - set_first_obj_offset(page, off); + set_first_obj_offset(zpdesc_page(zpdesc), off); - vaddr = kmap_atomic(page); + vaddr = zpdesc_kmap_atomic(zpdesc); link = (struct link_free *)vaddr + off / sizeof(*link); while ((off += class->size) < PAGE_SIZE) { @@ -970,8 +970,8 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) * page, which must point to the first object on the next * page (if present) */ - next_page = get_next_page(page); - if (next_page) { + next_zpdesc = get_next_zpdesc(zpdesc); + if (next_zpdesc) { link->next = freeobj++ << OBJ_TAG_BITS; } else { /* @@ -981,7 +981,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) link->next = -1UL << OBJ_TAG_BITS; } kunmap_atomic(vaddr); - page = next_page; + zpdesc = next_zpdesc; off %= PAGE_SIZE; } From patchwork Wed Jul 3 04:05:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8962C2BD09 for ; Wed, 3 Jul 2024 04:01:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 51E0D6B009D; Wed, 3 Jul 2024 00:01:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4A6FE6B009F; Wed, 3 Jul 2024 00:01:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2F9FC6B00A0; Wed, 3 Jul 2024 00:01:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0BF226B009D for ; Wed, 3 Jul 2024 00:01:49 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 8E0B6A2775 for ; Wed, 3 Jul 2024 04:01:48 +0000 (UTC) X-FDA: 82297092696.01.02C7666 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf13.hostedemail.com (Postfix) with ESMTP id 9248520023 for ; Wed, 3 Jul 2024 04:01:46 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="h/vn2l9o"; spf=pass (imf13.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979295; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TJqG5dwhfFg0PHzR6PQawnyu+ajyYcBzzmI1swN5VQM=; b=j+UEVMhUfWy9YctgTHhj9NC3X2Q6L/wKpAT6WsJcMvxAdYEbBTLhWsvDqQvpzyZjhV+utS 5DDiQJiaF2LcPMAr5RjvgXKm5JE8bFZz2TBZjsgKRybEkpfCK9opmgTGkFoezIbpzK+9BX xp3HgqkZDMOcLC2mQpXOSTM5tccyENk= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="h/vn2l9o"; spf=pass (imf13.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979295; a=rsa-sha256; cv=none; b=lF1hpRs5Sbx0SAQlRHwnG2NLqbjvhiZEWkTP41rgAQxyVytFVTCJMeCuPmtW6QJHwVRC4w MCvHFX6Z6bC+v9kx9xeQSFERvfHMCrVb/tfc9fhScFmMIHIDJovCg5gntIbOSX/PzZbxI3 OMkN9UE6XNZnEz8XKTz/8e7YH+j3jaA= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id B0BD5CE289F; Wed, 3 Jul 2024 04:01:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B9464C4AF0E; Wed, 3 Jul 2024 04:01:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979303; bh=cpsRKinL3S7LbxZnzQMdHFXcR2a2bxLsePv8cUuJa+o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=h/vn2l9o3YSdVOjafhW6qL2RUpxiwj6dLECE60UFuR7t8SZfG4edNba8A/qzaWYOu FtDjnMdg7TiMbLrqMf4E5P4XVWzIrH4IQ2A8brhdPHmObwWoKyZSngO9YkKZdafPBC VcUl09u2ChPvdiYH8WCn/LocRf/ff/wUNAE3SdawSUx+9HQEWHIUOw8LeF65Z15POI 1EmLkS2S5y55raDFiV3ohsvIKoFNH3jumJfY+f7TobhdeFJyrjTqOoF5oH6ZlIsSTt 3Jq7Xy/bhl2AnZ8am4wTsCkWnlp8TBUcPD/dlOh2lY3B5IaxKoCmRyBdS0hpGQz5od KnkYo94ZbqgjA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 09/20] mm/zsmalloc: convert obj_to_page() and zs_free() to use zpdesc Date: Wed, 3 Jul 2024 12:05:59 +0800 Message-ID: <20240703040613.681396-10-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 9248520023 X-Stat-Signature: iwk8rc6o874ycjqdfwnrbsmea45riskw X-HE-Tag: 1719979306-530432 X-HE-Meta: U2FsdGVkX18qxKAxskZWzh8P0uAxGZCr2Rcee7S4HDm4KaWRYI4Qp6pJPXW/Z9509Hbmpf1evO6/V5DZLp4vv0hpQf/nAPAavY6hz1pksNxtaXbRAo8Kc/kiiCg6addFCyMbV2oQ0w6XKKCY5BvXLQHvzBEiX57rwbfIIDVPnIcXtKi2BOdZyCBpfnByRrLcC5ESkWOzcJiuN3Uy4v/8wf0u0TnNHZGQcJGvH1KMDUAVexNGwZfZqJf0rdwXVMIPaMkxssJloHEdBAAD7rBDFLOBKP9Hp8fqRnNUml53G778lwmWddz8Mx1It44OjPnKNf2yeGkqnmivrvedDgQDRf5hut7bs/x+oKHFQueP+LMuVBmo86jmM6P8LHVharOp75hLzdrT5PiqwtjS4mFtOvZx9J1LfcIj0OrtoKLO+lXtDQOfH1amu5S3ZeZplPWjEMrCoLyYy33RSjj77EvsS20eMjN+mOWKZeGCt7mVIXRlp3za0KB7WW08qg1ZUy+ObGlf+S+byCajAYnwQX3yHSZl94xwJcYusz+ADX0nxmpiHBj/QPTDvub5yOCVenOGRtD4k8lY33ey0mTWuY4/gM3KpUsXg32fdc/5TiVim6DOw/fyJdBINZ69SWTR/Drso77E7JW+EQINasroRz9Rk3FO/bVPa4sWXfmBARRXPXqMaEEJ6qpK/5da7a1FcdGQLdNotsJndrYDAQZv/IH8BcP+BsAb39vKAuaGNu6suGw77hyajURm4Wd0JYBTmEpkNV+sH0qSZOJisgTGzHIvaFGL7QS23c3HkWvwUeoQHS0luXJyUx5NLIa6HcW1oN4xu4EDtC8zUs6+RvGnWymDR/b1EZaIiD4Zo0pHrLwNkRN1Szimr96pHvgomGqZRK0CA878LQhTVIaJwxN96U3DI5+QqUubmNWJg2WgNe3WnPWm1g3X6pIPsA2ErrpjHGx7Nwihs2gpwvoaksD0vuu B/7ap/dJ qkquZc28Q2MPCzU0n9+hCGAU1R1yAVQBbOJouxxyGckiOlUZO+Z7F5CVE6158S+OlSrKzUDhH6CNmaMj6FArcGBJr+xZW9D+YBDtZlJyIuqMsrn/B886WxlQfbInP+AVYaAHWkxBDtiwQpO7LMFc5r65OD+S5pM3GgGHXRdE+SP4KIpwKeQB8v2MstaQ0PgV62Lq0koMN2drYUecKEXH29UxoSBeM0bTnYijGOJdG29bMya3mS9f8aei36OsHsOwwBYet28hOUwErJv4kPTW61Y7SGj7SwdJ8I5Z8kc+dj+mgpR4I56fNTPBfEiZYSkjyWaj005Jr0t4Br89jycexmAu0lHBYJlsiKjq3ltKQX96OPz686kFzBP+8UQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Rename obj_to_page() to obj_to_zpdesc() and also convert it and its user zs_free() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index d3558f3f8bc3..7aa4a4acaec9 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -820,9 +820,9 @@ static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, *obj_idx = (obj & OBJ_INDEX_MASK); } -static void obj_to_page(unsigned long obj, struct page **page) +static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) { - *page = pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc = pfn_zpdesc(obj >> OBJ_INDEX_BITS); } /** @@ -1496,7 +1496,7 @@ static void obj_free(int class_size, unsigned long obj) void zs_free(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long obj; struct size_class *class; int fullness; @@ -1510,8 +1510,8 @@ void zs_free(struct zs_pool *pool, unsigned long handle) */ read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); - obj_to_page(obj, &f_page); - zspage = get_zspage(f_page); + obj_to_zpdesc(obj, &f_zpdesc); + zspage = get_zspage(zpdesc_page(f_zpdesc)); class = zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); From patchwork Wed Jul 3 04:06:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721080 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 912FCC30653 for ; Wed, 3 Jul 2024 04:01:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 260266B00A1; Wed, 3 Jul 2024 00:01:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1E9A66B00A2; Wed, 3 Jul 2024 00:01:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 039F36B00A3; Wed, 3 Jul 2024 00:01:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D24206B00A1 for ; Wed, 3 Jul 2024 00:01:52 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 883B9120725 for ; Wed, 3 Jul 2024 04:01:52 +0000 (UTC) X-FDA: 82297092864.08.25C46B7 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf26.hostedemail.com (Postfix) with ESMTP id 45849140002 for ; Wed, 3 Jul 2024 04:01:49 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=G0B04OIt; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf26.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979291; a=rsa-sha256; cv=none; b=dZPID4YcwJBun8m5ji9fRxZH0rhcWObp2Q8DLUhDqzWe9Ot1KhaOGlKH8EJOnmbg8jtxSp DP5ruO1j5P1OgOZ31+rGTjWyaqO3REhjoeDXXRWFNI5f6BId+R8pX7sQTfMkgXKJRQrcqQ X0cP4HHYx50Kl62nFWq1S0JbgDEZg+c= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=G0B04OIt; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf26.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979291; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SP6hImZmIrRQvoib1xZzKsa1/Sewrov2fmekSGish90=; b=yiQOMKWT8UdxtdrX52TgFIfxNrs94M6dRyLvqY/2jmLOwpyO+nMGIlRh+b8dV1H/q03p8p v1cq5B2QYcrZ50gUJH4ZLfk4prxjjoNEGiJcRXfPTcmd4vSB74vNJ8mMIpeUGn3eMxhlM3 1O+wsCTLW3WPF2VjClfw40cahcEcH1g= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id A7F21CE1DEB; Wed, 3 Jul 2024 04:01:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AF65AC32781; Wed, 3 Jul 2024 04:01:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979307; bh=X6q7HiV/JKEJWx8fJkY9r9rnNkPwseyxy5Uf/fZ/T2M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=G0B04OItwHRPyh+8sULIxpolV9zSD+VmOfgGiZ3Y3gonxy225OS8tS5EBr//Ixoi9 CGJLSxDUAUyX0z+bVgXNHpaQFluz/fuqNE+w2SNDereXSgVjKrHUl5iUM9F+xalXIg HUj4yZYZT2U2YtJvt7grh2npA+NoNsjvRWnII0qLazNTDES26JXJqTmwLz90DAg0G0 c/YrOuXaWyExREG3YPDdvo751cKKvV7kcj8kaFVEH/LpVWeNFJO0YWnOtY/qa7UjP+ z7d1o1kNYc+2puqu1lkayptxyVMKKn9z4kj7znaF0PH6/Ey8uz6qESrnX9LdeDE3NA JoLmzRg7/SuGw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 10/20] mm/zsmalloc: add zpdesc_is_isolated/zpdesc_zone helper for zs_page_migrate Date: Wed, 3 Jul 2024 12:06:00 +0800 Message-ID: <20240703040613.681396-11-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 45849140002 X-Stat-Signature: op6sjihqxf6qopjhg6kgsp3c5763yxk4 X-Rspam-User: X-HE-Tag: 1719979309-551392 X-HE-Meta: U2FsdGVkX1858yy1Y8UnZ2N1RcMSVO+rzZGyKFRNgzwwdVuBQy5ZHzY2fFaQwOQk3o0m8wa3FTP1fug+y6G3Lz0CLlBPcY7n54BvlSJO8l4ncMVgHso2eiMWBNDv0o/91XLiZrrml9AFVgotqHqqoDCuZpxvQiwY5IzTb02oCBtdlkqjfjTcmgl627Vvxx1hLIqVkl49pSUCmEo6om0AdXz7enMO7sJ//Eo3xC/lvn8kpqv26Z+0VeB0+2Q0J50z0lZS0KMu2CCuUUEfBc4/fhnAEvRuTKZNlStCG5QSDcZg33PpntUgLM+yZQhESKYmQYgSfQ3h8Xc2OL1Wo14STE54e6eO8yGapSzg5kqoun9rD9dkr0Warh5PM5NFyJu0hTi6i6BY8GrVdeCy9/YxliR9KHbD5bFpXSlMW8Rxb+EdMTaDHflATtEZN5E01CdpXTqLObUrquIjfx5ksImo3uSWfzrGk6wWg5Syp/4f2fMOJqMAI1FHIW+c89zaFjGMlgreeWUD6DVVBkyQt6Bl1rWihSVNsyrzhAVJEfFzko5R5Mb70He1AEsoT38/nwrIFHPt8vMXpFqShzgik+ynH9evIFYrYBLwmEXk9BLcaklBdSy3KwZJbK5re4KLWxe2kLmhSsMP/XBLhLbelgpi3ZygMpIhvi6ngkAAzvRNLP079vVkxq97zj50ITBzFAau/Duy50Em1wYVlvfH/gJy0yXaDHchPzcm2dLYqU9D4YXr5EGovieg7kceEWZc83keoLsrFqFLJph3XkUHtxVv/wjmoemaTJXZ95OLEz3Cadla9bLe8y1eVLkjavh1N52cHUnud4qjV8Kb5PFCa2iF+BpgQWQAwXyJ979CBCH6CHqQ8aVibRDNdk2Dj11zdV0hVUR+zZEFSVYHaHxjnSTU+kwvtDJ44VApL3e5U226UVYy93DQUu+GuqDKJzUpne18lqA9kGkQ2PPjAxwF7q8 gqfCIQ6U NoYEqTzzcWp9kRS2XFr9AK4dtPH/at6+CYW+PjfJpRyt7jwtwcxahDYvqae5VwTJZpxTgzZaAeF4+oy+w9ma9tqz7qCMEDitBex9/Sv1MKIYQeVS8zp3/txCYGLRBKQKShamKm5DjDMmmzUKICsAMykJRK+eTiqtBk5du/ofw1d01EZ/wR2GHAEKR93Xg+as6jM9gbwUvVC49uD7nImUMdOfuvjehhyjTmkf1pMoZKaWHlxt5OUR/7vFEd6up4p8LZiGIWnLwDvmvHCrhULV7PCLwY/o3cbtJjYWn7/AVu8+PzacAn8rQjyrCb0MavnHon0olNpfT+XDA1CvHt1JctmW2UA/Qe8cC6kfmFuir+AyjnQetkmwTUyKgSA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To convert page to zpdesc in zs_page_migrate(), we added zpdesc_is_isolated/zpdesc_zone helpers. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 11 +++++++++++ mm/zsmalloc.c | 30 ++++++++++++++++-------------- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index a2e320f24dd4..61d7ba0c626e 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -106,4 +106,15 @@ static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, { __SetPageMovable(zpdesc_page(zpdesc), mops); } + +static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) +{ + return PageIsolated(zpdesc_page(zpdesc)); +} + +static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) +{ + return page_zone(zpdesc_page(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7aa4a4acaec9..9bc9b14187ed 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1830,19 +1830,21 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct size_class *class; struct zspage *zspage; struct zpdesc *dummy; + struct zpdesc *newzpdesc = page_zpdesc(newpage); + struct zpdesc *zpdesc = page_zpdesc(page); void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; unsigned long old_obj, new_obj; unsigned int obj_idx; - VM_BUG_ON_PAGE(!PageIsolated(page), page); + VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(newpage); + __SetPageZsmalloc(zpdesc_page(newzpdesc)); /* The page is locked, so this pointer must remain valid */ - zspage = get_zspage(page); + zspage = get_zspage(zpdesc_page(zpdesc)); pool = zspage->pool; /* @@ -1859,30 +1861,30 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); - offset = get_first_obj_offset(page); - s_addr = kmap_atomic(page); + offset = get_first_obj_offset(zpdesc_page(zpdesc)); + s_addr = zpdesc_kmap_atomic(zpdesc); /* * Here, any user cannot access all objects in the zspage so let's move. */ - d_addr = kmap_atomic(newpage); + d_addr = zpdesc_kmap_atomic(newzpdesc); copy_page(d_addr, s_addr); kunmap_atomic(d_addr); for (addr = s_addr + offset; addr < s_addr + PAGE_SIZE; addr += class->size) { - if (obj_allocated(page_zpdesc(page), addr, &handle)) { + if (obj_allocated(zpdesc, addr, &handle)) { old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj = (unsigned long)location_to_obj(newpage, + new_obj = (unsigned long)location_to_obj(zpdesc_page(newzpdesc), obj_idx); record_obj(handle, new_obj); } } kunmap_atomic(s_addr); - replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); + replace_sub_page(class, zspage, newzpdesc, zpdesc); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. @@ -1891,14 +1893,14 @@ static int zs_page_migrate(struct page *newpage, struct page *page, spin_unlock(&class->lock); migrate_write_unlock(zspage); - get_page(newpage); - if (page_zone(newpage) != page_zone(page)) { - dec_zone_page_state(page, NR_ZSPAGES); - inc_zone_page_state(newpage, NR_ZSPAGES); + zpdesc_get(newzpdesc); + if (zpdesc_zone(newzpdesc) != zpdesc_zone(zpdesc)) { + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_inc_zone_page_state(newzpdesc); } reset_page(page); - put_page(page); + zpdesc_put(zpdesc); return MIGRATEPAGE_SUCCESS; } From patchwork Wed Jul 3 04:06:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8986C2BD09 for ; Wed, 3 Jul 2024 04:01:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 485C66B00A2; Wed, 3 Jul 2024 00:01:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 40B676B00A3; Wed, 3 Jul 2024 00:01:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 237C16B00A4; Wed, 3 Jul 2024 00:01:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id ED9EA6B00A2 for ; Wed, 3 Jul 2024 00:01:53 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 975E5160588 for ; Wed, 3 Jul 2024 04:01:53 +0000 (UTC) X-FDA: 82297092906.18.D3A2B00 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf11.hostedemail.com (Postfix) with ESMTP id E9E9540014 for ; Wed, 3 Jul 2024 04:01:51 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=NpDb5OJz; spf=pass (imf11.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979300; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kJQiG/5I+17lbFKCGyYuI+CpVs0RVUWdddaEyCf/eUQ=; b=XVE3MsFtOfyO5XpNrREaig4AhLPsmCSWoU5ZIAS8vA8fB9vHhd1wDhyAXTnzGWl0y2Nyks eXC3HQ7DnG73rWV3VYJlXhNJ4Ga0FHTHQnHp3pQtKYOE+5MMaD0A21vIzjNxvU+XOqeii5 olHtNpYKJhXjpKZwKeYOevVPAdMGEys= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=NpDb5OJz; spf=pass (imf11.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979300; a=rsa-sha256; cv=none; b=6TFgxpFK0ehGc+x9NyocHGsOhdfjLG/n8wvGiIzX96j79DblhRuah5BoYJ7sGbM22EWtDP ULXHTz201yThfFwl9VQRtEaHEICLTLh4DPic0EtD/CU014glwmvuJAyBPmLtD8vi4c94mg RZskuWnXQ8hNTnzlfShACLjByT72Od8= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 2060C620DD; Wed, 3 Jul 2024 04:01:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AF31EC4AF0B; Wed, 3 Jul 2024 04:01:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979310; bh=d7GxkLwHXdQyb6XsJEXUauGOBeH6/SyL3Iuz9vNzPwE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NpDb5OJzKlBHlmXy9eSfpYJA66+cDPSqPT/zmcenz11JqH6X7j4xggFBVksX4hHDr hrjebT56ic9Us0wcGrhTL9GrCCBcw2YKhHWj5Y/X7PXOoNg8AJDlaRCV1/bY5oVTjW 7reZHFTfGlB5U+5CYeaRbolfCWAWin7oIgpneC1lVlHKzPLBeY8NnMzeV6hQm2+Uq+ dyrqduH1axDvKQqlOLJqHrP2wLCJgyLxxzsoUfUu5MC/7ZxEDhAjZdajB8AmIPr8yV YuDrR4c//Kevf275D86Y0FW1sKewkadQl9n708d6mCQmWAu/FzFHMBk7NsFvLKFIyz O1sJ+oFdqQ6xA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 11/20] mm/zsmalloc: rename reset_page to reset_zpdesc and use zpdesc in it Date: Wed, 3 Jul 2024 12:06:01 +0800 Message-ID: <20240703040613.681396-12-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: E9E9540014 X-Stat-Signature: x1sx5ap3a66gr43gqhrb3jpy4oxy7h3p X-HE-Tag: 1719979311-632897 X-HE-Meta: U2FsdGVkX19Zk9iEgCeHnb+1+V7PA/vAEVsg3xXn/W2b3/nAtWqv9jZI3b9ZmsonD1q3GcIs64ouczr07A1mxH2OVjUT1a9jeyyR0KWuykJXjwPMzWOrSNqtoFMz58WeoDrolYgsUwBEXVAvA0A1JmXKUYN7BiPzRgvLX7yrU3ZksBplgy1KDHmYjoDUQqdJvpH4uYXUG8u4oWR9C5QvlrJ4PfkUxf10kt9oGTA1JmkCDbVE4iNKjI27VNcFbhZd1uh/ueMF5fqahrOWKCXg+kjy7KIKcEsIhSa5DtXeUV+wUMCSs8BT9UWBrnyeSITua5DjYtTbKaRJpsNNqRG1kaZveuEFkhUMxDi9hb81caGGEYftflYkxQS8mbddiHpb0IdQCuFcPQIwAG1Pd/bRkFCOc9h7QTwtMt1f2jlMQFA4g/Y4LzkV2CJK5k4IoYYZWF1GqE99TfcEHLd4ykif/zn0gcmVpZojqc+pRdOIK+bpebSSTLAek/+irgkn8aglmkirDfqf89ggWaXgWAtMVeYDJai/xJlMlnREhbLzyPV5SaU7pYafe2kW44VYEZeU9gIPCquowtDnn1h/8acraJENQa0mYUNavybhHb/IB1+ib6WHetXEBeLK7xjRybYahySU7canCTEX/ArlKnIX3uhghg7WX8jFRHXzOjkZbftvnH1sDlKfGbvPIZwpqXjoDtvl/WGarz17/h+liThDF4bmyAu2CETBrS8pJe41IOJng4qTJAe9Z/84fei2sDZG6A1JDNfEWQ7oF+rjHHLH3VVppRdBSGV7O7VXTm6G14K2rKuuOvaWPPh73IhxCgwa5kUjaQThCYJtQoJWwHo7k9MHXai3l1BjOQOVWA0fyPbr6PayAJsGt9ZG1OEQmJeZ5Af6U1fa0A/4vtHB7zILEhK0yjl0oguB6S7WSZWViAaMBE34Q2YJzK1pyBicyy04OvxW70ckofG7/7G9XBx VNUez5Sz gf54wiS/zR9ngQ3CbVtQq+yMQ4F69oaLT/c8qHdGjPPDrGqGxRz5HnyxHUIXLpbbqkr2Mb6phbaBJs1L0u8Q4lssIiuEYBnFDZ3LC/Yh895E3/z58W0n0R08hH/c8Q1TUIi1I/KiysWkWx9DXdXXM8dNO5O+V3W7OO59t5IkW7DxNqcizGYy6oH2Gj1LBcbHJmWQPugFtw8ZhiGMxLSk5EgyV+IdjVVeS8ae9q4TKR+6Ui8Bexik9JY7VE0Xfj1E1atjJAuE41OM0WDFpz1YjnUzNvniafEUWsYAW3Hu9pLIA0QnCjTJwZbuoeGen5PMLtwihA3GblLEqJ3J8AvA7gWh3by2rkCYKyuf7 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi zpdesc.zspage matches with page.private, zpdesc.next matches with page.index. They will be reset in reset_page() wich is called prior to free base pages of a zspage. Use zpdesc to replace page struct and rename it to reset_zpdesc(), few page helper still left since they are used too widely. Signed-off-by: Alex Shi --- mm/zsmalloc.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 9bc9b14187ed..6d1971836391 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -865,12 +865,14 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, return true; } -static void reset_page(struct page *page) +static void reset_zpdesc(struct zpdesc *zpdesc) { + struct page *page = zpdesc_page(zpdesc); + __ClearPageMovable(page); ClearPagePrivate(page); - set_page_private(page, 0); - page->index = 0; + zpdesc->zspage = NULL; + zpdesc->next = NULL; reset_first_obj_offset(page); __ClearPageZsmalloc(page); } @@ -910,7 +912,7 @@ static void __free_zspage(struct zs_pool *pool, struct size_class *class, do { VM_BUG_ON_PAGE(!PageLocked(page), page); next = get_next_page(page); - reset_page(page); + reset_zpdesc(page_zpdesc(page)); unlock_page(page); dec_zone_page_state(page, NR_ZSPAGES); put_page(page); @@ -1899,7 +1901,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, zpdesc_inc_zone_page_state(newzpdesc); } - reset_page(page); + reset_zpdesc(zpdesc); zpdesc_put(zpdesc); return MIGRATEPAGE_SUCCESS; From patchwork Wed Jul 3 04:06:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721082 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90220C31D97 for ; Wed, 3 Jul 2024 04:01:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 216B86B00A4; Wed, 3 Jul 2024 00:01:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 19A906B00A5; Wed, 3 Jul 2024 00:01:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EB90F6B00A6; Wed, 3 Jul 2024 00:01:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C43F96B00A4 for ; Wed, 3 Jul 2024 00:01:57 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7828640786 for ; Wed, 3 Jul 2024 04:01:57 +0000 (UTC) X-FDA: 82297093074.14.61073C0 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf24.hostedemail.com (Postfix) with ESMTP id D3563180016 for ; Wed, 3 Jul 2024 04:01:55 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=f1t7hQXJ; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf24.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979296; a=rsa-sha256; cv=none; b=vZf1Y8JCRbGtwEY4Itkt2mSS2pBXRtr75bZGpFbspVVgogf5x5ohFQf8Zt9LET/oAcRtT1 FYwp5zYpkt+NnCnknCE8k81m75c52kvdS9Sv4Jp97vfnWjGXUaKJPLzqNED8ZwBw0HhfHX sxoaHSRPUZsHqjXGzXTY2caym/H/4Sw= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=f1t7hQXJ; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf24.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979296; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KXF/dnRNKSbeaumpIz17Eg4SmWxMyzprpVBUw6ZMybs=; b=CD1+ZX7MtYbCDnej1euVKx/IhhNCEZwi+qxio7tNjqfVgQ+pI1PhpZXOI9XGWWaFq+uk9Z BXuogXIh0Cx6LuxFx50dF8UrLpZGQTW2YOlb1QNTgMlgZup3Qt9F28oUgDkhcNpSPlV+Vl HPNV2EHz8+yhTmCv2GaL4ojcR6L70J0= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 18900620E8; Wed, 3 Jul 2024 04:01:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7B8A6C32781; Wed, 3 Jul 2024 04:01:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979314; bh=rKWZuiJgMM3PFhIFbKbki7RwNRU1BYQ1UAI5bUa3ql8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=f1t7hQXJQvhCEgu9b1n3nE1YlFOxjEC/c/pUUWyfXG22Tw8QUwfCTUD9VZwL94fSY jv3/8d/eHpn1e+fQuoItWy8ZyrfsKIGYgO1xWlKssWAy9Twg9jtJDGp5cvknh2HJWd W5YnUonQE4WtBpZK6/z/Pci54CDg8ZV0li79TV2nbsIfnaWFZcSmGtqrgFhEgf8bOO OgWHm/ztVeQ56/PIb+6/rkbf8o6xP5OOTqlSNjwL6lQI1m8k5SNG5ocMCkzRy7TSwM Nqt/bERDQCtXlJL3/j3SpvyzKvCox2IVzTHeBDk2M3v70gQMtckTQ2XRIaKeFXX8LZ FKT2rzHyQimFQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 12/20] mm/zsmalloc: convert __free_zspage() to use zdsesc Date: Wed, 3 Jul 2024 12:06:02 +0800 Message-ID: <20240703040613.681396-13-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: D3563180016 X-Stat-Signature: fs6jtoffiprbxowur4st8pnf6fop3wqw X-Rspam-User: X-HE-Tag: 1719979315-241829 X-HE-Meta: U2FsdGVkX18r7uxAFAILwtQYSbzk06DwBIFvxlWiIHVkn91ByUs8FTtkv8DoVLGswhztMl2kDkdH8TkAtTNeuxE820ZIe4i1XrisXXOyIQKMrvAHYg4xclwnQgpELH4lAX+OeERh305qNuyTrxE/FlrI5Z6ETcMcWWbUsnvdJfCiR1g8HB2NIxpFS3ZBbNcv+/+ikYrHfwcL580XzLZWHpDmuNyVRwyWxz0Txu4oxJV5m9Cfjg4zPWBNqjVUGrK7kdS1Q281pMis8uEriBuAmX9WytRqhCggGyc4xWl2VWdMiukijDEl8g4q1y+WdmRZPcFoKh2oxdsRBGX1CPp2GP3q8o8pyzgyNiEw05zc/C66f/zCUw7S6ZEzPbLdlfei+BthgV+ct80QUiIyPbwAliW+3nXXovY0/meSZH/ls05/YgTPhr0qxltNrOoB9/wTJPi7VAnQJLr7MxRS2f/xfoE5sBKHEhjQrsfwUrFDiAmURgyYOZi2b+GzvdvkSPTeVrGh3T8+MpgcQITata/ansiQKyZEdUfq8wOZJMCCrkYok+3f0qDs3t0LdXsJT9siwVxEnzFDm/cnOx4hZk4NlXJDkzDnQwXEDwfREWNLQR4n8WdPBCyAfmzVDCrlYGQEyo7tuSOw2nn/wmKt6tYpzrp4jBpIiwOnBZo/V2pHUrUhW75LcqFl+rzte1ZNsprzUONQsE4sKFlgVXkNT2Dve14GQ5MlvhNZcUkE9OQ8hD0kOj33xO2PdsLYQW8gAzXCRwcfR/aWV3CajyYtX4K8qaqW6JbxE5EFvNDOldHHZPKvUG/OoZxvdTh1B2d4XdVsR1auYRbCqT+/3Q7v7zc60vbTlzTUKBYGrXnfPE2a7T64mlNZqrxWYAYMbWTtbZw5S/ENNmNL1RvHvZJ4lRDlQSGab2CJfvQi6kFygOPoqC48XQuc8ZhaK0V2KDdaz/LsUGG9jIvMc9yfrNrZekh zhflcCIC wf+3Y7Es5/0uAf8vf+aboRgB0AlMPTMJfL9sMWffSpot9Ok6q9IFYM5Qxjs/jf4oxDB/ev5L0RadxiJ47KHVV+C9Zmrj3ZO16FHHA0w0AqG6kP5Y2yOXKzAOeo54l21NXsJHDhkrbHC5fRoxRA4H8Av5ZVhYWQfI6OES3HNS9VnjqXI6v7C/6Nh/kcQdDGRIvQTEN08Qxg7RChbQEJTlDBHmneNXgsaLKMvzLOI7cDHI5WcMLy9qzHoke1bCGd8lqAE9XQ24AnJwr1AMYzIoeGIpmOwVD3wNlmCDvMBM4cdvjvEB0/bAFoMTZHOxrONSgHAvmyCZds/Qo6k4rgmNu6hL+FXrmZHYieXt3/ucx44Qik3pC3O2P94TX8JTsnUH0CMGn X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Introduce zpdesc_is_locked() and convert __free_zspage() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 4 ++++ mm/zsmalloc.c | 20 ++++++++++---------- 2 files changed, 14 insertions(+), 10 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 61d7ba0c626e..6de3c5d095bd 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -117,4 +117,8 @@ static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) return page_zone(zpdesc_page(zpdesc)); } +static inline bool zpdesc_is_locked(struct zpdesc *zpdesc) +{ + return PageLocked(zpdesc_page(zpdesc)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 6d1971836391..68fdea7b6e0d 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -901,23 +901,23 @@ static int trylock_zspage(struct zspage *zspage) static void __free_zspage(struct zs_pool *pool, struct size_class *class, struct zspage *zspage) { - struct page *page, *next; + struct zpdesc *zpdesc, *next; assert_spin_locked(&class->lock); VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness != ZS_INUSE_RATIO_0); - next = page = get_first_page(zspage); + next = zpdesc = get_first_zpdesc(zspage); do { - VM_BUG_ON_PAGE(!PageLocked(page), page); - next = get_next_page(page); - reset_zpdesc(page_zpdesc(page)); - unlock_page(page); - dec_zone_page_state(page, NR_ZSPAGES); - put_page(page); - page = next; - } while (page != NULL); + VM_BUG_ON_PAGE(!zpdesc_is_locked(zpdesc), zpdesc_page(zpdesc)); + next = get_next_zpdesc(zpdesc); + reset_zpdesc(zpdesc); + zpdesc_unlock(zpdesc); + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_put(zpdesc); + zpdesc = next; + } while (zpdesc != NULL); cache_free_zspage(pool, zspage); From patchwork Wed Jul 3 04:06:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C5F1C2BD09 for ; Wed, 3 Jul 2024 04:02:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C23416B00A7; Wed, 3 Jul 2024 00:02:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BAC796B00A8; Wed, 3 Jul 2024 00:02:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9FE496B00A9; Wed, 3 Jul 2024 00:02:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 71DDE6B00A7 for ; Wed, 3 Jul 2024 00:02:01 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 39F1BA24C9 for ; Wed, 3 Jul 2024 04:02:01 +0000 (UTC) X-FDA: 82297093242.17.C4548A7 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf11.hostedemail.com (Postfix) with ESMTP id A138C4001A for ; Wed, 3 Jul 2024 04:01:59 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=gVOc+ikQ; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf11.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979302; a=rsa-sha256; cv=none; b=PFEY2UwQbWtNK1R1RPr1cCOaEalhVS5wZCpDP7SFWjcFz2EGaZMD7oziJ29TwDl/k+Dnv/ cDYttgqZ9rD1ZnbpeBIy8P8TeYj0sjBFX4BMLM9bxX3eS2ZyBHWSZTFo9leEuqPFkmpx/x kVN4w7UrumvvNRpJyWBuhxzAaLezyxo= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=gVOc+ikQ; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf11.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979302; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wXagLVGUZ9eebRl/j1Zz+wClVG4D6z7RAED/mWVOrM4=; b=hmIkb9IWTHG/Yru6hnUUKMhzxh0tMgraHUhiFM+POE2YtZDZ5hC8bwWh2fLpXkTAEW5PQG PB+F70Cr1sPCoA/lxy5KEOzBxGilD8y3cyCQf3+X9/PycRCY55eIZ/uae6QsfXN+i6DWnV s113428VCDWL3a8Ra53nyfy7uLrARFA= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id E818F620FB; Wed, 3 Jul 2024 04:01:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 74B10C32781; Wed, 3 Jul 2024 04:01:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979318; bh=wTDI0eYHYt7QQuqfYVSh3nHEsVuSO77v1MHzFFaeoFg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gVOc+ikQ0e4uEf4lRf4VFTkicUepq4dYTbwtWmScFz/shEGC3tPJ2Tv3kp+jVx39J 02cNh0BzDrMvoMSkuZTHsBlPZovMNzYKU9VGCsNc17yGimfEVfQQ3CNb5lzuXAIK+c vLBE7IJcOkdN1BRsM4pIX8l88EuGTlaSwpUqFmKj6l44HiFqFjGNl68SmrkQ6JlwlG YEv7zNYxkcmEBnWJ62sBfJcYrEnUJwxuyTJf2Kb6E3+qL3UZzdAq666Nr3NlRFnIgS P+qbUaMur7AJrvXSuEPhRwv5psWkuwXQGkQc7n1tj5nhvojIeOMofVoEn+JGscY7nv qedKqn8ZvIBgg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 13/20] mm/zsmalloc: convert location_to_obj() to take zpdesc Date: Wed, 3 Jul 2024 12:06:03 +0800 Message-ID: <20240703040613.681396-14-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: A138C4001A X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: dbjarntndzgikxj3tb6t9p8un39wfgea X-HE-Tag: 1719979319-985193 X-HE-Meta: U2FsdGVkX18bpWYbIlO9QcAX2r/oNLzCtPYVDosus453bAIN4xi42PU04pDaLaLGNCRT+cSZ7N3J/TwNkpv5MVNmtEtgmDpRcfgG7JfPBdgFHrQpipP0Nx/79LPuGDxWO+Ds+mQki//JIjiR4qu3DFuftYIdj9RESl9RF//NzQyQ1NUu0webyUkkBu3+1wb0MVpmVsM7U+YNjYUpEIuam8HqGmdkfvjC/5eses/qll08yH8pW9CsQlhNaM74L2Ge+XZZoXv2810UAlZo5+6aQaL3gAkLUct0P7Lf9edeAdOZZuBl4yBokZK5FEXcVx55SMBO1UnWJDfRi8+HZTFAjJKovd5M3RM1tC6uA7fYwAr5GuFuMGM6Gtp+OQwU854QB2+dKUn7cu9pzD+zRRgSaicBLsGrH/12MtmnYnG/JwPeoNpTbnVBUSWPwQeQ+JgK8MCgy8R0kJHwH4f0ZxiI0agD0Sn5IPZApi/XRboRdMNlvWFriAQ8TeAGuEriOQFUhJl/wc9aLYp9hiK7n3E8d6Dznc6BViFR/z7SJjuyfrald4KaOwHkK3/RlMf4AxlEpvfvWh3HNi1hzll1tJ3svcUe5KQ26Mik6lqjAMNb1dM9KlweO72QElRWRkVeS9RFq5rhtW3gAEHTYLr0TeympCXUaJ8FnGQ/6YSFp87BDUfjozwmuMi6QWTK7a2w1vt/WlcCuiOpDKCCqjiPrQBxCyPcxMsxl0E//6kHreXYtFsczCH03tCTRQN9pmX2+c0kwMqV2ijdtpm9E/8+3HbiUZuGLjaa8g1lNIAUqk13FHpz3tKMWMhuDynjW2aUM4h1Q6s6B2L2oaUeWOF7t7cTIeADuYrQtESQwEXPIBZSFK2vigP740jN0KC/vJ0oy78qoLqHSXkV2baXzRJEJeQJdHlqooi7Q+wPau9DIBpogy0uK7DbSmHxyxiDzFihsldcSM90GIWuc64ps0PbNyo UwlDA7jy 4VIDf8bBEfxXBVSUh6C9f+0BGtk1ZxCaawKzBykbwenwEcB+2ysX2JCGnsrIaW8/e3bMiImAGcwRgKJme825RDSJYd8ZasQ87Yef2RRQjzcvIoKJef6diQ8eopqQV7syLuFrMDlXnycvIEdB9l4YboXq4QwK1YAAtEtIFQQMYMLFM0LuSgaTLFnNxwjvv0lYM9GSawVic+K5nTkWz8WBpI166DTUM1pfEes1dmI56nNRskxJUAz89K7/ePz8HQGnSFltAhaCSOD6OQSy0DcVbcDuOXn8n1cPy721pmkmxY0Ked/cjktojev9T79dS1Wymfd601sEJ+sNwWSSFvBwK+5KidSzEiqjHaclXkMCm5JuLouo6htS8zpnZo7ZaxgRHdBvP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> As all users of location_to_obj() now use zpdesc, convert location_to_obj() to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 68fdea7b6e0d..e291c7319485 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -826,15 +826,15 @@ static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) } /** - * location_to_obj - get obj value encoded from (, ) - * @page: page object resides in zspage + * location_to_obj - get obj value encoded from (, ) + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static unsigned long location_to_obj(struct page *page, unsigned int obj_idx) +static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int obj_idx) { unsigned long obj; - obj = page_to_pfn(page) << OBJ_INDEX_BITS; + obj = zpdesc_pfn(zpdesc) << OBJ_INDEX_BITS; obj |= obj_idx & OBJ_INDEX_MASK; return obj; @@ -1392,7 +1392,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); - obj = location_to_obj(zpdesc_page(m_zpdesc), obj); + obj = location_to_obj(m_zpdesc, obj); record_obj(handle, obj); return obj; @@ -1879,8 +1879,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj = (unsigned long)location_to_obj(zpdesc_page(newzpdesc), - obj_idx); + new_obj = (unsigned long)location_to_obj(newzpdesc, obj_idx); record_obj(handle, new_obj); } } From patchwork Wed Jul 3 04:06:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721084 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA59BC2BD09 for ; Wed, 3 Jul 2024 04:02:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 545A06B00A9; Wed, 3 Jul 2024 00:02:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4CD876B00AA; Wed, 3 Jul 2024 00:02:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 346046B00AB; Wed, 3 Jul 2024 00:02:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 111556B00A9 for ; Wed, 3 Jul 2024 00:02:08 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BBCBE1C1787 for ; Wed, 3 Jul 2024 04:02:07 +0000 (UTC) X-FDA: 82297093494.06.9102DE3 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf25.hostedemail.com (Postfix) with ESMTP id C1183A0013 for ; Wed, 3 Jul 2024 04:02:05 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=UAi8VdfB; spf=pass (imf25.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979302; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C6FyzEKOSWI6MCvUbbMT9Jgk3QqCa38I6z6Mp91Wydc=; b=foq4mGcJWCSP1oK/3wwC+En/Bn0NI1CBkipUtebpen7gu+NdZe4Li83lt9wGVxkUFmUHaL xTRHHmPYLYMPCa9mj8ypIZjtZDbKd3MTEKY3GpHm2coDdZyhHE0pH7eqpWuUBnLHaLhF6V AwBmya50eLydUXOFUQo1ecczkCuD834= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979302; a=rsa-sha256; cv=none; b=eHe4ZYwo/3rubxGH982aieTUcbQbcRUgdSKUWAkCN5IKbn35dsvDw8I3qTg7r+T4DxTNL1 2SlaXzEcZ/BcIRiVkn4DfERYp07VEy17kVg1aB5ZvNCyhMosO/BgzQIVyWssubxOXrydFj hIDmr4yOOv8FGG+/J3HkxoK3oc4OGE8= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=UAi8VdfB; spf=pass (imf25.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 37D77CE21B2; Wed, 3 Jul 2024 04:02:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6D940C32781; Wed, 3 Jul 2024 04:01:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979322; bh=K3GUIiHemMLy8fa8RNH072Iq/pXEu/mTjv8z/3myXkU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UAi8VdfBMIZrg0juq8txVEnN9W0EIBUjwpsOIiZcXYiKs8oR980Yy+uHO+fcF89px gU3DFwUpEt4cZuy68LLq7XCBh43WC3tWvg/b4RPgShtXhVSLkiqtHB6Cw08GmHBaVy t6TqfuihXAZJUFK2cGbb3I5U/kDa7LC25OR2aSeCr9jAJietiU90ZjWEIYzzKKbxcN IWqeUfxpYZiX8g7ytLWs+EQNJ2gLMuYGRelGem7bQ9Fgma33GDoN8rvUrqdybqMNJr Ao86j0XjV9Ea2MaRgEYnIk5y5CDjM9z8tG8mHiz0IxMP9HeX7MTYQa268tc5x3zpae OkY1Nxw+l3/6Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 14/20] mm/zsmalloc: convert migrate_zspage() to use zpdesc Date: Wed, 3 Jul 2024 12:06:04 +0800 Message-ID: <20240703040613.681396-15-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: C1183A0013 X-Stat-Signature: 91bhiupzqaheahgdwh51inb99jh4odep X-HE-Tag: 1719979325-10968 X-HE-Meta: U2FsdGVkX18Dvg5Nimtkm4ySLQbZljxuZlJjhX7pJ8c0kkDd+boizGrmPw78HwjE+SYELzkXdXnq8y+xmdbEDKgRBUiZtIlu6MjxrbOY5xcOAA3PhPpuI/4GaHYi6gYVxdD6Fyzy+ffmpoBa0a3boW2AT6EzhJkdQS+MW9SWFn+i9My7GgSKBBxeQnZUC2zbsJV8gMq7OgxsuOSTJxLdT2jlC/u6IO/ntLt6N1u4Lg7ViendFIJny9AFPlV1ipImk+6ILM3WpGy9kZ68dV1BNLOb2mxgmgdd2hIQhyQz4+SNHVx/oTDy1r6Jdor9pu1fNU/Lh489z9aROttJMAUw8N0Mwz4ESnbeDFoumRKnv7nX3/nryC1FWAOyAafD5GNXxc6mmMpCSi4bIYmqf6wvNgkjrGHP1WGxrcijdA6LAixophPJSJbb/J4c4cb3yOhPtWtO/RY/8rJPsBkFZ3sBCdbDpB8l5r5y2A0fJdj4r1Bnzw7amC6WY5HLKlWVY7zUGNXdz6bL5007kOsEVPrm3VHO3n7/ZRv3mT/suCYSFCINkmU8w9rR32NUJ2tEltnBBsxTxo4ycWFxrNQbSqFf4rILG8ggECKAaYddvtxo8pJ1FT1CAXzRg+OwTre0SpexZav9ATSKbUi7HvrQ3nAcG6/8f8gJv/WBGF0HhkefAwKRDzP18ocPBPE6kSJkwEcIG/CiXnE68GiusQ1olYkLj5ADGYhmyPQI3y0i2vTCTqO7MsBt5T9sOq7apnPYedhNXUOyib2T4MSS7KedOrqWlIV+mWxWLweqiESsvfTc0PDXQN0y8PXxqEXP62QyhOTWVsZ0zurnWFH1JFuGj6OFn3fqx4CxxCUvd5ASOnv/RuFtNProi1yupbzFOcFlAJNkjxqZtiBIi++r7UhiuWWbq5kjp2Y36KLQJsNnO7DoXk0LJLrXOOD4+OfJxQD2FFAwkQ8ZuvWzqce9R1AptQf 7wOjKb9z 5mo2i+LDXwWQ4hWYtmfZRwgwV5rp5+poYjnITn5UF16G0Ijo5y1oMK79Pez55fyuSxExulQqWml2y8NhZkls8KSLNf81ZR0Ym6OKcHnEi+okuvT19U1g9fU4gXaomSlweOVppQQe9kf5b3phinOuErXUPIRLBfNfFjZ0PYSpynF3zVT7ecKqqJLJ6zw+xoP3vbyslKf2wTvuPzwNQNuOlch4ls/lnMLMCVpPfaPq5oTUk7U7p+3o+OeUSWFkxus9tw50smcprnRgwz8sLy0/CoVldxSb9m/BPI8Dwd7g4DumHjcPojBig/MaaX9PLbZhpcx/FWOCe1F6VpjVATcuhCsLQxAPurnj0GTdHIfMF+zi6zgbDfXGZpfvRgA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first/next_page. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e291c7319485..7d039b0c66db 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1636,14 +1636,14 @@ static void migrate_zspage(struct zs_pool *pool, struct zspage *src_zspage, unsigned long used_obj, free_obj; unsigned long handle; int obj_idx = 0; - struct page *s_page = get_first_page(src_zspage); + struct zpdesc *s_zpdesc = get_first_zpdesc(src_zspage); struct size_class *class = pool->size_class[src_zspage->class]; while (1) { - handle = find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); + handle = find_alloced_obj(class, s_zpdesc, &obj_idx); if (!handle) { - s_page = get_next_page(s_page); - if (!s_page) + s_zpdesc = get_next_zpdesc(s_zpdesc); + if (!s_zpdesc) break; obj_idx = 0; continue; From patchwork Wed Jul 3 04:06:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87227C30653 for ; Wed, 3 Jul 2024 04:02:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C7B756B00AA; Wed, 3 Jul 2024 00:02:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BD2316B00AB; Wed, 3 Jul 2024 00:02:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9B0186B00AC; Wed, 3 Jul 2024 00:02:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 762666B00AA for ; Wed, 3 Jul 2024 00:02:09 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 2ABF6A27C3 for ; Wed, 3 Jul 2024 04:02:09 +0000 (UTC) X-FDA: 82297093578.12.6F9D2CE Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf16.hostedemail.com (Postfix) with ESMTP id 89DD8180005 for ; Wed, 3 Jul 2024 04:02:07 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=XVwqKhxq; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf16.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979310; a=rsa-sha256; cv=none; b=7bAzgJKSLUhWqqU7D3wcff8WTXJh681NsLlNS6jeCqaop69ddfFOpjgUUioDPfRd6ls4Qf COekU0nG5NlTw65Ewrkgj/+z5vGcA54QBk+UEHhrx7f48jOqeZAF+D0Ix/mZAmneOkSR9O D8Z3pGa0AH5ZRQB6hrbkR/T+Jaam/NU= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=XVwqKhxq; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf16.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979310; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Q9lrHJyVnJJavlQtfTrhM6Cg7QJsRwBhFfAPgSitRB8=; b=Ei8uwhVyZmhz4uStZzqKDGfzol5ig9/Gnw+PJA7jmQw02Y29oTX7aGS4YnegHOhpPLE6RZ B3MS9rPTttU3aevvIqH2XZzm6NCRGiUoOsbs/agNBSE8QanoDQ0c7ZCL/DZmFEUT6JJ8Un /Xpb7AD28sWR6HXlxZLIWF05rnk7Zpo= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id C4AA1620E8; Wed, 3 Jul 2024 04:02:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 369CFC4AF0E; Wed, 3 Jul 2024 04:02:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979326; bh=0AvWtekA1fBYWEoFvuJjRv8WAqdxESdkBRdAezOabRE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XVwqKhxqJJIYI4ImJo3+psdvnBbfEXhEGJDSfDKi962ESzTtF5/hbZ2P7VODY/3r5 4I4jG8LqmmhAtzostIdPRwMaxNugcJ7W3Lw+obRV2bqbBj41gU9MiaOVMBBEmkrvAP uUfWcaK1g6MOHtA4amW+Zy5h3GHC9ilT1fvyNp4vMZF045n4UCOPHLdeRlWk1ATdJf ZeKKavWcPN4rFm71clrnhy3cPlRVjb7FwjSU4mIeC/PyYBtQN2Mj0FXTUNviqAhJYi 2GI9UT/7KubfIvR8WPuJlMYEUEnCU4gkEmis4XUM0cBgldgN6W+/DIW3ub3foDmaZT kRSS3fBZLRMnw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 15/20] mm/zsmalloc: convert get_zspage() to take zpdesc Date: Wed, 3 Jul 2024 12:06:05 +0800 Message-ID: <20240703040613.681396-16-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 89DD8180005 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: jqj4wegk9wrbijog4t5fhmazwmg63sk9 X-HE-Tag: 1719979327-722974 X-HE-Meta: U2FsdGVkX19CbyQN9oGSgqZvws1Xy64y/m4PLXqHS2ekq0ixpHvPZhUoCZT8Vw4yaYCJTLKk+lY+zt/698mWjrXM6whGm7rT8RnRjC0+7S0CjOp1L9USMinsMYK8LFdJf29s73gJli6cYIHk+8yP5CJR0ZW7IB83qZa1z50aeSOBX3vJtO5655BBkRp1jAV/kcnHU0DA9jUKv6wY9jhEqBFSXm+9FckhkU76WWbAwztScHj6AW1FBQ/HERka+kupK288P97qmUHx1mhxSga0fZJ4DqkWqpbL10RFeqaFlilYoO5ks9VcVr9ywGvoxqPJBDzEedsg50obCPxWD3hp324O1Hjbdt5jc4RU1raI6LbaNVsRaWSxlFEi+vKwivUXw9g5mSsNlnakEkdXRzlFap67Awu1laxl3SeOsTObOCb5Iu8VSaPWu3zNuMXQBjwYLUyMV3ZGHIC27O1btS+5FSFV+bHt2tsZtPxexzqkV9zxPILk9a+cQmloxddkLuw45bkPQqD9Vd26gB1sBWS3p6lbZgEa9JhP/pOi/6JCqo8mSTrJpmhsmnVKhLKcyD9ahPz/uoavCW0lrQKa2TNeNj08pvVllxy/hLqwVBXxMCp0gz+vbgyI1ziv3ALK8Ijd/jIr39+YcJshJE3vuP3K+mYrsLX0KqL++LYParSKyUEOSDtFwasJyYHVHsRJj4SGeurtQH1HzQD3+MpTqkXuiNeaAjxwdBpsoxk+uBzRFcuF+XW1a33vLWk6zJFA32obsUaIud4BG+OPa5SUfgtZzq7GTCe2eGrf4tBkqU9sMilsXYArnzrP1gqvAhSR24WoAklNtvRFrqd56EHXwwl1oxbNiM0UtlHXSbuIAPLm9u7Pc6lFU87SQhbeRVq0yQZcdtxu6wNs+7qIfgCFCqCuqfByamthr6tn9TwRLRb/AlyMs9rPAPJURK9iSp0RzKuUocxXaX0820OWjlKee9M GWK1nPwq 3olHoHC6amNvmzTSSlbEiySL5E/vj3GwOamOphU3EC1ujWYtLcOY6RdQaZvkybfhUuEZtN/+npIUhHhkddGJMrqy/sWTRTKHVexXGhsAeFKZJsd3GzLajvglCx4C7DquDO3CRk0RrTuTqvgfcmSAE25b5VpVBJFCAiEKS72vxdMuTU2vDNA27E1SO+lutpAT1iQ/2sjLwJcuTkjd6sb0CgPLCCePFDDB4QcJkVt7CEbw4WGqV19ezPplvwzJvXl5XY/FgFhDBQiU5QQ5keAtmQCAGoXZW3mUm1hMX5CIOix/mdFDyj0wzsG9C6Ugi0HQcEWb27aBHO7+LwUtGB9t+dOfqKPyjOogceXR7te/kdqOnm0sDK6ayqPybk/DILHh/l4lR X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Now that all users except get_next_page() (which will be removed in later patch) use zpdesc, convert get_zspage() to take zpdesc instead of page. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7d039b0c66db..458ad696b473 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -779,9 +779,9 @@ static int fix_fullness_group(struct size_class *class, struct zspage *zspage) return newfg; } -static struct zspage *get_zspage(struct page *page) +static struct zspage *get_zspage(struct zpdesc *zpdesc) { - struct zspage *zspage = (struct zspage *)page_private(page); + struct zspage *zspage = zpdesc->zspage; BUG_ON(zspage->magic != ZSPAGE_MAGIC); return zspage; @@ -789,7 +789,7 @@ static struct zspage *get_zspage(struct page *page) static struct page *get_next_page(struct page *page) { - struct zspage *zspage = get_zspage(page); + struct zspage *zspage = get_zspage(page_zpdesc(page)); if (unlikely(ZsHugePage(zspage))) return NULL; @@ -799,7 +799,7 @@ static struct page *get_next_page(struct page *page) static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { - struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage = get_zspage(zpdesc); if (unlikely(ZsHugePage(zspage))) return NULL; @@ -849,7 +849,7 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage = get_zspage(zpdesc); if (unlikely(ZsHugePage(zspage))) { VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); @@ -1265,7 +1265,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1315,7 +1315,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1479,7 +1479,7 @@ static void obj_free(int class_size, unsigned long obj) obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); - zspage = get_zspage(zpdesc_page(f_zpdesc)); + zspage = get_zspage(f_zpdesc); vaddr = zpdesc_kmap_atomic(f_zpdesc); link = (struct link_free *)(vaddr + f_offset); @@ -1513,7 +1513,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle) read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); obj_to_zpdesc(obj, &f_zpdesc); - zspage = get_zspage(zpdesc_page(f_zpdesc)); + zspage = get_zspage(f_zpdesc); class = zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); @@ -1846,7 +1846,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, __SetPageZsmalloc(zpdesc_page(newzpdesc)); /* The page is locked, so this pointer must remain valid */ - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); pool = zspage->pool; /* From patchwork Wed Jul 3 04:06:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721086 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33956C30653 for ; Wed, 3 Jul 2024 04:02:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AEADC6B00AC; Wed, 3 Jul 2024 00:02:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A71A96B00AD; Wed, 3 Jul 2024 00:02:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8C4C56B00AE; Wed, 3 Jul 2024 00:02:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6A8A66B00AC for ; Wed, 3 Jul 2024 00:02:13 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 2FBE4A072E for ; Wed, 3 Jul 2024 04:02:13 +0000 (UTC) X-FDA: 82297093746.11.ECA0731 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf17.hostedemail.com (Postfix) with ESMTP id 92B8F40018 for ; Wed, 3 Jul 2024 04:02:11 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=iPsY0Brd; spf=pass (imf17.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979308; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kTm/sX+DW/w9pIMx9ScGftKWcPwLIh0BNYGkIGj7FVU=; b=q5lqyxu+OkDG6O8XL5UL3n8sGEbM/d5VbMclRj5mWANjiTnvcEXwrfw8JfG1Gi2fMu9Iot XYTXFW75FGC4FjQTD8mA9OhcYuuMkCftkVA0HGFplV7ds91uPZ1sK8V8IlXUEX6zjpU+dB JfqPLJ1w+0aNgICvYsSfDzZrOUagYqE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979308; a=rsa-sha256; cv=none; b=vWkIkB6iPJF9vzlhlAtdQ51URki4ZMNMKgd0fJRjwX7JzWCvgTv2UOoHxQPNKgTeC5vpp3 hGApKV5g42XdPRmvmgEnZXYMrk2qR3xZVyrgC42lxLhmqkihixHvPwS/TMUQMJ/Q7vtYsU 2nTqqulgz+b21xZ4UAPPaWRNcV2ov2Q= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=iPsY0Brd; spf=pass (imf17.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id BE3A0620ED; Wed, 3 Jul 2024 04:02:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2E17BC4AF0A; Wed, 3 Jul 2024 04:02:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979330; bh=uH0rZNnpHFMljEmSyKQuSSlLph2dNP0xTWvseJvNDaY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iPsY0Brd5KWtyIOk5fV/0otAMuF46d4KG5B5pddwxSDNlJ1E0cbaAnUHd208Zws0d 9GHVhVF88u6OFKosO7B9jELp3CHwrLGdn4h4C4xb34PtLyYwVbQV+IsjoAXADnV9j9 ZffY97ScQAwMreWa3NmKgcpzng+WKLp1IgAjr1hdxnh4yioFj0tdWX3klI0o5/wT3s XSD0p939k4/5df5GCsmd/HriGXYfyFLk0zojxelaHRW2zZ1egLtRTBjYri2bNAjJLG 3ywNEDpuwtzOPd+oo+1gxRI63Ppyldz7y+1CeuJAmASlFlF6pBtMSPY1r+x5RnQZg2 zkf3T4L1vs6PA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 16/20] mm/zsmalloc: convert SetZsPageMovable and remove unused funcs Date: Wed, 3 Jul 2024 12:06:06 +0800 Message-ID: <20240703040613.681396-17-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: bi8djptgndnqnx9agnzj4mqryghy1ybx X-Rspamd-Queue-Id: 92B8F40018 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1719979331-566427 X-HE-Meta: U2FsdGVkX188cvDEv+UvosmrJS3w6DtLMMmmbKv8z2iyKMGHugNG8WLg/IkEa/pu9jhT52LcaLcyfAHKPRwC2XqYrx/zJSpZSusHSFB3hk6W6WryoCSKPrPYhwYObsYr7WOY63XdcF3FH7kU1WtNSyafy+hzBd/YWVO25MpZCIBu8qsigwcZnQtyvSiY4TjYZ6zgAHjgUJrZfRjEeubeytx8tC69qWW+vRDdk580PlgqM28bge3xWEyglb2xG6eaxvwyaKWlV8YWBMMpaL625LcodoslH1oHGRTWFTEAdXb0X15UcYigOhHeWWvQQmIPlfqScyhTfi/q/TA1Fvuifpnwic10TZ1PDYAMW6OTpqpqy0epedp3PYgLZspZ7UcZW2BiZVp2BzeT1l+2+XS3OCXOoll1otnyWRwkfEFKW5q3pmodjMzyIm92UxjFKTb77Cab7ZEK/woZbBMVA0Gwq2PQUAfRm1CmVjPqSeKm1/6NUG8VV3s3CNWs2G+Y4CQLNTGQHv6Yj15mZrqJ+IpLHZFLj/ITt+3qoJWCMwWe+NR+yJNXKJQ96gWwQFBzT+gOaBcNI8UEuR+0vrV8PM9+/exGG93G2Lh/1qhJedlXjf/d0F+d+gj1bgEif9i5DovlXLUuLzHBsHE0zmZJvsJ9Axu0eYeC7CwbBcWoOCfI/AGqXhY4SgUjamBxNF7Np92khPBnCiIAqcM+iH8xXPAs8mLcCqxq6LlmHE7Pw2diCxiJ6dZuDq8KPBTU6tng2Iw9YWPR2QNpoZQqZbDgtya3aYN5VRDSLDzVS++MKtWMM4rhN8Mazn/wyMZbY3SFEKnQ6dXGOAFEPN0adJDjNos7AqJU6UiU1f3/dByiEVBbwjm/h8bX+7nBoKDd4g7yx+HkWin0n5z3kkJcW2cAXw7eyqQoGTC8LP0sITObTKCS8NSdtCJdmmlil3VTV2PgEZE+AFCWSxynZ2zXy/Al4Do 0iNczibJ nQ6kBAyD6+J73GUKl039ZQXSD+F8n6/asSnfbLQA8T1FoGmC2KQumCaHXnwDKeLdEZvY2oD5V4DYWyBilwgi4Mnya/ylje7eUNPnQ0aK9QEi64+ZLXiC2nX6Wrmmh4KSzUW/Ezj7WC1tUtLgR4M+9CUIJlige0Wnyal8jROzOV5geIvtcXjichirHX9E0rShFFZldEI/O4rqFOkXb5l15Fscjux53jagIdsat0MmsAe7wG8jcjmvjqrIwHEmAZe8xir5V4ALrslWnjLXpgYfrGB9AtnSfQQxUOSc4ROFMtZf799G8/pMSqB22VtzgvdLNejnSO3l2t9GTksw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Convert SetZsPageMovable() to use zpdesc, and then remove unused funcs:get_next_page/get_first_page/is_first_page. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 33 +++++---------------------------- 1 file changed, 5 insertions(+), 28 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 458ad696b473..8b713ac03902 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -468,11 +468,6 @@ static DEFINE_PER_CPU(struct mapping_area, zs_map_area) = { .lock = INIT_LOCAL_LOCK(lock), }; -static __maybe_unused int is_first_page(struct page *page) -{ - return PagePrivate(page); -} - static int is_first_zpdesc(struct zpdesc *zpdesc) { return PagePrivate(zpdesc_page(zpdesc)); @@ -489,14 +484,6 @@ static inline void mod_zspage_inuse(struct zspage *zspage, int val) zspage->inuse += val; } -static inline struct page *get_first_page(struct zspage *zspage) -{ - struct page *first_page = zpdesc_page(zspage->first_zpdesc); - - VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); - return first_page; -} - static struct zpdesc *get_first_zpdesc(struct zspage *zspage) { struct zpdesc *first_zpdesc = zspage->first_zpdesc; @@ -787,16 +774,6 @@ static struct zspage *get_zspage(struct zpdesc *zpdesc) return zspage; } -static struct page *get_next_page(struct page *page) -{ - struct zspage *zspage = get_zspage(page_zpdesc(page)); - - if (unlikely(ZsHugePage(zspage))) - return NULL; - - return (struct page *)page->index; -} - static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { struct zspage *zspage = get_zspage(zpdesc); @@ -1970,13 +1947,13 @@ static void init_deferred_free(struct zs_pool *pool) static void SetZsPageMovable(struct zs_pool *pool, struct zspage *zspage) { - struct page *page = get_first_page(zspage); + struct zpdesc *zpdesc = get_first_zpdesc(zspage); do { - WARN_ON(!trylock_page(page)); - __SetPageMovable(page, &zsmalloc_mops); - unlock_page(page); - } while ((page = get_next_page(page)) != NULL); + WARN_ON(!zpdesc_trylock(zpdesc)); + __zpdesc_set_movable(zpdesc, &zsmalloc_mops); + zpdesc_unlock(zpdesc); + } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); } #else static inline void zs_flush_migration(struct zs_pool *pool) { } From patchwork Wed Jul 3 04:06:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36EF1C30653 for ; Wed, 3 Jul 2024 04:02:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B88466B00AF; Wed, 3 Jul 2024 00:02:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B11576B00B0; Wed, 3 Jul 2024 00:02:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 988F26B00B1; Wed, 3 Jul 2024 00:02:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 723536B00AF for ; Wed, 3 Jul 2024 00:02:17 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 21AE180714 for ; Wed, 3 Jul 2024 04:02:17 +0000 (UTC) X-FDA: 82297093914.03.72E8BCD Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf27.hostedemail.com (Postfix) with ESMTP id 78F7240013 for ; Wed, 3 Jul 2024 04:02:15 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=TUNlomcQ; spf=pass (imf27.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979304; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nORsaZlIaUA3idoDw8T4vH/VEYYd+wWIyfYZsPkiZ18=; b=zmh303lC6CbMEDWa1cD+8R6J6zu2Y3D4NdMQL2iHnydOO62v/7Cp4Ck9yYPdE2ZlL8Zl56 zj/0yNfpE38EJ4hTxdb+ToLkvc8hjIUlCs9N+JqVNuItz1rg2YZnAcPYOxDncknOxrWtm0 wFo33LcTL0IFFpJKv/7h9hDGLK8htSk= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=TUNlomcQ; spf=pass (imf27.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979304; a=rsa-sha256; cv=none; b=DXz5DDW6fGPVX/ldomJ5HBfiSPH/Ywztiz4BCNSvXQAFBCtACWeKaIF6WimcjEAEYzNjQw ZvStFvQX9Urp2AcrEbDE8ss0+NkQz+t4uI50RoXoZqMV4ZEaQVfYWvHwd5WJro4bvrUBND jfv//FekV+ighCvMom6jwd3m6l7kyGk= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id B4EBA620DD; Wed, 3 Jul 2024 04:02:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2527FC4AF0C; Wed, 3 Jul 2024 04:02:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979334; bh=vsXuVfMJsSuMD7UFdk0WVE3RDVFvIl+YscGm1B7bHVw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TUNlomcQNEckWzE3pkdaNY/jWZemVDlbi9bxsdknLLDnG7+ZhTC6n5f1YKOBHPscF BhxNJlC+MBOjvNvdjcvvweVfGrQI08RQxthpm10H45a59hqsrL4GTJMugS9+1KtZdn 4M+riFrBTzpbHYxAIryEuoUqrDZckEejZCNBzptjOWLMeGv1EBzgq2ScxBviaaWrQ6 QWZDoYbgUQhGonitWAa/cUrUYW0++s9TjxH7obnaIBdamvY460PdXvBVLA6X1x6nkS B4DRFGqaiG9bq5kwyikisFoKGiRznLujqLzzynay9P0uddufucPKK3vVj2EgfV4as3 vtHSH2B65Q+mg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 17/20] mm/zsmalloc: convert get/set_first_obj_offset() to take zpdesc Date: Wed, 3 Jul 2024 12:06:07 +0800 Message-ID: <20240703040613.681396-18-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 78F7240013 X-Stat-Signature: qfmd5wewhedjr7wx3xb9i5cxqm3rdjpq X-Rspam-User: X-HE-Tag: 1719979335-84735 X-HE-Meta: U2FsdGVkX18cTUjOdozzpBxU8qphNth/kNMI4DTmYYc7Ggy/thNRIAG8RZdFaTFdJ9SKORxPZo24/LwBmXt3f/UG2TCM7P25lYgU1RFaFH8FW1wEIvGd/WJlu6/kCMMBsO9ADEY9lMFImK9iHIulvZH7RlfChUqhdMzU+x0ylDFI0t2wZNqUshiBICdujK99cgfKrPrgP0SZnu+45JtshUXCOj/FSnzu3izVa5+XgubysrDJ/AGg1ZkUMWKTGHf0zpQCK8NtBKtrKpw073yTfBhkJBUDyFjP3x1G54ksfaCiW8eKw7vLxxmrE9pMB1DH8S7kDqdjIc2c2t+fvKH696njh3OJK9p7aZKpwkNV0jOe0TTIj8JDzB9pZAigrg25psoGfY0Usy4VRTN/eMEzdT+lwlIeTtWSulgP0AAmOF9jR4DN3wuDC69CiONzgE+7n6fPE0PPOCkQ7OJRDN+gyUqHqKXwNVG23OdnN+aOrmLOF0BK0FvOIMIyYqoVT0vwX8Tr5MfEPOWaHkedJVh/BVkpaqPrbm+GPJN+TevgMK0KAYhmlxSHs6cJvJvGdlSCp41jsvyCH8lNuxDaLvDQ5Ra/AAA1L8Z1HYj04DVbLRVVsexzMbZbKS+64udfAoWd+1jfempP+ZwkSMpUR+kpvWFYNWIoD5PPEWv6dMKUXzrHBPqwVu7GVPLjskjYAZzW8Nd5221kg4wyyc4fx2fir9OOh/MWDb3qN/Jtyebr/PYS4OGhUKVl9IvDhX3QfuIZtuE1VjfBdH7IS+5qtQE1wYq+OO0/F/kAljnAnZpcHI5oDChz7eI4oPuuZ//exdP0kl+8QZHFtGSSoOzuYJvQx7ugShoJEuc4WG08awD96MRtQyJpxkbg7EdbsIZmCC4eaOi6WgIYnO9VU2BFOEawrbIpoK4M2jNMX1KpOF5fzU/YxGn8Mv4dnjWNTuBrUXU5MuEs+e0ov1UPZeqYwW9 3r0wlkr4 2DNFLXXDGYt4gpoq17JMXPff4r7ROTJpVSNnk8n9WASuX0LJLMm18VN+H00e0iwUpJILSgJNmn5jo2WQFb3mgTD4pbQbAot3ioKwEl0DIuyf1hhNuCtwFbGCsufM6JrUm+I9YUbKq5XZqUX0srIZL+Mn/JDLnMWaTY+g4OzkBekoRPnGrOmsgvbNqwz9gucbEoeQpbgn1laLQgMK1ecr9ViH6Wszm9sa3ZhuqHbWsoUTZerOiRQqri/OeaLg9K5Ek/wxKeDfHKOeGvTSXTKoz6JRIPTQVHZ3wn5P1Svcsw1WMAXLw5SrNgm4zvLZaBX3/henA+jtjf936LqLZYKGF/qsKaaU+lU15i8e/2ZcDxMRRo908nxD55apU2vaECrQm4ywZ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Now that all users of get/set_first_obj_offset() are converted to use zpdesc, convert them to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 6 +++++- mm/zsmalloc.c | 32 ++++++++++++++++---------------- 2 files changed, 21 insertions(+), 17 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 6de3c5d095bd..296cb1ff944b 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -15,6 +15,8 @@ * @next: Next zpdesc in a zspage in zsmalloc zpool * @handle: For huge zspage in zsmalloc zpool * @zspage: Pointer to zspage in zsmalloc + * @first_obj_offset: First object offset in zsmalloc zpool + * @_refcount: Indirectly use by page migration * @memcg_data: Memory Control Group data. * * This struct overlays struct page for now. Do not modify without a good @@ -31,7 +33,8 @@ struct zpdesc { unsigned long handle; }; struct zspage *zspage; - unsigned long _zp_pad_1; + unsigned int first_obj_offset; + atomic_t _refcount; #ifdef CONFIG_SLAB_OBJ_EXT unsigned long memcg_data; #endif @@ -45,6 +48,7 @@ ZPDESC_MATCH(mapping, mops); ZPDESC_MATCH(index, next); ZPDESC_MATCH(index, handle); ZPDESC_MATCH(private, zspage); +ZPDESC_MATCH(page_type, first_obj_offset); ZPDESC_MATCH(memcg_data, memcg_data); #undef ZPDESC_MATCH static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 8b713ac03902..0493f953fcb5 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -494,26 +494,26 @@ static struct zpdesc *get_first_zpdesc(struct zspage *zspage) #define FIRST_OBJ_PAGE_TYPE_MASK 0xffff -static inline void reset_first_obj_offset(struct page *page) +static inline void reset_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - page->page_type |= FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + zpdesc->first_obj_offset |= FIRST_OBJ_PAGE_TYPE_MASK; } -static inline unsigned int get_first_obj_offset(struct page *page) +static inline unsigned int get_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - return page->page_type & FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + return zpdesc->first_obj_offset & FIRST_OBJ_PAGE_TYPE_MASK; } -static inline void set_first_obj_offset(struct page *page, unsigned int offset) +static inline void set_first_obj_offset(struct zpdesc *zpdesc, unsigned int offset) { /* With 16 bit available, we can support offsets into 64 KiB pages. */ BUILD_BUG_ON(PAGE_SIZE > SZ_64K); - VM_WARN_ON_ONCE(!PageZsmalloc(page)); + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); VM_WARN_ON_ONCE(offset & ~FIRST_OBJ_PAGE_TYPE_MASK); - page->page_type &= ~FIRST_OBJ_PAGE_TYPE_MASK; - page->page_type |= offset & FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset &= ~FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset |= offset & FIRST_OBJ_PAGE_TYPE_MASK; } static inline unsigned int get_freeobj(struct zspage *zspage) @@ -850,7 +850,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) ClearPagePrivate(page); zpdesc->zspage = NULL; zpdesc->next = NULL; - reset_first_obj_offset(page); + reset_first_obj_offset(zpdesc); __ClearPageZsmalloc(page); } @@ -934,7 +934,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) struct link_free *link; void *vaddr; - set_first_obj_offset(zpdesc_page(zpdesc), off); + set_first_obj_offset(zpdesc, off); vaddr = zpdesc_kmap_atomic(zpdesc); link = (struct link_free *)vaddr + off / sizeof(*link); @@ -1589,7 +1589,7 @@ static unsigned long find_alloced_obj(struct size_class *class, unsigned long handle = 0; void *addr = zpdesc_kmap_atomic(zpdesc); - offset = get_first_obj_offset(zpdesc_page(zpdesc)); + offset = get_first_obj_offset(zpdesc); offset += class->size * index; while (offset < PAGE_SIZE) { @@ -1784,8 +1784,8 @@ static void replace_sub_page(struct size_class *class, struct zspage *zspage, } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); create_page_chain(class, zspage, zpdescs); - first_obj_offset = get_first_obj_offset(zpdesc_page(oldzpdesc)); - set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); + first_obj_offset = get_first_obj_offset(oldzpdesc); + set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) newzpdesc->handle = oldzpdesc->handle; __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); @@ -1840,7 +1840,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); - offset = get_first_obj_offset(zpdesc_page(zpdesc)); + offset = get_first_obj_offset(zpdesc); s_addr = zpdesc_kmap_atomic(zpdesc); /* From patchwork Wed Jul 3 04:06:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81BF8C30653 for ; Wed, 3 Jul 2024 04:02:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1262A6B00B1; Wed, 3 Jul 2024 00:02:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0B06C6B00B2; Wed, 3 Jul 2024 00:02:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6AB16B00B3; Wed, 3 Jul 2024 00:02:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C1DAE6B00B1 for ; Wed, 3 Jul 2024 00:02:23 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7872E4022F for ; Wed, 3 Jul 2024 04:02:23 +0000 (UTC) X-FDA: 82297094166.17.8EDA6E4 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf16.hostedemail.com (Postfix) with ESMTP id 79B6C18000E for ; Wed, 3 Jul 2024 04:02:21 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=SClcJWWi; spf=pass (imf16.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979329; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5+mq8l0P/h1ceq93DGl+UGxVgmYgd1vtsPsWyFFfnTI=; b=DctbWmu2rb0XDrZQSynBQ+0gWlcP7oCrjg0ssRVujBRSM9vnahaaL4Y3Ps36i/EYcbKKjT LqBKhDy0PkeAJSfXiryEcPlgkmLKLG1MBYOYFbDJ4hRQ4QQJu4Dh0WAz8HrN/kf5AMaUfe JHHMzp27YdGrZYwxxu+Zsgvf5NAtPWQ= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=SClcJWWi; spf=pass (imf16.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979329; a=rsa-sha256; cv=none; b=LV8X1BV1qzi+dY4y72XThtPqoUQoes+c08jkOZE734oCkhx9qoucZzcoSeetb9Q/Ujhv2Y yVXbykzNywQKkwmGKBabPpMZMeoH10NWgpdKPF22+QA4gGk2EqT58XCmodrXRzES+WRtle /KnFupN3psgkiFWcI8WfcDLSXpGdYsU= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id DEEE1CE28A3; Wed, 3 Jul 2024 04:02:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1BD1DC4AF0B; Wed, 3 Jul 2024 04:02:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979338; bh=WADQG+KFeKRI6MYgRuTdfJ+b1YmyjJ4toh2DCgrdz3U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SClcJWWiEa+5YwdOV0tQyYXItfPMvdIJTAnwNUIfop+4H0HQMX6ZlDZTA9hyInDUt n474nw2Xna1yNdCGIxwVeOL+AsXksKOajIh2vblG663lUPirNCunRmA1yrrPDSvY4t tAKnQr8OFrdsHf20r/qE1JXENe2kMo+E51gUSfzDg6a/UXszSkdWHn34wdnr1yKv1R kF12ECffDFlANRMIMJ2uT+QygpKzV52sLDqzBCGLDPV5aJ6Wt98lGqf5iak5vpEClN dcelnlI5gfXuQnyhUIPfGTQxg211qR96KWLBHvR4OwbGn8W4VGlyU6VOIxCFdv9j9B w2mMYKGx6JW7g== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 18/20] mm/zsmalloc: introduce __zpdesc_clear_movable Date: Wed, 3 Jul 2024 12:06:08 +0800 Message-ID: <20240703040613.681396-19-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 79B6C18000E X-Stat-Signature: 5zqrd3nhx4uthafixamsm8eraehcg73u X-Rspam-User: X-HE-Tag: 1719979341-448507 X-HE-Meta: U2FsdGVkX18wieW1LMDUjOC3P0KvNmVdxR0dwvMmez8ghd2z+Rpsq6g3EAGbvXHwVMrdwvCHYgguoJFM+MMjskc5yqVAC5iyZrUGIGSYVZGGRIUBQmyJKoNwTo0uJMpglDRYu1aD6poMOm+XXlFUViBQU6289P3Kgblp8KLCX92IuhbgH3cy6slHg9qnRLNC3phJP5DJKf+bhRkEWncMt+jKbgVZQlio3kr6tjXrBYmzPyDWjFaHU26sbYGYZekPg3RaFhlTXKqpQymAAU4j0KUdEeCbe/Ycbn+lNOJfCDzJP4ntULVMHUZfyAMNZCiBhOkut5FjEbKx8PyWK9fqaytmxnlVDJeX5E1IY57U74ggVrds2mqA6t8Hog50TwZUYLyq5DDskBIrTdkXspICPrppB2odXuAS4fVefdJU5Q+kOyGvm52J5Daq/vv5AtiFSgm2mBnRD29sqdRdHvHfIYorYRAkPe/Vy2KU+h7fhgJXaDoppupZYWQZ5CJsV8brMC4WmHz6nrHYPVZrosDaMT6apPwtDQzqJgAAfPm1uq//jUxKPZzrH645dDn7oEwO0Ldv3Huv/VCkBRVucjW6QlEHxL2376Z4+CH1ZOwCUU217nC2E5OpGnA8knoVdC2NiAROl/rT2VW/CoVvzxCXuqZfvYygwqqXhSt45KxwzH5lTUl4SqtFxCpXATOAROF34w3zvEnyqdRvdCXxlSsL2e/BoMWODTJTeUkkCMLSZ/rNtd8kOVs+VuwRcgr1zmuaVJbcjXUTTNf48peHhTBD2tzIZJXKsJo/gqiRDeL8V8aIEo+WlJnCdP7cw7VUmMFs6Oecs9dxr8PgO+rRIWxAZp8EMBP7EOu9i+p52elnAzqOWHekLvdxG5vLrma0I0FqdBCQDxFQuoQ7tztNBjuUz2vyXpDKmuSHa7ueXg9gfN0Z+YJgXDsMdwkh97yjVBMX2bHbF/jCFdbWHJ7arxT CfHK3bYi kiro6PXiqEsXzPEbTSaLGvlLJJatUc+qCdShcfgm49uwIYeC/zkF9tvrdochJb3mtoi92FwTmTiRMXY84f5wePdGEN/FdW9m8tx2mz+uI8bF63wvAk1+hxzyBIWWxCOn0WNLrXV/Q+1lt2MvMc2CIrdkn/xaUte65teRqyb1ZEmzCeZkogWly5b597ryJIh5y25DL+yUZsQz7FPA0sk31imAWSCBS3YaRvV0xYY4NmrkFf1r4tumTqxRCNrOtmMf/BAVKjUMC37iGPYqrgpNcMYxWrqD/ISEnE3zKJqdjac7ggdftSIthvmAXGsIotfMDlABE/3PVKrkj3SCVLw6vsiTKx2eeFXLsIRs/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Add a helper __zpdesc_clear_movable() for __ClearPageMovable(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 296cb1ff944b..4780d7015267 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -111,6 +111,11 @@ static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, __SetPageMovable(zpdesc_page(zpdesc), mops); } +static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) +{ + __ClearPageMovable(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 0493f953fcb5..08ab22f0ab7f 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -846,7 +846,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) { struct page *page = zpdesc_page(zpdesc); - __ClearPageMovable(page); + __zpdesc_clear_movable(zpdesc); ClearPagePrivate(page); zpdesc->zspage = NULL; zpdesc->next = NULL; From patchwork Wed Jul 3 04:06:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721228 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F0C3C2BD09 for ; Wed, 3 Jul 2024 04:02:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 014EF6B00B3; Wed, 3 Jul 2024 00:02:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EDEC56B00B4; Wed, 3 Jul 2024 00:02:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D345C6B00B5; Wed, 3 Jul 2024 00:02:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B0C906B00B3 for ; Wed, 3 Jul 2024 00:02:27 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7308B40786 for ; Wed, 3 Jul 2024 04:02:27 +0000 (UTC) X-FDA: 82297094334.13.2FB2B0A Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf02.hostedemail.com (Postfix) with ESMTP id 79DA180003 for ; Wed, 3 Jul 2024 04:02:25 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Rxj4KWb+; spf=pass (imf02.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979322; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6sIt+/gQb+mmHbUvaMTN2rMqRFssnW21+1PAy7MZPc4=; b=aSDss88CUZ9hSJVAqbC51hZzksN4hR73cSIoez23kFd7bgFHcL29oHgmJTZs7REfvHl1B9 3SNqTv9eGPboP/fBw346wnGXmtZtbZmnU0KJXP7pQOvHbngGqPetj4TnInhPXzg8VX5WVK EwRLT3IqhFGsDOJzUai6FtWDBiZgA9w= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979322; a=rsa-sha256; cv=none; b=GRfezggCGXDQ5+CoCeY5xRETm8j1rApu85WCFIryljIQhZT/6Z2evAFAsYMluu7SVzi2Km oUl2brXoY6Wv1b5VAvDoRAssKxosfnbU4rXiOyXTID+FY0QI6iHMrwO6GYut06gaT3DDWM BjViw4v/WEeF+gthtZsbBxa0vwKKaD0= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Rxj4KWb+; spf=pass (imf02.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id AC6E4CE0E1E; Wed, 3 Jul 2024 04:02:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DCA06C4AF0C; Wed, 3 Jul 2024 04:02:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979342; bh=hU+tJyMJYXqvt89j51XXi7aJ8UUcH7P41TCm8J5zHmE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Rxj4KWb+WvTzcDWLevWOhyIZvPre2S3FEfiFSoQHVdaD48MPqIaESXA0lOeqDWz4J +PV3qdO7ClN5HuEXB8ZUTc93SoSHRCXfeyZI6fUCocUSJvT7GQ3gYuhhwAhBmiA2gQ 7Yd7yqxS9s9YMDPR2GjZ+8cb7cxFRTqV08nofEfWWqVvquRaIH5UowQJOAm9rb6Gk6 EqDDetEeLI9t2PnL67Cb00PJ3IQcUEQU3tIyO/IsiBlD8O1p6pNaYiknY4a9ISb+wO E+VQvCq3i2xi4qwqy6BJzM1lJTJPnVc4iWDAyklxFyGuvDzcv8yzDdWvCLl69l39Ja +6L9lNmpQ87Pg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 19/20] mm/zsmalloc: introduce __zpdesc_clear_zsmalloc Date: Wed, 3 Jul 2024 12:06:09 +0800 Message-ID: <20240703040613.681396-20-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 79DA180003 X-Stat-Signature: sn77dfygiousu5puzw9q5h9bpto5mnxs X-HE-Tag: 1719979345-807486 X-HE-Meta: U2FsdGVkX19SFV6Lc8CH1DM66QcsMM2yaenpzAVTn/LZjc62y1/4hF+z3QtTMarK2XpAu+hBPzoJjLw1NL3lig/N0qZwqHjs1GgKYtO4ygaCY0BCyI7RcGAen0IbSRoAt3R6RWJErVc3ElrgUSZ9ugOlLZfrGmEeJrWx5Dny2Uq/vfIY/ZTuvypTnrvWOdBW+E5cbtnBnEKhekU/rhq7Nnc8y/+Ie0roWtDcOqRarJwFbhtc/EsLp5zsUKOn3hxojVQH8u92plJO85Nyygb4oILPZfyf7TFxRPwsgzQ9shm8cBLjPs2WP9tV2m28+G387Fvkw6OjGNYKquzW5lXEe5Vlx4Nfupc2uJvXgl/IFV+oxKKfVgmuY+jAMtJorn40y3YD4bsmy0Zec9TLUz3RmG64dE6Zw2GtZ8wI/i8yQZdQ2HSVyMEG24qYZXFmJ1czJLrbYUl8nN30WVciLSA9ORpFk5MGlf5C4MHsJZ3yc+HoD0g/USV1v7/huJPRaew0LxXuqPqRWm9BvyPdH/1182lj2yRitXsnAp1KfCBA+wHY+EuTzCurHJVhKqi/r00ow1drC3c77RGgbBF1YgSR5BsIjudi1u5nrftbooZlriolxWYLfntPZTzgHPMBy40nXOzxbAAOJnMjvO2e6q25Ji1oREhwR/y195uC1rXw2IarU4JXPbfX1ip0xHZhFrwkWZD6rR9l+Jwz+U+GiX50v5ywEFuTAsoCh0GYbparubjQAE3h+vn0h6fivWTRM2a3HBpGH6eMpXLgLClAdMqKZ4Gv1DLG3BeBns6iGZl5SELvWXUpmIKHAD9n8rtsLamhF5UBw2bXpzCpmXfuqiREKH/VQgRv/+CVwCRSbGyiAcaxRF5ZUW5L+s6Ese6verfiX1H5KYhLpZyHnBXePuk9K12TQiZw42fo8lA8h7Z/hFBXlMJqDRQL7C9uk7Bloz0tnw2oCLYGy2CjnVuA5mG TyTMFrsU X9IJ55YBuhSvO8jyTx6P0hSwaxWTcOx8tS2/249NSeE57bUpZoPEyYDnDlYWyVRdNpH1wIs8nLWArWMNGlg8jslBzRct2vDtdEzOCwUeZxLRNIFUGS2TJSwvXjB/6IqV98IMQk2ZQX5oScbKfL9lEXqiFoI4+Rdv1zof/IDIitlP52qJUoqH89yUrNY5Fekw0TsnzyIrZswCG5rwl6ZzMnUNHBGQ0W8chIm97ibpAuzynbyTWP1s8RoPP//Y20ySBQByv6XBADWikfp8T/QSmmDd4l195fRQktLQh5ZcQ2dLdGcN2Tmm/xfiLV75+YcZptEf19c8qHCnyY/Rvw72/wXFKeX2ZuDcoGzwY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Add a helper __zpdesc_clear_zsmalloc() for __ClearPageZsmalloc(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 4780d7015267..e6a71c7a3254 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -116,6 +116,11 @@ static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) __ClearPageMovable(zpdesc_page(zpdesc)); } +static inline void __zpdesc_clear_zsmalloc(struct zpdesc *zpdesc) +{ + __ClearPageZsmalloc(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 08ab22f0ab7f..9e87a17ed802 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -851,7 +851,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) zpdesc->zspage = NULL; zpdesc->next = NULL; reset_first_obj_offset(zpdesc); - __ClearPageZsmalloc(page); + __zpdesc_clear_zsmalloc(zpdesc); } static int trylock_zspage(struct zspage *zspage) @@ -1024,7 +1024,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, if (!zpdesc) { while (--i >= 0) { zpdesc_dec_zone_page_state(zpdescs[i]); - __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + __zpdesc_clear_zsmalloc(zpdescs[i]); free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); From patchwork Wed Jul 3 04:06:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13721229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58324C2BD09 for ; Wed, 3 Jul 2024 04:02:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DD2DC6B00B5; Wed, 3 Jul 2024 00:02:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D59D36B00B6; Wed, 3 Jul 2024 00:02:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BAE386B00B7; Wed, 3 Jul 2024 00:02:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 8E9736B00B5 for ; Wed, 3 Jul 2024 00:02:31 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 4568F806F3 for ; Wed, 3 Jul 2024 04:02:31 +0000 (UTC) X-FDA: 82297094502.25.03D360B Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf27.hostedemail.com (Postfix) with ESMTP id 3E7C34000A for ; Wed, 3 Jul 2024 04:02:28 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=S95km11Y; spf=pass (imf27.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719979326; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UTf/HlUyEjbIfUKANcNriOGEF7uISZlhqNW1oQ4W2Ao=; b=YaP9LDo7tyoa+dVbImlu0uMRywQYmqCAa8N/MhblWceLniKLBkosn9D9QIWvS5FL0OTjhQ Z9DtyWwyV82OeTA8UKUfxak4vZyoEzd0i1eA1cZ0YkcTsu6vt9P2Wu1jf5TrCU5jsSzek3 Qm2LxUzJYY0ygA5oAgZZZFYXhw62g4o= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719979326; a=rsa-sha256; cv=none; b=wBQuLQetnUT7a8vT24XrproIcOrhEdeHBwWgwuohHDHmIo5NBytzh5vRzNseAT6RJV5RjM Labu2OSxqorR1z3sODwRYhrJedMUc0xDauUnJ32+eJSeN2mMrQOhb0PuqhLKn4iEUal3Z3 TEzqgCv1/9PtMEUNzYVIky1iYFZfwd4= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=S95km11Y; spf=pass (imf27.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id C21EBCE28A9; Wed, 3 Jul 2024 04:02:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A954FC32781; Wed, 3 Jul 2024 04:02:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719979346; bh=xf3aTHyp0ikeJrBlj+XtoBw8uMm+YJ3YtskUpiov5H4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S95km11YpkihSJvZhp/uEBasmUDA3sdyzUblqTJ0PHexxmxxWnXUj5mtykzJHpNrL XD0RWT9AlXI5pqwVWqcqGazjfUItM50KJOKLYBKitIgobArYWP0maVhIJWPj3Yq0fS sybAhV09B9V1Q7sDjIyjXElsWNUCNqumEPPQgXAFRXYD4otlCHYNLmBIiEUCaOqA/k 2eWyrQRHBNjkRAGZQG7t7PJnxkokERgGGw3mKOEfwa19w9CEgKNLYpKeNNXg9xqttq 1XwiKETYPDACzuEAZsYk7Nw+G5Bg9RICoxyHVdnSsINHRH3Qe+11uFGhN0hZY5+La2 Xii7jQTm0oudA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v2 20/20] mm/zsmalloc: introduce __zpdesc_set_zsmalloc() Date: Wed, 3 Jul 2024 12:06:10 +0800 Message-ID: <20240703040613.681396-21-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703040613.681396-1-alexs@kernel.org> References: <20240703040613.681396-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: 1a4eshprx8597cqre7o4xzj6n8wr7b18 X-Rspamd-Queue-Id: 3E7C34000A X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1719979348-206046 X-HE-Meta: U2FsdGVkX18al88XHmY6eCCjeuLN9rE+AU8rTC+FqW0VHN0QQv+pgG7AMpeYuWjZQaVd7vKJxfBeUOXaBOvigFhwC20bxRZGD1GWamksxN7xBxQnxYVX0OVBW/eNXf5UmrZ+3DSXYr3drfHUXqK+oUoJnkE151fpD06VfZF1UC90iPuIRX5UktVLbtY6DdKgfcwvgbqQZ78iaogJsDAMiwA8KFtw+C77+ymB58LNmgNcIMfUcCadRLSVbnXBisNwcikBnKpAcBFE4W8V949oaAoGpLPwJ3/8T3W44l1sSBb0gk1iNmx5I1raAkS769GBfHenEh07tGquaBTeEs6FFa7BK6jIC/p4ptkI00oFJOzE3OsTX7oBuyOkjhfb2HGFSafod1UcF8728FcaaSPQKoVBomGHxy3+LCJKHBgZf30ghjXHfULTrFMImJf2tQq3/50uqF7ZhZ4MH3NaB3pCZT9Uvll7K2El6ZPhHkCjH5nxFAm7ONtcWX73kE4M9n6zvYKnAYTMyFXyYuL9qbMxpYzduVZ2mSSdzxyJVNPkRTb9pQ4SL3cBcy+XRWqazP1Wf/IAive5fFxt/lvZF2zIkaq40ig/B4HiGjF39Qr4YFOwvjyZ13ayc7WkODuQ+Yx1vcagU5gclvrc3d02qr8YlD5rhO9v7QlyILa5IPabZPD+iRrmxaeDTXpByg7759ApG0Qx+Y4Y/O2JiY2GS7n8VYuxlbicW0Ps5PoveXcvXLTjW6Nylgx/4NwsDpnMKRDQeveOlel3ofKwHAuUVwr65NwkmgrldCSN6bzxLnFgaHp1zDPEvtlR+VM4psUTdn062jSwsNBncG0DlkupWnKQaoLUrbBBnD97mIb6cU9i6TCq00B0kcywDWPeNElmYlsDzb19wdQUEWUFLpYpDtteOJiJwfKOMF3wUz7+ymjW1aaNuztQjk2YyI++LDuBG628t23g8EDj3BpSx/HbdQH zCRMVuPS dafffAn369h41mFdhvcoPmAI+1DEolQ3YLRhc6RpzeBdeYddCm0k2gm3o9IArC21JTlAkZGSZsjN8e8gXfTK4wTTJjFcwPDUI1xy2JkaqiNBcT5YVrbW6a8FYwKCHTT8inx1aqBK4kD8VfRIcnkpkATW0nVmZOIAxW9wRUZPjqYd0p3LsU1TM5dqnlgrwpLdHgQW+QXfYzaGkGW+VUQVquA3fgcDnWASj9xCUC7P66a9tcftG216Bsb9v5F9D/H5fVcgtJQPri3XgaQTlFs/VRayP1iWvXIfm5CA9wxINlITW7Txt3bhEdBnJ1QNEpViT8IJZNUTvjHRDK5rv36f74aThDPoch8gQnXj5 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Add a helper __zpdesc_set_zsmalloc() for __SetPageZsmalloc(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index e6a71c7a3254..d0e5929c1666 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -116,6 +116,11 @@ static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) __ClearPageMovable(zpdesc_page(zpdesc)); } +static inline void __zpdesc_set_zsmalloc(struct zpdesc *zpdesc) +{ + __SetPageZsmalloc(zpdesc_page(zpdesc)); +} + static inline void __zpdesc_clear_zsmalloc(struct zpdesc *zpdesc) { __ClearPageZsmalloc(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 9e87a17ed802..7bb85447de94 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1030,7 +1030,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(zpdesc_page(zpdesc)); + __zpdesc_set_zsmalloc(zpdesc); zpdesc_inc_zone_page_state(zpdesc); zpdescs[i] = zpdesc; @@ -1820,7 +1820,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(zpdesc_page(newzpdesc)); + __zpdesc_set_zsmalloc(newzpdesc); /* The page is locked, so this pointer must remain valid */ zspage = get_zspage(zpdesc);