From patchwork Tue Aug 6 02:22:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754318 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 221B5C3DA4A for ; Tue, 6 Aug 2024 02:18:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A63CC6B0096; Mon, 5 Aug 2024 22:18:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A142D6B0098; Mon, 5 Aug 2024 22:18:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DD2E6B0099; Mon, 5 Aug 2024 22:18:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6932A6B0096 for ; Mon, 5 Aug 2024 22:18:01 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 1F489160555 for ; Tue, 6 Aug 2024 02:18:01 +0000 (UTC) X-FDA: 82420210362.06.33A7D20 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf05.hostedemail.com (Postfix) with ESMTP id 6DFAC100009 for ; Tue, 6 Aug 2024 02:17:59 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mv6dhj49; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf05.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910629; a=rsa-sha256; cv=none; b=c8VcSwxBrJ/uzbNMudSrzPf9EnBEFi32WPPWji1QOLsSVP+JKgUQQmyBRKXhVr0oGbcaB2 PNLpQSbpOMA+33RUK9UC5/GYclYoulqrdx77MdXcspB+V4o9KiYJ/Oh3G8d8Vf5f0SX7jb 09eswU60G3ErXJ/i0YLnBvSar7nj2+0= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mv6dhj49; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf05.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910629; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MaTte8CSelhyAQBCysDfjAEZZat+5YdBDjKsR1Km+s0=; b=L2HN2sGX1Y5mRl6Go7ORE9TYzNzrJfHDwZclYv6CWeedqnkYFmBqOJQdbD4mqwhIvk6JXS mty8ZwXReODoZKSJ6FbgWOFSIhoeSNPWG3fms0+aexyfLlr+tjYMIFg0jOEcNaUAIJCStK d7WOYZZMabUBo+ILk4YzGYwP1jl6kis= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 874A460ED2; Tue, 6 Aug 2024 02:17:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 33563C32782; Tue, 6 Aug 2024 02:17:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910678; bh=QBwdQc1dwuHyM9NzB3E244MQf6Dsu1USXdYwK6ATMGw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mv6dhj491G+wvqkCVOQPcZFOz90e8QwbdU1OqQYGUfz7bXpOGtE/EKXo3Wl+4C3V6 BmgsoLz8/5KNioCraKsuYBX68DG/tHGuT80cKnWOFrFUl24nhSPxsbQT+IpZqhyw9s djar2TNNIMLvYlhpdzqvaZu1mGka6ad2IKV6/YAuqvOyajAPMt403Lx9onkOcbne3o 2Ak4H1jmzpVLx9QzvOTp6sWdpHB/Rwn1koZaJ7boU1PW5DpL8llCiW7Sjk42llhlEg fNniplVkiQ/TRyTiJj4PXqqq7f2EISBarWVIHCgoKWQOnpMSNON9/kEp551CIq9iWz 2izL8p4n04eGw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 01/21] mm/zsmalloc: add zpdesc memory descriptor for zswap.zpool Date: Tue, 6 Aug 2024 10:22:48 +0800 Message-ID: <20240806022311.3924442-2-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 6DFAC100009 X-Stat-Signature: p8bw1mhe5zk73ybx4zfyrspcddg4iyc8 X-Rspam-User: X-HE-Tag: 1722910679-57574 X-HE-Meta: U2FsdGVkX1/40xrOFffBjdyDoflANZZaFu68+4Wn/oLJBruKTDkuDMJ2pNMzG7wkIBy66RW/pdGEmUwfOumutc9Doppo0M5CXvDsLebCyshnmlQXhbMFq2n1S2EY1CCVOqZdYUlAL/sg3wmk9eR4WtaPuJtp1QwYxHkXmtzdGfH1iKv7hoOVL0gsfk1vYlL6vF4E8N9soljqDSrXa6f26CQaIKFVbaTDMGkLg0FwobyzHr2HGOE1Ah5kUkFITNmw3QDcCPNEWv3EqLTiVFp0HMOinGF8k0VS505mww9Gib6GJwB3QveJCI6Z1USyaopQCm2GMRcEzazrpGsEtIaLPK1gWmt6UXNaZ26E2kLMzVOY943NJOa8FiYZLfj7tTvq5LpWnuO2F6KD2+V/98/CWI5UAQyKrq22AkHoA+jx2hzGn/QHKukXszVEMEZRMniL227YwPCbeLhssvawpqpRHSc0xse03MpVCzaZg+iyPcLpuDmhkElZox3lKgn2bzZRukEjTxpgBWzgLh6AJLzH/fAiryZA+BXQGBMH0diDXufaV7kvbPjwpO3jkHONfYjg17uCE/iyMyQGH8etj95FRCl+z836/syeklfNIzOpzWwtNn9eVzWpT9erwc9R2KTDOVbT0qOlXsOP2Gz7wpXBll3zmgQ2g2cjdt3WCroOs3YBpDwvM/JQ5esSDBWdr33aEaTus7avbv4dXGrmh3uStFV0tFC9KQenMTaEVPs8O3Kc3jvMZHZa3N5AChrbFrvCNzmZqPZfh2kGmnOzSfMNR8Bv0GfpHykzNJDcKqfZ+7QwgiLV+1CTklpURFa4gWO8/UocPqBl1NrENhD1NbQRFjMES9wi3pdEJIIPH9IvTbUdovMzyfAAVU/1zDa0oznnFTXwz2KegP47k4KjML4b+F9FPUtdXewv5QGMf89iZ0nOW6nt7yQryhEFYsLpyf80B7LF7+ZDvRuTneTk+ix kw/o37gM 20PIFii8LiWSEysUAhfZY6/dXXR5dwlvCXxA6Yoppz5n/G2a8E3DZty8+kPosBPpWfwYsySyGG1VPuGqhqavs0sGKwyZda3JKHN/MzR4eJXoYcIqJq43ugjvRcqWSallKnKUqIRFcoK8IwhN0GsN+49QVyHsljatoQv69aa4CNfyun35hXu3pnAO7uqgWHdpaX72IvjIHC5jHt0R5kMqcNu0RBtsBnkhHv4g4IgRmKqh7YQDm6AqBAk66jjydQYNDq9jGZQ0H3Gg/rZ5u1qZw6GGqtZbYilH49oPF9y+QmW0vyY7xjvy7fG3Hk7IVYctD/U4sjAaGqz2tbNM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi The 1st patch introduces new memory decriptor zpdesc and rename zspage.first_page to zspage.first_zpdesc, no functional change. We removed PG_owner_priv_1 since it was moved to zspage after commit a41ec880aa7b ("zsmalloc: move huge compressed obj from page to zspage"). And keep the memcg_data member, since as Yosry pointed out: "When the pages are freed, put_page() -> folio_put() -> __folio_put() will call mem_cgroup_uncharge(). The latter will call folio_memcg() (which reads folio->memcg_data) to figure out if uncharging needs to be done. There are also other similar code paths that will check folio->memcg_data. It is currently expected to be present for all folios. So until we have custom code paths per-folio type for allocation/freeing/etc, we need to keep folio->memcg_data present and properly initialized." Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 71 +++++++++++++++++++++++++++++++++++++++++++++++++++ mm/zsmalloc.c | 25 +++++++++--------- 2 files changed, 83 insertions(+), 13 deletions(-) create mode 100644 mm/zpdesc.h diff --git a/mm/zpdesc.h b/mm/zpdesc.h new file mode 100644 index 000000000000..0df54a5b1670 --- /dev/null +++ b/mm/zpdesc.h @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* zpdesc.h: zswap.zpool memory descriptor + * + * Written by Alex Shi + * Hyeonggon Yoo <42.hyeyoo@gmail.com> + */ +#ifndef __MM_ZPDESC_H__ +#define __MM_ZPDESC_H__ + +/* + * struct zpdesc - Memory descriptor for zpool memory, now is for zsmalloc + * @flags: Page flags, PG_private: identifies the first component page + * @lru: Indirectly used by page migration + * @mops: Used by page migration + * @next: Next zpdesc in a zspage in zsmalloc zpool + * @handle: For huge zspage in zsmalloc zpool + * @zspage: Points to the zspage this zpdesc is a part of + * @first_obj_offset: First object offset in zsmalloc zpool + * @_refcount: Indirectly use by page migration + * @memcg_data: Memory Control Group data. + * + * This struct overlays struct page for now. Do not modify without a good + * understanding of the issues. + */ +struct zpdesc { + unsigned long flags; + struct list_head lru; + struct movable_operations *mops; + union { + /* Next zpdescs in a zspage in zsmalloc zpool */ + struct zpdesc *next; + /* For huge zspage in zsmalloc zpool */ + unsigned long handle; + }; + struct zspage *zspage; + unsigned int first_obj_offset; + atomic_t _refcount; +#ifdef CONFIG_MEMCG + unsigned long memcg_data; +#endif +}; +#define ZPDESC_MATCH(pg, zp) \ + static_assert(offsetof(struct page, pg) == offsetof(struct zpdesc, zp)) + +ZPDESC_MATCH(flags, flags); +ZPDESC_MATCH(lru, lru); +ZPDESC_MATCH(mapping, mops); +ZPDESC_MATCH(index, next); +ZPDESC_MATCH(index, handle); +ZPDESC_MATCH(private, zspage); +ZPDESC_MATCH(page_type, first_obj_offset); +ZPDESC_MATCH(_refcount, _refcount); +#ifdef CONFIG_MEMCG +ZPDESC_MATCH(memcg_data, memcg_data); +#endif +#undef ZPDESC_MATCH +static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); + +#define zpdesc_page(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct page *)(zp), \ + struct zpdesc *: (struct page *)(zp))) + +#define zpdesc_folio(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct folio *)(zp), \ + struct zpdesc *: (struct folio *)(zp))) + +#define page_zpdesc(p) (_Generic((p), \ + const struct page *: (const struct zpdesc *)(p), \ + struct page *: (struct zpdesc *)(p))) + +#endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5d6581ab7c07..30f0a7abbda3 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -13,20 +13,18 @@ /* * Following is how we use various fields and flags of underlying - * struct page(s) to form a zspage. + * struct zpdesc(page) to form a zspage. * - * Usage of struct page fields: - * page->private: points to zspage - * page->index: links together all component pages of a zspage + * Usage of struct zpdesc fields: + * zpdesc->zspage: points to zspage + * zpdesc->next: links together all component pages of a zspage * For the huge page, this is always 0, so we use this field * to store handle. - * page->page_type: PG_zsmalloc, lower 16 bit locate the first object - * offset in a subpage of a zspage + * zpdesc->first_obj_offset: PG_zsmalloc, lower 16 bit locate the first + * object offset in a subpage of a zspage * - * Usage of struct page flags: + * Usage of struct zpdesc(page) flags: * PG_private: identifies the first component page - * PG_owner_priv_1: identifies the huge component page - * */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt @@ -64,6 +62,7 @@ #include #include #include +#include "zpdesc.h" #define ZSPAGE_MAGIC 0x58 @@ -253,7 +252,7 @@ struct zspage { }; unsigned int inuse; unsigned int freeobj; - struct page *first_page; + struct zpdesc *first_zpdesc; struct list_head list; /* fullness list */ struct zs_pool *pool; rwlock_t lock; @@ -448,7 +447,7 @@ static inline void mod_zspage_inuse(struct zspage *zspage, int val) static inline struct page *get_first_page(struct zspage *zspage) { - struct page *first_page = zspage->first_page; + struct page *first_page = zpdesc_page(zspage->first_zpdesc); VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); return first_page; @@ -948,7 +947,7 @@ static void create_page_chain(struct size_class *class, struct zspage *zspage, set_page_private(page, (unsigned long)zspage); page->index = 0; if (i == 0) { - zspage->first_page = page; + zspage->first_zpdesc = page_zpdesc(page); SetPagePrivate(page); if (unlikely(class->objs_per_zspage == 1 && class->pages_per_zspage == 1)) @@ -1324,7 +1323,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, link->handle = handle | OBJ_ALLOCATED_TAG; else /* record handle to page->index */ - zspage->first_page->index = handle | OBJ_ALLOCATED_TAG; + zspage->first_zpdesc->handle = handle | OBJ_ALLOCATED_TAG; kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); From patchwork Tue Aug 6 02:22:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16E70C3DA7F for ; Tue, 6 Aug 2024 02:18:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C78C6B0099; Mon, 5 Aug 2024 22:18:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 950796B009A; Mon, 5 Aug 2024 22:18:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F14E6B009B; Mon, 5 Aug 2024 22:18:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5B98F6B0099 for ; Mon, 5 Aug 2024 22:18:05 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id D53B1A8A29 for ; Tue, 6 Aug 2024 02:18:04 +0000 (UTC) X-FDA: 82420210488.21.A0903E6 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf06.hostedemail.com (Postfix) with ESMTP id 39459180025 for ; Tue, 6 Aug 2024 02:18:03 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=rc1NJYVS; spf=pass (imf06.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910622; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=l89Rhq7PwDAQV1pwzUwf0f+8cr+wUGabqM+6g+/d3Nc=; b=CqK1TONLaUz+5aE8Lw/ogqDdW/iCwySTeJD58INLmiJlArfe7v1yafxWz95c6GaQZ02DD1 DHWmEJqCIxrsiOaE01y7gjR3wq27xBJdA+RYOqSVcsE2pxgr01Ki21y4Jk30iMH3MSzrNj 1DXQCY0L7dFmMlmViA4WbDSZNOclYKg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910622; a=rsa-sha256; cv=none; b=wEVnJNmVMncM70t6QblNKvESFuhkpOxY8IGVpFyGeyMRuTMWHc7BhT34d9XTO6l4Tb5vNY P68fOyw349AjtWTwHTP6j3aXp8nU0tC0RbpNRLsZkzHVFk/z2Qj85vJ16rivyySL90nJ38 MB500RvePybxvvXg5dU5OlnRQZ91Tz0= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=rc1NJYVS; spf=pass (imf06.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 76A8460ED1; Tue, 6 Aug 2024 02:18:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 07238C32782; Tue, 6 Aug 2024 02:17:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910682; bh=W2CmxaC60Muhq4cmp7NdT+PQ9UwnM6tRFRsuxLeQy0s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rc1NJYVSBoTuAOpI656XakHjjozJdwgj0cfNQKibJWY1LL8jXwY1WbssbpvpbER1c 073HAUwpSPdOO/ObCGVEy/V2WrdPUN9CE0imEwfF24LSChnSVfkxdzE3Hi/81h+Qgw Tv72ubRfzJgh7QGq3G9bcSqWtpY2v/me/APYMaPnJ/tiUbMf3WZkk3CD8BZ4SjGTI5 T9YRt/NxHjIMqT9neXat/llYvBdantwiqmgBW8/yvpPxzyqg4Lvr2NqYnBteDvTNtS a/Ur5BNX8XYzh08MQfeB+baYhmi02LXVkt5rTusJok2SIRQvbcncnRMzdl7hZA8hUM 3vGbZy+RQvA3Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 02/21] mm/zsmalloc: use zpdesc in trylock_zspage()/lock_zspage() Date: Tue, 6 Aug 2024 10:22:49 +0800 Message-ID: <20240806022311.3924442-3-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 39459180025 X-Stat-Signature: 9gqobpqrcogbs4j3yf1mukz8u11pgcf6 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1722910683-557431 X-HE-Meta: U2FsdGVkX1/oEZu6iurpcp1CBttkB6i0vyySP+RyDBwjKvgT/7R67eS1XVJYCwtR31FUzNb0oH8PE1KnnvDITM0m4v0YZ4ZcuctX3A96n/IFmTCLbPTLMps3hEtLc+Mm95Qrf3CasP86PlXqxnBEPBAppMwqR8SEEGo9nx9x7sjHjwiSwtbSrF3+yF1koD3mgHQkEKAZ5p45pH18rqIJ92pTVi4EHCpogWrbtfFFsCPpocPiB/Km/rszzuQTZYIY+LVDGRTMuPbNpTAU7g6ls/24Rklocvf+2QsNn6TCE7O9VraCOAE+DgKcj8Z588If/juyEVqEYmYIst015yQb/jmBKRYQavhkBaNooVLM1Um8wZ2ShB6D49wk4sh94Kzu05cRhFswtaZiLXSTIBeg7VIXqvPH6Zz7I0H7/RWTEI0ChCD85V/NmjqQ4i0SthtRF402ciHP1VB5Vw/ZAgi1dKyMlIQFW2nJrcLE2MPDDTBQYzVRKf0bkgcj5Qi5mYTLuidn5ZdDSM0oHtBkj7zZ15O4geg9ELX2umDGVvHKk3izG4Y2D3RIcfVX8F0wwqEJ+tOk5uRHABp+Cv7ngh9LiFf2AqjlRUk6we5Qoxh1gyKTPeQ8sgYbL7VawFEXhikPxXnb6Zmgi8pYn5mSbBrZdMKAMoTOfaAH6N4QclQ344xhLIqM+aeXoPef8hRzWd/Ho2SRTBbPARXGdjy9A/Wv4gbN5sZkcTnja+/4zID4/UuYzUXX+ibMhW8NeEK9OzjfTzxVAp1ooPKB9a7Eo2KoOsmLV8BGovrwngdluH1TC4vfl2faYthODJDri+2C92/wix1xlC3WemQkseQN53w1hb9+65dD6k31TQeYixMPrGFdf53Vofx1KJ6qHXXIy3AoHMMMUVAhOHXMRUYRDq9WjgEByHzXIi2i120LyenM2MPwdCtVBquBAadJo+yZSiy0EQ5x3N/0ILuLoSxpJ7G D6jfl7MJ V/whRBpS0EeTb1l4hKHzuAAgCnjfMCErzE3VzBpBzATr7Y84xMyv35NEhQTMGdhTE1b++GH+wEKGtfUGx9TuMZwaKOTshGNStcngu3kNnnxIXnQYZP4t1fGYBbgfyynojZjW23BGjRvf1nc8zmycabIj/J8KESPDgXLCdU9aCKa2F6NnFmdBvWOcuse2Eu5tCCQGmFrQWDHkNfOT4FtBDAGbPb4whH+9KGzT3uQcT1tSkk3Jy0734gvUb8Ri4e8HAKn8MFoPYOhVpRBjLH1ZiM/IFTNagO2efdxqPvKooKDw/mwWEbol3r47CNLx5p9LSp1l/ZvCACzCjcc1lMnB39ygJtzm473bsnm9YaYOdpIBuSNkQYiJMZ4aE+LqaTYW5ozGC X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi To use zpdesc in trylock_zspage()/lock_zspage() funcs, we add couple of helpers: zpdesc_lock()/zpdesc_unlock()/zpdesc_trylock()/zpdesc_wait_locked() and zpdesc_get()/zpdesc_put() for this purpose. Here we use the folio series func in guts for 2 reasons, one zswap.zpool only get single page, and use folio could save some compound_head checking; two, folio_put could bypass devmap checking that we don't need. BTW, thanks Intel LKP found a build warning on the patch. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 30 ++++++++++++++++++++++++ mm/zsmalloc.c | 64 ++++++++++++++++++++++++++++++++++----------------- 2 files changed, 73 insertions(+), 21 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 0df54a5b1670..421eeeef6f8f 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -68,4 +68,34 @@ static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); const struct page *: (const struct zpdesc *)(p), \ struct page *: (struct zpdesc *)(p))) +static inline void zpdesc_lock(struct zpdesc *zpdesc) +{ + folio_lock(zpdesc_folio(zpdesc)); +} + +static inline bool zpdesc_trylock(struct zpdesc *zpdesc) +{ + return folio_trylock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_unlock(struct zpdesc *zpdesc) +{ + folio_unlock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_wait_locked(struct zpdesc *zpdesc) +{ + folio_wait_locked(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_get(struct zpdesc *zpdesc) +{ + folio_get(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_put(struct zpdesc *zpdesc) +{ + folio_put(zpdesc_folio(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 30f0a7abbda3..25c90224f21f 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -433,13 +433,17 @@ static __maybe_unused int is_first_page(struct page *page) return PagePrivate(page); } +static inline bool is_first_zpdesc(struct zpdesc *zpdesc) +{ + return PagePrivate(zpdesc_page(zpdesc)); +} + /* Protected by class->lock */ static inline int get_zspage_inuse(struct zspage *zspage) { return zspage->inuse; } - static inline void mod_zspage_inuse(struct zspage *zspage, int val) { zspage->inuse += val; @@ -453,6 +457,14 @@ static inline struct page *get_first_page(struct zspage *zspage) return first_page; } +static struct zpdesc *get_first_zpdesc(struct zspage *zspage) +{ + struct zpdesc *first_zpdesc = zspage->first_zpdesc; + + VM_BUG_ON_PAGE(!is_first_zpdesc(first_zpdesc), zpdesc_page(first_zpdesc)); + return first_zpdesc; +} + #define FIRST_OBJ_PAGE_TYPE_MASK 0xffff static inline void reset_first_obj_offset(struct page *page) @@ -745,6 +757,16 @@ static struct page *get_next_page(struct page *page) return (struct page *)page->index; } +static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) +{ + struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + + if (unlikely(ZsHugePage(zspage))) + return NULL; + + return zpdesc->next; +} + /** * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value @@ -815,11 +837,11 @@ static void reset_page(struct page *page) static int trylock_zspage(struct zspage *zspage) { - struct page *cursor, *fail; + struct zpdesc *cursor, *fail; - for (cursor = get_first_page(zspage); cursor != NULL; cursor = - get_next_page(cursor)) { - if (!trylock_page(cursor)) { + for (cursor = get_first_zpdesc(zspage); cursor != NULL; cursor = + get_next_zpdesc(cursor)) { + if (!zpdesc_trylock(cursor)) { fail = cursor; goto unlock; } @@ -827,9 +849,9 @@ static int trylock_zspage(struct zspage *zspage) return 1; unlock: - for (cursor = get_first_page(zspage); cursor != fail; cursor = - get_next_page(cursor)) - unlock_page(cursor); + for (cursor = get_first_zpdesc(zspage); cursor != fail; cursor = + get_next_zpdesc(cursor)) + zpdesc_unlock(cursor); return 0; } @@ -1658,7 +1680,7 @@ static int putback_zspage(struct size_class *class, struct zspage *zspage) */ static void lock_zspage(struct zspage *zspage) { - struct page *curr_page, *page; + struct zpdesc *curr_zpdesc, *zpdesc; /* * Pages we haven't locked yet can be migrated off the list while we're @@ -1670,24 +1692,24 @@ static void lock_zspage(struct zspage *zspage) */ while (1) { migrate_read_lock(zspage); - page = get_first_page(zspage); - if (trylock_page(page)) + zpdesc = get_first_zpdesc(zspage); + if (zpdesc_trylock(zpdesc)) break; - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); } - curr_page = page; - while ((page = get_next_page(curr_page))) { - if (trylock_page(page)) { - curr_page = page; + curr_zpdesc = zpdesc; + while ((zpdesc = get_next_zpdesc(curr_zpdesc))) { + if (zpdesc_trylock(zpdesc)) { + curr_zpdesc = zpdesc; } else { - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); migrate_read_lock(zspage); } } From patchwork Tue Aug 6 02:22:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754320 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91AE8C3DA7F for ; Tue, 6 Aug 2024 02:18:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1FFCF6B009B; Mon, 5 Aug 2024 22:18:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1896E6B009C; Mon, 5 Aug 2024 22:18:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 04FDD6B009D; Mon, 5 Aug 2024 22:18:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id DA7066B009B for ; Mon, 5 Aug 2024 22:18:08 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 95775C07BB for ; Tue, 6 Aug 2024 02:18:08 +0000 (UTC) X-FDA: 82420210656.27.FCA839D Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf12.hostedemail.com (Postfix) with ESMTP id 04BD140013 for ; Tue, 6 Aug 2024 02:18:06 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="Y9bMfZD/"; spf=pass (imf12.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910656; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cAi81bWlDHMmhHwcU64lYTDVUKa+q1KUucaRaXn58z4=; b=tvfnubfIWOPg5/jVB4tsJ/nqmrbqsD7xwKTVAq6YLlApHC6zf5VNlJ0pulK7mzXAhDy7rT ShrtiN9kNkOwF2UCoTfk1IRHHJ0RVZZLEr8e2ENh30XY3xZdD+PAmAx5WBWPRIeXSlN273 JMgJ9FDEOQV8H4GGSRhxlN3n55STxf4= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="Y9bMfZD/"; spf=pass (imf12.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910656; a=rsa-sha256; cv=none; b=b8Ktsa3IQmzEmbz9POBLiM2AQfxlxgok24RgiY4dIjVsJUxSHcnLy35wDOwCQ1WXIwopyX HeYPU7QbqBW0P8g+9g8XoHTtpNN+NF5XfzOTBdZVbkTlWZMWU15xOki+JzmlZzcW4C4udV kYFWRLq2Dva6IeOIkQRSVpu6Nbi8drE= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 2E39D60ED0; Tue, 6 Aug 2024 02:18:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CD578C32782; Tue, 6 Aug 2024 02:18:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910686; bh=rMm13SX4aLf3oZ/lBmcjJeeV3CGO+2RCAQN5DP+xGmo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Y9bMfZD/5HIEGcjAndQ/lypfE4f+loM6ikbINUx7e4ZVxqmabtcM3IJIvPbKyohqh R0lCElOP1IqSvv4eN2T5KrP+x7QIi/l/v+tKP2fas0PoG5toYLpB4I8hyrj3HHzO4l i3LfF0/shOAC/fAypNubA8AmCHtevJ6wxFiamGL7VuIWUQzJUzakpEWwkMwS6V270S 6YI85bUWbzpkTNsuW+b7q0KKgWs4xbBRXMUf/5dTRjxYEjlzTdU1434myFq2/48qy8 9UhQf2lLEHTOM1xIgk/f1Tgn8WKUcPUokYFcZtIK4WiIPOvpjGiU92Bq4SJtbV7Sl1 uNm1Wpab5IGyw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 03/21] mm/zsmalloc: convert __zs_map_object/__zs_unmap_object to use zpdesc Date: Tue, 6 Aug 2024 10:22:50 +0800 Message-ID: <20240806022311.3924442-4-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 04BD140013 X-Stat-Signature: jk65fpdf1m4ao4otum4u68a1ft3sjbm3 X-HE-Tag: 1722910686-273969 X-HE-Meta: U2FsdGVkX1+bx6QbvEgiIVBRyh79MMkTTRRvbaHZxKbsGWkwKk6N6YsUZ4bfd7CWX4J0kJtnHpJnycRf1vsLZV3rcwzaRwl4ZYQLRra/bkUmtcntyYX/a+S7olPu6zPJ3n5grmtLuNuyMH/m38r3pwX3YS9wN8lC3EnFKnTyoe/4tU7NHCBO5rE4qizzfob+xlWVr6XnmUMfZt+CAX4QCpVn4cgR0/VulRo4ntoAH8wZoXrumpO7s6BWzGcwwfoyx3O3NpWyjBNvH7udm7AvL5XLetkzb8LGenlTa0d7HGc7sc1vKl7orl8dgvSKyHlhzo5KEKi4BXfh2AYdPO9QUNycQsBTzt6Gp3JS2Epi/4Z3MzRtsBBG3B83B01V4/RCVR322EQ1DpyMSQEJg5VPk0u8hfyT/UL4uczh8qhYGw8QTH7zguPjEuLLMQuca7uBzVeo/xAFyiA0SbV0uGUKyDrJsC6262Xatki5E9JqhiXGQwBhRGBqAuu7YSZJod0o1vAk1/wcbs5O9W/0zAZdzk+oY1UI5vvdVJlKUMIhDzUBizgQASOVSVYkMOgEgJ071zXQGFBYLNmX87aNY+9RZ2Rv4j8n+3N1qVWK12FlZUBgrx42V/CPwO0i/anH1AdDGs5uo42neTQ5szxzHT949oPEgVam8R3h5N+H7q8FTpngmgx297VxChQpV3HbUESFlopLVNrRXcwlCscTUvfbRDDj+wQtdhGQcH3RvtG23FASWE7wfESCkqQzCkXgNKAR1G2XQzJe1YZyNFLyMjken79cSR5FuY5UtCAoTjLOIo1Ak+yhJY1+w49LkQKi/vesd8q4dQqHj8ZaHJIIHr0+fss2Tl+ErZAHXtuU9+X/aGebYKdgSd2ojO/ZhrT/4hmeO0UbnzlT+WGIHLYasc7NXfwCjXpPWmQIcwpLJT0E+yEVtAsUTA01ru1+24qc57+KpJ3Y8OKs03dHdl354HG 8H8S3gwg vfJhkUuIcp3aoPOhC1QbQ428FaCOKDglS18gDPyLi05hyjfVN0YsyNFMErlUuCoKya1pCL3IGDCQvJLt2B5me5qeOVvf4ItUrTJFeQfsHzZrnhdWitmrcyyrKO8+W6iJs8wITaX45GuDDv5ZWmyc7+2l0O5q8LQCeZeNW1Y6YvWqIaPx+2Z7iexk5GDKLMWe04RyLha8xgHP5Hq/ysdSuXWmy5KBuXDC2YL+//qRB5gSiXpU3q2YDsxAI8aUpMoq0bgRKYOEqKgNeRHjCCPH9nuCcjQHoyO3uUzQ8jGC0Lm76j+ZUEIgaGNVIPdOc2A1RfE2CLG7c0UKfTnLRgBbcU1xtMuKTD1PaVRo+GATkd7JkhnxcOsovo6NW0y31VYE1tXQm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> These two functions take pointer to an array of struct page. Introduce zpdesc_kmap_atomic() and make __zs_{map,unmap}_object() take pointer to an array of zpdesc instead of page. Add silly type casting when calling them. Casting will be removed late. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 25c90224f21f..b9b5e2824f2c 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -243,6 +243,11 @@ struct zs_pool { atomic_t compaction_in_progress; }; +static inline void *zpdesc_kmap_atomic(struct zpdesc *zpdesc) +{ + return kmap_atomic(zpdesc_page(zpdesc)); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -1061,7 +1066,7 @@ static inline void __zs_cpu_down(struct mapping_area *area) } static void *__zs_map_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1078,10 +1083,10 @@ static void *__zs_map_object(struct mapping_area *area, sizes[1] = size - sizes[0]; /* copy object to per-cpu buffer */ - addr = kmap_atomic(pages[0]); + addr = zpdesc_kmap_atomic(zpdescs[0]); memcpy(buf, addr + off, sizes[0]); kunmap_atomic(addr); - addr = kmap_atomic(pages[1]); + addr = zpdesc_kmap_atomic(zpdescs[1]); memcpy(buf + sizes[0], addr, sizes[1]); kunmap_atomic(addr); out: @@ -1089,7 +1094,7 @@ static void *__zs_map_object(struct mapping_area *area, } static void __zs_unmap_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1108,10 +1113,10 @@ static void __zs_unmap_object(struct mapping_area *area, sizes[1] = size - sizes[0]; /* copy per-cpu buffer to object */ - addr = kmap_atomic(pages[0]); + addr = zpdesc_kmap_atomic(zpdescs[0]); memcpy(addr + off, buf, sizes[0]); kunmap_atomic(addr); - addr = kmap_atomic(pages[1]); + addr = zpdesc_kmap_atomic(zpdescs[1]); memcpy(addr, buf + sizes[0], sizes[1]); kunmap_atomic(addr); @@ -1252,7 +1257,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, pages[1] = get_next_page(page); BUG_ON(!pages[1]); - ret = __zs_map_object(area, pages, off, class->size); + ret = __zs_map_object(area, (struct zpdesc **)pages, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret += ZS_HANDLE_SIZE; @@ -1287,7 +1292,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) pages[1] = get_next_page(page); BUG_ON(!pages[1]); - __zs_unmap_object(area, pages, off, class->size); + __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); } local_unlock(&zs_map_area.lock); From patchwork Tue Aug 6 02:22:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D502C3DA7F for ; Tue, 6 Aug 2024 02:18:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C8726B00A1; Mon, 5 Aug 2024 22:18:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 951D06B009F; Mon, 5 Aug 2024 22:18:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F17A6B00A0; Mon, 5 Aug 2024 22:18:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 620FB6B009D for ; Mon, 5 Aug 2024 22:18:16 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1F2461C5706 for ; Tue, 6 Aug 2024 02:18:16 +0000 (UTC) X-FDA: 82420210992.01.ECA5818 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf17.hostedemail.com (Postfix) with ESMTP id CB6FB40013 for ; Tue, 6 Aug 2024 02:18:13 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=kWdNfAWj; spf=pass (imf17.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910663; a=rsa-sha256; cv=none; b=JGZg8XpPV+aJdeZ11OSsoacAjwHBerFE01lMIbZJh8KuDRZekh5W9WaxqLkN4I+FCGNe8a jz0vUv/fHrQF8JIqTZzBV3ZrRiLQ3AIcIpyzLIc7uSt9KHxkcI2cShP+1xzqRf6MwTYS5C x6K5Gf4Z5NU0VS0KciU7gmcYByyqgbg= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=kWdNfAWj; spf=pass (imf17.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910663; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/LCNfaPFs+qw+vdOY2hLSzyV/jZ/R3hzUjVuyUICW5A=; b=ZeZndzRqJNTve4XZt334H//glnfodIBwtyz9xd4wGRt0SimHtvFJ9V8ZTwOkZ3SkpggI1t JH6XcOwNoG3YhIyzm43UQ3Y8YcSDtqW710Ou5EfTh352JMMcuHVqVwDK4V1x+2FYrEFSMC XoYHOrsv97ow6md8+5FEq74ukNRGo0s= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 7F2BECE0B30; Tue, 6 Aug 2024 02:18:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9F567C32782; Tue, 6 Aug 2024 02:18:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910689; bh=xRbL5zVs+Qjgf9i92JU0jHMe8Ob3o1agljGLF7p0KNc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kWdNfAWjBaZIXiutIjz8LzEoghILI0Dj9+anUzmCrCkxd2qtOypIhrUAueUbH9/AB SUaM6AzVwPMda0h0Fp24pvVUNJSHvPsPeONOsGBBdfOM6WKARobWbhHgGiw0N24ciB s3peCG93Il1VzKl2xancD0/l4WcPam3WIJ6VxrY7s64MRlHHqfgauOTcbXQdTGX+0V jFm+qWpku0GRsDctyTcWfoTMSz3ku0661lDy50QNwj+ridqElSa1flaoSpkm7/wTvk 9T7sWf9dGhH1BuNHnBlTr/MEgx52hGdXT2HzQRDzx2fMk8DowDnotoKNHgihfWmglX 0GFSUutoDztSw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 04/21] mm/zsmalloc: add and use pfn/zpdesc seeking funcs Date: Tue, 6 Aug 2024 10:22:51 +0800 Message-ID: <20240806022311.3924442-5-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: ie8kajbftfwu89rak99bdbs65eyt875h X-Rspamd-Queue-Id: CB6FB40013 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1722910693-962711 X-HE-Meta: U2FsdGVkX1+uPMuHBuTFKUe43/ZivBD8XELfDl1a9IfbMOm5N06BPoRklhQO4R/DI6mbwUlH9R77pEhNsAGQCjfa+52VaazdsHSG2BoTbf7qOsZvNOiCXOc2hLyNdNACzFYEBFo5udeVjjyR7kf42wb3sG4x4qko9dF3kRi+XCWVmdKhPZWpLw8JD8OzTm8ABuMe91UczFB8xzEayGOBCCKBPzIQomlWM6yjqfj2lrwCNWO1bUEWwFXFnrop6ibSUbgHJIAY6bstzLBKOrYQw/SuEpAwIpMj7ZP/z7brovRG5dwnG/nJVkZsUNgV9KW69p/XG125jGO3+Dzo6JAmpNUDiL0XQ1nhKn7E7N+mx89gVCWKUcGWs8cVkQIPv0Jh4sqV/YV3T7RX4W1o9Fs5svJ46aGwNUWKgTwlB97+cHO+vw8Ra8rqTKAqelZjEwuQAI1RNvJphzI8z2zazJO6nuwD5DiVdIOkNZuAJLtuKfVHnUjeMnduCoRJEBB2Rnhsurpv7uH+nOMAgj6qFxh3bBKjN/B5AFkcz53/7jaWpkNi8SWYIiPzq70Mk1RxYaTyxEswBu0QZkTP/UNTmmZZ1ya/oZ7gsJB2Zbf5r/a1O+t25rw7TSvYyhOWUqznHf3ppT/iNyDJejys01O+6xkJqwOtHpnUq/8XfZH76IVOhisgOcQNIRqg1pYCi1YPpJ6lDANQBg0XwWRVTtf4wY1kyj3X+Ay3DzByXCEak5k+LbGCAJlNQK7AFdPWFFfecr7qcUYjWUJAP+HM9FhjiS+gic4hUmHHM1koMHvnl8qSDkaX32+7qreNMOH1Fb7z/4Rydmwo7BFxr+8vrvdDcZFtUbnWMbhFD+ka4xuvwPQp66H6aEzi8/b8m7MIzJq8Ti6PvoNEZt1VfX9ATYkiprEGD8KThCkO9guB2qRL3jkocH3Cbu+6cdsL+65gIzRO0ww8UPsLt4bfGT879xOT4kp 9Rq9yrFO PV7m7QpUSdgRPEf+hLNIpEJnVZ6BgmVaAz2UOqXJRuUZRo/Wi/IaU3R0C1FBuxKPhABiE5hlWysGKla0pTut9XrmK6xl2w8g0zj4UBa8fmEFoP60neVWLd19hDyzhGpIpTvgTT0ltXQd1BaCJ5ND1w4HYwxBm48YC3fIGx8EA7vdyzhFd7eILtmpZH/BBPNidJrFwTm14aY89C5CbA5464IF4W2zaFSB0l6bcjFt2VNwglZXaX5lb68ixFDwZr/JBmOzFVM0UlrL9hNt/89ktlWdXSxsz0MSSvfQYfJ6UqAzHgeK0pNwWAIXVvD8vVJqHQdeu4xgK5UzC1pc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Add pfn_zpdesc conversion, convert obj_to_location() to take zpdesc and also convert its users to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 9 +++++++ mm/zsmalloc.c | 75 ++++++++++++++++++++++++++------------------------- 2 files changed, 47 insertions(+), 37 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 421eeeef6f8f..2101de23d16d 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -98,4 +98,13 @@ static inline void zpdesc_put(struct zpdesc *zpdesc) folio_put(zpdesc_folio(zpdesc)); } +static inline unsigned long zpdesc_pfn(struct zpdesc *zpdesc) +{ + return page_to_pfn(zpdesc_page(zpdesc)); +} + +static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) +{ + return page_zpdesc(pfn_to_page(pfn)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index b9b5e2824f2c..384a5ba49788 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -773,15 +773,15 @@ static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) } /** - * obj_to_location - get (, ) from encoded object value + * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value - * @page: page object resides in zspage + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static void obj_to_location(unsigned long obj, struct page **page, +static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, unsigned int *obj_idx) { - *page = pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc = pfn_zpdesc(obj >> OBJ_INDEX_BITS); *obj_idx = (obj & OBJ_INDEX_MASK); } @@ -1208,13 +1208,13 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, enum zs_mapmode mm) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; struct size_class *class; struct mapping_area *area; - struct page *pages[2]; + struct zpdesc *zpdescs[2]; void *ret; /* @@ -1227,8 +1227,8 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, /* It guarantees it can get zspage from handle safely */ read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage = get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc_page(zpdesc)); /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1247,17 +1247,17 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, area->vm_mm = mm; if (off + class->size <= PAGE_SIZE) { /* this object is contained entirely within a page */ - area->vm_addr = kmap_atomic(page); + area->vm_addr = zpdesc_kmap_atomic(zpdesc); ret = area->vm_addr + off; goto out; } /* this object spans two pages */ - pages[0] = page; - pages[1] = get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] = zpdesc; + zpdescs[1] = get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); - ret = __zs_map_object(area, (struct zpdesc **)pages, off, class->size); + ret = __zs_map_object(area, zpdescs, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret += ZS_HANDLE_SIZE; @@ -1269,7 +1269,7 @@ EXPORT_SYMBOL_GPL(zs_map_object); void zs_unmap_object(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; @@ -1277,8 +1277,8 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) struct mapping_area *area; obj = handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage = get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc_page(zpdesc)); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1286,13 +1286,13 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) if (off + class->size <= PAGE_SIZE) kunmap_atomic(area->vm_addr); else { - struct page *pages[2]; + struct zpdesc *zpdescs[2]; - pages[0] = page; - pages[1] = get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] = zpdesc; + zpdescs[1] = get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); - __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); + __zs_unmap_object(area, zpdescs, off, class->size); } local_unlock(&zs_map_area.lock); @@ -1434,23 +1434,24 @@ static void obj_free(int class_size, unsigned long obj) { struct link_free *link; struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long f_offset; unsigned int f_objidx; void *vaddr; - obj_to_location(obj, &f_page, &f_objidx); + + obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); - zspage = get_zspage(f_page); + zspage = get_zspage(zpdesc_page(f_zpdesc)); - vaddr = kmap_atomic(f_page); + vaddr = zpdesc_kmap_atomic(f_zpdesc); link = (struct link_free *)(vaddr + f_offset); /* Insert this object in containing zspage's freelist */ if (likely(!ZsHugePage(zspage))) link->next = get_freeobj(zspage) << OBJ_TAG_BITS; else - f_page->index = 0; + f_zpdesc->next = NULL; set_freeobj(zspage, f_objidx); kunmap_atomic(vaddr); @@ -1495,7 +1496,7 @@ EXPORT_SYMBOL_GPL(zs_free); static void zs_object_copy(struct size_class *class, unsigned long dst, unsigned long src) { - struct page *s_page, *d_page; + struct zpdesc *s_zpdesc, *d_zpdesc; unsigned int s_objidx, d_objidx; unsigned long s_off, d_off; void *s_addr, *d_addr; @@ -1504,8 +1505,8 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, s_size = d_size = class->size; - obj_to_location(src, &s_page, &s_objidx); - obj_to_location(dst, &d_page, &d_objidx); + obj_to_location(src, &s_zpdesc, &s_objidx); + obj_to_location(dst, &d_zpdesc, &d_objidx); s_off = offset_in_page(class->size * s_objidx); d_off = offset_in_page(class->size * d_objidx); @@ -1516,8 +1517,8 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, if (d_off + class->size > PAGE_SIZE) d_size = PAGE_SIZE - d_off; - s_addr = kmap_atomic(s_page); - d_addr = kmap_atomic(d_page); + s_addr = zpdesc_kmap_atomic(s_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); while (1) { size = min(s_size, d_size); @@ -1542,17 +1543,17 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, if (s_off >= PAGE_SIZE) { kunmap_atomic(d_addr); kunmap_atomic(s_addr); - s_page = get_next_page(s_page); - s_addr = kmap_atomic(s_page); - d_addr = kmap_atomic(d_page); + s_zpdesc = get_next_zpdesc(s_zpdesc); + s_addr = zpdesc_kmap_atomic(s_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); s_size = class->size - written; s_off = 0; } if (d_off >= PAGE_SIZE) { kunmap_atomic(d_addr); - d_page = get_next_page(d_page); - d_addr = kmap_atomic(d_page); + d_zpdesc = get_next_zpdesc(d_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); d_size = class->size - written; d_off = 0; } @@ -1791,7 +1792,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct zs_pool *pool; struct size_class *class; struct zspage *zspage; - struct page *dummy; + struct zpdesc *dummy; void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; From patchwork Tue Aug 6 02:22:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754322 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 429A1C3DA4A for ; Tue, 6 Aug 2024 02:18:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C5F246B009D; Mon, 5 Aug 2024 22:18:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BC6A06B00A0; Mon, 5 Aug 2024 22:18:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C8306B009D; Mon, 5 Aug 2024 22:18:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 714256B009E for ; Mon, 5 Aug 2024 22:18:16 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 1F3EF160846 for ; Tue, 6 Aug 2024 02:18:16 +0000 (UTC) X-FDA: 82420210992.29.554B3CA Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf08.hostedemail.com (Postfix) with ESMTP id 84F5916001D for ; Tue, 6 Aug 2024 02:18:14 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=SxdVNQz1; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf08.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910648; a=rsa-sha256; cv=none; b=w03ZFEIM5r+vCNQcCMoTg9/HjnIxtFt+PiTcmNQqaaXCmq8MliJpygPp7E7mqW6Lf2GyH3 uAhl/n1xpjwcHLi9f4paIjmPLuPpVrcc/+EVqRnkHkC8iwTmsqheX7edbCL/+1pd7MdSch ffKZD9NvTkHhwN38QEqZYXyi1IccbBI= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=SxdVNQz1; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf08.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910648; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Yh2M3ze6FPP2pb5gtVIfrEf+1wX6q2uwXp/9ylWsfqo=; b=4fC+35zO2/qH6YJhBZIJbq/hWHIHxW3BwWL8M87JWY5GMPjUCInGamzNrt9GDrR1ztNauO p/HJRJtso4/ftW9My+VP+N5RuWHJKF73x2a+QnuOgJ4YOp6A884TCzVER5JNM3YO1GuTt9 /SrBIDslmC/pFRLkmu+2L8YNQrTXNYk= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 9F19360ED3; Tue, 6 Aug 2024 02:18:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 735E6C4AF0F; Tue, 6 Aug 2024 02:18:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910693; bh=j9RlWiCV98maWH46GH8LQqP9KXOBrIEnE2eMblguIA0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SxdVNQz1yw2axgmM9F16czobZ/4nc0Pb+j/p+kCd1AYPxtcjl3v5C8Ac1KS+WxEJA qHbVrLZrGxWROP255woP/8UgVivsjW7fQKSmK/u3U1YhM2Y5yfKSBcPBnm9hSvwETb tHwuQb3AVPme+ZKcFSzgpl9kj5oiBQM4daNIhzVFI3bO980a3e7XE7995BirXv6581 ZNtaCZgVAX8hzxySY1qD5hfTenjzEHUfm9R+fwANUH/T/lBx0ZWNyrtlZH/O8s3ftT moOpR1aUdc1DFPo49MVi4sKcDA2ExbcNzbWbRSz81ROx6l14RmY1ZjK+/txBfF/I2J DfRyE8ct66jZQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 05/21] mm/zsmalloc: convert obj_malloc() to use zpdesc Date: Tue, 6 Aug 2024 10:22:52 +0800 Message-ID: <20240806022311.3924442-6-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 84F5916001D X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 83eyit7a558n855mp6d1pgrd6j1jewo6 X-HE-Tag: 1722910694-394186 X-HE-Meta: U2FsdGVkX1+pQmD3+pL40VrGdOW8h2yzXwMjSlvkfpHGUIMf5dbN1kooer7s9xKs34ffYPzmEZZiKvaCmJ4sAMM4X6W7RWeuzpaV3C+jdtnj+/hOb2/xn3zPLL05gLCBIZMOvcLg9p/0+wSUiMMiAf4xx7Z1LKDVUiYQuZv5bvVg4dNxV3OCI+lDp/iIxK7BjMhxmfEd5F8dF+iWeWbvewlhY3hOMOjHuEZ4lqey0soX8eO1TX87jESgWp9T8VXDp46+1dtKy2qzaZGy29+e5i2NJYSQPDYINBQqoZ5WTXP7TTcGzp8YETWaNZ9GMdKLkWfIuYpBFlrT1pdInuvTzNwoJF90WY9ubqzllvXd1IRm6HWUlFiGJyD9jZktXi1/ONJ34ylwNpfEDw3IOJ8PmVhR4vgnwCaSGi1RqrcqYFFpkL/lcSOHcO0i6vnx6zz5wjLEJrdwhiRs3quYC4VwwmmE7OQcLr++CVlqKrbkLJLjgYKOOI+gWcGUOok2hajfZG+6BaNh62r9CX3rsfFIlY1gdV8osW6n8D2at6nq2aBqclBOksZ2r9jSTegdWMHf3t9vYwtAYTAF5BY7TygbHTWzexVbDhHeJ0V4mYOXRp9Pon8sCuCMMZYiEz1U3FhjsfUqgaovR/asjCorRGpmhQD0NshlkA8nkUpyIfExakJi3YvBQGh2MqGN5jQaACA/YupBWva4zYSwlvILd0jhLDJmJi11RW2at44bQ3d2Ekx9cLBkra195ziMcVtS+FCQoQcoI+Lyhay1MvB9B8ULsHHMisqN58bvA2stBrtWhiKAg9QZWULqX+yjEOp/XE6YcXL/M4fTiDs2ut5QzBgxPRnyA0hmNMvmC/fFokVxHHyxll68A3JNs8PELcokbh+sun8jLPwSFemfD6lgpki9DDBUnU258PJBJme3Ux2kHPATN0m/kMnAOqwJt5ED/X/eyg2ioBAtl63hRtnjm4e fYyZHPth rEZTQLIx6XhdmAy87kNf4hIFfXqspEF6OT+c4noE6OtCOB6Zfr1H2LsFXXSfQpIinn2NzXWipu9nq23S9aUp70/UXLOPdvyLmtAVkDaVcCxSvou1Uw2VpFkS/bpKKE7Om1U37PAFg4PMaxD55WaDmzqc1KFX+vdqATv+JTfPhjF6d8jhjJzaOP9gWxR7wVofzIetq2bIFb6Q2rQkqoVgMDEAihheWLsE3ugtcRfzjdaxqDb7Bq+HEYFphGApvFIFJAetRsOJMwknToznxnbLiNRba0n7TeYmd0onYYTcUr4DQMG4yR/Dw0OP9QkopCUP3fSwxOt4xeVIcZKtgQskr9mr00YbNB56W58cgaUgaqrQhin9CdymuYMSZdmdAD3gBTf7e X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first_page/get_next_page. no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 384a5ba49788..7421d7678880 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1322,12 +1322,12 @@ EXPORT_SYMBOL_GPL(zs_huge_class_size); static unsigned long obj_malloc(struct zs_pool *pool, struct zspage *zspage, unsigned long handle) { - int i, nr_page, offset; + int i, nr_zpdesc, offset; unsigned long obj; struct link_free *link; struct size_class *class; - struct page *m_page; + struct zpdesc *m_zpdesc; unsigned long m_offset; void *vaddr; @@ -1335,14 +1335,14 @@ static unsigned long obj_malloc(struct zs_pool *pool, obj = get_freeobj(zspage); offset = obj * class->size; - nr_page = offset >> PAGE_SHIFT; + nr_zpdesc = offset >> PAGE_SHIFT; m_offset = offset_in_page(offset); - m_page = get_first_page(zspage); + m_zpdesc = get_first_zpdesc(zspage); - for (i = 0; i < nr_page; i++) - m_page = get_next_page(m_page); + for (i = 0; i < nr_zpdesc; i++) + m_zpdesc = get_next_zpdesc(m_zpdesc); - vaddr = kmap_atomic(m_page); + vaddr = zpdesc_kmap_atomic(m_zpdesc); link = (struct link_free *)vaddr + m_offset / sizeof(*link); set_freeobj(zspage, link->next >> OBJ_TAG_BITS); if (likely(!ZsHugePage(zspage))) @@ -1355,7 +1355,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); - obj = location_to_obj(m_page, obj); + obj = location_to_obj(zpdesc_page(m_zpdesc), obj); record_obj(handle, obj); return obj; From patchwork Tue Aug 6 02:22:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1424C3DA7F for ; Tue, 6 Aug 2024 02:18:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7F6136B0089; Mon, 5 Aug 2024 22:18:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7A3BD6B009E; Mon, 5 Aug 2024 22:18:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5D12C6B009F; Mon, 5 Aug 2024 22:18:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 422D46B0089 for ; Mon, 5 Aug 2024 22:18:20 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 09012140836 for ; Tue, 6 Aug 2024 02:18:20 +0000 (UTC) X-FDA: 82420211160.09.59F247C Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf19.hostedemail.com (Postfix) with ESMTP id 4C5071A0018 for ; Tue, 6 Aug 2024 02:18:18 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=CNjv4Yt9; spf=pass (imf19.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910636; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7v3g5GxWR0jl5mkBSc+FR4m7cJUyvgF1oZpCugoaep4=; b=TE2UpcZwaXVb4IUNyktnvKutZym/zEejWWp5Y1ANGnfGGJpQICAtCN+3nGcx8zE/FD8x1e 6dIw1sjTmLUyUEUTtclnKcN9+TXw39NUU54JbYwChs45hyu+PrgUzgt247lPE9iHYV1+EP H/Q5B5yZzVgG7gKN43WgcNvEREqskTQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910636; a=rsa-sha256; cv=none; b=OfY+yNkFw39c8nBi1E/dXDEKLnuwDSALdGCbyKMR84cDe0IY5Jc/3S1i5EmJnIqlD5mDyj u96W7gxEELj2OncK5pzPfC/e3sYPBKrg4GjZcVLs+mSWdgPLUAJJn9SmNKeHbG2klpIMYS kx+uhD6n81R1lpypbOmU1wIntSNWt38= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=CNjv4Yt9; spf=pass (imf19.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 917D760ED0; Tue, 6 Aug 2024 02:18:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1EF94C4AF0B; Tue, 6 Aug 2024 02:18:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910697; bh=Hcz6YDLKpn+zwSbhAqxdxpFHMpHIDCe8idbfi+gvSSM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CNjv4Yt9eT8l1U83xyPDy6BZuirk5dnxQU81nTlT/y5ULIt1rjedy+R5ACeEBWVBi VjVKk5NmdQP/WWEUzGqrzT/Ogyykzw74HoNE1d2Bb7YfG/8xWI5ax3dCf6Xr7roZDJ Dk0iu3z0Ib7D4ATLidLr6RPt6z1oEzNj6mPt1ao36Kolg7F5BG4mpOYNep/gw6xur2 NfdoPAuQsnTHxwzxtvJfi1g/1TMDQVBATGFt2nKp1jWMG6DhjD9cuTQKfitgFhUrXi I8gLccD4SMhAWkrUe+LA6YXXw2sZJsJLCqF2lkEQ87ruSuH8FI/KSKK6CgG2hLMUbn LHxQHsK6dtWzQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 06/21] mm/zsmalloc: convert create_page_chain() and its users to use zpdesc Date: Tue, 6 Aug 2024 10:22:53 +0800 Message-ID: <20240806022311.3924442-7-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 4C5071A0018 X-Stat-Signature: hf4kdmmgjiuw5ryii1iywfqw81tirgs8 X-HE-Tag: 1722910698-977852 X-HE-Meta: U2FsdGVkX1+7MnAhifGowOc1BQkH454qw9Z4/uZ8OzqGYFbjx4aYSAhS27szO6A03xkQJOPVzN2CMQl9oRxsgX4BMeptMPchzvsxGDkWBvTrrLgreZm/Vsyh8K8tENpqmeiVFlq6FgldCpUo/L9WhVCM3/Be7MSNigKpMWud8bEVt69TMZQQPs4R5QoZ4U3jdELS4ZAYJYHn2FMEPOPd0eYPZ3EvlucfSyQAB5Qz2TMhPu4tb5ijPMS8GfzcurAv7ZH++n/zbSGF3PrFKFrGO7FOUrafVC3I0cxrSC4D/Wd+RNw566fPa5P2eelLs/Rm/bbwyalaJ91aoaoenfPsAgseeV2Qz7NdB7DasBfXMVD1FvMenL979uwCjuPWnRcZjHx66e4epNsxOD0dFCDk+SpHW1T7PbNu5qgnUEAQ6KpPSkHAFUAdTjeBEjJrSzR5TZEYyXU0x/UWJ0KjyrndYtOOj1vAYi0ubs1ojL9QXmQyQaGw1TFhi7J/8r76Md+8voiAgyxMU9NQTQUUvZed73Wowrylafq7J33R4VRqyGjq5zJRAWQZO+f4emyFkcYsfiX/aOvRProeLKmNS7OlSI4UXYukuJJ8YKodQSMOSwux33YBFRv9W98t6NFVhKCxUhh3k8dRNhRvcasBUHaJsklftWLkIEZPXK4KlN+JyMvhE4knfSz0qV9vcYG4rBtoYXVsTorc60fbWbPxy93ZLL1pRHMA9cQDSnS/iTta5cgvuyKyfsIGkA12kXSFAZLVcjJIeCn31qQur3tsipjUaY6eqEgZ6W9ocIAVQLb90mkahBuAfpO4qSjVJoi9ducPa0THRNnIlEPbC6hRJ6T/1AWT/HSBB5pefWlboY08L+3NXgKAuRlfMKRTnbFrenGt66nwqGQhu+bDtMwoGsLtgCWs3QjFWUP0nniivpqgBwuGy51zUUpeiXH++Tfl2vAOheYEEHWorc78cDsa+3M /KkYzmMr Uev+7XzQWDSp9seUUDxzs93lPmTQTtFCV5oikZ384/1WRw5ozI43Lc9x0/ab/S+OKSIBQ0zEbWaRHVXdKA41mzVYqZQvCIO7AXsAhGOdS4xiMAk6/wp/tPxFciq+Gpqm3B7TkNfGMrh0H5YIFVUkIq9sK2FrChAX2RkBeweC3qd4+vfAhWKJ7ptZ70m7fPMwxCIvyTPXb0oHu5Psn3Ak/st2Bb29s85OLh58e1cbyfJSHSYEokILkHvqfuv/i77kuLNjhB9IpWLhIgHvh3Cl4YlI+BIqmLBT6cuT0bYsvh6eR2Qr6R2Ns1gzxGEwr6kr2BD/6O73dBiDMq+AjxqnKUdj3eqHQP54iifJmKZsa+es5+A9HrvEKS8BqU3FPyFrx1IqJ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Introduce a few helper functions for conversion to convert create_page_chain() to use zpdesc, then use zpdesc in replace_sub_page() too. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 6 ++++ mm/zsmalloc.c | 95 ++++++++++++++++++++++++++++++--------------------- 2 files changed, 62 insertions(+), 39 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 2101de23d16d..6a01e09e4beb 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -107,4 +107,10 @@ static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) { return page_zpdesc(pfn_to_page(pfn)); } + +static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, + const struct movable_operations *mops) +{ + __SetPageMovable(zpdesc_page(zpdesc), mops); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7421d7678880..33a4f7d026e3 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -248,6 +248,21 @@ static inline void *zpdesc_kmap_atomic(struct zpdesc *zpdesc) return kmap_atomic(zpdesc_page(zpdesc)); } +static inline void zpdesc_set_first(struct zpdesc *zpdesc) +{ + SetPagePrivate(zpdesc_page(zpdesc)); +} + +static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) +{ + inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline void zpdesc_dec_zone_page_state(struct zpdesc *zpdesc) +{ + dec_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -954,35 +969,35 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) } static void create_page_chain(struct size_class *class, struct zspage *zspage, - struct page *pages[]) + struct zpdesc *zpdescs[]) { int i; - struct page *page; - struct page *prev_page = NULL; - int nr_pages = class->pages_per_zspage; + struct zpdesc *zpdesc; + struct zpdesc *prev_zpdesc = NULL; + int nr_zpdescs = class->pages_per_zspage; /* * Allocate individual pages and link them together as: - * 1. all pages are linked together using page->index - * 2. each sub-page point to zspage using page->private + * 1. all pages are linked together using zpdesc->next + * 2. each sub-page point to zspage using zpdesc->zspage * - * we set PG_private to identify the first page (i.e. no other sub-page + * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). */ - for (i = 0; i < nr_pages; i++) { - page = pages[i]; - set_page_private(page, (unsigned long)zspage); - page->index = 0; + for (i = 0; i < nr_zpdescs; i++) { + zpdesc = zpdescs[i]; + zpdesc->zspage = zspage; + zpdesc->next = NULL; if (i == 0) { - zspage->first_zpdesc = page_zpdesc(page); - SetPagePrivate(page); + zspage->first_zpdesc = zpdesc; + zpdesc_set_first(zpdesc); if (unlikely(class->objs_per_zspage == 1 && class->pages_per_zspage == 1)) SetZsHugePage(zspage); } else { - prev_page->index = (unsigned long)page; + prev_zpdesc->next = zpdesc; } - prev_page = page; + prev_zpdesc = zpdesc; } } @@ -994,7 +1009,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, gfp_t gfp) { int i; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE]; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE]; struct zspage *zspage = cache_alloc_zspage(pool, gfp); if (!zspage) @@ -1004,25 +1019,25 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, migrate_lock_init(zspage); for (i = 0; i < class->pages_per_zspage; i++) { - struct page *page; + struct zpdesc *zpdesc; - page = alloc_page(gfp); - if (!page) { + zpdesc = page_zpdesc(alloc_page(gfp)); + if (!zpdesc) { while (--i >= 0) { - dec_zone_page_state(pages[i], NR_ZSPAGES); - __ClearPageZsmalloc(pages[i]); - __free_page(pages[i]); + zpdesc_dec_zone_page_state(zpdescs[i]); + __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + __free_page(zpdesc_page(zpdescs[i])); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(page); + __SetPageZsmalloc(zpdesc_page(zpdesc)); - inc_zone_page_state(page, NR_ZSPAGES); - pages[i] = page; + zpdesc_inc_zone_page_state(zpdesc); + zpdescs[i] = zpdesc; } - create_page_chain(class, zspage, pages); + create_page_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool = pool; zspage->class = class->index; @@ -1753,26 +1768,28 @@ static void migrate_write_unlock(struct zspage *zspage) static const struct movable_operations zsmalloc_mops; static void replace_sub_page(struct size_class *class, struct zspage *zspage, - struct page *newpage, struct page *oldpage) + struct zpdesc *newzpdesc, struct zpdesc *oldzpdesc) { - struct page *page; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE] = {NULL, }; + struct zpdesc *zpdesc; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE] = {NULL, }; + unsigned int first_obj_offset; int idx = 0; - page = get_first_page(zspage); + zpdesc = get_first_zpdesc(zspage); do { - if (page == oldpage) - pages[idx] = newpage; + if (zpdesc == oldzpdesc) + zpdescs[idx] = newzpdesc; else - pages[idx] = page; + zpdescs[idx] = zpdesc; idx++; - } while ((page = get_next_page(page)) != NULL); + } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); - create_page_chain(class, zspage, pages); - set_first_obj_offset(newpage, get_first_obj_offset(oldpage)); + create_page_chain(class, zspage, zpdescs); + first_obj_offset = get_first_obj_offset(zpdesc_page(oldzpdesc)); + set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); if (unlikely(ZsHugePage(zspage))) - newpage->index = oldpage->index; - __SetPageMovable(newpage, &zsmalloc_mops); + newzpdesc->handle = oldzpdesc->handle; + __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); } static bool zs_page_isolate(struct page *page, isolate_mode_t mode) @@ -1845,7 +1862,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, } kunmap_atomic(s_addr); - replace_sub_page(class, zspage, newpage, page); + replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. From patchwork Tue Aug 6 02:22:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754324 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A960AC3DA7F for ; Tue, 6 Aug 2024 02:18:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3DAC86B00A2; Mon, 5 Aug 2024 22:18:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 385A26B00A3; Mon, 5 Aug 2024 22:18:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 189FE6B00A4; Mon, 5 Aug 2024 22:18:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E6DE36B00A2 for ; Mon, 5 Aug 2024 22:18:23 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A69A0A8A29 for ; Tue, 6 Aug 2024 02:18:23 +0000 (UTC) X-FDA: 82420211286.08.C4F56F2 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf07.hostedemail.com (Postfix) with ESMTP id E3E2040018 for ; Tue, 6 Aug 2024 02:18:21 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="FoXry/KR"; spf=pass (imf07.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910619; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TK3D9BhHCud4UbGasCGJxl/q7H+vBmcvyIb8ehzvADg=; b=bvi9Z4AgdWTZGuF9tIkmVFR6JN5WB8KhvnrPevcx7revdC2pX/p02+O3zUo06RsT6/CzPP N7VWuaOjsgbVNvZnoY2gpbnf458KJ00NmN2uA3X3hoMqmzWVBzJe/gx3tGtQ0IknKoFftx jqH5ogXKReELmRDpQ2JrAGeJlN278SY= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="FoXry/KR"; spf=pass (imf07.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910619; a=rsa-sha256; cv=none; b=V7EaXlOhaZCI6ZwzNpKVWzy7k7MXYt43soS+OM9qE0VaBZApmphzs188mGhFqaOOUr5fWp /65ktn4OiAO/us6Bl41QbhcOWYk8BGyOLPf9JgdTXXkWsgBJexex1CNHASfyK+kyOtgi+O octdOF4f4nl+K3e+1xarNEUZaB4UHkk= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 1F21E60EE4; Tue, 6 Aug 2024 02:18:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E7EBCC4AF0B; Tue, 6 Aug 2024 02:18:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910700; bh=S9lb/ZM2ddFk5ykU4pBIY4i0BSbXkh97cFi/e+1K9AA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FoXry/KRy6McNtYGfI2Z6enXmj2PWAy7b9Lk8XnIwOINqaCLUi2tLUaqmmtgbrpG/ MxbLCcc0DyuZARER7WhzK6gSVOwnOsHFc6/RIAwicCSRIGdGeoRD1WcpwBuRgSYNlB kH1juNG7IOd1xdgQDziBMjpAYa4BgcHE3F6FXY4SanlKYCBX8Dg00AfglkRQ9Md6O4 lxZENUCGEbNYf4+TLt3zOZ2H3MKXoY9KXp+kd38tme2pa0wo3SFHn68t6XoGliUpbi mfFhW6n+LPNt4zSKnSof8Uoueji5zXnGJ0yBZbj3+XJqo46Rfrfv+PmvZHTQMa1OTC 4fc970Hz8u/Hg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 07/21] mm/zsmalloc: convert obj_allocated() and related helpers to use zpdesc Date: Tue, 6 Aug 2024 10:22:54 +0800 Message-ID: <20240806022311.3924442-8-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: E3E2040018 X-Stat-Signature: z7r7hcehkzmd9ion1geg7zynuux85swu X-Rspam-User: X-HE-Tag: 1722910701-298965 X-HE-Meta: U2FsdGVkX1+giJcXh0g1FZwqR1OxUZ4op0VDgoo65Rug2pSKJhc/bfvFSI9qiwDNU0f959MBRZZr4EW9HKHtf41R2XknLDp6FVQdloOgCztwQSeByzkhHTh6fvXXQ5S9ccvGmM+gK3zw2rvrRk+yPjnJ3mifW4rr9bbHDPwaa8wFdk83kiWUMcdbtPqNN/kpHCgha8YAbKZspA6cF2+KE63CyENzpsZEA7+xzBoSp/BX90lML7teVy/kawhI2RzEKl49DN1K6BCylGbgsKtvF5G0g4nl56czDAQxYUuPYVbMg0PGWB8pIj3yl4p0M5ak8iDtEtBKg78NB19AURs2c+Pu9bAV9b7eT6PuOTL1OjpoRWv9HJ6MUgJMjYKI+GLLEu2bO9qAzW1n1CLvaZuR5usFoyrxkFGcM0WE9yuEIs0gjVY9WFNilNgSkTFxx7fuRMTtMuN3Tp0QGmi5wNzI5oDIrks5BSmhprxUHtAg1x5UclIGQpq6RBI+Oa6FPImZFfBy2UsZCaqZArft+gZ1N0dAKbbCvMHZ/6w/QhzWROySiAQpT5TqP8vGBQL2pfeeadBAuYv9Nav8q0gJUF1zmdiJeWrUUlQ+oe/Oz8vfoGpnEfJ9gxIYxxIB5P/4Kpr3iPcJaasawVs1evVqYAmi4FV8jr4cM/dST3GsCa2BrhM9bSt1VHY9CqT4xcdBd5ilVBOLs/YJ/aQAnR8Xde7hmmJR0BRS9cB8eBTKAbXeenUKzjDeIfpOxbssJFjXLrmUjKjnpjgD50K2jjw/zgHLbYR1y9vXkVacDKY5ifmuLnOjQjW8M68BYxr46XfXXU52vbFRmOIKMUvBUukDJg1/6rc5oZnkaezMRiuyHH00aFYtQMzbZtKM1XXQeGMLhmJF5NqdC3u+/Kkxc2YkkuQ2LTDRY/Tu1/nbUnjoVgENYFTRn9BKFVEBbFvfxwwbthTeNOAktt+N+re9sTsOLgM RZelhVU4 M7vrmwyfoo7zuo/hE9w0OKr3qDzDszk6KNOvcE4F+4zQsx4UDgVBX5cBof01GuepZU+2XoMpCGqPKosgRlDzbTKF4x/gSH0J1XlzsOPY24c0AclZ7CT/9VqJDz5T6cSlYgpngJSMCTig7O0aswFCE4XbFRF2twh7AuWx7mBVx4ggcynTCvoL0SMKp4dECCyZT/k60ehOiP1E7vX1a20BdCOkS81A7Nq0fCBY1LIZ/xTWaitx1qoEG3rRmPhUk+8k3J1nVKjki4mEL4DRo3rlgL8NhnVg09kaG+oDn30wOC4tUi+r+HPfa3ElVUzX/z/GTmYsS5eYQGMgd2bEK5pRPdykmkuWgM2qvndlyCjFHf8iFiiJeLLhMgl0HOLvaPKuzo+S/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Convert obj_allocated(), and related helpers to take zpdesc. Also make its callers to cast (struct page *) to (struct zpdesc *) when calling them. The users will be converted gradually as there are many. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 33a4f7d026e3..3c1755227706 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -825,15 +825,15 @@ static unsigned long handle_to_obj(unsigned long handle) return *(unsigned long *)handle; } -static inline bool obj_allocated(struct page *page, void *obj, +static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage = get_zspage(page); + struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); if (unlikely(ZsHugePage(zspage))) { - VM_BUG_ON_PAGE(!is_first_page(page), page); - handle = page->index; + VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); + handle = zpdesc->handle; } else handle = *(unsigned long *)obj; @@ -1583,18 +1583,18 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, * return handle. */ static unsigned long find_alloced_obj(struct size_class *class, - struct page *page, int *obj_idx) + struct zpdesc *zpdesc, int *obj_idx) { unsigned int offset; int index = *obj_idx; unsigned long handle = 0; - void *addr = kmap_atomic(page); + void *addr = zpdesc_kmap_atomic(zpdesc); - offset = get_first_obj_offset(page); + offset = get_first_obj_offset(zpdesc_page(zpdesc)); offset += class->size * index; while (offset < PAGE_SIZE) { - if (obj_allocated(page, addr + offset, &handle)) + if (obj_allocated(zpdesc, addr + offset, &handle)) break; offset += class->size; @@ -1618,7 +1618,7 @@ static void migrate_zspage(struct zs_pool *pool, struct zspage *src_zspage, struct size_class *class = pool->size_class[src_zspage->class]; while (1) { - handle = find_alloced_obj(class, s_page, &obj_idx); + handle = find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); if (!handle) { s_page = get_next_page(s_page); if (!s_page) @@ -1851,7 +1851,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, for (addr = s_addr + offset; addr < s_addr + PAGE_SIZE; addr += class->size) { - if (obj_allocated(page, addr, &handle)) { + if (obj_allocated(page_zpdesc(page), addr, &handle)) { old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); From patchwork Tue Aug 6 02:22:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754326 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65967C3DA4A for ; Tue, 6 Aug 2024 02:18:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8BEC66B00A6; Mon, 5 Aug 2024 22:18:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 707B16B00A9; Mon, 5 Aug 2024 22:18:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4BF156B00A7; Mon, 5 Aug 2024 22:18:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 228636B00A6 for ; Mon, 5 Aug 2024 22:18:31 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 923F440512 for ; Tue, 6 Aug 2024 02:18:30 +0000 (UTC) X-FDA: 82420211580.10.879E037 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf22.hostedemail.com (Postfix) with ESMTP id 98CB4C0014 for ; Tue, 6 Aug 2024 02:18:28 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=HsMWYRZJ; spf=pass (imf22.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910647; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IavPEFbNVTr4w9OfQ24+1A+3hl7N9lwrjw5Ixde814s=; b=QQZRZ65ftybBTcReTJG7EHXkf8tAHMiwoPsa+vp6znMNWGunh8D9Fp6dBN3jgFxeXnsRRd 3oXPAa237dJxlhESXjcRh4Rws2ooG8saJ1/9CO39yCT7ugHEqtmXYzR8K0tPB6wxH+uCkA /O1OSa3H+ARyQn+625Qb9WKo6KoGel8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910647; a=rsa-sha256; cv=none; b=izwsjyIxkL3sSyH4l51+8xjQ3ygV4ZWl7Jl4KXFpBo+LfSDHkwA0m/DKJA9h4sD3SoijpH 7+3sVqDRDSU0Uudxe8VDM5IGW5S0JcOwZXWAiLVl3FI/Bz6FctaaQ6dQ5VEAkX/dCZmDng bal9mSgZ3QwBzRjd0FGQ6qLsfDk8snI= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=HsMWYRZJ; spf=pass (imf22.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 71A3ACE09AF; Tue, 6 Aug 2024 02:18:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9359EC32782; Tue, 6 Aug 2024 02:18:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910704; bh=vNwIGyy8uUag3zQP5bXF/PNe/niCN1bQWoZ3dk5XDpg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HsMWYRZJUqU0Itrt3J8567cHgcx7ZWcgbPSnbOQZy70q7mwkndUTL+/3BqxOm3jLe 1JRXz3aiwZLokB5iJOIAxSzflNxhwrdBNUmQ0yr+fTHmP6UHPnfZXf7ixPXMndCYtR Xvx0NYmeEH8VFJaWIPEcpkE44VAf3Iq0589/x9KYqXkzoR1v4hFCn9gXAp43FRTRuB 3gwiRk+9fT3CtV1V1KDMvvYsPFFrtc+3sl1sU18WkpAkEgFNlSYItJMcheDweCDUcj x6cS/1tCwsSUmImOLOvc1iIP45X2TJ1nhJOZhMkYGSHW81OmWHotixv+CqQ+uO3/q4 WUgiwGNqY5X+w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 08/21] mm/zsmalloc: convert init_zspage() to use zpdesc Date: Tue, 6 Aug 2024 10:22:55 +0800 Message-ID: <20240806022311.3924442-9-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 98CB4C0014 X-Stat-Signature: 19gwm3si68zj5en4ezogz68jcoikzeo8 X-HE-Tag: 1722910708-667819 X-HE-Meta: U2FsdGVkX1/iFVOFMPd2mP0LVsr/+21mxA6kYihHKalT2ukPgDUDfXk/DsYPbGSG4twse/ftfitsplSrC8uMOd9T4zfT7WrwDyqjNVcMZoDtKf+2lQbY91098R5jbIkcfmddb5ad3vYLR3aY/sE9hPKO60d/Jeu1NhweQxDUo0qX2v1S+bFuCJ3iaEnwuN8QDx03Qs9cLp6/KHn8OHjMZYKMOEu55eJY+ED9L42W8j5nmdWY5uuFkd1K7Z5SKNFx6hz15JyFl0J49maKuAShze7rQCNp5pj0xc8p6DYiYlXB5Qy/l70cIUzVe8zDACZuq58FaSYBEDXvTcJHXBHBFDvetPFtgVvzAbhAjdCCzQIzEh1Og9h7J/2yaMBoMGhr5N4OuBjxFio+Ky4WFn9NWvpLGPQtF9P2IbAGA3kAPcxK0Xeb/PFdXEg3ju/nRi2fPU8b5cf3Vl1Y1xCIxoE5DQXVluARA2qsv4uLqRlM0RB9xzJ2ROKKdIODJ/O+/rsLGgBF75gbfxbsMVR7xhYxbju0heHjNuE1ueoNSOplMi9rMoK11bl07pQ6SnJgxdU4h7Gkx0480wqK2Is2a+LDcTLOOitw+o4b4mv7JWWou7aMJFqOpm/pi4k81kXUHIjvTdIM0wwqgSxKVXfGm695ZGVJFatVkbwtIggRStB43MNWys6shPlGzbxfAdrz9eYCBSz7RhtXWkSpW2Rg+IeJhtY3/Ij8caHKwSZZeVQ8NIEAvM86sfOihISXMparqJkJcT47jGLcpCiZLlLQ8j0enqycln8JowqbLCM30JkkLU5sbEb+8elep9+fVdfIVkdHsWtXd1Stb3OusraiJea+3kGP3DvA6UQZ/1fROzjMutuWWTvlq2EdFDNoaNAgBRM+PpWe4lx/XEzBFNGpj+FBsOl+X5kSajj9arkmdQ8Qf7fC7xJRdUjjmzAl63Z/fQ2XFUQ8JcDi2Gzrw+Najjk O4k4UU0Y OvUxjM8y5lU+jslOCQi58qxfa+aX/CUtpQXJGlYo6E+vbHj84Pp79fAsaUK6oVV59okMVdhjS9aEe7EnhvSDRTNNkVbA7GoxqTxBnd68bXMug8/KHsaWd9S/RfWkElCtRaufVfcqDg5QFp/r6L5R2YDIY+Mg5+gYgD0LfjXRAXqRioiihgkAMn947d6ZlJcr0BSKTbMW3zkVYhPFSEYIM2+WlR6L+GhHJPFAk4O9pPwSYfy4NDX6up8mD13dpHNyWcdRca6LKaoFrIpKxYhemv+pkkUlNOjePTP1p6ca+00zj9HqjrTpaTQuJEjQwUH2pMms2lSOQkMVe3/msLY8ADRKCW2EDicvWcIY05wv6J2Bk9LyO+Cp6SuWKt60z4vrd3w78 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Replace get_first/next_page func series and kmap_atomic to new helper, no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 3c1755227706..d7400acd0f01 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -928,16 +928,16 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) { unsigned int freeobj = 1; unsigned long off = 0; - struct page *page = get_first_page(zspage); + struct zpdesc *zpdesc = get_first_zpdesc(zspage); - while (page) { - struct page *next_page; + while (zpdesc) { + struct zpdesc *next_zpdesc; struct link_free *link; void *vaddr; - set_first_obj_offset(page, off); + set_first_obj_offset(zpdesc_page(zpdesc), off); - vaddr = kmap_atomic(page); + vaddr = zpdesc_kmap_atomic(zpdesc); link = (struct link_free *)vaddr + off / sizeof(*link); while ((off += class->size) < PAGE_SIZE) { @@ -950,8 +950,8 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) * page, which must point to the first object on the next * page (if present) */ - next_page = get_next_page(page); - if (next_page) { + next_zpdesc = get_next_zpdesc(zpdesc); + if (next_zpdesc) { link->next = freeobj++ << OBJ_TAG_BITS; } else { /* @@ -961,7 +961,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) link->next = -1UL << OBJ_TAG_BITS; } kunmap_atomic(vaddr); - page = next_page; + zpdesc = next_zpdesc; off %= PAGE_SIZE; } From patchwork Tue Aug 6 02:22:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA78AC3DA7F for ; Tue, 6 Aug 2024 02:18:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 68B6B6B00A5; Mon, 5 Aug 2024 22:18:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5CE8F6B00A6; Mon, 5 Aug 2024 22:18:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 421536B00A8; Mon, 5 Aug 2024 22:18:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 211C86B00A5 for ; Mon, 5 Aug 2024 22:18:31 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id D4319160846 for ; Tue, 6 Aug 2024 02:18:30 +0000 (UTC) X-FDA: 82420211580.25.2482A03 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf28.hostedemail.com (Postfix) with ESMTP id 4E970C0017 for ; Tue, 6 Aug 2024 02:18:29 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=QBV8krES; spf=pass (imf28.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910660; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=loEoxznJVMRwxobcpeKv3MGdFFg8glW/PKiOuSU8DAo=; b=wOhzsF9YIIjvzSxgkonpfxeamn2C+BjgfchTK3A9KUtZfBZCbPvaFeoeV2fpGItqRpo8yV j2yM6WfKSRHl3TxSrjapmJehyKqxEe8ZdJQgCOVc4RIcPBub3lIoaLyDH8OOE0ij6ThRzO zb8lUdTKfVIP9xtq33ESUfShOqIbCcA= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=QBV8krES; spf=pass (imf28.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910660; a=rsa-sha256; cv=none; b=neCEgC0CD08gQgD8ynWAzVtRrf9qUS3K75Jzj2RHpPtCW1TbVB58On+wBNSkdxckimBT1P 0SgO6pDAGDcT2HxV7eAa1mQAO3o8vXs2Jgy4j4qkLlcc8g+iKZQMPf7EvEhEooHX9rvdJj BoEkDVDKhuHCHpWOHCOIau6W7zYpa5M= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 93ED860E93; Tue, 6 Aug 2024 02:18:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 66673C4AF0F; Tue, 6 Aug 2024 02:18:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910708; bh=2XuUwCpmZ+NvY5bJ3CJbiV5pb57fdunEaUBcKFqNcHQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QBV8krESkVQBI1btdudXp3KnYlXCSSkzzl+6/65HHV3ctpQCaq1VOxmims7maxwHY wrBU3toUz1xv2/xTaEdWgIGqSdlbQT8ALHlvL6k9uSg7FUZoD2n3R9jk12A6+w3PPG 4p9Qjo2J8NR/4mJT1z2wuIu0hKQsn8J025ArD7UIENklvSYRCOSEarp8LcmPNjEITN fnG0iA0wTar2Lb2bdYWWc9r/imZYQ83co9ypl4Ct5PTBSrK62sZ/wIQDLljIBPQAks EsRG+glMxynyTvNGO7qfPVnLA3IEmk1uoQC882anPHSZqeNAp7qpEJjUJ8+3Lp09gw whxLi1wfKYSfw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 09/21] mm/zsmalloc: convert obj_to_page() and zs_free() to use zpdesc Date: Tue, 6 Aug 2024 10:22:56 +0800 Message-ID: <20240806022311.3924442-10-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: joas78mgqghe3twqqkorrrg1jsaakjfr X-Rspam-User: X-Rspamd-Queue-Id: 4E970C0017 X-Rspamd-Server: rspam02 X-HE-Tag: 1722910709-239819 X-HE-Meta: U2FsdGVkX19b7e/iaReVmwTVDqPkT1S2uV2IHYvVrBOa6UDJhCtCVogWvrrMj+x2qYbvLYr3fXblLcQcqQoJJfXg2cjulCi+zxp5MrJfn5FXoTsRH9geNbmoUU609jIATA46M0Zw+5MGsvbCQ9TUq2n3dMt1TEVZu/9IYluzZye316nRzJ7xmT+k0r0gw6B70sHHwcYn9lDyk3CXnVKc7tVIeyDjoeKyUFVUMIiDj5hb8fvSCgtmPutmYiadFM0aM06VyaNaa2VA75DtF/GLuVefa/48PyRWu81jH5PFCWstOSfqIt6HvGGSuPQXaupzJAtK1jLCIbQ6BBXzXRw0qDrMBUyHRoMtrJdRFGHwtnRgts+EYJGsgHCq4bil75UMhrlBhaLb7paIZ8ToCPwEPvq95E+UAI/jZTho23LaSP8Mm+Dv1cd1KKl6s1aBPSp/aqwm5B3ZxeTj5e6JVe1Zv7g/3gCEYBQkf0vzUdTMaFnXNS2bKIxKM+S4mB7HAy2apSbD+VpTdQQyJMMX3llAOy3aH/r9PTjI1PAznW63S6Rmb08f22G9DvhAVlHT16uIl9tqqYDP8LRQ0LBBAnZZclrelS5/qhdbFTqZ0tuv9BUQcn+ZXCf/CN2QK/TrhR4KcZ0YD18oXDQvfbvFWiv4Voav7qFc8q3fk0Dn7GUStcl2zxaY6Gssmg363G0/eCoW2IGhHw6rSvLpwF46K9fgnFYjDjVWgWL9JHAxsIMRLW8CEWMQh28OEEPAZJMSTl5LMROQc+cq1B6b8NFaiVztxpWJzu/ORkE+4/ghI3Do+cq0jQhTyWZh8VkahBDSGjs+mWcEoL8r1Ku9jzgftFsW/Yr+qo4kfT5F6nJIhQP2rW0UdrXA4GThLBPWXh2xNGkG8ChqEmmYGIswtYw5pocm1SXW7N+1OjO6DoiCsvCrOuHZFdDpaFH2LF5BCILrxGuHSdCKhKOh7donHHLQkjb EW6ELkpg yQ0LEIQGDjdDD08D4scN4WRF8AE23UOnap1eG8uISSwUErJ2jSmRIq7lFtGqUdCkX5UrqYmEtowGJMq2Mfvc/oXbL9FLVZx70+08g1Va9o2ymlzSPWEINJF+28Td6p2j+272olQnZkegiYIIG/5sdCbWbBQ+0usYZ/L6d7KPuvryWH/4Cvppo3oytukqr2Jm9dJZhF2/tEXaWzJqcvCmveDT45UC3I0mWo+Dbn4OcLLr1cBqWYFqI3Ll7rbMJmOe4Yuxq0Rx/kJrXr4T3m6pyADTtUh3aHsL/gWOgq0AUAQC8DYqIFfIQjkvwbJwE5t+GS0dRR4kBUUvi5t/WEoEGxUTVHJ5563YqI50HRakB6fXE6TOth4ABW0tDpvpxRl2sGlkE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Rename obj_to_page() to obj_to_zpdesc() and also convert it and its user zs_free() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index d7400acd0f01..5fac2bb436f8 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -800,9 +800,9 @@ static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, *obj_idx = (obj & OBJ_INDEX_MASK); } -static void obj_to_page(unsigned long obj, struct page **page) +static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) { - *page = pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc = pfn_zpdesc(obj >> OBJ_INDEX_BITS); } /** @@ -1476,7 +1476,7 @@ static void obj_free(int class_size, unsigned long obj) void zs_free(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long obj; struct size_class *class; int fullness; @@ -1490,8 +1490,8 @@ void zs_free(struct zs_pool *pool, unsigned long handle) */ read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); - obj_to_page(obj, &f_page); - zspage = get_zspage(f_page); + obj_to_zpdesc(obj, &f_zpdesc); + zspage = get_zspage(zpdesc_page(f_zpdesc)); class = zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); From patchwork Tue Aug 6 02:22:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754327 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4B76C3DA7F for ; Tue, 6 Aug 2024 02:18:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 61F4E6B00A8; Mon, 5 Aug 2024 22:18:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5DB7B6B00AA; Mon, 5 Aug 2024 22:18:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3AE2E6B00AB; Mon, 5 Aug 2024 22:18:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 14CE46B00A8 for ; Mon, 5 Aug 2024 22:18:35 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C16CA40512 for ; Tue, 6 Aug 2024 02:18:34 +0000 (UTC) X-FDA: 82420211748.03.9E468AB Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf23.hostedemail.com (Postfix) with ESMTP id 2E073140010 for ; Tue, 6 Aug 2024 02:18:32 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=O01JxTsF; spf=pass (imf23.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910664; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iWgbOZ2QAxS+P952aw1NkEslEDRfDsVglr2/2vNMBu8=; b=C3PcfDPGOOqiz02HyfGarl0CWGv5hdGb6XZrkw0mC8ohJyx4qaDOyu9qieQqEinWfIdrRj KKW+4BjXuuAjoAfoX12auibYxhpWaOAfKKQ8c7APXbCLlpoIwBCSZq4KBWvrJeciMUqrjp F7k3Rlg8C9TFvsnrRJF4MwbTScDX7vo= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=O01JxTsF; spf=pass (imf23.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910664; a=rsa-sha256; cv=none; b=FxwSnIrvPyk7ROEa7vctTvySDNF2yFyCXi8ER5ByT0usmVMansTJPCHe9CzwrWoBhu6uH8 MWAYB9dOyYZlaaqhS1BMk6kU/O3VW3wmVjon9jtLOCrrvLtgmSWLw8f5j5/HZ60wGuHCeG UD5q+88qP66LSExqLSgMXDVqYqC4TYo= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 67D9360ED8; Tue, 6 Aug 2024 02:18:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 13C64C32782; Tue, 6 Aug 2024 02:18:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910712; bh=jd//d/JKjS/cL1w5x1hSjGz2Vv8a8C5I3yz/l4s7SBA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=O01JxTsFcnRCOHKuPEnby1tCq8aWX74P/c0exfYMK/J/CPxNyd8q5/bPamjiRPu/D Jbdq5g/rE4MKhzhnGw6Y+VEGx66ZKCcPQDJlAoDr/Ho7DFKt3N+Ej0/Y+xm+s8hkZ1 2pbU44Dg9XOKAaX+Ya+IpbEvo2BwfU0yv8ESApwHCdGk424aYgHyqyBtoK8f/UUPay XPT+/plQr4/OmfpRfFVDj3hPa3nI0wbRBKhBeb0/doYypbQPMnKCJu4N90NLDyF7ER gCMoDrX+hji7qleXNI8IYn1jvUXVJzmAni/q1ZINV1LIEht6CBgEA6xQtXk/vEztSv 0Xwt2TumKKccg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 10/21] mm/zsmalloc: add zpdesc_is_isolated()/zpdesc_zone() helper for zs_page_migrate() Date: Tue, 6 Aug 2024 10:22:57 +0800 Message-ID: <20240806022311.3924442-11-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: i9y8uw6mpaxebzok3hfqbg8mk4oduc1w X-Rspam-User: X-Rspamd-Queue-Id: 2E073140010 X-Rspamd-Server: rspam02 X-HE-Tag: 1722910712-246390 X-HE-Meta: U2FsdGVkX19yqkPzLHEzH3FqvdoZ2TPWRBTV+P7uGGEnE8Fz8FWLsD619CPWY2U/AKRuz6uTuI37RFrObN6fj8NpeGbjaUIII7CeI6/FoW7KOdFa+gWwuS1XLxyChMqw0XCRipZhfjKRrqWh1s4rGUTdTWvgLEtrpcdRTfpO5MCiGq042mirQH0jYUbt8l/4+QujCA0gBTPf7kiULqmMN48HJiv7BMtcU9z4YvhsdJ/Zs3wgNJmEvNUbJ8NBzZQV2YnWp9O7NtTsZtSfewZwysDrnH8jWfR/cXlUbIVQ+rDSE6IpJkSDA/Ool9o5O4pXHN4FtWdvGAA5mBdX8tyMYyFh2ixKN/dZSOs4FWaY9AxXMv3udD4DKJ9/Z1tQXqsTi0JHkjzcq+/fXHwHWLKvQfXTcGVfk15Og0poguuPsEvqxnARfOssyY3i+4kQw+Tb0VfbZZI4vA4lBpbi4WESle4mzOYa1xjIXuwynvav1SrRhvZt+y/0S55StbQfKTsaecWMJtRwSm69tJAJe7zbM4tJOno4jOjg8CLK5jDg6r08EcDnH2A5bBqOp1bdXoaGDF2syqUaZKU3kjaE7pEWmvd18pkGBKjjXMdhOiNbWUEQBEcwfgFGc5khi2x5hl3M3BtrtDkmtv6cP+R6x0dda/oMhELIxhR566agntswvyybJWC5SeeeNOtuV58vFzvO8BuA78aZcK3SIagImpBBnv232p+xQpFSFVihQfC6u17/Q5pZRs2izg/7l2BGkHlJFAz4GJ23OIJR8QJvUX2dFV+YrKGw3mRArbw3d/nMpYnXWIB2l5Jiogb/WGPf3drDCJnONlbBXXKyX9dmoUQcoP61Re+1DAjZN6YPtsSKRUKFCL8DMXT0tjAgWmLNb2Xwur90KZbg6h6m2dehrlUOBa1a2jE4968ri87+N3HUurRrkDkH1jwSWDb6pfBUo2ROnzjTttDeM2Utc+4BnTm H75aczll wBzZ8WmZNi4XwLsFzt32XLV755d4ewJJzhF+MDg2TDewDsX+DX48Su+/FX/OIe1G/5JnYO83jUpwKG6k4Sor1Qo+zX8mW5kGEIMUAYxkmr3oiB2asCIydYYYmmV0mMXYAZoI3GCKo2cMj8wuaqQT8zmPv0Do4nocBAlFY7BKdnz0hhV2TQJGfTweNKVnS0qvB+59gMwVN/ras32lQoq0qllh11OLHSXTRoTd1pF4f4y9x9X4hghcz5VuTV6A7M8+pKdEGw8eP2E5HWxCXosxln5A2SbVKgLOOboOfrcCdg82BcMdGzHh4rdmoT17PIb7FNgyQ8LReiw1LsiZk8H3Zem3IBjYP/bWYM/Y4h15W4BkriNORNGThNKNp/Rz369yd7yx5 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To convert page to zpdesc in zs_page_migrate(), we added zpdesc_is_isolated()/zpdesc_zone() helpers. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 11 +++++++++++ mm/zsmalloc.c | 30 ++++++++++++++++-------------- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 6a01e09e4beb..20881e7e42be 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -113,4 +113,15 @@ static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, { __SetPageMovable(zpdesc_page(zpdesc), mops); } + +static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) +{ + return PageIsolated(zpdesc_page(zpdesc)); +} + +static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) +{ + return page_zone(zpdesc_page(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5fac2bb436f8..398b602a3b4d 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1810,19 +1810,21 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct size_class *class; struct zspage *zspage; struct zpdesc *dummy; + struct zpdesc *newzpdesc = page_zpdesc(newpage); + struct zpdesc *zpdesc = page_zpdesc(page); void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; unsigned long old_obj, new_obj; unsigned int obj_idx; - VM_BUG_ON_PAGE(!PageIsolated(page), page); + VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(newpage); + __SetPageZsmalloc(zpdesc_page(newzpdesc)); /* The page is locked, so this pointer must remain valid */ - zspage = get_zspage(page); + zspage = get_zspage(zpdesc_page(zpdesc)); pool = zspage->pool; /* @@ -1839,30 +1841,30 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); - offset = get_first_obj_offset(page); - s_addr = kmap_atomic(page); + offset = get_first_obj_offset(zpdesc_page(zpdesc)); + s_addr = zpdesc_kmap_atomic(zpdesc); /* * Here, any user cannot access all objects in the zspage so let's move. */ - d_addr = kmap_atomic(newpage); + d_addr = zpdesc_kmap_atomic(newzpdesc); copy_page(d_addr, s_addr); kunmap_atomic(d_addr); for (addr = s_addr + offset; addr < s_addr + PAGE_SIZE; addr += class->size) { - if (obj_allocated(page_zpdesc(page), addr, &handle)) { + if (obj_allocated(zpdesc, addr, &handle)) { old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj = (unsigned long)location_to_obj(newpage, + new_obj = (unsigned long)location_to_obj(zpdesc_page(newzpdesc), obj_idx); record_obj(handle, new_obj); } } kunmap_atomic(s_addr); - replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); + replace_sub_page(class, zspage, newzpdesc, zpdesc); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. @@ -1871,14 +1873,14 @@ static int zs_page_migrate(struct page *newpage, struct page *page, spin_unlock(&class->lock); migrate_write_unlock(zspage); - get_page(newpage); - if (page_zone(newpage) != page_zone(page)) { - dec_zone_page_state(page, NR_ZSPAGES); - inc_zone_page_state(newpage, NR_ZSPAGES); + zpdesc_get(newzpdesc); + if (zpdesc_zone(newzpdesc) != zpdesc_zone(zpdesc)) { + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_inc_zone_page_state(newzpdesc); } reset_page(page); - put_page(page); + zpdesc_put(zpdesc); return MIGRATEPAGE_SUCCESS; } From patchwork Tue Aug 6 02:22:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754328 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2312BC3DA7F for ; Tue, 6 Aug 2024 02:18:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A17C96B00AB; Mon, 5 Aug 2024 22:18:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9C73D6B00AC; Mon, 5 Aug 2024 22:18:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 867E26B00AD; Mon, 5 Aug 2024 22:18:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 642766B00AB for ; Mon, 5 Aug 2024 22:18:41 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 135E416193B for ; Tue, 6 Aug 2024 02:18:41 +0000 (UTC) X-FDA: 82420212042.21.2F8BD00 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf29.hostedemail.com (Postfix) with ESMTP id 1B63012000F for ; Tue, 6 Aug 2024 02:18:38 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=eumQagZu; spf=pass (imf29.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910657; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=a2/GB7+WY56RfUVdcNlABSbDEOw80OJBksB/qmP2y4E=; b=HQOLDuic7l8XzUSBkAcFdRSMYWmzbSqWaz9/7Tsk3awJg+1Ukuf9KV0cTEhy/7SU8KaL3V Sb7ohOau46mE8AEyt1gO695Oi39BO+2C9J/iJK4jJYVjfBmVaKKdEgLdWWtov0LWPBUD0l S0lidG5VPrHhmdVP87AIuy6zoUioBZ0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910657; a=rsa-sha256; cv=none; b=i8fu5fxXhtw3rE2hCxmq2Jf7bDDTANTzSgl4ot3Q9F3rU5h/GfGoyJwyWDjvMyAZKa6ylK UNoxReoiUg0bTuCyZzKngfQ2fGuX5VyYAm8bATVxbud+C/GbOLbvBgIaAX0SANuIdyYT7u etJO70c49X4RaUE1Fr2rYiW/ewd46ss= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=eumQagZu; spf=pass (imf29.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 9434CCE09AF; Tue, 6 Aug 2024 02:18:36 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DB7A6C4AF15; Tue, 6 Aug 2024 02:18:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910715; bh=lIAmwXpKjxS2Hsoj/dTwcnmyf7UY5ner31n75hmLtY4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eumQagZuRPiE1JGpb1pdxG+/99Ht5pD7RRzlqeURY/va7iPdIX7vB4gKhr06VJQ3r UWoA8msGRp4eA8sccL2/GzO5mQxkVqzheMiDuAC3F5rJW5N+SCFZtKyAtFO+Lf67rc KTHxlNvMvXZB/J55XLBEOsY745Nro5/dQ9+M95DGwMx671Ot4J8rtSAseOVLtktMQF oFfqu8itoW6TWi2Fvx8KvB0l7TgUW74oWiWh598DDOWMLJ7jNCPpg2iwUlurfumYlt StcwxXKGxg+yrDJ9bj0qQnx/7/ncGtbXfNpvFRVea1X2Pg4QthRUrw4kec6lJhMnIZ D6Wh2QaMAul8g== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 11/21] mm/zsmalloc: rename reset_page to reset_zpdesc and use zpdesc in it Date: Tue, 6 Aug 2024 10:22:58 +0800 Message-ID: <20240806022311.3924442-12-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: fjfnp4qdw1bo98pghb1y5qn5g33ryau8 X-Rspamd-Queue-Id: 1B63012000F X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1722910718-100412 X-HE-Meta: U2FsdGVkX18fB1E9STzZHcHNkzXCN36Ral9R/3JJjjofttRKARng7BxEK79o9LHxoVDkGEdqz50aENh2pFobbHT1wLVJuMYzwIEcCSJw3gKAtDWitMr6ChnBGiIRBayL/3kyyS2p7+ZUUkdTpj0VQdCrnH213OaW8+kJbtv6f2Q6GgCqhTjlE8QiTH5fOZuxjYt6uvTKEvum7GQx2CqbdN1Xnn5NjzY0Dfvnqki5d4Y5taiI9O/e0IjeQgohQE9henm2gVUGw+pwYk4abTeO3HCBeJHdiJSoVj8i4Ct0PsISg8oPULPH4dQIHQM1bq0IDvnyyfE813Pn//ADZtlJDlwo3I1fCoi7HySVM41HAgiKdGG9rDOfSSZypgkvNSe5PxeTVnYISGqBdFc2BmspZu39islkph7AgdK3NnUbSIlCbtZAKwDlNsY/tQGGCdzfZQnfOzbBvzjraKFQFzLpfIuInFaRVGSNl8TE/YIaMC0HfYPW5O9NnNq8bqfegVmDMDQvv2AWcp/HP2nTE0IVN/7qn2xfwa6E1tUUAbnaMV9+HBxgvWiXdxPoe8IvCmZrFWR/bxPMDLmuoEc8aUBO8CwIWKDE20DbAtchY+PLoE4HDQpVtpPE9aCGzuX6xCeD313abD6512YCWl3PZtE1Lzg7Vi+HnE3tdFGzXjBmGar4bLyU2JUv9me79agK4NjJulQcmwVFfUTIFrfSJQABvI0I5JSkLF+uyc3uay8Wz7CYHi7zSz2ovK90SX38wKJ5i8Cliu990Qx5w86edjh0dyiEJP4WvzbjLLbyyB191JOUHpKvDSjpbjvph5CAvC7qqeV2AVnkH9SkNFboq97xzfsS4FSApcg1dCikCHca1ufDXsCdVS68Gyap6MXsGqpor9DjoSmz9DzKfu1idTLZhuRxONxUmsl6JLjgJNHy6/pG0qvklAYBMVoqI3Cg447Uj+zQRR9GnLLiGhftJGa bgSpNG9w f7sAJ+a77y4gPzoBSyl6qEi0o5v2emLWqlS3FsegKQ8f0wR7ocaoEE2c0wH7rg6X1dmlb4c9f6gSkSO38510NhelSj0tl1c6NtPgvajdpYwFAcp3efIR5OqWVrnIKLUR4faMofGpTeZc6iYbxzOyOoa7y6/mLLn1ADNfC1qBbMoK6F1eY15kgIVhKuv/Z9yGHnBAqKDMhhP4MzC2harzLUd5sxYAc40thzEP9P91lC7b78dwUotU1D4mJ8YIvvJ3396qZxJ0j1djvlLPM++vbCjwnrA9mWx7DUONTqk+qb+TO5D548p94jlkVMfkBVWWfQvLn2gBzZHSFCKZ8jQIXBdthjleVimlHYGIP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi zpdesc.zspage matches with page.private, zpdesc.next matches with page.index. They will be reset in reset_page() wich is called prior to free base pages of a zspage. Use zpdesc to replace page struct and rename it to reset_zpdesc(), few page helper still left since they are used too widely. Signed-off-by: Alex Shi --- mm/zsmalloc.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 398b602a3b4d..f4fc79c0bef5 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -845,12 +845,14 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, return true; } -static void reset_page(struct page *page) +static void reset_zpdesc(struct zpdesc *zpdesc) { + struct page *page = zpdesc_page(zpdesc); + __ClearPageMovable(page); ClearPagePrivate(page); - set_page_private(page, 0); - page->index = 0; + zpdesc->zspage = NULL; + zpdesc->next = NULL; reset_first_obj_offset(page); __ClearPageZsmalloc(page); } @@ -890,7 +892,7 @@ static void __free_zspage(struct zs_pool *pool, struct size_class *class, do { VM_BUG_ON_PAGE(!PageLocked(page), page); next = get_next_page(page); - reset_page(page); + reset_zpdesc(page_zpdesc(page)); unlock_page(page); dec_zone_page_state(page, NR_ZSPAGES); put_page(page); @@ -1879,7 +1881,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, zpdesc_inc_zone_page_state(newzpdesc); } - reset_page(page); + reset_zpdesc(zpdesc); zpdesc_put(zpdesc); return MIGRATEPAGE_SUCCESS; From patchwork Tue Aug 6 02:22:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754329 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C750C3DA4A for ; Tue, 6 Aug 2024 02:18:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B290E6B00AD; Mon, 5 Aug 2024 22:18:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AB00E6B00AE; Mon, 5 Aug 2024 22:18:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 951DD6B00AF; Mon, 5 Aug 2024 22:18:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 74BCF6B00AD for ; Mon, 5 Aug 2024 22:18:45 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 2DA81C0516 for ; Tue, 6 Aug 2024 02:18:45 +0000 (UTC) X-FDA: 82420212210.27.DF5B62F Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf03.hostedemail.com (Postfix) with ESMTP id 1E98E20024 for ; Tue, 6 Aug 2024 02:18:42 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=cvE9YuJi; spf=pass (imf03.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910693; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BdOLw1XUqDePi7WDy52zOYaOjyt2X3wfg8YgocalmFo=; b=ehuW3GNHn6BVSppbZ1n0FW3m/baje2cfMkvZqoi/GvBU6368d2GF64A1T5ujTJvB9YAoCI 33xnsXS/hJkqLbx88eUymNNSbsPmVZIRDR4Y9BxkcYjeg+Lh04r5p6hs4SaEtOPzI2ugub zmh47+Lhba3tRo09AD+mRW3pVOZHViM= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=cvE9YuJi; spf=pass (imf03.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910693; a=rsa-sha256; cv=none; b=Z9zRjJB508VY1Rk+VwYzM2gja7/sjPCQ7nlI9GITuSmr1Wn5/JLXRzm6beIRqNzdmQmZgj Ynw7+8lZcEZ6jxr3xM26947yPwLhzKAAlYbUpJPapXAzMkVCBCpmFdCyGq62PqxlD/n1sz EFaJRFMlGCG+8XTR+niaEed9Kqh5vHY= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 3EEF0CE092D; Tue, 6 Aug 2024 02:18:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 88D72C32782; Tue, 6 Aug 2024 02:18:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910719; bh=aFVskGOXvp/05Uc/SGs92DOGYm3KvLJCDxXlSNzTuX4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cvE9YuJiftrDSE1MdCbV7FmQDTvQ0+TQ0arxnPZndZbNu/MTxGTtjGpfe109H63gU EYlozhNKkNouVY2+GUzBSvklk1lOwJte5Wbw/cdE+8jzyW43VHDfNpeanSC+8PlmFA T1z6KNjfNtMt5abA5/7hG2ehSUx5Ru2Mv+oQLyP1YuD7GTKasFewskGQUq8gJrEt/8 5VzOBU3CEAoX4BAIxOH4oznwxYA5GhplD0r91533J0skWDt1FnOyWfJlOEvCQOx7AZ jotgu/PQ420IcxYrbHzQKf4mQnk4BqKD21NVym2jYYv8egk9cSHF8LUQayTgt8GFm+ 0L1DkJwRroClw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 12/21] mm/zsmalloc: convert __free_zspage() to use zdsesc Date: Tue, 6 Aug 2024 10:22:59 +0800 Message-ID: <20240806022311.3924442-13-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 1E98E20024 X-Stat-Signature: wky8u9tax7sknnwm5jo3mrgk8kifeeg4 X-HE-Tag: 1722910722-902039 X-HE-Meta: U2FsdGVkX1/XXXyE5NADJg2C3LA97yxckl6h7gSJxZpVzizH6bI/0vjPe9wZqBkkxvl3JJ7+BvqVbyUwWwf00UOp5czupt73/PTWYyEv5DnpJgOIFB9UXUxbbcN6Q6yOHyP6u6lAgV7tb6HgcgsdZvX3zgPG7XrxNvotdO+4b3IBGJqZrqeT/5yxJOcJiVpN5gUxiYhRsXO076qkkl7353CBMiQLGWHZkv/G39r+b00yTBMvrBQvxPKk/zh3WcNwEU5JttjOmQ61S74ZH0iGQgNSWuGoGq7kpbUNVzgzKd17Bf7336BE04sRvtx9bMeJB3t6wtwnVITyzqvJi4275TwXI9BhMrUotenT2/msqERuMBw6En4dBAa8LE7duWhA1/drtsl+lPOQvJ9024QguAWMiOMvRmwaAyKWoDD/v6ADmDe5nVVnTITbNJyoYux1KQdyusdzzZi/YrpFnSrpfros0MMdzZmRxLPW3FNnByzbqXYHzrGv0dVsemE7xJNVWieSVTo7++DT1v6h8r79+P6PovSN/Bw+DNi8blrUOxxfK0U1oZlx05YonhzZi+dXRUuPq7Abi9htOh+SP2iQrYFXyG1xjlMKsU9e/eUrTYdpvV073WnOTA50/oLk1UeGwh5wUj2XQKPT10yBB00otqToXsMhi4mevYNBGdRUjmg7juimILgRlmCZppw2iQ+Z5Wkld7zJdl1KOvQoGlrLJLJpOCIfF3PseJzDh+qRJMnQX69h9qMRJGQIM8DNDY4PRU2DJPQQLgtG8q13rl/99pQMvsFlGLmkZR05oEtGmqquMsumB1QavjOtcXMansCjEDVG4otdlkUBq+/CoqDFDuqO0ZcPcmH1MR3BZGR7oHUYMKesP16ixkEF2pc/5Z9PKLUg5vGOXdB9TJLM8JpbNXRmQCsQnzl0Dm4gzxk7DlPBcMeBIPL5+Na7vffPgQr/ij4hwgtbkTLrSsqNcdV jZsDbs7i 3nMkSCDjo7U9M+cE9dVnfavAev/+ewmVbOp0DUMcem2bWTTp3EXPONf7xDT5prXv4VePk7jdnsIG70c02v5tgHZdQ2LADot8gS5faiKEve8hsV5dCTlTBC04KOlGjtocv++xMtWRFECt59qN/n5jI9voZNm8EwBFNiusnicabsCS+tHbe0ZaMS8XVqeTMD2FtNeF8SV7X5btsL2Vb8DoZjNHfIxcXI2uQc3NuQQQ6f9TS9fT5k0QnsgGvuvYzGFw0DkyhugaJXc+yyyqqxcyP0KVoMo6Mc1G4OMXz/k2YOc/4f05FfOQW0m3oAHI1VVj4SInKO+ZBrrbolAh3fjc8IWNERkpZq7e69yISKBPJMAelaA72k3ybQo2VWQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Introduce zpdesc_is_locked() and convert __free_zspage() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 4 ++++ mm/zsmalloc.c | 20 ++++++++++---------- 2 files changed, 14 insertions(+), 10 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 20881e7e42be..640f33b5c8bd 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -124,4 +124,8 @@ static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) return page_zone(zpdesc_page(zpdesc)); } +static inline bool zpdesc_is_locked(struct zpdesc *zpdesc) +{ + return PageLocked(zpdesc_page(zpdesc)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index f4fc79c0bef5..4b948b84e7f3 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -881,23 +881,23 @@ static int trylock_zspage(struct zspage *zspage) static void __free_zspage(struct zs_pool *pool, struct size_class *class, struct zspage *zspage) { - struct page *page, *next; + struct zpdesc *zpdesc, *next; assert_spin_locked(&class->lock); VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness != ZS_INUSE_RATIO_0); - next = page = get_first_page(zspage); + next = zpdesc = get_first_zpdesc(zspage); do { - VM_BUG_ON_PAGE(!PageLocked(page), page); - next = get_next_page(page); - reset_zpdesc(page_zpdesc(page)); - unlock_page(page); - dec_zone_page_state(page, NR_ZSPAGES); - put_page(page); - page = next; - } while (page != NULL); + VM_BUG_ON_PAGE(!zpdesc_is_locked(zpdesc), zpdesc_page(zpdesc)); + next = get_next_zpdesc(zpdesc); + reset_zpdesc(zpdesc); + zpdesc_unlock(zpdesc); + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_put(zpdesc); + zpdesc = next; + } while (zpdesc != NULL); cache_free_zspage(pool, zspage); From patchwork Tue Aug 6 02:23:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754330 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09087C3DA7F for ; Tue, 6 Aug 2024 02:18:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CD59D6B00AE; Mon, 5 Aug 2024 22:18:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C88036B00AF; Mon, 5 Aug 2024 22:18:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B262B6B00B0; Mon, 5 Aug 2024 22:18:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 887816B00AE for ; Mon, 5 Aug 2024 22:18:46 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 0C546C07CD for ; Tue, 6 Aug 2024 02:18:46 +0000 (UTC) X-FDA: 82420212252.21.113CAF7 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf15.hostedemail.com (Postfix) with ESMTP id 6ECB0A000C for ; Tue, 6 Aug 2024 02:18:44 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=CRmUrjTQ; spf=pass (imf15.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910663; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=26gYv77REOrP+tSsekzSzMy/5K3dzn0Zx7gMUrX39Rw=; b=HWMFXFrQNkHauOvjSdGcjFc/0ffY/yXEObdxEDoa/C2Pj9aNehNJNJLVgzmETJediOlS5e yQGhKduxp4tsJ6sNykz1zq2PDbPzHIUylse1m8Pgpvlm6X+YfQqZwDmo4GAdDvxc0Vudgu 4z7GoxgPUm9dBU1g0l6/v1wkcCzGsbk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910663; a=rsa-sha256; cv=none; b=qJYJtwlkHbQNMIyOU3cgl3qiGV1NaXw7pjhaNDLFDPrzN2Zy8Afpsmvjhd1x+cGo5sk6zA cLGqFj/YEQrsN17iZbn8LhP8B53kTAWHbzP8lLKmc9npIEZIhN76cwSHUMj6/0udXper+5 xRsDY2PXrS7ZsfvPn/Kvfvd3cORS4Bg= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=CRmUrjTQ; spf=pass (imf15.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id A4BE160921; Tue, 6 Aug 2024 02:18:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 34CA4C4AF0B; Tue, 6 Aug 2024 02:18:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910723; bh=7R27PI6t8PPGIzjMbda+XNUteaxs5t0PfeOafdtKA9c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CRmUrjTQbKnuPQmOwufiAx4cYw8y+lYQTG38+7FslMxb3MSpH7GyZODkN1/ZB59fB s8imNPAIKMRXmNKRZK0mUiaLhD+0PgbBtXwT168RS2gd2u58Rg3m104ej3wIe/snzs NjG9ikTjq5ki9Gnay0a4shcsPz1sNZb4rTFy/MQsccnSfB32pVwCaqCPwnUL4La9DS TWtv6u+zy+LKEbXrV6o32O46jbrvR2Vg73CGnzTQFa+/sr9+klLHfr9ul2ZUl55YGl dcLUtMdq/ti4ucNX5c9ViRpDob04PaaSR2Fjff3vT5pmKjMGlolcDuytY0hd9wT7Fe 85JxlEARg1HcQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 13/21] mm/zsmalloc: convert location_to_obj() to take zpdesc Date: Tue, 6 Aug 2024 10:23:00 +0800 Message-ID: <20240806022311.3924442-14-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6ECB0A000C X-Stat-Signature: wmxfpfzmrgdgunjo8we3uzi7w1ahuy41 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1722910724-487558 X-HE-Meta: U2FsdGVkX1/wNu0153g2RSyufI8GfjXRkNVNBiTeFYtaVY8mKs3FaochLVee/RV1SsfdD1egnlASZ1tLOraIg358QilkfhHl2aPlTxapb6vBc8qKNPxm9quDW+OWnksohkanbeiSbCrglEl/BpLvnBxAxkHC3RwbZ2tGgJP1qGT4rmnpJNzz1FUp2SW5JxBTlPyI+T6+IRW1beGxUjYMNlxOHERFt8FWWRzoHGmLMaTsXCM65q7aMzfyWi+Aag7eBswif8mQqQAm5LeyPl8csMV75JhG/oWTjpD2z8UwTSWAtUwyiwe/EBgQs7/l4GK8sd9YER/+9W+PvrPMdkWbnUsTWFEz+SlGqAGe8dndP3w0yBazeYCBseh4qsTsytod+XPGZECc7o991GftAO9Ekk+k7w5u0jUHKw2NF0UBV5P5lZUtxHJ8W75wsZjWj0hqMeHtaHDhnKr99FAlDR5YFgRtMiUKLz/KIH23LjEPTBWh7nq8pQl5+JIPho41Q+UMnfMvFGGyBKS/OphMLTZtLCqhPMPl4A5K0UqvFJXe6ysOrpO8EDGQnSnjng4zk0L9BMDcDbpxTjmSogHLG94jJHi7XTHiBp3BL0Mn7l5ocUPRX/qBg2xdxfNR/Z1ZZLmgPy2jJQxnXchnqwmgsMM9ZUsGdR8zHhh5WRMh2v17tFp8x6bsLD3HmYbjAbNYKtRZ/XjjLLRdyZzNoTly50O5iUbOdiw+QnVFhtJTw7aWYs5SGQkoT6/9tYnkDb59QrI1vrr4PFXXoqmTUXi9nmSnYEoEH+bPjVgBDiOaGodMcTHbpPSNQK4MjmP+2teFTjZt2PyH24POOjX/34yM4bRhXUNypREHI4zPJTjk6OOFOyQlEHfPtJ8fDCq56ePd8w6mi61Hj+hdeKm8dDXX8Xj0tLRX5P/HRSQNbapEImIHjTton0WpKn0LR6vCyeG3IoW27J4Ca5LDLDEE9VR/swL YrJUyJ43 JXCySFpiVWpg/tL/o8/+ucKPX39GNI8NaNp4003fxSCURVOUeH2e5J1UfQWCGU+YaMVzNo16noCJPpSmwpItEKl5jkhM8G8zbuJJUtKrIaNUCmX4lOx0MAUcPRzwQX3uKF399mzDReX7MS4Ifnle0CnlPt5I9OZvsMV1MwMGFpxoPPer/r2XGBHQ0g2pu1W5yUthnOFbjAQsw4DJYEvRv8eAYVukcRB4udtcndvo/PIRMFO1EhtCAf3m563/UEhkb94mm9krerMh3UIe8OtNXalMF7+gV0fe8P3pWSSvVWcx1O2cGnGEcM3pOwU3cZAgnXT2J/7+tUkU8FzwyFlOK2p8jUeEtvSiqWmstQRD3JVljqOwTYQNU+Cn3y4glOEIoVi58 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> As all users of location_to_obj() now use zpdesc, convert location_to_obj() to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 4b948b84e7f3..e6d1bd8969e5 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -806,15 +806,15 @@ static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) } /** - * location_to_obj - get obj value encoded from (, ) - * @page: page object resides in zspage + * location_to_obj - get obj value encoded from (, ) + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static unsigned long location_to_obj(struct page *page, unsigned int obj_idx) +static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int obj_idx) { unsigned long obj; - obj = page_to_pfn(page) << OBJ_INDEX_BITS; + obj = zpdesc_pfn(zpdesc) << OBJ_INDEX_BITS; obj |= obj_idx & OBJ_INDEX_MASK; return obj; @@ -1372,7 +1372,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); - obj = location_to_obj(zpdesc_page(m_zpdesc), obj); + obj = location_to_obj(m_zpdesc, obj); record_obj(handle, obj); return obj; @@ -1859,8 +1859,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj = (unsigned long)location_to_obj(zpdesc_page(newzpdesc), - obj_idx); + new_obj = (unsigned long)location_to_obj(newzpdesc, obj_idx); record_obj(handle, new_obj); } } From patchwork Tue Aug 6 02:23:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754331 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65CCBC3DA7F for ; Tue, 6 Aug 2024 02:18:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E5C056B00B1; Mon, 5 Aug 2024 22:18:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E0D4D6B00B2; Mon, 5 Aug 2024 22:18:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD34C6B00B3; Mon, 5 Aug 2024 22:18:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id AAB136B00B1 for ; Mon, 5 Aug 2024 22:18:52 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 5F01CA0830 for ; Tue, 6 Aug 2024 02:18:52 +0000 (UTC) X-FDA: 82420212504.15.897A291 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf14.hostedemail.com (Postfix) with ESMTP id 48ED6100008 for ; Tue, 6 Aug 2024 02:18:49 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=QC1dagSP; spf=pass (imf14.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910682; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rgwNPyWZEA1v6Q304XUxXEQlyQ7bgtgwHhwp+EhAfp4=; b=xMjqoMah7AOrutJymEqHwpUoD6a+du6Fll7nHMGoQ1X6hq46Uy78T6DHGfVxNn8UtOb+PZ Di+IozDx8V7/yZN/XUGkyw7Lq2bGWSFc497d7iNpvRj9m01dCzd6GyEKtOKNg90NU/zlq3 oX4WFoufL5dWZIjqlHaciu21yZmhwOk= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=QC1dagSP; spf=pass (imf14.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910682; a=rsa-sha256; cv=none; b=adC/eSbMnRMQCPVxBoishVifc6Jw7QhU+cjSmZhYPP5nOheSVFMU8HTrH4FEMsT1KXhx3+ s3NiHwteg1goPxb+gIFrypnTuICr5MnrZ+X/goW1KZ2DRxvBHSgKNmTKGvHyHHZ6fJi3We /qAZ72umsgdM6n3TaPBNg6Dd8vb4fBY= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id B2D16CE0BBD; Tue, 6 Aug 2024 02:18:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0788FC4AF0F; Tue, 6 Aug 2024 02:18:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910727; bh=UE8t+seeM35rl2H7JTm6eRVAxobMBm5PBazGBdplkkE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QC1dagSPDYyeNnZc8QopLpLhTX37estqkQpAdmcGKNJbVDWh/qqd4LXDnB9rPkwo8 a5aYS0P0SofZMckIif6UjPr4WkZzfpHrUzYIMglACX2iFlDgqrD7MmCC1PSWaPV2XR 4s7xbWyqt0Pf2yz0mPj8FaMGg0br36C84Q2I2Sc6N0UVKDmLr9mQ91j5HduyA7RXQf AJgeqEdBcb0yfzmxjDNiltYEvbGvoZFOfiIJiYLj8FzzUrjCgp90Qm3abdIcIP1/U7 vHzJUWdv9NoTlwyaXv/hHPPUokBHRBunGFV9102zXdOcI3xZq4MGWE5tUhw0uLYS/A Koq9n6XXYdWFA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 14/21] mm/zsmalloc: convert migrate_zspage() to use zpdesc Date: Tue, 6 Aug 2024 10:23:01 +0800 Message-ID: <20240806022311.3924442-15-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: 15dz6pwdnifccjqa4fdusedm7ub83y4x X-Rspam-User: X-Rspamd-Queue-Id: 48ED6100008 X-Rspamd-Server: rspam02 X-HE-Tag: 1722910729-827064 X-HE-Meta: U2FsdGVkX18DOAM8SEKEcvGmlYujuRkjPkuQSdz0gInHLLbiERi9pBk+szB10feA3LQoP3lCiCH6HUsvJ6pVsfBKO6XyWyC+Qgcxa1QG062JF2t6eeVSvnglkDq49SsU7afa6VUSZtwcNwl5DcGPqPg4SDxLVdjteCzzgosmPJ8/GnvJbgVk+DjuLheXormluJHoUyGgZJgqWv6EdnG5JNabBnwFEa/oA0q3FQWDgJm/EXXrgGKUj1RXnyW1UWXLPp4FPnT4toRomYzJv0u7Cw/RcHpfxQVN6RXmphGyNHf1vC1lbVDofNkI+Sqr5VX5xipXybaTAMTUVmLAH1oMuMAzlUGmGOYIT3+hOlQgm6YTvo07vuVv6i0U3DnpSqDD6S1maJfSRrMBjClzQcK80KziWbSj4f5OUykc2YdvwMNKwpppUlznPeWYJf9qwdx7BTLbaENOlX88oEWu+k080R0tc42ueyhA0NdI2O4T6s+CXUy7GMt1kBlLaw9E7+SbezKQi7tiU+I8LdDRSHIjHJgJJrIRwEpdyY/XoajHku98N00jaJoHNcWo7SD5CMOBXSFbHqjB8ts4neJfB8f9F7MiuZxp1NeOaAvihOzFCJA0Xh6Wc/l9g5IMxHwJY9XRqQV2JG0ahjY0PuXxVMjxVTgi2DybZhCismgD+LwiK8w3roooCZafQgroV/02wLtdUMiP8mYyQR7HIipoHVErTjcge45ufcOn2LVdGt4Yv8QcWwbL7w3Jqeer5RAaCC0UIlmda1Oul1BWKvLkLnx06S4M8lpTj1WhJ5jkgs96uvOOHAoxmEyosDkr5h8W+O2X9HOUSKeoNGLpCdhRjjHoK+XoAeIrdEEaDxQhg9nmJw5nXgBZKviPRTTZXfcYtwYhYs7yJXMUuxx5O0MJ5d+MufO73nSxWE1pFoQc8IPfaib18Q7rdchiFLMrKxQyJn9GgvUYFFozh9wVyO/yNh0 QaJHYlsE HMD6C1VUl0flVfzHfyqp75u9fdE8/USdbnrzRohNSw9BKXDQB1Ir+CFwdPWw0uAli0BMgBLcw6WTMgqYBKVZRokl9N4dE6azbg2rUVcHP3A+fH+cbZFCEYP7sMQZ9a4TQrsVjEMqKRYGS8+2szIJjs5smkjrJRX7vb5CWWqL6AyjxsUzAMTA1V4RMvRnAIZe+7n5UJfqjc+6Kbi6ZLiTYlBYDhZZ5bcbMIizeVOnsBBBa+tCZZBe/0hU9Ae6/LXeMZMjoruEGi1nrsW2S1S7htbbGRST80hu4YK6IJqv661vIKIwVHrEbjFV1LmwcdhawwFBxK1wAM5HdWvF/Exeune7KcnwLrACQeDKA0StpBdxbM2nX2oWzoEAsTd/gKkKetd+j X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first/next_page. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e6d1bd8969e5..c8a533b516ea 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1616,14 +1616,14 @@ static void migrate_zspage(struct zs_pool *pool, struct zspage *src_zspage, unsigned long used_obj, free_obj; unsigned long handle; int obj_idx = 0; - struct page *s_page = get_first_page(src_zspage); + struct zpdesc *s_zpdesc = get_first_zpdesc(src_zspage); struct size_class *class = pool->size_class[src_zspage->class]; while (1) { - handle = find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); + handle = find_alloced_obj(class, s_zpdesc, &obj_idx); if (!handle) { - s_page = get_next_page(s_page); - if (!s_page) + s_zpdesc = get_next_zpdesc(s_zpdesc); + if (!s_zpdesc) break; obj_idx = 0; continue; From patchwork Tue Aug 6 02:23:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754332 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 265A6C3DA4A for ; Tue, 6 Aug 2024 02:18:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3FE2E6B0085; Mon, 5 Aug 2024 22:18:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3AC686B00B2; Mon, 5 Aug 2024 22:18:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24EA96B00B3; Mon, 5 Aug 2024 22:18:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 062BA6B00B2 for ; Mon, 5 Aug 2024 22:18:54 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id A56EE81D46 for ; Tue, 6 Aug 2024 02:18:53 +0000 (UTC) X-FDA: 82420212546.05.06A0001 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf30.hostedemail.com (Postfix) with ESMTP id E611980005 for ; Tue, 6 Aug 2024 02:18:51 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=DWgzmYgv; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf30.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910686; a=rsa-sha256; cv=none; b=XoGkipiM5lL/TKYuazrzNAysruNoTTZRzlRJUj7pweAGMJwlVDxTYzp518IAYTKqXAk8bV UQyHKwaOKuZJkS18zRZ7NAC9O9Io0j96R2dMzixb+BsQxoj6iK11Y7919SSSy2NsNNpJbr ulw6h3DuUquU3oXC3d7yNKrIVZwnkaI= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=DWgzmYgv; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf30.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910686; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4qvJssN0OKf7J45cyAlp9jz3d5IvZkEnKH1088GlueI=; b=tCGGRO7mANzk2SYMwV+T8SSlgT95QeKZXj02NkxxjCoJQS0bfk5HVjZb71dwlOCJxeMWuD U8m/7SJtEsNjJpI5pzDkBPeYna8zDmdnEsj6tzdipEZzKbaEUnbwBGl1xH/Z+UYk5AranK cwHflSxZPs7FR6DQrojZr+kUSd3QRfI= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 23A4560EC7; Tue, 6 Aug 2024 02:18:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A7538C4AF0C; Tue, 6 Aug 2024 02:18:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910730; bh=ogZXEVMMW4vu3mmfL4ddTVfOtlodq8fb2Ek8tntegjk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DWgzmYgv43cIb1Yiwb7X3nxE/pvVIH2czvwhcvBxw+2XWnAWknGn5a5R2zBnvcHX2 AN2G5vS7GEbGmHtuuwf4IQ+El+/Yo9rRX+lAukVEUWkiDn5Bk0XTE61vlWrhSNXtsw h0RBltoj7fNzr7nTdqweZjWmT4+rVA5Ye+/YKu7LpAc6OM+i0DtFuwv/Xjziz4XPMJ yykfc6l+Byh1PYyMscKofhgssFWWTA1a90SfMRPHgJDCsBwDIKx9Y7d+Jtld/GETLC eaOgOHDsyvsotZXHSHWJd33trvlCMq2Ie47dpzUtR/Gs3xRP3Vg2pqkk72SXbUh0fa OCih3mDoloK4w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 15/21] mm/zsmalloc: convert get_zspage() to take zpdesc Date: Tue, 6 Aug 2024 10:23:02 +0800 Message-ID: <20240806022311.3924442-16-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: E611980005 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: mzbk35xjmxyirgd7yi8hckq35kejcrud X-HE-Tag: 1722910731-304529 X-HE-Meta: U2FsdGVkX19J1YhoqoYwIYz72hQRVWunFxQNtWQlsdsc4e+ZmZO3VMbGo+jSGrUTvPqGZIO7sX4hlCMF04J+/Lkcsa94kkvL7qoPYkJKlLtEfJFz2BWdhw1PmkkwINuyTPH2rg9bbID3GoDTD8jPNDjg47eZ/e990X8WBP2kxREBscLZIsjrIQLak/6tf+XwclzMqEvl3qypqwFPxBaqaZIws9wg9Sq86QP9oJ5KuQ/iYk3Gq5yElnka7dOHD8x+/I3QSeIgfzjLUApe2r5MrEMb1bNLO7NdiYZOUd+n3lSnBGw1wQFcuy1mOH0llMhLGUgm7OwByyFT/u2FDjXKsA8XiWhE7LgZr4TpspJi/oJGjSIAWf8nLcN29C8U6cC+tbl/l6USmi1ck/YLLlG4PGPbal6gtCb8f6HZLrPNNAk/oy90VgNEauHipyqC2L6hNvcXLcm5oG7hFsasZLMHUU89y4gHCoJRIhP4tiNDoHjKVJwudXEv8pMQRHGscqr2DCR+1+9UKjVmmaj6lo9HJDjnD1xINoEFXs3V/PsJvuMaUI+qIxi+QU9kI6tt8cR8+18VYFiHu1leZFc/hlgynRK8BOlx2yPALn0o69I+7P1b4SHXl3cQCjCe20Fg4IF7oH09+AQT9eAq6ePMxomSbnIB0CSHPA2X3Ma3VgTSsWT2n44tlRAWs3CRotJHZ+SMFwFVYfAQkLqyP0fNzCGJjHtWcpnwONZv5xazeufN6X1UxNcCuvCnOIZF4dHKdSuV2hhbRf1Ru39VhUsLhcYtiqfmwYEWm9n0CH3xWV6qQS0u5oQ07uejUqxRxqZJou0GpdgUFGryyK9swI0JA+u507hiDSDR25rPdhfotOZ235xEaUn+WMIBSt9D/kSQxxdSq73IO09Wzhs0aaOrnnoJdtvbtSsK7dTmb9k5rToLujMuGR4+5xVQPaM6LfOJZxpIG0qFE08zcJIaFAQc5uI 5vf00kmm HifMVqIEYPMgiYlf4MPRCVLRAUXs0qg5sqJtI/ejLolaodHJsqkIdoPvqQ98QFEjnfA/CpO2/VeXwb0kdt0bDzHm8WlUFsIu7Qqa6xQph+/pcPUT+SFD65Sq5kOGXxvDfTBxXx0Lc6lNyF5sSQLsqrPL9HsGCO3n2VgnjK3CTKVCgPtSkoU9PmrReN2+vNR4A2Ax07Szp452+HOjj3whvlj3xFrER8O3Iq1AmeZ15WQckGVOc/I+XbSyhvn3Bsh76jg1vCoM8LF0dzo+anLWXywToH+OeuStz14o0jJZYiQ5MBqNDcIMKvr4XBOaGIJZazXdLRSoJOS1HQL0xobhNVPfE+q7UdEE3oZRzlwTgL0byvRR6z4RHGr2ch5l3kM7ZnqvQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Now that all users except get_next_page() (which will be removed in later patch) use zpdesc, convert get_zspage() to take zpdesc instead of page. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index c8a533b516ea..7ae98d048590 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -759,9 +759,9 @@ static int fix_fullness_group(struct size_class *class, struct zspage *zspage) return newfg; } -static struct zspage *get_zspage(struct page *page) +static struct zspage *get_zspage(struct zpdesc *zpdesc) { - struct zspage *zspage = (struct zspage *)page_private(page); + struct zspage *zspage = zpdesc->zspage; BUG_ON(zspage->magic != ZSPAGE_MAGIC); return zspage; @@ -769,7 +769,7 @@ static struct zspage *get_zspage(struct page *page) static struct page *get_next_page(struct page *page) { - struct zspage *zspage = get_zspage(page); + struct zspage *zspage = get_zspage(page_zpdesc(page)); if (unlikely(ZsHugePage(zspage))) return NULL; @@ -779,7 +779,7 @@ static struct page *get_next_page(struct page *page) static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { - struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage = get_zspage(zpdesc); if (unlikely(ZsHugePage(zspage))) return NULL; @@ -829,7 +829,7 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage = get_zspage(zpdesc); if (unlikely(ZsHugePage(zspage))) { VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); @@ -1245,7 +1245,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1295,7 +1295,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1459,7 +1459,7 @@ static void obj_free(int class_size, unsigned long obj) obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); - zspage = get_zspage(zpdesc_page(f_zpdesc)); + zspage = get_zspage(f_zpdesc); vaddr = zpdesc_kmap_atomic(f_zpdesc); link = (struct link_free *)(vaddr + f_offset); @@ -1493,7 +1493,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle) read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); obj_to_zpdesc(obj, &f_zpdesc); - zspage = get_zspage(zpdesc_page(f_zpdesc)); + zspage = get_zspage(f_zpdesc); class = zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); @@ -1826,7 +1826,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, __SetPageZsmalloc(zpdesc_page(newzpdesc)); /* The page is locked, so this pointer must remain valid */ - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); pool = zspage->pool; /* From patchwork Tue Aug 6 02:23:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754333 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15E4CC3DA7F for ; Tue, 6 Aug 2024 02:19:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A24D6B00B5; Mon, 5 Aug 2024 22:19:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 952116B00B6; Mon, 5 Aug 2024 22:19:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F2E86B00B7; Mon, 5 Aug 2024 22:19:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 60A586B00B5 for ; Mon, 5 Aug 2024 22:19:00 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 1B53D81D92 for ; Tue, 6 Aug 2024 02:19:00 +0000 (UTC) X-FDA: 82420212840.29.CF89A6D Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf24.hostedemail.com (Postfix) with ESMTP id CF52A180009 for ; Tue, 6 Aug 2024 02:18:57 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=lO72cahT; spf=pass (imf24.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910707; a=rsa-sha256; cv=none; b=TF5UabcfliUvPMytzoke46A4MyobN/whqCvUJBZuPJ4QTN8uyFcfqe9QUfPQCcn5klTRlz gdPGtCh7QPD9SdGefcdwrD+s0XQ+jCYzqYMOnHzOgt6nX+t10gMC3jlK9EFtzQlYxIIKFy HAAgHlBSJucGDxYJ4q6X1/iYAhTNTzc= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=lO72cahT; spf=pass (imf24.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910707; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mECHyMHpJrdQfrlljgr0KIclcNdpBi3Rbw357iNHdTM=; b=5NH/8jeA7uWvjwu11IQXTxT+c6VCq6127Fd0xidXSniEoh7+sNZbu9k2bl+E9Fk/qwMcHB hdatssFDKGN0BY8QuU1QHAcYrpXRdLeeAus/QDy/dDEv4X8D5rHA8GW30nVb916pfK5+dd Zd5IePjhL+P5ExSU3RiJvuglslK/N8E= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 556E8CE09AF; Tue, 6 Aug 2024 02:18:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 79ED9C32782; Tue, 6 Aug 2024 02:18:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910734; bh=dtYWibYOHtP7k2L7bRrRJ0o8M+qJDQY9usNXaIwbwBY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lO72cahTLoWkgC7Fa1GVC+0hzqrLY7guqKdalHp+19GYloDBXVGgnVMkA6OEbgxLz I0omHi39eDE1hwAIsmiyNC3lQerM4A6pVo0uXUNxZeUyQTI5Rll9kQOReJhxdbTnse 3riuCQoqgpy1oFQmmD0ZEN/HioRasC0sVhguV0gCV6kwUReZeO65WOweOVH7DebCig buBiZCpWOGWi7uSflgP1duhhn/JiZAMVXhBSAcntuFNtApPZZlo5fb18wvMFe1NzvE xLJCqvCx1jHwdpI7Qb+YpqvafIVMoIj7yj4TEfqqTAuyAcFqW0znvZNr7xRenq/Hh5 ERruHjKKT35BA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 16/21] mm/zsmalloc: convert SetZsPageMovable and remove unused funcs Date: Tue, 6 Aug 2024 10:23:03 +0800 Message-ID: <20240806022311.3924442-17-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: fdtpptgs48kri6q64gkmb5dh98wtikpz X-Rspamd-Queue-Id: CF52A180009 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1722910737-268088 X-HE-Meta: U2FsdGVkX1+fnrOArcM5nE9wAxNf25GD/gTvPOsOuieHUGNz/SBMOyHJy0GXnX/vlWhgFznmDpsQcwwbb70Pf7i6pa68R/Kc51qgk6CJk8RuAO301AgNjDx2GiErSQtBNwMOD0mjqKYNhOeObxZHDV8tN0gYSQMU2JHPmLKAcjsiaiXv0PSce/F1B6Plfg934AF1K/4oJ1WEZyKia878Y+eIesRTBgJ0mqat73lpiA0sJCI90DdAcg+d4cJNFAlz+jqq3S4cAk7fs2huyxLbMlUAGIY5wzhmGNThM8PjK8VazLjrTw2xSgpGeGwZnhUEUn/Sq3UnvE0cmah1PYdSGOE5RmKBUJOizMyokPProi+kveMZInRh0r7eP3AJC0/8SQHVQAuodIgTEsHUboXX9hqd0rLBu7GQT7rs1osSlrGGryXBy7eiIlKFx7ob5HSfBXcHHtkLK6qK/Vkdc94bNwV9msB8armhVhCwmxzuRF/3qisB1vTVbQMOlxbAz9alSUFSGf9Ip0kqsVSAQsu/3YWoNRUjWIMW3FhBYsUppByHMfjWKJ6Xv5mF4eqbbk0wu0U6YFrYizvrGWxh68A9eJYYGKm5YE7ZglarwL4ckRyQmabDzAZPE7qsvhia84++AHTU2KufT0ZvFNdluNjTdmJHuwLf6W6Q7QltC3w1hil77vkpCmV8Ln3sH/M0bK8XonL9HFeNj8rBaTNfk9K/cHVemaLiA+KACLXQzPctb+fFxBIV8lRdx0SrC6rT92Yj09oX5aYWe9er/dbquHfW80CqUzVA1KnqZBH6olh9E4PF303sb7N1rNrbA3LU129W7AUN9K3xJfP9vntMEv5jCKagxro86atpoy+98GTshpK9m5lCvG7ee17MvPzfU5kIF72EYz0ber7IZspIcJ5bw3bhJz8JPwol953zVan+o91F488QDW1cxIt7/FOR26i6Jz2fWHqdJIJQ04K3EWv BK9Lb8RU +i9LB9Qbgt1hhwlEWmeXUbtlzXw3SOzwi4ON8KyXqubn5ddRr88NjFynAqqUbzQyA3GaWztHmhUbZ52nQ5bncOY8wARtswh4PtEU/z5x2ju9WBQ7TFOf+wJTDgsKIiUCF+CkEgSrdqLMqIsWZL7TeP8k1/N942rT3/tw59OCUyA4viHxvSkVP0HLjtezuZzQ8AzKJ6HFLTKo3UJ7Ay6e00WbpvVnbDFrAYvFBE6CKu8yOcO/RMEfoQh0ODcvRFpQWTmxM+3Vd43+joTp1NS3NFTMECV7MY/klx3GgxaDxImVQ0EmhrlrQkD0lLv6NVLAS+ItShp1wQ7Fg3pE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Convert SetZsPageMovable() to use zpdesc, and then remove unused funcs: get_next_page()/get_first_page()/is_first_page(). Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 33 +++++---------------------------- 1 file changed, 5 insertions(+), 28 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7ae98d048590..1c53a58e6473 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -448,11 +448,6 @@ static DEFINE_PER_CPU(struct mapping_area, zs_map_area) = { .lock = INIT_LOCAL_LOCK(lock), }; -static __maybe_unused int is_first_page(struct page *page) -{ - return PagePrivate(page); -} - static inline bool is_first_zpdesc(struct zpdesc *zpdesc) { return PagePrivate(zpdesc_page(zpdesc)); @@ -469,14 +464,6 @@ static inline void mod_zspage_inuse(struct zspage *zspage, int val) zspage->inuse += val; } -static inline struct page *get_first_page(struct zspage *zspage) -{ - struct page *first_page = zpdesc_page(zspage->first_zpdesc); - - VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); - return first_page; -} - static struct zpdesc *get_first_zpdesc(struct zspage *zspage) { struct zpdesc *first_zpdesc = zspage->first_zpdesc; @@ -767,16 +754,6 @@ static struct zspage *get_zspage(struct zpdesc *zpdesc) return zspage; } -static struct page *get_next_page(struct page *page) -{ - struct zspage *zspage = get_zspage(page_zpdesc(page)); - - if (unlikely(ZsHugePage(zspage))) - return NULL; - - return (struct page *)page->index; -} - static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { struct zspage *zspage = get_zspage(zpdesc); @@ -1950,13 +1927,13 @@ static void init_deferred_free(struct zs_pool *pool) static void SetZsPageMovable(struct zs_pool *pool, struct zspage *zspage) { - struct page *page = get_first_page(zspage); + struct zpdesc *zpdesc = get_first_zpdesc(zspage); do { - WARN_ON(!trylock_page(page)); - __SetPageMovable(page, &zsmalloc_mops); - unlock_page(page); - } while ((page = get_next_page(page)) != NULL); + WARN_ON(!zpdesc_trylock(zpdesc)); + __zpdesc_set_movable(zpdesc, &zsmalloc_mops); + zpdesc_unlock(zpdesc); + } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); } #else static inline void zs_flush_migration(struct zs_pool *pool) { } From patchwork Tue Aug 6 02:23:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754334 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46D18C3DA4A for ; Tue, 6 Aug 2024 02:19:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9F0556B00B6; Mon, 5 Aug 2024 22:19:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9A3416B00B7; Mon, 5 Aug 2024 22:19:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F4456B00B8; Mon, 5 Aug 2024 22:19:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 60E946B00B6 for ; Mon, 5 Aug 2024 22:19:01 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 14480404FE for ; Tue, 6 Aug 2024 02:19:01 +0000 (UTC) X-FDA: 82420212882.04.34F4AE3 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf07.hostedemail.com (Postfix) with ESMTP id 72D2140016 for ; Tue, 6 Aug 2024 02:18:59 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=BaWMFlpO; spf=pass (imf07.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910690; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uWAjWyRacCJyVicWR8YrrX2R+eFZhZcd/reMVPUk9oA=; b=2CevD4g3+pscqmePepZxpih+qStJBpVcISVVqZOGWN4jtewBWFkfBNBzQKBgDr5YE9j9jn +e1oq+mljKkFq1Z9YMbp9LwBBETvNYHuU6Q05ru20rigsgxHe/1+Pd22Pfbt4rSBBH/5K6 3tGUjlHCxBf+IwiNkvszMaLgLqCj3mw= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=BaWMFlpO; spf=pass (imf07.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910690; a=rsa-sha256; cv=none; b=m9GM+j4ecxy6ubXww7bpGuH0SNqt7yDcT2t64Xf93tXgtA+EK2LbJtsa/Gvx9ZlLhFQeSZ DfMpCDphXwGO3uUkwPn7H59MtpHbgpqDlVki/a8qkNlZI4L+S9dZbN3FO1728XbJtVp6jD J73+Zh4ngVghTb5RjbS+jKVmIqmavPw= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id BAF8360EC7; Tue, 6 Aug 2024 02:18:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4B6A3C4AF0C; Tue, 6 Aug 2024 02:18:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910738; bh=xBRbiPI9Fj5vFVKbb/yDx6Gk1e8RF8irjBVrIsL9n4o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BaWMFlpOOvQxiuUrEdt0K4kqxnxvptrUSrrBTxZQhjnf0InUS+rHp/YA9L18z0U/j o5GFJeiz6YFu/eejUX1D3OwapL+PjeF8zf/2vZEs8ccrA84UfOySV0QI9jatpp15Az xz8nX1jKA+eAf3fiP9Br2fwstLZFEXiPeFHtLhgnOBmcy9ACATYNVDLet8rtdgtY2p fwnDQi+SpyOlv71NA4smr8Y+8jIBLZ8Pa3+WRph3nNejjXqF9EnM9X3i52KFE/XAX7 PJ0RCylZSMz2WHISlIgceqb5JLJl4q0/+QKv7mFavS/oXXL4FUDFILThvNu7v/30Tu xYIIdvUjvEPvQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 17/21] mm/zsmalloc: convert get/set_first_obj_offset() to take zpdesc Date: Tue, 6 Aug 2024 10:23:04 +0800 Message-ID: <20240806022311.3924442-18-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: qt1dyku48xrxf9m7tbw8x11bh6h9u3a4 X-Rspam-User: X-Rspamd-Queue-Id: 72D2140016 X-Rspamd-Server: rspam02 X-HE-Tag: 1722910739-510505 X-HE-Meta: U2FsdGVkX1/RYBzP6cvooQ/ObE/DKdrTUc6/vzRrYaA02UyHKUGcJQkgvotRm21YBMi2lzrc3TZX/uNDNYjDUmqfylzTJ9MXTFdz9G+1eFVSpTSd6kiVcqMYhRg505RXBfsvqORlGWo2U9Eu30+/fm8FuypwZBE4IJ4jxzA6IArdjBbQk97L9Kpvnzeusov8ppfNq5GXF5pN6v2sYNaemPeaVOU6jqW0zi0sO05wQShZh8WddhXESvnmzgYAixDA53bjOLLECfrCgfUVoZYO9v4eAcSzmQh8bTwL4c6hqGcM+bNXLDp/AqIvvUGLLTh/Y7zbY/BaRkLlPtMJgsCueJPnkcmBPLnFyrLo7LCQqf2/J7mRQEPoOdyPapHn7bYkEE+6fNgfg8kyHqPFKwKMYMk153QLSCLjfys4t8kZbSblUBHp7O48y1DXL+YE8xifGMD8Q979ad1VXVME1Qj7eABcKxd1C0rn7KoQ9qd9G4OW5Zbn3Z027kwqyNVQh+wA7/SFnSju/J14z8TYecGhAgCj1tOodTVCmyDrqrIGjg6JO3fZJPsyvKQu7/ZAaA4POJ3woexD4j4wz38oHDKq68dI/hcswYjQxNkhDoATFL4ROJ1k1h/b6XlDyWgMxpA7sUv2Q7bXOHFXgPP+CxYD7cckgMdbB26zSwE4A+SwaeAahxyd7gtY4u+m+dtadbrSBIOHQPOzaKhExZATql+HWYk9WRitt8Hmh8Al9nYjHCtNJJW6PtuD8v1zOtsPfPtXdvjg2tpqcnoVL7FTuDN6hDc9hrgkoEoRSXgj03N4IJMLpm1HHEUdjn5OgdoDcQMYQBwSzLsTWAbSjQ1SB1VlW/iAozEcY/Qi+tlxYAabjm7k2RZJ2zw/bxqFNV8A6STNImYnTg9/rYBp0rAgjTW5UAAvfYpQQD8PMvKWxoqrNr35XG8gUIFFOb4S2cZFXI2rNWCa+ZweMKKX0X+UJsk sat4pWT6 ngydpwf2dMUk5A890608dZqaKf4R18D6bMheXDmh/cwPMEtMUUTvrP7ISbf8+QQzYfjcQI62eGatBQVZpcqnuNvfspCYwybyzqDnaIXQgnt73ij+VYRRl8Z0w57H07wHaLd/AASYAzSrMRV82F6HnQyebCw9j0IrJHSmObCl47UBN2jB9FB8YclPzZI2E7PFJvpJtz5kDRl1rZJn+2mECQq1CxlvrIrkrJZ7HqBsK2/NmG3oGDsLhO2nqGOns3oTmFK23L0aCPcLVcvPPlPod6G4j7tZUMiUERE0tCG4217zJSQeUhk8NJHLO8esl8lMhvEcErrv7T5smJxO98pboSTPLiFj1Vcm2mKCSLWDzTgpSakgPWkcQz1bfzjDM907CfOzl X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Now that all users of get/set_first_obj_offset() are converted to use zpdesc, convert them to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 1c53a58e6473..e2cfee57a39a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -474,26 +474,26 @@ static struct zpdesc *get_first_zpdesc(struct zspage *zspage) #define FIRST_OBJ_PAGE_TYPE_MASK 0xffff -static inline void reset_first_obj_offset(struct page *page) +static inline void reset_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - page->page_type |= FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + zpdesc->first_obj_offset |= FIRST_OBJ_PAGE_TYPE_MASK; } -static inline unsigned int get_first_obj_offset(struct page *page) +static inline unsigned int get_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - return page->page_type & FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + return zpdesc->first_obj_offset & FIRST_OBJ_PAGE_TYPE_MASK; } -static inline void set_first_obj_offset(struct page *page, unsigned int offset) +static inline void set_first_obj_offset(struct zpdesc *zpdesc, unsigned int offset) { /* With 16 bit available, we can support offsets into 64 KiB pages. */ BUILD_BUG_ON(PAGE_SIZE > SZ_64K); - VM_WARN_ON_ONCE(!PageZsmalloc(page)); + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); VM_WARN_ON_ONCE(offset & ~FIRST_OBJ_PAGE_TYPE_MASK); - page->page_type &= ~FIRST_OBJ_PAGE_TYPE_MASK; - page->page_type |= offset & FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset &= ~FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset |= offset & FIRST_OBJ_PAGE_TYPE_MASK; } static inline unsigned int get_freeobj(struct zspage *zspage) @@ -830,7 +830,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) ClearPagePrivate(page); zpdesc->zspage = NULL; zpdesc->next = NULL; - reset_first_obj_offset(page); + reset_first_obj_offset(zpdesc); __ClearPageZsmalloc(page); } @@ -914,7 +914,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) struct link_free *link; void *vaddr; - set_first_obj_offset(zpdesc_page(zpdesc), off); + set_first_obj_offset(zpdesc, off); vaddr = zpdesc_kmap_atomic(zpdesc); link = (struct link_free *)vaddr + off / sizeof(*link); @@ -1569,7 +1569,7 @@ static unsigned long find_alloced_obj(struct size_class *class, unsigned long handle = 0; void *addr = zpdesc_kmap_atomic(zpdesc); - offset = get_first_obj_offset(zpdesc_page(zpdesc)); + offset = get_first_obj_offset(zpdesc); offset += class->size * index; while (offset < PAGE_SIZE) { @@ -1764,8 +1764,8 @@ static void replace_sub_page(struct size_class *class, struct zspage *zspage, } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); create_page_chain(class, zspage, zpdescs); - first_obj_offset = get_first_obj_offset(zpdesc_page(oldzpdesc)); - set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); + first_obj_offset = get_first_obj_offset(oldzpdesc); + set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) newzpdesc->handle = oldzpdesc->handle; __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); @@ -1820,7 +1820,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); - offset = get_first_obj_offset(zpdesc_page(zpdesc)); + offset = get_first_obj_offset(zpdesc); s_addr = zpdesc_kmap_atomic(zpdesc); /* From patchwork Tue Aug 6 02:23:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754335 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F62BC3DA4A for ; Tue, 6 Aug 2024 02:19:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1B36D6B0092; Mon, 5 Aug 2024 22:19:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1626B6B00B9; Mon, 5 Aug 2024 22:19:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 002E56B00BB; Mon, 5 Aug 2024 22:19:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D49636B00B9 for ; Mon, 5 Aug 2024 22:19:07 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 8DB50A1D04 for ; Tue, 6 Aug 2024 02:19:07 +0000 (UTC) X-FDA: 82420213134.22.A70BD4C Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf07.hostedemail.com (Postfix) with ESMTP id 93BEF40002 for ; Tue, 6 Aug 2024 02:19:05 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=GJ4wy4eK; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf07.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910695; a=rsa-sha256; cv=none; b=bf5RKBEck+poq1h6vPYKRAodUHpI8M9c55fZTJvoMmcsYOMQXIaBqWPbrBUMZODZMvKJ73 kyqUFEJlohBdmDc0JGRWHiSTJ51nhOzlJ/ZO9LsUk8kQHlJ5ZkYqcThPwKyFEHJShlFkPh v2pgtp25f0DWprzPQ6YGfpqE13B3dg4= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=GJ4wy4eK; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf07.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910695; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uBWX8k6Fc/SXtf9GHhfeSJeVUi+MHdfG0m92u/YR1lU=; b=Mlb6Qvm8Us3AEiJAH1U2a8vSWaEs2Sfnd5nQ1lBXqHDXEiJ4/Kp9/4vI3iCQSG0p7RFJAr i9LZ9fAucVfetRmAXcgUmPgkVfPBLjCAtEddPF63olLAP7HJSn9ZoVgHqbGsGYzwUXKJBm pyBbjSKQRwjwm6vlco3MOABi1/grnNM= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id D77F4CE0BBD; Tue, 6 Aug 2024 02:19:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1C9B0C4AF0C; Tue, 6 Aug 2024 02:18:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910742; bh=gO5yNLlLwsCfZqvTP2RVzms7OfZiyAMCAW6p5U5JaUg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GJ4wy4eK4qW7kRgbBTYCXZtWl8AumXoo42ILqjgP/YgqVhQuJCoip2c2h9Nz7JxV/ Y+POjNOqPqoYGnzttSb8PWI6rk2905cz6tpbM6fPqqHkcZAESSGqoduGWf9qcJC5uy fdj1He4fUsZ+XfS2s6kIB6S2Vcb7Bxox8DH2QYPvI1HsZZTySzchbOkR4W7LNgBlzA VMUbijwI55mKeVswvC6FPobdGxl25MGMZkfiag52ZV5yHMqh0c+r1IyRmuyzBvSkot lmUQeCUa0om8RhsQedOxNuawS8lii5OvwvYr7I3/LzmSnTIKMnm+jenS/B2PwIE8So gKZLae5Vm8ryQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 18/21] mm/zsmalloc: introduce __zpdesc_clear_movable Date: Tue, 6 Aug 2024 10:23:05 +0800 Message-ID: <20240806022311.3924442-19-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 93BEF40002 X-Stat-Signature: bufmrdfoqmkh97hsjqc6kw8duaysuqwd X-Rspam-User: X-HE-Tag: 1722910745-93501 X-HE-Meta: U2FsdGVkX188jkNdHLIDDc2OV2sHZFB0t03jDSteqm6oJywyWtIljbzf0bLYX3IOsomw3t0ZOcI4wZOiU5yXKRnKQ4CnA5AIc6ZM+K+HH2uW1zubaHcwlGbqZ5lq+bY+PA4aDp9QlEsESDug67BfEpZ2M/k6NJMr0S4Jv7apkM1oNMxJMJROFKlkMSRyLgL7PEMLDsBSO93vr+S9csUIM1PrtT40Ejf37vIRMD0MdX2f27CLb1coLTh2wyHkzsnXmDSNJG1y2MDH7DyiPT3aeu0iZ+C0gsqIW3l97090oSQPaMnzLA4CjepKtTlU3TjDE2nmAvd3932qXftz8dcZy2EdKKZNFNpRC5yE+0rhXs2Pe7jbhCkFIi/3N07cdbCpDrGlsPoXWSBfHdohF7rmqMwudd1PQDUI9caBvZ8OdS55cfE4ZeVNT/snIVkf0x8JzP4GHASXegJQoh4kGv1ruwPieFmdErOKqkaVRUGGmzBMy6Gvn4Zlbi9gNa6PEb4SEQChQkaJ/bzBzn29ReDI3J68ZA53Exlz4MLDTymz99h48wCHImNpFcXJfEiE4vw7QHhbcoMJdV0XW1+OWaoYTjhDCX0NA/GzoQ59Mc5aAyy7WK7SEAsN7F48tpFRDN3xr6INjVaXDBhQdOafWu98ImKd+6BPhnp+EjHGTv53/MAsl2+VBGeUVmOHz4kK8aveMUxZ6edFFTrjamsWC84xtNsZjjeQ3CL57MT3wXCP34ti4hJ7JJ2He+YzxHGycmH5SFok4r2d06UfHoHu8H9Dw6yBrafgidSEk5V8P5Sjy/X5+h6kSZTdBjehL5veo1vAgczp+Z2IZ+bi2oFyOz3EtLQ9q188qz4EtGDrP2pE54x/AP11VEyx6TeL9qiqvqhsWsrrsutXznLQ2EP06tentsMai//EZgAUST5eoXqDEkAPmTmrBK6CqC6t1ga908LTPi1vm/liHqZyTUawzqh 1qZG8KVP z6tlpSIsERrzoizGiyNzjK28wvVM1dL5E219NSk9J8oJnSoPjH62Ltwpk3X7uYud3TlMfPAID+SXIUWo7rdRPCnRNg0XUu5ec7eQgkz3MlvNQZ5wZAb29+GV1M9k2NUVSicp07EUZu3SMT/3Dcz6HNB/5HWjWMxzOHYdg3pgH8yH5eDQzTAX4n5G0FIV5iM59FlwNi28PS0TbWi+QQRIFdEZSdm36F+Pa6hcLNM1UbgEP98jVv98gtlmR2m007V0wQTXRj0JCYWyvKWtFSdCSi6yfTrAVQr1kWbmM1UQbrkXAIEaTDaNEcK8oDcw0I9f4t3n0fXDDAI+uxBhrTQBmPtDXRrQLdaChs2Oj X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Add a helper __zpdesc_clear_movable() for __ClearPageMovable(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 640f33b5c8bd..1ab47faed49b 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -114,6 +114,11 @@ static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, __SetPageMovable(zpdesc_page(zpdesc), mops); } +static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) +{ + __ClearPageMovable(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e2cfee57a39a..7c7432a00a07 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -826,7 +826,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) { struct page *page = zpdesc_page(zpdesc); - __ClearPageMovable(page); + __zpdesc_clear_movable(zpdesc); ClearPagePrivate(page); zpdesc->zspage = NULL; zpdesc->next = NULL; From patchwork Tue Aug 6 02:23:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754336 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E41EEC3DA7F for ; Tue, 6 Aug 2024 02:19:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0C7BC6B00B9; Mon, 5 Aug 2024 22:19:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 079EA6B00BA; Mon, 5 Aug 2024 22:19:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E0D076B00BB; Mon, 5 Aug 2024 22:19:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C1BC66B00B9 for ; Mon, 5 Aug 2024 22:19:08 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 7EBB3A68DF for ; Tue, 6 Aug 2024 02:19:08 +0000 (UTC) X-FDA: 82420213176.08.C5C0F52 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf20.hostedemail.com (Postfix) with ESMTP id E49431C0029 for ; Tue, 6 Aug 2024 02:19:06 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=HznYQbSG; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf20.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910677; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ovfDqQxTVdKERQAKtqZwOsjviHhDb3m0AzMbr6CEq8M=; b=7jx7irllh2LZpw2VfXXt2GCCMJqB1oaJDOSXG9L006peWaDQvD6nTC1X8j/Ds1t5xVkdhC o7WAYbZUQjQRMW2dl8f/qCe4sVRrx4fZj5dvG887AMcB9G5Ni35XFo08WaNAiCHBgz287c oxH8XMoK2cbLHZayEbNtmYO1vD/BSU4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910677; a=rsa-sha256; cv=none; b=iLVzWKu8Z8k4N+RvlQJ8mRWStHv1Dq5GGoh92w1f3imLu/gaftBCWm8Atm25s3b5pG9TFZ ZEAMIG38QKOn3S31xSrD3a5RfPFjHDIztGwY0Zj3hFqPDWWpJ0iGo8D1VldsYzcD5U9NpN VNR95UtnWa2dOYRtmmv+9GOT76dBtP4= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=HznYQbSG; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf20.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 3850960DC4; Tue, 6 Aug 2024 02:19:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BC641C4AF11; Tue, 6 Aug 2024 02:19:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910745; bh=oL5cTZYBz5hcMd4ibMjrzRAOxUIAWeTC3GGP5qzznjU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HznYQbSGB5gQ2kI4wcaUCOnPgGomh9VJ4uFtU8YG1LSpwabhCQ5mxFNi8WpsZ6exU K2rMxdEaj7xFao9BCCnDrHgaXr9hQ4jYCcgFfLBKdAml2YEKJ69MLvRMrXZQgAQxqX RONxiHPQHIUEX/m1A/ofr7XuFO/YWOznmfR9zypSe7GLU6pMZS8Xx3aN/pypoyEzh6 kLyjt2xVtNmiSUqQtoB2AjBnSdr4Dcc/epjg/GoMU4SEg/Xv2KyIEHTI6BaDEVxS8k gT+v+/dqRQT+ymOFOQz2ckMqoZBnAoBylS3mn63NoXSeKUbYwjnzgDGBG/r5ZOElwK iG0VmI/vVGPxQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 19/21] mm/zsmalloc: introduce __zpdesc_clear/set_zsmalloc() Date: Tue, 6 Aug 2024 10:23:06 +0800 Message-ID: <20240806022311.3924442-20-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: E49431C0029 X-Stat-Signature: jrs37zritnjuwwg4j335j5g37r8yd598 X-Rspam-User: X-HE-Tag: 1722910746-791517 X-HE-Meta: U2FsdGVkX19r997sVyR9hUNA2800dzmBW1nSsDUOwtxH9Opy+zJY8aCjkqAjH6/Vgi5luYxSi5ynKzdBDLmuZ7NnAanzLu1m6QElrUB8kVJQcAqjK35qB/HbmGxr4pq8ixtdCPlPeomlhvypBK8vgAAXfGYasyTbppA+DtMyHIpHorTWqBoi+aAzV7j89Oj5biIrNLhavv2FFGxPLno8ih8asEsoZ2MhY3aRA9ghDpPOKPzOlScUBOP5SW+oWMBkabZescgoMPm2pNjgxnUHKoMWJqGWqgKBPVtwXiCKaJcF+Ge0lQdFKdJGMD2+Mj9t1Fq301zeATnjSRXRp0jBl08wc4o2il+EFUybSJ6oVW7jwLQ4Bue5kvrlPN34I8608pOhN0mVEcYT3MmsjsQJPwi9m+784R39AxmHnnrM9z+uPkN1iCQUAkzJxqcEwIXxkcf2zv1qKqsAHwUDtKSKsvTU7iEKiFXcJUGsntFoijIDjP2g9QE/gsjobpgi5yI9qktyxckPZZjfOvpNWc9iBrP34xuA0OBZSjm/EnEDKzWefjXnyv1RLr87HNU1/Y03klpvDr2jWEQMupjn4WR9Yj2Eg4sKmEsMofBYOXYVQfKfapvu8eNoxuORZ9YXDkchq31seJMN5tzH1J5q9NfBZboMopeJOpuJdClPKPyBCvUHKqHBlbYdvdh23AJRwzRTfrGdoXkuEWpKhXV32CtPUMvYIUJ0YtP8t/E1Ee6Ps1qvZd71aOie7djPpKoPrUjKx5FGhyKfOg5/ypuDwhzsI82KTzxTmTlTtXFHOf8awinR849f/3UDUKmeqrQaT1vmIQpkXm4ZI/YOXuJWaHA8DflC8hQNx6mxpsPxrxY9b1+Lvc/MPIFFDjmcoYt4csNh2jPgosR3Jy3CchHbGZ7MvqOLF8rbmw/Zoajl/vLlOmLhGJEe1e42x60VbBnIDr/l9rb1wOAGkYsX57Alh27 9gcaJmv4 L7ojqG2u3EvwQZDcL3nZS50QXab69vyR0z8Qhu1aggpErE2hvagB/3/BR6aslG3mxnAWei7JhJ2kVriikevW8z/XTgRlASZuPrF8ufONZK253fZtyd0rgQz1ldTZs7eHbyhAo9H62OS2sl1LLiBaODhHL9PfwULpBQ0hP5bmESNig81U4d8FFWkJUZ3uTWJSITIjaR1Fknjts2u/AogTo64FKGkzJ/cM6hRDR0m5e0h9Ed8a7sqP5KQ757jfT3HjubzZMC0NOSMPF65PDSaZ3tHis7At4Q6mb2WXvdL6eeQBY4lef0MyTqGlExqDwZcjb7Lrhj/NJCy60dltoyJFHg6RU55q7KHXqIanL X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Add helper __zpdesc_clear_zsmalloc() for __ClearPageZsmalloc(), __zpdesc_set_zsmalloc() for __SetPageZsmalloc(), and use them in callers. Signed-off-by: Alex Shi --- mm/zpdesc.h | 10 ++++++++++ mm/zsmalloc.c | 8 ++++---- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 1ab47faed49b..d0a1af0ee424 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -119,6 +119,16 @@ static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) __ClearPageMovable(zpdesc_page(zpdesc)); } +static inline void __zpdesc_set_zsmalloc(struct zpdesc *zpdesc) +{ + __SetPageZsmalloc(zpdesc_page(zpdesc)); +} + +static inline void __zpdesc_clear_zsmalloc(struct zpdesc *zpdesc) +{ + __ClearPageZsmalloc(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7c7432a00a07..0b318db894a8 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -831,7 +831,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) zpdesc->zspage = NULL; zpdesc->next = NULL; reset_first_obj_offset(zpdesc); - __ClearPageZsmalloc(page); + __zpdesc_clear_zsmalloc(zpdesc); } static int trylock_zspage(struct zspage *zspage) @@ -1004,13 +1004,13 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, if (!zpdesc) { while (--i >= 0) { zpdesc_dec_zone_page_state(zpdescs[i]); - __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + __zpdesc_clear_zsmalloc(zpdescs[i]); __free_page(zpdesc_page(zpdescs[i])); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(zpdesc_page(zpdesc)); + __zpdesc_set_zsmalloc(zpdesc); zpdesc_inc_zone_page_state(zpdesc); zpdescs[i] = zpdesc; @@ -1800,7 +1800,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(zpdesc_page(newzpdesc)); + __zpdesc_set_zsmalloc(newzpdesc); /* The page is locked, so this pointer must remain valid */ zspage = get_zspage(zpdesc); From patchwork Tue Aug 6 02:23:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754337 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBF70C3DA7F for ; Tue, 6 Aug 2024 02:19:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5FA216B00BB; Mon, 5 Aug 2024 22:19:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A8B06B00BD; Mon, 5 Aug 2024 22:19:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 44EAE6B00BC; Mon, 5 Aug 2024 22:19:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 244D56B0083 for ; Mon, 5 Aug 2024 22:19:15 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D9E351C4F96 for ; Tue, 6 Aug 2024 02:19:14 +0000 (UTC) X-FDA: 82420213428.06.2F53501 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf18.hostedemail.com (Postfix) with ESMTP id D21D31C000B for ; Tue, 6 Aug 2024 02:19:12 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=eogsWgAJ; spf=pass (imf18.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910691; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=b8F0v5gWthfH+h7xRuT9XYpictXvzfJNloRRGZ5+SHM=; b=7mdmxstkIitFWUtS++fOdLmZdS8X5VF4SYZZHWyyZG6ngo0QMR6g1/s0mEAXt1HytQ6YQL IwnIjy8gofUoArVVW8yYkIFLhzyW55WMUI7ANdi369M2Gjr4NrHgs/87TrZgzB64WmMP4+ O0d5wOkQ0r7GZYvWoiFR4l1seVveFQo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910691; a=rsa-sha256; cv=none; b=vmaupF/2yTye/paePYJBJwpQLKM9HwCGKoIX22ZzfHY4MHhEN7uYW4CUb+AGupfww0sxXU ywh4Ias9hMwP5eNpKycmbgzLoDdmOvQyrZpOVI4eSSJJ00kNidaVW+tKL5oeodzn9myv78 BV7pJXI9wqPB3Qmop9Xgq6W6GzXC9js= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=eogsWgAJ; spf=pass (imf18.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 45215CE0BB0; Tue, 6 Aug 2024 02:19:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8E4D5C4AF0C; Tue, 6 Aug 2024 02:19:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910749; bh=SIgiyPCW8QCBZtqr5fnMzvbajnde7CasC3gp99UMX7E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eogsWgAJOVMVgMm+88DiD7XkroE1wr8TVvpEQjp6NmnLl30eBFsFWj/nQJK+I8c5I jUhC6+ElxHccYMtnUAdfDvOCrIXJ1xMAi+HSzAgR5yAn1y686RgXFAQo4iYVVx44kJ dBNVnqwpiPqTyTKh3OA5LoUrYXaseNQYH3N0a+Z8E9/IdcZJ72fPC9/LMMjOC5NpMH h46X+x/17nvesgtbZ3mxjDf/f1XGF2Wp8TGFIPucQKhRAFPIUBJxVLCHLlEozMXN/2 yHKU2MvKQY37DOnJnxKAUQXYl65JxIApCyH3Cvfj9VuUr49mJFH14C+pH9E5/7j6Ly xUWeJVNSWL3aw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 20/21] mm/zsmalloc: introduce zpdesc_clear_first() helper Date: Tue, 6 Aug 2024 10:23:07 +0800 Message-ID: <20240806022311.3924442-21-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: D21D31C000B X-Stat-Signature: j7tmfknn4stefai3umt4qn4bzp9o95us X-HE-Tag: 1722910752-441154 X-HE-Meta: U2FsdGVkX18WMo9JreE2TcuN7kp8K/OqRvvNACJ8aaHMx+TOvXheVEaSej24V840ckBI6L7SRvat6JqPm3TBa/fxTyjR7Nqkibr/v7kRNoWk8mvIrkM/GJgj6ZpVEVsRjfdsZrrxfRyalT5Pd3cqL7hhx9/AXgYm7W8Fa5mF27OqwMa+Hsi4ZSdfNlG7cTQCJlLmkhh5uOs7Fm6EHGIy6NpeeDZT9hMUFmK86cKiKfKeY//VhAeWnLeXM/GQkEnOYHh5A0MxYGT8UekxyhrPfGrZmYybuboATFh0o9gh289P8uCjD65VonsXhGAHGn5L/OuB0hY0oMkTUfogQnDsKypw8i7G6sD+2BI6DoyWuDL6PdYuRvtOO+1XekptK74eOYBbuiUgISJqOIVgi6yDozVRPqL7i6YRP9dauY/JKtlK459vV8CanofIsURoBK7PbMciCOwdvWcvfJjQV0YEDb3aju/bElQjXL3RxMsQiNopbVcmFHHuJz4ornfxdvtWLF5H5L/0/2w+soI6CIsl9pu2d6/grAqvF6LgCdQqbiazrszTMOw10OTxjeYBW4xbIEjQBJhAW7ZHD4yABZbtMDjtkxWkKTepCc2JnKOhKJX1MP7jOz/cmDZS94FaDlIxUeSK4QN8k2xgTpH9a5ORiaxX6ETLkVYpcJkC3WMvelC5JFhvcLqoZQg4/8utFO0/l51hsD+9nKReOvyKPGyo9dqT2gsry23nHW5L9c43A1ksi+S5t5oRXGghtD5CJmRYm8qkytSpkJTEt6SF//aROlkVJ815lIzwBlXbFk0BQBAZ1Icr5Det3anVVhEUAR7RiGhHs+5Jty5VwuQMBzY3qFHA2AlVTQ7N59rUup9nNuSjNbyxoaPTA/0mB7GqwNHUZrUw43cPljUIvB0Itr6eGGYafCUa5D0acl2fNA6ci8KJ5sGQoGrMc+arDHi0ggPRukURtEMOdo7Jl959Qa/ VaVPsoB+ 2kiqqUMMz/YMQixUR/fVrwsafLIICosNPhIJ1aNdRhrKcc3DcPxj3QOchmpyrWxiR9kb2qQSLVISQYIu2j781onHPyYFkG/kvOHkoQ9Ofn7+vA4VHucQqVX/7ziNPebriF2t/WFGCckZrPc8KgqxAq8FLymLpsgoCayqJ1z5hvU4dTfUILuoYpxvlVlXYwm2o7C4ouQ2HBkW2KCixPVryiV/JZ3hwdDv0hiwTkptGD8UIemjZ0WIt0FILSCEBrAnxGkz+kpBaaCyAJIW3e2mk7U1I2jgZ6TGahWqTMM9wPDyCKohv06wSwRZjAfbC8/iA5856Ik/EZ7zrNx0iwzy3y6y3OI/6epXpuzPv X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Like the zpdesc_set_first(), introduce zpdesc_clear_first() helper for ClearPagePrivate(), then clean up a 'struct page' usage in reset_zpdesc(). Signed-off-by: Alex Shi --- mm/zsmalloc.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 0b318db894a8..e92451c6b69c 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -253,6 +253,11 @@ static inline void zpdesc_set_first(struct zpdesc *zpdesc) SetPagePrivate(zpdesc_page(zpdesc)); } +static inline void zpdesc_clear_first(struct zpdesc *zpdesc) +{ + ClearPagePrivate(zpdesc_page(zpdesc)); +} + static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) { inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); @@ -824,10 +829,8 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, static void reset_zpdesc(struct zpdesc *zpdesc) { - struct page *page = zpdesc_page(zpdesc); - __zpdesc_clear_movable(zpdesc); - ClearPagePrivate(page); + zpdesc_clear_first(zpdesc); zpdesc->zspage = NULL; zpdesc->next = NULL; reset_first_obj_offset(zpdesc); From patchwork Tue Aug 6 02:23:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13754338 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB6CBC3DA4A for ; Tue, 6 Aug 2024 02:19:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AB5BE6B00BC; Mon, 5 Aug 2024 22:19:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A164C6B00BD; Mon, 5 Aug 2024 22:19:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8908E6B00BE; Mon, 5 Aug 2024 22:19:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 699AF6B00BC for ; Mon, 5 Aug 2024 22:19:16 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 2D64E161D54 for ; Tue, 6 Aug 2024 02:19:16 +0000 (UTC) X-FDA: 82420213512.13.58E9F35 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf22.hostedemail.com (Postfix) with ESMTP id 77B1DC001E for ; Tue, 6 Aug 2024 02:19:14 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=fHD0Nind; spf=pass (imf22.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722910705; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vqM7Gkcszhk+4w/7mI0miJdp8Uy77XWXHsMSi6m4afQ=; b=7/h02sTZ7vpezVa8WEVVWKNOGmZk0giFf9Bom4yYKPoiu2qMoT3MEbSVJ8QK4uX9RCGfOg s+/Xu9KBa3UDCU8nndPXc0PFwqsafuZx32fCFHdrJ2Z/zttemhm/fj9od+rzOOj3v3SuKH L+Uiqfaxza/l4iOw0zCXAosa3ukcGDU= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=fHD0Nind; spf=pass (imf22.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722910705; a=rsa-sha256; cv=none; b=QVOEJZgnuIG8+TD8ZLDrhSWEhfphmyUCphPO/Ru4cGHAbO1Y4N3QHrm2G6ZfKAi6Pq7U4v 3c+COan6i1dApzSHnWW6tMNckkwUR4fL0V3s/nVTbkQ0E7oz7exeLZRkF9BKYhzlopdsqP bfB85k6ZcQeYFcw0cr/pmXjRirU0uF0= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id AC50860E00; Tue, 6 Aug 2024 02:19:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3BEC0C4AF11; Tue, 6 Aug 2024 02:19:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910753; bh=8EJ/o5yp3ZapndwWQeMuiNEuMIoUNdxMr4p7pDAhjMQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fHD0NindhMQW6QUkJ2u9oDL3HNMi/H8qnJWxEJrGbwtH1YlIb2amxcYwVifocYTdU S4LnhS1E+vxEX5IUdJBshGvy/4Dkn7bBOdoq6v4vqZpYCkKdW+tzpO/HAsF430o7YK iCQpenKaRCiHhk+pkrD9d6b1W7PAfmGD87rWCRlLrvfE8NInQwIObWeiw3kX9eifcF H/PzSqlHXVcXLUFXOgsJxQfLjOW1W40R4RhUgtvXZDEalOqqkVePdXSRBv33b1YYmV i6P2PWItaOhhBbbw2/aX1FW7p0MeXuJp6GEMv+4xwNRv0u7yh3UTkHwfIBgu1KrIoI QmZak2SKy6ucg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 21/21] mm/zsmalloc: update comments for page->zpdesc changes Date: Tue, 6 Aug 2024 10:23:08 +0800 Message-ID: <20240806022311.3924442-22-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: wswp1m1t5azgmaxizg634tay7r3ib3eu X-Rspam-User: X-Rspamd-Queue-Id: 77B1DC001E X-Rspamd-Server: rspam02 X-HE-Tag: 1722910754-187473 X-HE-Meta: U2FsdGVkX19qM9oCxyO5HSn7zSREP3YvU2IS8mOLz9RlHl90g6RfY2GB9yR20VKexMuGCcefpA6cRCwF6mfAvLcznvaeZUv1Cc1pdq4Rl0HU+JqTgMGMRf8QF6dCDY2f3X6COBha11WDYq9K5BFavrxlsRp+yGD5soORUnx6FjjeH4ZHKmOejGSbD8c9GE6BEUcWkvjOculBzVVZz/qHQcIzhqnEtvnUrqkBI4zYmEyGv9WHJuk9FOj7ncEQhMm2PrkZeJsRF7/VtB9g0LSx55UtUFd27oeoJVZ8IPlTR7DILwZQRg2FpsrPMUcAODMEVayhIq0ewJnfeSzb1kKlYqrSMwUkzBE4YBEdNRX2wSs/2C+pTyw9t0T0nRwt2IQ096T8D48SuWzRPJDhArLhThmIambQYMSctSJcY8oBktIyjqJcSPyfNDUG5l6t90fWgwbKgmCLMVtmiXLxr4n3ca4ZiLeOfhDax1iYvV7hkXw3yr8fhg2244gG8scA/PBMF/QbR3pw/9OlN2SCsq+Vub5arH8hBWpmtrJbqy5wHDDcOMtHtv85uA2EA6/QHmNtL6MZbShdKCWYLYSsl/9zEjVsY9uUszxcYQMzNzuQZRAeAgiDzqwqv+IKYhnW2gSQDYXPJCIC9VpOnRnhrDC3aajlDMfxOr1fYxpT9LwTtuXwvIMGOff58yigTu7zTI10mporK6n/TnosbQaDja9P4I+BbjRdYuJX/bbTARJoH+t4Tu9dkmqbKWyDSoPEVvjoNZzfXRVV2J4kysN59AoA679IHKdXvi2Cwt4zogXBUzAVOTEJAHZd76wjU4N+9YzCdXaK+DpNnCj51rdS02MR6Y8hRNTclzA9Xgi180eq4LDW44shWbt/eOkSwkrZUDefaBCmfuzEhkbr5AFwh5f91Slp5PIwTj34t02cCDbX00O9T8xmT5xFbCIU9RHs0RgaStgVB/1xpdkCqWs0Qzw wh/PRkLs aL9eaqHjJk/GArr+/OZRxSBoaOXJ5+uyYHQ0MpjKDO72m6hLQ6Ds7J+yEGOzyA4A0DiB6luzWSY8dc3Wr4R3adAAz/IZMzZYCXmvcg/9NHx3Ummt3T9qnHJP1zjsuPFEqix7u/MIjdcCz+J+B5TOexXSFPZfQz+cWAWEc5em7ZZFymtwSUFkGZwmBQlKhmDpBne/bYvtY25Jjyic2YR2Cc0X4WXiQOD9KEpDnvncR0v+R9Hps1MhZA8iTrHMEqpZYKarL/OKqo4mMR7he6nS2upnnjYect0iUy7bTRCpwiVOiuhvFy8LzYZEmbw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi After the page to zpdesc conversion, there still left few comments or function named with page not zpdesc, let's update the comments and rename function create_page_chain() as create_zpdesc_chain(). Signed-off-by: Alex Shi --- mm/zsmalloc.c | 47 ++++++++++++++++++++++++++--------------------- 1 file changed, 26 insertions(+), 21 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e92451c6b69c..a3d9431d29ec 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -17,14 +17,16 @@ * * Usage of struct zpdesc fields: * zpdesc->zspage: points to zspage - * zpdesc->next: links together all component pages of a zspage + * zpdesc->next: links together all component zpdescs of a zspage * For the huge page, this is always 0, so we use this field * to store handle. * zpdesc->first_obj_offset: PG_zsmalloc, lower 16 bit locate the first * object offset in a subpage of a zspage * * Usage of struct zpdesc(page) flags: - * PG_private: identifies the first component page + * PG_private: identifies the first component zpdesc + * PG_lock: lock all component zpdescs for a zspage free, serialize with + * migration */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt @@ -191,7 +193,10 @@ struct size_class { */ int size; int objs_per_zspage; - /* Number of PAGE_SIZE sized pages to combine to form a 'zspage' */ + /* + * Number of PAGE_SIZE sized zpdescs/pages to combine to + * form a 'zspage' + */ int pages_per_zspage; unsigned int index; @@ -893,7 +898,7 @@ static void free_zspage(struct zs_pool *pool, struct size_class *class, /* * Since zs_free couldn't be sleepable, this function cannot call - * lock_page. The page locks trylock_zspage got will be released + * lock_page. The zpdesc locks trylock_zspage got will be released * by __free_zspage. */ if (!trylock_zspage(zspage)) { @@ -950,7 +955,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) set_freeobj(zspage, 0); } -static void create_page_chain(struct size_class *class, struct zspage *zspage, +static void create_zpdesc_chain(struct size_class *class, struct zspage *zspage, struct zpdesc *zpdescs[]) { int i; @@ -959,9 +964,9 @@ static void create_page_chain(struct size_class *class, struct zspage *zspage, int nr_zpdescs = class->pages_per_zspage; /* - * Allocate individual pages and link them together as: - * 1. all pages are linked together using zpdesc->next - * 2. each sub-page point to zspage using zpdesc->zspage + * Allocate individual zpdescs and link them together as: + * 1. all zpdescs are linked together using zpdesc->next + * 2. each sub-zpdesc point to zspage using zpdesc->zspage * * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). @@ -1019,7 +1024,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, zpdescs[i] = zpdesc; } - create_page_chain(class, zspage, zpdescs); + create_zpdesc_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool = pool; zspage->class = class->index; @@ -1346,7 +1351,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, /* record handle in the header of allocated chunk */ link->handle = handle | OBJ_ALLOCATED_TAG; else - /* record handle to page->index */ + /* record handle to zpdesc->handle */ zspage->first_zpdesc->handle = handle | OBJ_ALLOCATED_TAG; kunmap_atomic(vaddr); @@ -1679,19 +1684,19 @@ static int putback_zspage(struct size_class *class, struct zspage *zspage) #ifdef CONFIG_COMPACTION /* * To prevent zspage destroy during migration, zspage freeing should - * hold locks of all pages in the zspage. + * hold locks of all component zpdesc in the zspage. */ static void lock_zspage(struct zspage *zspage) { struct zpdesc *curr_zpdesc, *zpdesc; /* - * Pages we haven't locked yet can be migrated off the list while we're + * Zpdesc we haven't locked yet can be migrated off the list while we're * trying to lock them, so we need to be careful and only attempt to - * lock each page under migrate_read_lock(). Otherwise, the page we lock - * may no longer belong to the zspage. This means that we may wait for - * the wrong page to unlock, so we must take a reference to the page - * prior to waiting for it to unlock outside migrate_read_lock(). + * lock each zpdesc under migrate_read_lock(). Otherwise, the zpdesc we + * lock may no longer belong to the zspage. This means that we may wait + * for the wrong zpdesc to unlock, so we must take a reference to the + * zpdesc prior to waiting for it to unlock outside migrate_read_lock(). */ while (1) { migrate_read_lock(zspage); @@ -1766,7 +1771,7 @@ static void replace_sub_page(struct size_class *class, struct zspage *zspage, idx++; } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); - create_page_chain(class, zspage, zpdescs); + create_zpdesc_chain(class, zspage, zpdescs); first_obj_offset = get_first_obj_offset(oldzpdesc); set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) @@ -1777,8 +1782,8 @@ static void replace_sub_page(struct size_class *class, struct zspage *zspage, static bool zs_page_isolate(struct page *page, isolate_mode_t mode) { /* - * Page is locked so zspage couldn't be destroyed. For detail, look at - * lock_zspage in free_zspage. + * Page/zpdesc is locked so zspage couldn't be destroyed. For detail, + * look at lock_zspage in free_zspage. */ VM_BUG_ON_PAGE(PageIsolated(page), page); @@ -1805,7 +1810,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* We're committed, tell the world that this is a Zsmalloc page. */ __zpdesc_set_zsmalloc(newzpdesc); - /* The page is locked, so this pointer must remain valid */ + /* The zpdesc/page is locked, so this pointer must remain valid */ zspage = get_zspage(zpdesc); pool = zspage->pool; @@ -1878,7 +1883,7 @@ static const struct movable_operations zsmalloc_mops = { }; /* - * Caller should hold page_lock of all pages in the zspage + * Caller should hold zpdesc locks of all in the zspage * In here, we cannot use zspage meta data. */ static void async_free_zspage(struct work_struct *work)