From patchwork Tue Aug 13 08:45:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761550 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70F1FC52D7D for ; Tue, 13 Aug 2024 08:41:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A984F6B0092; Tue, 13 Aug 2024 04:41:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A20546B0095; Tue, 13 Aug 2024 04:41:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8766A6B0098; Tue, 13 Aug 2024 04:41:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 66AFD6B0092 for ; Tue, 13 Aug 2024 04:41:13 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D3E6D1C3111 for ; Tue, 13 Aug 2024 08:41:12 +0000 (UTC) X-FDA: 82446577584.05.C32DA9A Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf18.hostedemail.com (Postfix) with ESMTP id 39D5A1C000C for ; Tue, 13 Aug 2024 08:41:11 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=sctRuWuO; spf=pass (imf18.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538435; a=rsa-sha256; cv=none; b=1Fna3Myt4klOkF/ZTBg5DbipXMoM3LNcYSQvJ/2kiMUk+oy4Wf2eTlU5Xg+BUXq0rjXfgG 3Bz95O5c39QqdbkHEa3SdW2neM11puX6WViEIyG9KB8zDNYZ/qf9Qctew3PGQ+RpJ9NZpD oHPBDVlBvCEikCgOtVD4SGV2oOSCiTc= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=sctRuWuO; spf=pass (imf18.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538435; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YpSr9HxqtrC1WeN9TJ7/XQW/IWivlFJeqtO3+TGnico=; b=8Z37fKeUrS1+9T011tC9Vmpkz+04qoOqZDL/QfTsgKeqE2OdhQRyZHfInBnDf96yWT0AT6 HwhEBoTvVjaS7SEIqV0GmpKlpaQknLhkWxBAtIzLMFxswzf2m01FsRDeL5sEUhOWeVu2c6 gVbfDuaGxGh6YgtdcMd7+K+t6TUV1/w= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 57AB7615AF; Tue, 13 Aug 2024 08:41:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C85E4C4AF0B; Tue, 13 Aug 2024 08:41:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538470; bh=lcqEVHRvpSRBTNxWYIoFd8H0J5LADCF1j8AgJLJ7lyU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sctRuWuOZUMHpgEBqQgzFcyirs9bs/BF2RsAguyaDVXZh8TyHBKHlq0mAx6t/cAQw stINPQtq33vZoulhTzvNOnn8M4MantaAoEpY3p+JOyQ9F8NZbvsaIlPdr4HXyvUvjk NPevCP8yIulU0G8VM9YbZ7C1O0x+RO3pN8CgrlQ4u8hctVFYUu9O1Tub2ebG6P+/64 Rdrg6bNJneuMkPj5szU3VOE1WynC3YoCtCzqSx12ZTaNtQA8fRSPtWuSV02P9bX9Yt ezIXBlQInIGxfpz1NkDMEMPMmeAIqEDGaiB3RXvQKSzASlDi3SrXYdkYgV1kpkMUE2 GQTccEpwQdpuw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 01/21] mm/zsmalloc: add zpdesc memory descriptor for zswap.zpool Date: Tue, 13 Aug 2024 16:45:47 +0800 Message-ID: <20240813084611.4122571-2-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: yzsq7yydko1oxa9gf3gopu11f17dtdzx X-Rspamd-Queue-Id: 39D5A1C000C X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1723538471-260952 X-HE-Meta: U2FsdGVkX18A4ihfDEa6Kp0XeuMK0OM7E2u0rM3Gaev8/liWxIbhvsu41e6vbncnDK50O+yGvakgoYwHlnSMy0VOSy5eWzOnuciQkGwvMdeYw15qv3L9B+IjrILeSQuROcgrzwsnThOswjsU0Nh1//lcPmVq+R4zIPCMwyVOa7Vpd521g6USqX0FlA5Jp0oKIIiELACznAJZ8ohyR4yRqAV2TUTUlNqU1GN1ndMJbR949FF5M9inQWjbXHZtr5/udEVR/lE0l3C1OxdmN6jYWuu0k2WVwwT4yWYuq1F/+RkoRMlAH6eyU7hJnL+8OuWYzJTf+JtaiAHQJp9TT6Rx9mkHj7jun1EXPcW9AQnTNJ/UdYVJLtqoVRjtuO/r7g1ikAE0t7GelNe2SLFapgUBE11bpMQoSZ6qWSpawnbqPE71BoM02ZSRkpwb6FbIekXco+PiZcr+g+b7qRwCkSLyBYrIeMZ8qlIlIzMS0+HoT5y5famzMFYUYN/MQg81/e+eykU5ypldKal+HPt3gzzZlnQJtFGDT8L66d16OYSCfUuSRV60kJZlGi+044ilWnKoiKSNpf52ZSI9HZdV8sMX4Z+5Ikqwcej6d0HdCxHHBUg8aelwj3ZcGc570B2vCk3QjyzUWgrCJb2k8gxm3ICEjq2HjDUIY/tGNO1DNA2O3UjrdrtPMfdDt5K4F5j/lseMc7wmNeAxQgxujpdW2vnEBmYKer4Zx70fty0Af7cKaSmNaRn9O70mUlPjgzBfhmSX+kpTfz1Aj66fnuYZ4jqq6NoDQG2ukS0UpKIC+bjls8lf6ciRooSHZHTvljGEcsX1UsQz7+/Jh/vj9uN+bhf/DZZFMvg7CjdR+jQN560/PZqEiwwkYBUuAG3bAVZLmBUbDOLB3/vdPUfqL2LnAhC7rxxldhHT2t98dmlg4MJgbI4Fw5XYPqP6K8eIT2BgkEXRN+8Hj4ZN/0NsVXsSp14 SL0GVZDA UDBD1GIjLXcgy9L/OOphJbUuMmTNi91ip6WYVz3xugyy/MztA2YjerJLEaNzbEBlrBbjusI3qD6cZ9p+jTLOHjUwBuJ0sp1NXRL4L9Bgg7JEdYYnoArVI1dXfZb6gYN+RyU5dOXyoleyKW7JLrrX4xzQVILPVZxMIHYAaFinGCzJvGHox4URZl2tWqgUt/S4HJesKXaTnfsIvoBVfPlyM1lFO3i3m/6mIFhBwjLfBPJh/Ox92PkJw3d8P0WJoT4b5eZweIg3k1Dq8zE0yIaKLzvVQsrgMW25iGN/tjFf6LvMLVnMZnkuVPjZvCalRrEukQG91/3JE2FC1Lws= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi The 1st patch introduces new memory decriptor zpdesc and rename zspage.first_page to zspage.first_zpdesc, no functional change. We removed PG_owner_priv_1 since it was moved to zspage after commit a41ec880aa7b ("zsmalloc: move huge compressed obj from page to zspage"). And keep the memcg_data member, since as Yosry pointed out: "When the pages are freed, put_page() -> folio_put() -> __folio_put() will call mem_cgroup_uncharge(). The latter will call folio_memcg() (which reads folio->memcg_data) to figure out if uncharging needs to be done. There are also other similar code paths that will check folio->memcg_data. It is currently expected to be present for all folios. So until we have custom code paths per-folio type for allocation/freeing/etc, we need to keep folio->memcg_data present and properly initialized." Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 72 +++++++++++++++++++++++++++++++++++++++++++++++++++ mm/zsmalloc.c | 25 +++++++++--------- 2 files changed, 84 insertions(+), 13 deletions(-) create mode 100644 mm/zpdesc.h diff --git a/mm/zpdesc.h b/mm/zpdesc.h new file mode 100644 index 000000000000..721ef8861131 --- /dev/null +++ b/mm/zpdesc.h @@ -0,0 +1,72 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* zpdesc.h: zswap.zpool memory descriptor + * + * Written by Alex Shi + * Hyeonggon Yoo <42.hyeyoo@gmail.com> + */ +#ifndef __MM_ZPDESC_H__ +#define __MM_ZPDESC_H__ + +/* + * struct zpdesc - Memory descriptor for zpool memory, now is for zsmalloc + * @flags: Page flags, PG_private: identifies the first component page + * @lru: Indirectly used by page migration + * @mops: Used by page migration + * @next: Next zpdesc in a zspage in zsmalloc zpool + * @handle: For huge zspage in zsmalloc zpool + * @zspage: Points to the zspage this zpdesc is a part of + * @first_obj_offset: First object offset in zsmalloc zpool + * @_refcount: Indirectly use by page migration + * @memcg_data: Memory Control Group data. + * + * This struct overlays struct page for now. Do not modify without a good + * understanding of the issues. + */ +struct zpdesc { + unsigned long flags; + struct list_head lru; + struct movable_operations *mops; + union { + /* Next zpdescs in a zspage in zsmalloc zpool */ + struct zpdesc *next; + /* For huge zspage in zsmalloc zpool */ + unsigned long handle; + }; + struct zspage *zspage; + unsigned int first_obj_offset; + atomic_t _refcount; +#ifdef CONFIG_MEMCG + unsigned long memcg_data; +#endif +}; +#define ZPDESC_MATCH(pg, zp) \ + static_assert(offsetof(struct page, pg) == offsetof(struct zpdesc, zp)) + +ZPDESC_MATCH(flags, flags); +ZPDESC_MATCH(lru, lru); +ZPDESC_MATCH(mapping, mops); +ZPDESC_MATCH(index, next); +ZPDESC_MATCH(index, handle); +ZPDESC_MATCH(private, zspage); +ZPDESC_MATCH(page_type, first_obj_offset); +ZPDESC_MATCH(_refcount, _refcount); +#ifdef CONFIG_MEMCG +ZPDESC_MATCH(memcg_data, memcg_data); +#endif +#undef ZPDESC_MATCH +static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); + +#define zpdesc_page(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct page *)(zp), \ + struct zpdesc *: (struct page *)(zp))) + +/* Using folio conversion to skip compound_head checking */ +#define zpdesc_folio(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct folio *)(zp), \ + struct zpdesc *: (struct folio *)(zp))) + +#define page_zpdesc(p) (_Generic((p), \ + const struct page *: (const struct zpdesc *)(p), \ + struct page *: (struct zpdesc *)(p))) + +#endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5d6581ab7c07..30f0a7abbda3 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -13,20 +13,18 @@ /* * Following is how we use various fields and flags of underlying - * struct page(s) to form a zspage. + * struct zpdesc(page) to form a zspage. * - * Usage of struct page fields: - * page->private: points to zspage - * page->index: links together all component pages of a zspage + * Usage of struct zpdesc fields: + * zpdesc->zspage: points to zspage + * zpdesc->next: links together all component pages of a zspage * For the huge page, this is always 0, so we use this field * to store handle. - * page->page_type: PG_zsmalloc, lower 16 bit locate the first object - * offset in a subpage of a zspage + * zpdesc->first_obj_offset: PG_zsmalloc, lower 16 bit locate the first + * object offset in a subpage of a zspage * - * Usage of struct page flags: + * Usage of struct zpdesc(page) flags: * PG_private: identifies the first component page - * PG_owner_priv_1: identifies the huge component page - * */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt @@ -64,6 +62,7 @@ #include #include #include +#include "zpdesc.h" #define ZSPAGE_MAGIC 0x58 @@ -253,7 +252,7 @@ struct zspage { }; unsigned int inuse; unsigned int freeobj; - struct page *first_page; + struct zpdesc *first_zpdesc; struct list_head list; /* fullness list */ struct zs_pool *pool; rwlock_t lock; @@ -448,7 +447,7 @@ static inline void mod_zspage_inuse(struct zspage *zspage, int val) static inline struct page *get_first_page(struct zspage *zspage) { - struct page *first_page = zspage->first_page; + struct page *first_page = zpdesc_page(zspage->first_zpdesc); VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); return first_page; @@ -948,7 +947,7 @@ static void create_page_chain(struct size_class *class, struct zspage *zspage, set_page_private(page, (unsigned long)zspage); page->index = 0; if (i == 0) { - zspage->first_page = page; + zspage->first_zpdesc = page_zpdesc(page); SetPagePrivate(page); if (unlikely(class->objs_per_zspage == 1 && class->pages_per_zspage == 1)) @@ -1324,7 +1323,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, link->handle = handle | OBJ_ALLOCATED_TAG; else /* record handle to page->index */ - zspage->first_page->index = handle | OBJ_ALLOCATED_TAG; + zspage->first_zpdesc->handle = handle | OBJ_ALLOCATED_TAG; kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); From patchwork Tue Aug 13 08:45:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761552 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4DC5C531DD for ; Tue, 13 Aug 2024 08:41:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F2BA46B009E; Tue, 13 Aug 2024 04:41:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E8D826B009F; Tue, 13 Aug 2024 04:41:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C94CF6B00A0; Tue, 13 Aug 2024 04:41:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 9753D6B009E for ; Tue, 13 Aug 2024 04:41:21 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 463E5C06D5 for ; Tue, 13 Aug 2024 08:41:21 +0000 (UTC) X-FDA: 82446577962.30.388EFAC Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf01.hostedemail.com (Postfix) with ESMTP id DA4644001B for ; Tue, 13 Aug 2024 08:41:18 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="JjhJrf/w"; spf=pass (imf01.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538467; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YWv08D5bwyusSokvG2XzW2e1J+umZPeFR1do5jiHGv8=; b=2TV/hhYV76t/iC9545yFO2r0GxrvZxEbHn8M/XItSdlN5RqYT13GS9itQdtN+nnMN2niku MW91e0jIBwl+84uHN/IwgT2TzzB3eE9hlO6fAhkcP1x1xDYGmxedkrKekeyv8r5RwRH8jo WViZOF/SqNKAQ97wSR478KHHcKbFOFA= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="JjhJrf/w"; spf=pass (imf01.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538467; a=rsa-sha256; cv=none; b=0DLFHm1l/3wMTugf6lVLXoLfX3YdaB+WNSCicLjG6+rAqjzVB2CuX6UWtW9w2ZVksczvls BfDSYpznGoGFnKefJu/vu6rPGL2MpO3i3QKDc4a47ipbBeO9QzD3sePZ5OGomOqpXYoRR5 eR5ytv2jok9LeFzAYnc/8S/6kT/Akmo= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 11BC0CE122F; Tue, 13 Aug 2024 08:41:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D2888C4AF10; Tue, 13 Aug 2024 08:41:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538474; bh=PQtx1dFIRykqdoAci1ap2Dlk7DGGX895wpZmxxTa/no=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JjhJrf/wtkBI9UEBsUcawJtJs8gmNmxHrh1Mo480fI0G6gJrj94bzUPkFtEItVqaS vvk0HHpWOyA7tNNTEQi1OU+L8oYA8et3hJzE0UaFiy78T/w0wveVetYoCybv1xZK62 NzuSeIFYgYQ6JtJxEw3MRxT41R0yHnWLIUbY2OTTnF8YYcp2OAEnS6xAlUbToI8mEI ot2el35HS7YnXfjX0Jak615XPEbamwZncUhoSUIXXPWDvOhKVojS7FDYeyhsjH78nq WuhR1Ueh4Scg40xQw0c26/I0EtwDGn1eQe2qGoIibIksi2gBQ+R+qISZyjaFJIxj1N bcX3wurSiw0Pg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 02/21] mm/zsmalloc: use zpdesc in trylock_zspage()/lock_zspage() Date: Tue, 13 Aug 2024 16:45:48 +0800 Message-ID: <20240813084611.4122571-3-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: a16y5whrfx9tnm4nh7bg8tgaomax9k3x X-Rspamd-Queue-Id: DA4644001B X-Rspamd-Server: rspam11 X-HE-Tag: 1723538478-830316 X-HE-Meta: U2FsdGVkX18xkZlWwgTxwn7w/sMnYou58dpYqiz3EEbGBaqX71LYzX5tLLkaDNPLMtnjPTAlqlp+GT5wxl1z7MxJrQPGN3bKDUR2DdADQCsdSUr9ik7zPJBdPTtH+gnqmnYNImz7X4M14fzKCD2R21MwY+kq8gnw4+45dWHIcasEQwGVyYfWExHjSRfL3WsWcMytlbXPQWgHv51TIg1sm0G8D8KsXn/hYjYkE40xqirT42HlK9E9mYIKOlNIneB2iSpWswAkfkrKysYsdfYDvP9EAcXm6MaYX7dQC7LcqK9QYvMolBae+b62NSTm77LnkHx2I/BYuvjzPQH1OlhcUPzPiiDqkco0+aquYkm3JrwAvZCR4XOoGGe2qIsQ7gWjbew1hbWEL/LB1UXhQwWLmpWNvf4n8WOQW7JB8hcX90CZuyBwuPP/CFBV1kF50MzMpDSaZR/m3RKBXh8y1Pn1reWH1CPLhdhIvC4BeVEavf6pkuI1aV2ILBD0Cj6hs+7FSh30El9SqAjuh83tqink4shYmY9FhuZbAxdae370Qa9NZ9XBeA9w8OleHd3b3mVoxkaPovd06naO1ba4uSfdBeJedShhStWqgQsL79n7ib1xNm4++SdVruoq5jiy19r06EOl1uLocv5AxEss3oAfuyTSSkbDq3TSXVD9nQafhrVHlbEEF7d9mLiTM4sjJ/yRTQaN6peWIgL1Zk3dxI18ZAEpNO/iCwMYytVyDavuCdeUI8vsTn8MNT3WEjI0BaoRsbbFZ0ZRFwXnOD3eb1DNIcQRBxhtEsLz7B8oyrHU4n5rvatbsiRoSQzZH3o3vW6IjRPpuuoGYu62STIpVTvT6VpeUK4OvWnMK3qIHJEhEsmxu0qlmhwiwEm0lzErRd3BHsOMWQfMq32kE/dqufaYnYkTL+Fy3XZsULqFn71r9UpMf6iHvgFxsbnQhLQh7mMORpq8dQ4foOlgOgXGVdL bE+B1DHE +C6tSVYEf70BEMOLiVwNlt7gGP5uWGjQTRyIOoPXjSxIEC0V2LrtsuXTuY1ieT0SugVbxXPkZAXIcgf6i7BXCYTFJtoBsAbqExdRzfFJM9PTJWpj/E4VCkUbcyj0NRxWbIo4LLbGv4szmfTy40v6xzbTLkndS5L/lGSLbBV6D1B3K1dTAwuFw8p8I8k08evFQQe3KFvkaBFw2EiupXHZcqQP6XckBhLrALr9bAfnSgvZiL9HAcMIAgW4qqbNoU0Yye5Su5PoLfVCZtNrtIGomDQXHmh+7VoVrEsTUU+UMN/Cc+Iyd30vrwkIi14oUwdifH3VfrpQtwv7PDAFo6biNZl5VYVl8kV6VgI2La7G9XMBUbpfIxo35r/9Cng== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi To use zpdesc in trylock_zspage()/lock_zspage() funcs, we add couple of helpers: zpdesc_lock()/zpdesc_unlock()/zpdesc_trylock()/zpdesc_wait_locked() and zpdesc_get()/zpdesc_put() for this purpose. Here we use the folio series func in guts for 2 reasons, one zswap.zpool only get single page, and use folio could save some compound_head checking; two, folio_put could bypass devmap checking that we don't need. BTW, thanks Intel LKP found a build warning on the patch. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 30 ++++++++++++++++++++++++ mm/zsmalloc.c | 64 ++++++++++++++++++++++++++++++++++----------------- 2 files changed, 73 insertions(+), 21 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 721ef8861131..782b5ad67cda 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -69,4 +69,34 @@ static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); const struct page *: (const struct zpdesc *)(p), \ struct page *: (struct zpdesc *)(p))) +static inline void zpdesc_lock(struct zpdesc *zpdesc) +{ + folio_lock(zpdesc_folio(zpdesc)); +} + +static inline bool zpdesc_trylock(struct zpdesc *zpdesc) +{ + return folio_trylock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_unlock(struct zpdesc *zpdesc) +{ + folio_unlock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_wait_locked(struct zpdesc *zpdesc) +{ + folio_wait_locked(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_get(struct zpdesc *zpdesc) +{ + folio_get(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_put(struct zpdesc *zpdesc) +{ + folio_put(zpdesc_folio(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 30f0a7abbda3..25c90224f21f 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -433,13 +433,17 @@ static __maybe_unused int is_first_page(struct page *page) return PagePrivate(page); } +static inline bool is_first_zpdesc(struct zpdesc *zpdesc) +{ + return PagePrivate(zpdesc_page(zpdesc)); +} + /* Protected by class->lock */ static inline int get_zspage_inuse(struct zspage *zspage) { return zspage->inuse; } - static inline void mod_zspage_inuse(struct zspage *zspage, int val) { zspage->inuse += val; @@ -453,6 +457,14 @@ static inline struct page *get_first_page(struct zspage *zspage) return first_page; } +static struct zpdesc *get_first_zpdesc(struct zspage *zspage) +{ + struct zpdesc *first_zpdesc = zspage->first_zpdesc; + + VM_BUG_ON_PAGE(!is_first_zpdesc(first_zpdesc), zpdesc_page(first_zpdesc)); + return first_zpdesc; +} + #define FIRST_OBJ_PAGE_TYPE_MASK 0xffff static inline void reset_first_obj_offset(struct page *page) @@ -745,6 +757,16 @@ static struct page *get_next_page(struct page *page) return (struct page *)page->index; } +static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) +{ + struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + + if (unlikely(ZsHugePage(zspage))) + return NULL; + + return zpdesc->next; +} + /** * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value @@ -815,11 +837,11 @@ static void reset_page(struct page *page) static int trylock_zspage(struct zspage *zspage) { - struct page *cursor, *fail; + struct zpdesc *cursor, *fail; - for (cursor = get_first_page(zspage); cursor != NULL; cursor = - get_next_page(cursor)) { - if (!trylock_page(cursor)) { + for (cursor = get_first_zpdesc(zspage); cursor != NULL; cursor = + get_next_zpdesc(cursor)) { + if (!zpdesc_trylock(cursor)) { fail = cursor; goto unlock; } @@ -827,9 +849,9 @@ static int trylock_zspage(struct zspage *zspage) return 1; unlock: - for (cursor = get_first_page(zspage); cursor != fail; cursor = - get_next_page(cursor)) - unlock_page(cursor); + for (cursor = get_first_zpdesc(zspage); cursor != fail; cursor = + get_next_zpdesc(cursor)) + zpdesc_unlock(cursor); return 0; } @@ -1658,7 +1680,7 @@ static int putback_zspage(struct size_class *class, struct zspage *zspage) */ static void lock_zspage(struct zspage *zspage) { - struct page *curr_page, *page; + struct zpdesc *curr_zpdesc, *zpdesc; /* * Pages we haven't locked yet can be migrated off the list while we're @@ -1670,24 +1692,24 @@ static void lock_zspage(struct zspage *zspage) */ while (1) { migrate_read_lock(zspage); - page = get_first_page(zspage); - if (trylock_page(page)) + zpdesc = get_first_zpdesc(zspage); + if (zpdesc_trylock(zpdesc)) break; - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); } - curr_page = page; - while ((page = get_next_page(curr_page))) { - if (trylock_page(page)) { - curr_page = page; + curr_zpdesc = zpdesc; + while ((zpdesc = get_next_zpdesc(curr_zpdesc))) { + if (zpdesc_trylock(zpdesc)) { + curr_zpdesc = zpdesc; } else { - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); migrate_read_lock(zspage); } } From patchwork Tue Aug 13 08:45:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 867B2C52D7D for ; Tue, 13 Aug 2024 08:41:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 20C306B009A; Tue, 13 Aug 2024 04:41:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 195D66B009E; Tue, 13 Aug 2024 04:41:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0366B6B009F; Tue, 13 Aug 2024 04:41:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D98696B009A for ; Tue, 13 Aug 2024 04:41:20 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 9FD2D1606D8 for ; Tue, 13 Aug 2024 08:41:20 +0000 (UTC) X-FDA: 82446577920.02.CE2E479 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf27.hostedemail.com (Postfix) with ESMTP id 0451840007 for ; Tue, 13 Aug 2024 08:41:18 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="jJgXG5/M"; spf=pass (imf27.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538443; a=rsa-sha256; cv=none; b=n3dZhnYc587KuYYxyFmINqHeCPSE+v72+hYbns2PeXzGiRlf0yoNrb6o8rOMjhO5fkrbwZ dBbLqM4VFElidkP+WL5Doa1TW2it2fKwJsqJ14986+pUsQdT5dB5HUHh/cjUD8FVrfO95q 14qjlTXTs1f7jEBRovX3sFvgJpKkIAU= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="jJgXG5/M"; spf=pass (imf27.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538443; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cAi81bWlDHMmhHwcU64lYTDVUKa+q1KUucaRaXn58z4=; b=PlMgfe7jazp9nNTnbuEbrbs1QdDLDBcspFnkagOx4siTjXdfzrPaHm02WfhYCvO2+x0WWF 8tSyNMA8Hp8FpwU+xpEPPNGlJur51EdVnN2stwuYAwFNKOx625fi5zIth/4t20MqgmQeCu CWyh/LDoni5X/ge4VjTQjV76GzECTR8= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 38FEA615B6; Tue, 13 Aug 2024 08:41:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D74FCC4AF0B; Tue, 13 Aug 2024 08:41:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538478; bh=rMm13SX4aLf3oZ/lBmcjJeeV3CGO+2RCAQN5DP+xGmo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jJgXG5/MU5u93XZsl+G9CJ0mZ5Zsa5hW1D87/ECOlkqHb+Sr7/6G9JpU9JSleiaMM jeO2UHtuI8wPWRVqLVhxb3dyjcW8uiCm+Wre+EoFb8xV3wGDYcMFKa9RWetwLmAfpW 7wGshauiHkGZQEFx7I7wSWteKnO22jVG7YHziDbs66qRtrx9x/JGYl0r8S9g0dUrhw pe6jnDZAoZToI0zoFrugdc2rA141iOVEivfjDwK26VKKJn39xFkNkKoNQ5vYFHKL5I V1bo7H8J9Swu0EPIh8U+xtQC+GvyGGiBC1koj5PQ+iVCqihCzllNAuur2U0FggB05u phlueemlWNrtw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 03/21] mm/zsmalloc: convert __zs_map_object/__zs_unmap_object to use zpdesc Date: Tue, 13 Aug 2024 16:45:49 +0800 Message-ID: <20240813084611.4122571-4-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: hqnjsjmxkjoctqnsrt4hkobt3bzmd6u3 X-Rspamd-Queue-Id: 0451840007 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1723538478-825860 X-HE-Meta: U2FsdGVkX1+BF1kevXMGaO2ywGHxNEWmnvH+lfos7t9bgPm2IAtdAorOs5J7zG0bq5N8gjIc8R13dPH/O2/+MDuLQdpfy32tbPUgOAL4bDmuNz01/CBGA7eP1gO0Qoj5XIFUrl81Sl6IgozbIFiCfKchJGFDhMZJfapmqELN4rfreQlHjd/BRP56ddB6R+m5jQoXmMoQMJu0G8T7SEebF+zSonCoZOz/tBw8Y9xgoYvRe7t0UZNOlpYtmksMYC2XU11trfQRgfC3a7nv+noIGvvdJp5zjcIb7MQw8hN18n6W69S+Zhdc5mdcdBBfSJNf0PWw5YaMug815Dh3Nt8V5+aMVWAVEeusqYagGdTzll7vIFyACXyhs1m2Kn14zEccBoGYcVIVselI3b1VBItzXvtiCRFuDr0olvGTw/RDdeR36HJrGQ/sMMhKQSezvgnvPLYm8h4bm9410QfhcvvuJSa0ZLecIsJrkT18d5wKYtZz10Fi9tFKQQiyI/wZ1/L1IIhpkUZucvfVYlRP2Vr9jjNNTWLwfthl797saEToTq1+rn774F1+6h685oBvuj+kdRk5KS7wv0YPgL2B+wGw0P3VAA5J+euwVzc1uNIvqcCiWrz+vCK2KPtXha62w6uFJmjD7MY7F+HfzhZPmR1zwInwQIx1RJ2evjS0QXFBRrJpYvtphY0o7ASGbXrx95DbItCmpYKxS8WF54U1PBUYq+wgrWBG1mtxz1oOWF46BHXfM7prgsrqTpSyUKsZUccZQ1w7dyf9NhxrmGR6/xTlO4skqikMu9KQXsYAPHUsiNqUFeNO297Fw0mHlDvEfrFE86Unuu/53LHG5NbPi2P8jYfJWbE2rF8/yQ2FJfkhFzSI4DD4XYg8sub26EKab9ZV5tuMcChR+3YNdL/uR3GFWMO0c+Hfg80cy5Ow4LPOZm04sEsy9poOsK7uu6GhaPuPkJGBSNQ8Xhea050UZKY X3zmhAP7 46wMWnRCBczjU7PXYBqOsTprBBUu+t4WAyeghw1YMtAUE8WWwrj0iftbZqCXbSqF0eOteO4cK2gpMAmP0jewHzjKAU6VRghjwR7K4DEvcCCUsDQos2TxQQI9GoAm/LEW1VI5YGX5TJlg0PFKgfyO5BzPGy5ZM/2qUPkdLHczxUaH77oFDifjrWW8Ds7DUy2xvR5nXJtb2nTnTWpwe2WdC5QDCFDRQGkCflQKDB05Y3wfJ1VAtObVbOLFjuquyJqk27PspbwkuX5uInRp5iqGLPrichEzED0s8hhtCJX5MWydCie+t4OrbzbeVdl+8YnJjk9NLVrfiB4C6/UzMFw+COdpxcJvEYHDKnwHqeA54HkRXbv7VXWIuo3CuzsLpPRxrHW1K X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> These two functions take pointer to an array of struct page. Introduce zpdesc_kmap_atomic() and make __zs_{map,unmap}_object() take pointer to an array of zpdesc instead of page. Add silly type casting when calling them. Casting will be removed late. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 25c90224f21f..b9b5e2824f2c 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -243,6 +243,11 @@ struct zs_pool { atomic_t compaction_in_progress; }; +static inline void *zpdesc_kmap_atomic(struct zpdesc *zpdesc) +{ + return kmap_atomic(zpdesc_page(zpdesc)); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -1061,7 +1066,7 @@ static inline void __zs_cpu_down(struct mapping_area *area) } static void *__zs_map_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1078,10 +1083,10 @@ static void *__zs_map_object(struct mapping_area *area, sizes[1] = size - sizes[0]; /* copy object to per-cpu buffer */ - addr = kmap_atomic(pages[0]); + addr = zpdesc_kmap_atomic(zpdescs[0]); memcpy(buf, addr + off, sizes[0]); kunmap_atomic(addr); - addr = kmap_atomic(pages[1]); + addr = zpdesc_kmap_atomic(zpdescs[1]); memcpy(buf + sizes[0], addr, sizes[1]); kunmap_atomic(addr); out: @@ -1089,7 +1094,7 @@ static void *__zs_map_object(struct mapping_area *area, } static void __zs_unmap_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1108,10 +1113,10 @@ static void __zs_unmap_object(struct mapping_area *area, sizes[1] = size - sizes[0]; /* copy per-cpu buffer to object */ - addr = kmap_atomic(pages[0]); + addr = zpdesc_kmap_atomic(zpdescs[0]); memcpy(addr + off, buf, sizes[0]); kunmap_atomic(addr); - addr = kmap_atomic(pages[1]); + addr = zpdesc_kmap_atomic(zpdescs[1]); memcpy(addr, buf + sizes[0], sizes[1]); kunmap_atomic(addr); @@ -1252,7 +1257,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, pages[1] = get_next_page(page); BUG_ON(!pages[1]); - ret = __zs_map_object(area, pages, off, class->size); + ret = __zs_map_object(area, (struct zpdesc **)pages, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret += ZS_HANDLE_SIZE; @@ -1287,7 +1292,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) pages[1] = get_next_page(page); BUG_ON(!pages[1]); - __zs_unmap_object(area, pages, off, class->size); + __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); } local_unlock(&zs_map_area.lock); From patchwork Tue Aug 13 08:45:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C368C52D7B for ; Tue, 13 Aug 2024 08:41:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A4AC36B0085; Tue, 13 Aug 2024 04:41:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9FFA46B00A0; Tue, 13 Aug 2024 04:41:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 84DD26B00A1; Tue, 13 Aug 2024 04:41:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 64A186B0085 for ; Tue, 13 Aug 2024 04:41:25 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id DB76BA749B for ; Tue, 13 Aug 2024 08:41:24 +0000 (UTC) X-FDA: 82446578088.23.6CB6986 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf10.hostedemail.com (Postfix) with ESMTP id 39B96C0006 for ; Tue, 13 Aug 2024 08:41:23 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=ATf69V+r; spf=pass (imf10.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538448; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WbDWaoconi0g0luVfxDcAgmb3q9wgcC4jngmvATF3bs=; b=PJ0DafiuwPnYJXss09uFtsaIPNf47E8fBmSwfsp5CxzQjnfc6FgTnjJZqgulqZCzD3rfs1 rgDS6Jbw924vA+zm0wm1tdNPMVdbGHGzL/9mjohz8hSSuOKYSYSZGcItjJDqm1qsVqLfLA EQ4steiNxEuKsG9uIPDPGSqktNH+mAc= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=ATf69V+r; spf=pass (imf10.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538448; a=rsa-sha256; cv=none; b=6fxq6P5/wzLr9IiYljF7rmic9fTlrjJ+0Kaoo699Zhbw2/9ZZCjee2dLjhcLf1x+2jEE4D FmGBuG5E/tt2Tb3C09GZuGa5kcbRknT+pdid3dcHVdK90/7KAQiTdmzZjEhVKpGTaHNsQY P02NnWoNmkqYjccqcJrwGezFS1zZCzw= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 5C384615A4; Tue, 13 Aug 2024 08:41:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B392EC4AF09; Tue, 13 Aug 2024 08:41:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538482; bh=3JFITvoM2vdTLh0S1hX/hhJAVedfseKmbRV8MWLijFE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ATf69V+rkNHvcKEhZ4OMbfXyaIRrEgDaD3lPd3b2uhkn1uq3GX1I8PMUccr/8Po8M 0iZBkgTgW9zR333lSv5o+jCu5k9uX8zsChDh8AduidPjulDTLd5m1xAcPaXGCTdyYy TVJugJwFoO3juUkYtS5OGZk7tSTQAXO5h2QTpKSbtz9cA5E0Yq5+medYMzAwjWjUnD HkJybZOkjGucGmSQG3ReLrhCS2/qUELtVCIZz6IvrblljdqCeMtbplf3PZ7gCIG+Wj 7Q9oepphR0IgGUEh5B7utLIJbHJp1jLlBvNMCmg9G1nLHZ1O3/2Hy8E7V51l2FQi2P 834YXy5qSA1hA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 04/21] mm/zsmalloc: add and use pfn/zpdesc seeking funcs Date: Tue, 13 Aug 2024 16:45:50 +0800 Message-ID: <20240813084611.4122571-5-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 39B96C0006 X-Stat-Signature: t95mc91k4q1or38uapecwftz7ee4brn5 X-HE-Tag: 1723538483-66970 X-HE-Meta: U2FsdGVkX1+HOBj5Z9Q7Zv5WUive0gOOyL+hxdZqBhIo19x2KjNsBEF+NHxU95Hb/B0teO6fculeDD4wogIopgg0CpiiJmisD+d7nN4PWrmqPHKjCPjniiQhQVPsB1VqI8gwg5oU0f1gA/8524zGIW4BKlUwng8ar/DK9I/8lCdGL10SwKB/9SeQCqSM3jMAsRoeP77YrQ/neZfFlAAOofzQfeGvviy1Ak8e1OvvD/f2822luyeUI5faxNOeGupRRf3tI1GCWHleMD7ZMIL8XJNm/Y+S0CCEr5f8WSibBTmpsCPmpEWiYxFKtR+RomEb2eNJg3MT/hgKVSLhWNRdOlmso6xeKatcI9nbh31m9m5kUECXMM2au8bp7PF4v0aLW5ZNH6D7tV6ppS/3j3y6HsIJLZp2NG5EX4C6uyjPi9CHo/SjcE0a2hkapjz6kBhi0t/1AZ4TOIhv0v6B0NH8gwy2a01L0KG13uLzq3DUmc46z8SdqdOKTzeaUTiRW1Jj8fY8FLSNfLBbJCdXp1a6on37v16G+BgSuUzPhv+L5lb8zTuIWbXj0XDTDjG8FoeDkMy8BnDL6d2fV/w4yXAnvL7qf87Mn2hqvyQBc337T5g4gx6rbdfKnF18IxNizqAHV3OMA5ntPWwwa3qHyNNtQO3Cb2QRwmki8Mtc9THPmEmY67yTDlh67OEnRZR+zdu+u8jTXjCxRHJl/1eS5cmnYMgIelZcDG0QPED/IDGw8yx5snasIYbEMk970QFrrDiph1XhWel+VPif+WU/EJn1BgZ7UXaHcNVxsy87GfrBRvD/b9sYV0csgv7RE1t2eNAwYLWXMRu2zhhBv3E7XhwDZ+PTkJHtUjWdU6vUOpxuwMIkvuH1LMAgDgtmmCJvDCacKhVQXyYsLig5I6WuQdmi7iks5DggW661UMwq/hsusdQJlkXDp2DqFymgLJyH97NhuORCFpHF8tnFtnTTHO4 lqP7EcqU VcSSZ7d0sXWi4TT/Xz5kVLHoMag0LEeq4KjFBgYULqxBaWRy/W7f23lAutsk0JYge+MNWjnZ63ImTKTLFbnBJYsVEFAzvBGfTpE9EJMEizKlrjJBppbkOvPZHkZ2rkugiG6ZGCA2uP+Mln5beTxO575rOAHcY/maBtay4rw+c/aeWp57m7yqsfJUcPpJaDl5tGZ3Z2VKemKFopIuELoGXjZejcfAx0lk9pD/jeKby/Lk3TDiYpSHLrAUMO7j+d2EVMBVJO0YDPYl71UOn6sYUX7TAic38062jhvqce7C+ngUTzDbifmQXI/5zsrzjjoLnUGWWayVExUuTZe4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Add pfn_zpdesc conversion, convert obj_to_location() to take zpdesc and also convert its users to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 9 +++++++ mm/zsmalloc.c | 75 ++++++++++++++++++++++++++------------------------- 2 files changed, 47 insertions(+), 37 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 782b5ad67cda..11083a1c2464 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -99,4 +99,13 @@ static inline void zpdesc_put(struct zpdesc *zpdesc) folio_put(zpdesc_folio(zpdesc)); } +static inline unsigned long zpdesc_pfn(struct zpdesc *zpdesc) +{ + return page_to_pfn(zpdesc_page(zpdesc)); +} + +static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) +{ + return page_zpdesc(pfn_to_page(pfn)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index b9b5e2824f2c..384a5ba49788 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -773,15 +773,15 @@ static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) } /** - * obj_to_location - get (, ) from encoded object value + * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value - * @page: page object resides in zspage + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static void obj_to_location(unsigned long obj, struct page **page, +static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, unsigned int *obj_idx) { - *page = pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc = pfn_zpdesc(obj >> OBJ_INDEX_BITS); *obj_idx = (obj & OBJ_INDEX_MASK); } @@ -1208,13 +1208,13 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, enum zs_mapmode mm) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; struct size_class *class; struct mapping_area *area; - struct page *pages[2]; + struct zpdesc *zpdescs[2]; void *ret; /* @@ -1227,8 +1227,8 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, /* It guarantees it can get zspage from handle safely */ read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage = get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc_page(zpdesc)); /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1247,17 +1247,17 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, area->vm_mm = mm; if (off + class->size <= PAGE_SIZE) { /* this object is contained entirely within a page */ - area->vm_addr = kmap_atomic(page); + area->vm_addr = zpdesc_kmap_atomic(zpdesc); ret = area->vm_addr + off; goto out; } /* this object spans two pages */ - pages[0] = page; - pages[1] = get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] = zpdesc; + zpdescs[1] = get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); - ret = __zs_map_object(area, (struct zpdesc **)pages, off, class->size); + ret = __zs_map_object(area, zpdescs, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret += ZS_HANDLE_SIZE; @@ -1269,7 +1269,7 @@ EXPORT_SYMBOL_GPL(zs_map_object); void zs_unmap_object(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; @@ -1277,8 +1277,8 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) struct mapping_area *area; obj = handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage = get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc_page(zpdesc)); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1286,13 +1286,13 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) if (off + class->size <= PAGE_SIZE) kunmap_atomic(area->vm_addr); else { - struct page *pages[2]; + struct zpdesc *zpdescs[2]; - pages[0] = page; - pages[1] = get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] = zpdesc; + zpdescs[1] = get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); - __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); + __zs_unmap_object(area, zpdescs, off, class->size); } local_unlock(&zs_map_area.lock); @@ -1434,23 +1434,24 @@ static void obj_free(int class_size, unsigned long obj) { struct link_free *link; struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long f_offset; unsigned int f_objidx; void *vaddr; - obj_to_location(obj, &f_page, &f_objidx); + + obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); - zspage = get_zspage(f_page); + zspage = get_zspage(zpdesc_page(f_zpdesc)); - vaddr = kmap_atomic(f_page); + vaddr = zpdesc_kmap_atomic(f_zpdesc); link = (struct link_free *)(vaddr + f_offset); /* Insert this object in containing zspage's freelist */ if (likely(!ZsHugePage(zspage))) link->next = get_freeobj(zspage) << OBJ_TAG_BITS; else - f_page->index = 0; + f_zpdesc->next = NULL; set_freeobj(zspage, f_objidx); kunmap_atomic(vaddr); @@ -1495,7 +1496,7 @@ EXPORT_SYMBOL_GPL(zs_free); static void zs_object_copy(struct size_class *class, unsigned long dst, unsigned long src) { - struct page *s_page, *d_page; + struct zpdesc *s_zpdesc, *d_zpdesc; unsigned int s_objidx, d_objidx; unsigned long s_off, d_off; void *s_addr, *d_addr; @@ -1504,8 +1505,8 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, s_size = d_size = class->size; - obj_to_location(src, &s_page, &s_objidx); - obj_to_location(dst, &d_page, &d_objidx); + obj_to_location(src, &s_zpdesc, &s_objidx); + obj_to_location(dst, &d_zpdesc, &d_objidx); s_off = offset_in_page(class->size * s_objidx); d_off = offset_in_page(class->size * d_objidx); @@ -1516,8 +1517,8 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, if (d_off + class->size > PAGE_SIZE) d_size = PAGE_SIZE - d_off; - s_addr = kmap_atomic(s_page); - d_addr = kmap_atomic(d_page); + s_addr = zpdesc_kmap_atomic(s_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); while (1) { size = min(s_size, d_size); @@ -1542,17 +1543,17 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, if (s_off >= PAGE_SIZE) { kunmap_atomic(d_addr); kunmap_atomic(s_addr); - s_page = get_next_page(s_page); - s_addr = kmap_atomic(s_page); - d_addr = kmap_atomic(d_page); + s_zpdesc = get_next_zpdesc(s_zpdesc); + s_addr = zpdesc_kmap_atomic(s_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); s_size = class->size - written; s_off = 0; } if (d_off >= PAGE_SIZE) { kunmap_atomic(d_addr); - d_page = get_next_page(d_page); - d_addr = kmap_atomic(d_page); + d_zpdesc = get_next_zpdesc(d_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); d_size = class->size - written; d_off = 0; } @@ -1791,7 +1792,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct zs_pool *pool; struct size_class *class; struct zspage *zspage; - struct page *dummy; + struct zpdesc *dummy; void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; From patchwork Tue Aug 13 08:45:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761554 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E14C2C52D7C for ; Tue, 13 Aug 2024 08:41:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 655226B00A2; Tue, 13 Aug 2024 04:41:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5DCB56B00A3; Tue, 13 Aug 2024 04:41:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 40C576B00A4; Tue, 13 Aug 2024 04:41:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1FE006B00A2 for ; Tue, 13 Aug 2024 04:41:29 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A61A8A621D for ; Tue, 13 Aug 2024 08:41:28 +0000 (UTC) X-FDA: 82446578256.19.88777A7 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf03.hostedemail.com (Postfix) with ESMTP id 137CB20021 for ; Tue, 13 Aug 2024 08:41:26 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="tRCXz/JP"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf03.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538435; a=rsa-sha256; cv=none; b=jUHuSAfw9+NCoxJcPA11Lw0vgj4OYRKJZuT9Q4YVSV/deEWWxRQ7AvVGYNVlxMeGf1BhvJ JWQ/db81JFJRHcbOBPPz1OVKtR/Gz0CP5lrQnWe/S5NaCZyeExn88/Rj9D0AZXEEjkPWBy 1o7+ECM7QqmOBP6PQ2QnDl/+zzjpbA4= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="tRCXz/JP"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf03.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538435; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Yh2M3ze6FPP2pb5gtVIfrEf+1wX6q2uwXp/9ylWsfqo=; b=ixRzd/anIvf+gtIaJxhsCtgzERdkuvWXfO0shrHh0fzlSRFPcOVn/9sDEDQkG4ltuYRUG0 /5nLVpR25109+IRHczrvPYBQl4uCgY4EyeKBzoUsv4k+U8T/XxwigpIJNZWcXYMUtxuD14 8ngCqtS7ggOhOuND7mRxGaTyjevbJiQ= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 437F1615B9; Tue, 13 Aug 2024 08:41:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BCC2AC4AF09; Tue, 13 Aug 2024 08:41:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538486; bh=j9RlWiCV98maWH46GH8LQqP9KXOBrIEnE2eMblguIA0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tRCXz/JP3Ty7JtZpW73rKxJ1lhhYk9Ol1Cmy9UFewuVw/ovD7dnyJn3Fbc0FM9JyH mbyqFfGvvltrcRIkhc4Kqz8hdrpnbnVVrP2SWs7vG5U6rft8AnC4LlOKPQGi5ClCVH 3Mn+kEiwvynvpeD3qsCqqNS3lbmOLEQg0BQgJID9O7ZBmqUXu/9Ptl/7Ocu0ngA7bQ AxxymPJxLhXJdFG2aFdsZZLVsIwDpMKZsq9uEsIf4ZtIkaZnzsGRiYyyQzOnk/d8WL eoFkQ512SDtJmPOshzJRwq8BO4kzCBEv+6GJFUI/d80irjxb7cVWmkqi+WJHy3Gh2y VvTRBPTeO0Y5Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 05/21] mm/zsmalloc: convert obj_malloc() to use zpdesc Date: Tue, 13 Aug 2024 16:45:51 +0800 Message-ID: <20240813084611.4122571-6-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 137CB20021 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: giue37quxfnc5y13acotb89yahhudm4t X-HE-Tag: 1723538486-975729 X-HE-Meta: U2FsdGVkX188mOgpVBh/ViQ8hMn8qK/WHUlk7ccB0U011mV2EjXo+/mzK72T+sabHyCbQxSUylKQxJSYHo8V9K85smdEXtGqMqJAApZj3G0peTSJZEFUwj1Iz0D3GgGzS5J0jXBodDO5qOMhzxbH1S2Mj1PFiYXfEfmGTyjlUiSKHwZ6iRKQjM1jDXhf3f6da1/TFtHpLsL8JdkcVVndKYZTMvIgUSQHcGwOKbdvyoym/Th0xDFm4KcsyqqrMszpiAv4hkAe/Qc+LNKqAZ4ttX1xYU9VVjrB9ahs4R1YJSVig9c+qeuQ71sZvaqyQVmxy47RzoTpR/G74pe4Wwt4x3ES2qJqOW+rzeDb2kwqBR+yAVYQyQoBtxWmGV7F+yByjZxc2/jcoVRgaR3XYryPjv8p5IxthZyHHHVZE+ImSNurTvnxq9w8eMN9TdSBXsphWo55IfVRfCub/md8puH00hcPBOlZh07aULom+pEMesmAE1tFerKFbdBJB0LhTXqxYTcYUnWbu6cJlhUOEQTGuatZdPQf5WyVTCAjbOvVh7ZtNHRYTLn6U/hBsXNadCtQl20tXm08BnDE8fiXXLRiFctgF3Wz44uZ2aPSV+EuLG9QhfMgq1cfBocywjetpEUD20uxH2DptbGHaCmM9sSW8pnsPKGu6u0Ju7EkwPbDwcDHT6bWQSiANJ9RIZIFNa/AyC0jD9sJ2lraGhm6biJuq6+Es8r/8x8QVqZ/OLahxZ/CWq3VJBtYP13aYSXuPNJPKh+TVxRTRpO9kgTrAbMeF5aACkZsPf1UxF2FQ/FOC/qs1CepItaC95T2YxqGkfJac2QMhiXWoBu7DH226FHhlpCLG7DzgiVQK9iLGsnpKrPYWBaTXYCcykZWN9Ht82CceL/EuH1hRmMPxUlgJjBaMCuWqauaOdlpj/5SbT3+qZ1Xl+heuc1zDsePk3UpWlvEe+lHYTBscqSTERF8Bx8 ze/m6iSM JDI3BnDXl8ErHV4LIC78ZSuNM42nhrebbss3D+f7OLOiEdSk9lVAPGIObE3iUkkz50eICICUASZdSG9xT08w4dHbCJtLAUt/ziYApfkQbqH1PpG6R6q2Ck1yTLM4mgRm61jFddq/h6DUbhGBBsxBNcW5pk7O/Br5gPISj+64+Z+RWWBXGRBYAhZffgUMkMGhMVw4c+rQOJoL07QxOtJRL00P0FqbQYD5ajP7GhWTI/7wHeVkcgRcgtcW/izLFz+uVFgMytZnMw0PLHV4aIiA8BnC5qpNHAHCGKzPcpBOH7MWWQkKuB/d7u0DG11V/7agWezyS44PzEjMvyqg1kX7LWtpIwinoygUUnsmchL1eR8VV/X6+vjHOL4c0bzIH/S/sPTVW X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first_page/get_next_page. no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 384a5ba49788..7421d7678880 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1322,12 +1322,12 @@ EXPORT_SYMBOL_GPL(zs_huge_class_size); static unsigned long obj_malloc(struct zs_pool *pool, struct zspage *zspage, unsigned long handle) { - int i, nr_page, offset; + int i, nr_zpdesc, offset; unsigned long obj; struct link_free *link; struct size_class *class; - struct page *m_page; + struct zpdesc *m_zpdesc; unsigned long m_offset; void *vaddr; @@ -1335,14 +1335,14 @@ static unsigned long obj_malloc(struct zs_pool *pool, obj = get_freeobj(zspage); offset = obj * class->size; - nr_page = offset >> PAGE_SHIFT; + nr_zpdesc = offset >> PAGE_SHIFT; m_offset = offset_in_page(offset); - m_page = get_first_page(zspage); + m_zpdesc = get_first_zpdesc(zspage); - for (i = 0; i < nr_page; i++) - m_page = get_next_page(m_page); + for (i = 0; i < nr_zpdesc; i++) + m_zpdesc = get_next_zpdesc(m_zpdesc); - vaddr = kmap_atomic(m_page); + vaddr = zpdesc_kmap_atomic(m_zpdesc); link = (struct link_free *)vaddr + m_offset / sizeof(*link); set_freeobj(zspage, link->next >> OBJ_TAG_BITS); if (likely(!ZsHugePage(zspage))) @@ -1355,7 +1355,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); - obj = location_to_obj(m_page, obj); + obj = location_to_obj(zpdesc_page(m_zpdesc), obj); record_obj(handle, obj); return obj; From patchwork Tue Aug 13 08:45:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43E7EC52D7C for ; Tue, 13 Aug 2024 08:41:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE4696B00A5; Tue, 13 Aug 2024 04:41:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C6B9D6B00A6; Tue, 13 Aug 2024 04:41:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE6526B00A7; Tue, 13 Aug 2024 04:41:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 8972B6B00A5 for ; Tue, 13 Aug 2024 04:41:33 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id A08C51C3110 for ; Tue, 13 Aug 2024 08:41:32 +0000 (UTC) X-FDA: 82446578424.05.421334B Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf11.hostedemail.com (Postfix) with ESMTP id E5F8F40023 for ; Tue, 13 Aug 2024 08:41:30 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=LRJGVXPM; spf=pass (imf11.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538420; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hUQv6/Qaa84Jd44DFKs+zhut+z945vtrMEIZ5kVxxCY=; b=giyo8j2p+/CU/HaaPwhYKPeenZZ98aJVT1rOwQ7woG/l/gmtwNpkkTy62bV7+8Y9TmaLtq dg+IRzc8mt98Kwq+Q5qjTOIdgHE4DCpJ6AXPgiCmXovcPbhOnsl4Pdoz2xhXeiplWDGfmc e4rtZ2EkD12/zxhQLoLuee1RILH1BFY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538420; a=rsa-sha256; cv=none; b=jCkn3chqtHecJdRuSTGIqUtEuNElFr79tL8JFmkAaO3UgIWa0P+zxElfezXGlqpogr87k/ T3Q7S33zCpWpVNtN8jqGSVr+hqJW905KC5kTtt67nbA8qocQ3YIuJJqw1U5Zfm+7sZjXc4 2I6iA4gL8H02PdB57BQ3OidUAKhxpBw= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=LRJGVXPM; spf=pass (imf11.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 306B7615B7; Tue, 13 Aug 2024 08:41:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A4125C4AF12; Tue, 13 Aug 2024 08:41:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538490; bh=It08fyjD3QWexpsDd5b4wIE8APHhPnH+kLBEpOpVbWw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LRJGVXPMQpMirOcEYCVJ2PjrlSaeIZMkPeyEPqRJc10lEEzZG1mk7YLd4+e8qKzBQ o+Ow6SJpIAgPPpCi2JYGfEien/wgKV9DzVRW0yBRlAjzhSfnKVh0AP5vkUP9EBi8Dv T2GbJPU+PD3E9Dogt/B2sRCEr4oJj9vL6ea4uj/yRIl+IcPUd40hS1VvLQ88r2ZGnO qRx7IntvucBBuag5ICIL0WOK+kUcWoswWW78NdodJ0NSaO+BzQVOzg8FjyThLefxm5 7PqoaEF3XnjvBO+M8yHMz1UiYC81gAqB8xZxjI4tSaVrJ0BV3dQj7leBEYGsEuIhBT rYsAKQBs+LbLA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 06/21] mm/zsmalloc: convert create_page_chain() and its users to use zpdesc Date: Tue, 13 Aug 2024 16:45:52 +0800 Message-ID: <20240813084611.4122571-7-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: s978r5eaz4su6bpd11djobhqnyb4k17n X-Rspamd-Queue-Id: E5F8F40023 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1723538490-361940 X-HE-Meta: U2FsdGVkX1+DR37ImX5ZEFH1B9wIfgSayCCHzbpQuk49zRjpu1lx0SlOFVxJ0LAApdUgO3Utvm66FnU/ncnI5XDLSG8QRzK9zf3RvnUDToNKrCcWg5NMjVyuPfoTNxaENA+dJCLbDNIA35ev1/BUNld6sxM//MUnCyPSqIympe666q48qUWUj25yBI/v4qYaj4kaWT8IQg6OCT3eWZ9SVFpD30R6X9lsfZ1UzEdJDLFR1nGOj2ltRbaU7kYqBdyaNlU1SGkt/8E+Z6GrSi6gWe0wxyy7zLioDR97t4AeMK6fSOAMzbqCUFdGzTxJUOFD2r8gSZI8gIjOK54QRa+pj/UzhdaLidChHsX2TADsMFGPnUmoeMBeNeo2kywAKvsS2aAMOEHxk7FelFcLXmYWF5WAVbENjZXn5O23IXMi8OOYErFVnlN2HRn8xCazxljC5vdurnwZGYPHmk3VsvIaEjNl7dt+kZS4sIC/jZLCMOH+R0bv/vrdWLMYEBHvFMzbIrmmLnhVh1in7zAvi9jgFKRYT6eUhwkBKDY/sepTChzvozhVkScb2LMFPRzr2fLoc3sjIjrDhXaxVi+l2Paz64V85QHqP7Fi5x8x/8rP4bw7FulZh0CgRWpeLXCYueCA2g7VSnY7vVWxE3cBp0kidFLU9pryCv2jlM3PYS24m3OwhhThFWzSwd6E549Xm+WIejZZ6RR5olekoIjl1iAq0tMGfs/HH12E76WvtRJLtSimbn6VJu5aeZyMWNlSM186NRsVYlmS3ZtzuqF2D5pIBeCTO1JMG3kw0sQKG0I+0DLbjUv6a5ckyxvJ3uQ6rQpTh9J+ojh/Al76UsTGfNEtqvkQEiUzGU7eqwkwRatCMWJ8DyrtU+bT7O/1df8l3sDlL1va3OxQNtsSXQU9BvqFTZX254MCCgDTChmHry+oRbAwQ0sfHRVhwh8CvtKliS3zQTu708o1vpjZVdCRjnP 4Cswrm7U sK/prLxB+cIbqsXtEMen/Z62E9QbLKnf7KKwvYWIz2PP43yQTCgCFhBh25C5u3uy7JJFmzUr01/dwR4xBtokyVIPIWAmjZkyGzYXy4OJJvi17uW+R3MmoP8obVHZ44NsuYvfD4gj3psfO8WiEl67NumRG7Isxw7cJjxzNW+IAipklA3m/YrV7KJMKPbGBbs72hTh310qKWTyPg1PoiHivKB5+B5Lzt6M77jymivJNDGFhB6JlNXz6vvOHavtliIcgeROZQl42jqd1OlkRuEwTwc3mMgARvO1K8SUQtHaBfPoQv6ZahjKoFODPqVD3RWjpvhmWmKa+ITR1uuJ97VaBeExomY2XUSYf49Nw6slHZIJiItSlrniNAzTQpdje/y6RuAfT X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Introduce a few helper functions for conversion to convert create_page_chain() to use zpdesc, then use zpdesc in replace_sub_page() too. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 6 +++ mm/zsmalloc.c | 109 ++++++++++++++++++++++++++++++++------------------ 2 files changed, 76 insertions(+), 39 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 11083a1c2464..3a65a7d494b7 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -108,4 +108,10 @@ static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) { return page_zpdesc(pfn_to_page(pfn)); } + +static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, + const struct movable_operations *mops) +{ + __SetPageMovable(zpdesc_page(zpdesc), mops); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7421d7678880..2d0f3d9011ac 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -248,6 +248,35 @@ static inline void *zpdesc_kmap_atomic(struct zpdesc *zpdesc) return kmap_atomic(zpdesc_page(zpdesc)); } +static inline void zpdesc_set_first(struct zpdesc *zpdesc) +{ + SetPagePrivate(zpdesc_page(zpdesc)); +} + +static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) +{ + inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline void zpdesc_dec_zone_page_state(struct zpdesc *zpdesc) +{ + dec_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline struct zpdesc *alloc_zpdesc(gfp_t gfp) +{ + struct page *page = alloc_page(gfp); + + return page_zpdesc(page); +} + +static inline void free_zpdesc(struct zpdesc *zpdesc) +{ + struct page *page = zpdesc_page(zpdesc); + + __free_page(page); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -954,35 +983,35 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) } static void create_page_chain(struct size_class *class, struct zspage *zspage, - struct page *pages[]) + struct zpdesc *zpdescs[]) { int i; - struct page *page; - struct page *prev_page = NULL; - int nr_pages = class->pages_per_zspage; + struct zpdesc *zpdesc; + struct zpdesc *prev_zpdesc = NULL; + int nr_zpdescs = class->pages_per_zspage; /* * Allocate individual pages and link them together as: - * 1. all pages are linked together using page->index - * 2. each sub-page point to zspage using page->private + * 1. all pages are linked together using zpdesc->next + * 2. each sub-page point to zspage using zpdesc->zspage * - * we set PG_private to identify the first page (i.e. no other sub-page + * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). */ - for (i = 0; i < nr_pages; i++) { - page = pages[i]; - set_page_private(page, (unsigned long)zspage); - page->index = 0; + for (i = 0; i < nr_zpdescs; i++) { + zpdesc = zpdescs[i]; + zpdesc->zspage = zspage; + zpdesc->next = NULL; if (i == 0) { - zspage->first_zpdesc = page_zpdesc(page); - SetPagePrivate(page); + zspage->first_zpdesc = zpdesc; + zpdesc_set_first(zpdesc); if (unlikely(class->objs_per_zspage == 1 && class->pages_per_zspage == 1)) SetZsHugePage(zspage); } else { - prev_page->index = (unsigned long)page; + prev_zpdesc->next = zpdesc; } - prev_page = page; + prev_zpdesc = zpdesc; } } @@ -994,7 +1023,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, gfp_t gfp) { int i; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE]; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE]; struct zspage *zspage = cache_alloc_zspage(pool, gfp); if (!zspage) @@ -1004,25 +1033,25 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, migrate_lock_init(zspage); for (i = 0; i < class->pages_per_zspage; i++) { - struct page *page; + struct zpdesc *zpdesc; - page = alloc_page(gfp); - if (!page) { + zpdesc = alloc_zpdesc(gfp); + if (!zpdesc) { while (--i >= 0) { - dec_zone_page_state(pages[i], NR_ZSPAGES); - __ClearPageZsmalloc(pages[i]); - __free_page(pages[i]); + zpdesc_dec_zone_page_state(zpdescs[i]); + __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(page); + __SetPageZsmalloc(zpdesc_page(zpdesc)); - inc_zone_page_state(page, NR_ZSPAGES); - pages[i] = page; + zpdesc_inc_zone_page_state(zpdesc); + zpdescs[i] = zpdesc; } - create_page_chain(class, zspage, pages); + create_page_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool = pool; zspage->class = class->index; @@ -1753,26 +1782,28 @@ static void migrate_write_unlock(struct zspage *zspage) static const struct movable_operations zsmalloc_mops; static void replace_sub_page(struct size_class *class, struct zspage *zspage, - struct page *newpage, struct page *oldpage) + struct zpdesc *newzpdesc, struct zpdesc *oldzpdesc) { - struct page *page; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE] = {NULL, }; + struct zpdesc *zpdesc; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE] = {NULL, }; + unsigned int first_obj_offset; int idx = 0; - page = get_first_page(zspage); + zpdesc = get_first_zpdesc(zspage); do { - if (page == oldpage) - pages[idx] = newpage; + if (zpdesc == oldzpdesc) + zpdescs[idx] = newzpdesc; else - pages[idx] = page; + zpdescs[idx] = zpdesc; idx++; - } while ((page = get_next_page(page)) != NULL); + } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); - create_page_chain(class, zspage, pages); - set_first_obj_offset(newpage, get_first_obj_offset(oldpage)); + create_page_chain(class, zspage, zpdescs); + first_obj_offset = get_first_obj_offset(zpdesc_page(oldzpdesc)); + set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); if (unlikely(ZsHugePage(zspage))) - newpage->index = oldpage->index; - __SetPageMovable(newpage, &zsmalloc_mops); + newzpdesc->handle = oldzpdesc->handle; + __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); } static bool zs_page_isolate(struct page *page, isolate_mode_t mode) @@ -1845,7 +1876,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, } kunmap_atomic(s_addr); - replace_sub_page(class, zspage, newpage, page); + replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. From patchwork Tue Aug 13 08:45:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761556 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F70FC52D7C for ; Tue, 13 Aug 2024 08:41:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 05EE46B00A6; Tue, 13 Aug 2024 04:41:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F2A736B00A7; Tue, 13 Aug 2024 04:41:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA50A6B00A8; Tue, 13 Aug 2024 04:41:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B4B456B00A6 for ; Tue, 13 Aug 2024 04:41:36 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 704301A06E7 for ; Tue, 13 Aug 2024 08:41:36 +0000 (UTC) X-FDA: 82446578592.15.82BB81B Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf24.hostedemail.com (Postfix) with ESMTP id C670B180013 for ; Tue, 13 Aug 2024 08:41:34 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=oBiwvACn; spf=pass (imf24.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538424; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8zG/EU/V1Tv+2Gp64iQ8usdpg3DGgcILcWSP8Vl3WXU=; b=fX018f1KvgiKYgFFk8X745BEVse7mj8o/Bkv1f31iSTyEe++Hk0zQWyet9iKFnAxf8UuCS ojciVOQz4ddb0awVIzcFbD/3fSrd4gygbYH41s4S48si4AakfZzh8FkUtHPbrrOx9wK+x+ Yr+JQlEMLhRYRnVeWrDti5bjBgh9w88= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538424; a=rsa-sha256; cv=none; b=IWlz70YKaNeTn6zNq/jdoDR7oKg3CTYQuiZwmnYf4m1K5h0rwopxoRddSCEqdgBftpvojj uAlajiXhRCdoqNLEjPrKPVtf8vAv367A/ODKHwnCTZy94x144w7avuNhoRKdxc4q55oPpn 7IW7vIlOOV3P5H+EizuHWJeR7WYqmhE= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=oBiwvACn; spf=pass (imf24.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 1228F615AB; Tue, 13 Aug 2024 08:41:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AB249C4AF09; Tue, 13 Aug 2024 08:41:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538493; bh=NHbMPinJZbaSf10+aKQqs+WxIPwaHLYUrrf2vkQMe8s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oBiwvACnUOx9fF6WAJKC3Ncc3QfJT0w+4/qElt/JAmOLrk/TUUoDtXszV1zYfezaL mXeZYTtMykO9vw1MJ0ZryLAbrLv1yQmWSXLhYgkk5XkQ2fzFKHYmGgOotSgGy1y7WT 9G+VzpmkPb8FjojkNTnSlkAS7pTaMN+oR7H16qLpjNBHtuNKW1L/c/dY39laZ4V4uf g5tsJyjl2CE3UaiBoR8IUlYBHYpUFtzwYlVR2TXMTSsXAHcyCHx1rinO6IQ+bFPgbJ yQAMoEoXrz+7v60ZL3sHg2Ud0fA25HisV1hIksvPKXlAGbd0LNhCPYuUiX05aaqzKa Vqku0Xt3M9vtg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 07/21] mm/zsmalloc: convert obj_allocated() and related helpers to use zpdesc Date: Tue, 13 Aug 2024 16:45:53 +0800 Message-ID: <20240813084611.4122571-8-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: 3aw1z85h81jicetzyy4md8wh193ikzy9 X-Rspamd-Queue-Id: C670B180013 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1723538494-1630 X-HE-Meta: U2FsdGVkX19uAPWC80wnZ4yj1Pxly6EM2mXumvPo3gnBz6ABKpHavR8Hnh7UorG3yvHsxnwG3dVMHFWL1LY64frwWWkOugJjxjbDmscmAkGnEMQsuN5FlTt6pTOanfr7SwpzptjQM1lrz56WK2UqceImrjH7PopbQMLfQfs+1tMobSj+63BN1WLBdgOWmhMTBGJKx0kXet02LQBzPhQdfD+AhqOZ734ygzz3vZgQ4rYc3isWTT17foebabxGdABWv6KLe4MkoijTUtRxLl/Fqhfey63LR+O0Dl8uNjtO8+l4TPlSw9IH7VcBpbpqwuzJoW3E4sqfoX8XRtmcyBGGdqgT8ZqmWMWmfm/gCIScOG/YSWKSF8MDSSwBbV3QqPnX/XeTnrgXIay9EvdOjkmJv5PgwZW12g06d4r6RnKkvR3hPEr992ikR8SABBBYg1Ysq4bm0FvQ7817wmV394Vqee6PfcenOVKEfpUCnJOQdu8BRtg3HGoJy8wgfQ/aD9cb0Iofnv2xBBlX1QaqqNFa9FkdLtYj4lgxhCBypvMrVB9sVwHCwvzDSEfFauLwUqpSEcONnMyg9JkMWR0h0bi7UgAXW5SWKmuwDGRsPsV79mHgi4WglbZyfFUEM1gP2R5DvOB7/2yaNfU6HipjaNRfQQ8L5LRxmMQF2vn62ogx4NnHH4vEFCRxy5603SdpYOTRpNJlWIrLRruiyciIeWBnGJ+Mn9o95uojNBMx3zvRAtM6O/DvyacD87D6Xq3H4myHa98wuumR8iEWuAMV90AElLnQgGFfickPTZDbxyxZCRKgX5fxPTN85BioYihP4Zlb0i1JeercTrc719PML+H/5YKUFgrxBFcRQ2iBRF3KHxibqtPiXcLtxiKsF8jb/1Wtr57lppTwWFvWcATPswNsGymiIL1ReF1TfRxOJDcqzJEzuB1PQGTtNLx3lDPR8diaF+WiP5SHl1p/HpqCN9y uI+iIZyI WuwsZZfk1XyFBEi+2pzQwaYKSe5R6ZMrM8TqchBkdF/Dxnpwqx2bIkatDWyJgnnil90u2H1IZmE1q1er1YgzjEsqx2QRsRhe8ecyneozOYaGwmxhlwEnU7m4bcZU6JnENx1vucQxrGn78o5hUdxnS6Qdv8aKwHdzthvoTpaEwcm2rvQTjWz9jgfAaAedV4A0b4aikadTu07w+Po0hLRHxZYk2ek9VMpbt3aVUh0kA4Wt33A6ZbvbY2R9e5INZ03e7/ZPkTizEM5lR+fWd/I+ZlrC3vPX1Rj+wsqtlGX/ptk0nNzhCaO3HHOo5GqCEwz7PdMIqpbiHOe9Z+mS93yKjCuqHA97BWSWeITojuaNLpRMnbM1xnI4PK3+Z91rLtYuAUNY4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Convert obj_allocated(), and related helpers to take zpdesc. Also make its callers to cast (struct page *) to (struct zpdesc *) when calling them. The users will be converted gradually as there are many. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 2d0f3d9011ac..69305ddc0e81 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -839,15 +839,15 @@ static unsigned long handle_to_obj(unsigned long handle) return *(unsigned long *)handle; } -static inline bool obj_allocated(struct page *page, void *obj, +static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage = get_zspage(page); + struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); if (unlikely(ZsHugePage(zspage))) { - VM_BUG_ON_PAGE(!is_first_page(page), page); - handle = page->index; + VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); + handle = zpdesc->handle; } else handle = *(unsigned long *)obj; @@ -1597,18 +1597,18 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, * return handle. */ static unsigned long find_alloced_obj(struct size_class *class, - struct page *page, int *obj_idx) + struct zpdesc *zpdesc, int *obj_idx) { unsigned int offset; int index = *obj_idx; unsigned long handle = 0; - void *addr = kmap_atomic(page); + void *addr = zpdesc_kmap_atomic(zpdesc); - offset = get_first_obj_offset(page); + offset = get_first_obj_offset(zpdesc_page(zpdesc)); offset += class->size * index; while (offset < PAGE_SIZE) { - if (obj_allocated(page, addr + offset, &handle)) + if (obj_allocated(zpdesc, addr + offset, &handle)) break; offset += class->size; @@ -1632,7 +1632,7 @@ static void migrate_zspage(struct zs_pool *pool, struct zspage *src_zspage, struct size_class *class = pool->size_class[src_zspage->class]; while (1) { - handle = find_alloced_obj(class, s_page, &obj_idx); + handle = find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); if (!handle) { s_page = get_next_page(s_page); if (!s_page) @@ -1865,7 +1865,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, for (addr = s_addr + offset; addr < s_addr + PAGE_SIZE; addr += class->size) { - if (obj_allocated(page, addr, &handle)) { + if (obj_allocated(page_zpdesc(page), addr, &handle)) { old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); From patchwork Tue Aug 13 08:45:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54BA0C52D7C for ; Tue, 13 Aug 2024 08:41:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D5AFE6B00A8; Tue, 13 Aug 2024 04:41:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CEBF06B00AA; Tue, 13 Aug 2024 04:41:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B826D6B00AB; Tue, 13 Aug 2024 04:41:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 966696B00A8 for ; Tue, 13 Aug 2024 04:41:40 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 48B1580695 for ; Tue, 13 Aug 2024 08:41:40 +0000 (UTC) X-FDA: 82446578760.11.0F3D5F2 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf29.hostedemail.com (Postfix) with ESMTP id AA08F12002F for ; Tue, 13 Aug 2024 08:41:38 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=F7UIAcQ2; spf=pass (imf29.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538443; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WwpC/yQMHyHuan9IsFC4y31Y+QMOJ0jRvo30Z33pQSE=; b=QXPNoBNnkWgxg7BHq1ljHPn13lnImgjC/NN1mKFwfwpnAgERp6sJHLIlyU4i9cgVnPfbKU Z/yNX2gH66nSXh9mom/DwT96IsX9mUqYhR8tHxw8a/4nnaAWFOxpNIQ7imBOfwXuDQb9/u lMmW3wmT5MX3fQ67lkUMUNmXU7n9+fU= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=F7UIAcQ2; spf=pass (imf29.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538443; a=rsa-sha256; cv=none; b=maUXT33EzF7N/eEEevROwbqPdoSgOb2pGNPU8mm4+yD0+3m8e7qJ/ZZwJneiqevvWbZbFb CGFytXVHXt+8F8BUX1dCqqcuOPYHUnuiIp/+Gp8MT9ELeMkTky8miXqZxPIFb3mq0Aj+Jg Ee9ubIN6aNSW/cwIscMgbi/QVMt1tOk= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id E1CD9615AA; Tue, 13 Aug 2024 08:41:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 84F52C4AF13; Tue, 13 Aug 2024 08:41:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538497; bh=WK8OGotBTrkxbKYhrb1ooiiEWg5GM27+9CUbteEDELE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=F7UIAcQ2ktZdGn1OWLy0E/zasMsvSF0kijBroHF8EHYacYxlB2MSlCDYeT7dZ5PEW hKROAwVNCVMZVR+fwI0yTGPNoBq5EBrBecd+M52TBLNgx+wkv56wwvC12zZhUQ8mwT +F+57D62/AqdkqlVTqT1n79ttTF99im4A3EcDPvAFtxW64q4kWkdhyQBTAd1UPNn6J Jsjm3wHPRJ86FBV0xo8SRnIYWL6ysi77/wGwMHyBkg0DBonGQ/oe4+WZG14R5xU9Md Pjw4a+uz84D61qulH54+gpXX+rMLGjKv/rwbp+re9WUDj8w72WRHBXQP9CHoAlNvBv ceaxbBtYv1Beg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 08/21] mm/zsmalloc: convert init_zspage() to use zpdesc Date: Tue, 13 Aug 2024 16:45:54 +0800 Message-ID: <20240813084611.4122571-9-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: ufkcw87exjf4ao8rrntdzsw5dafwbpu3 X-Rspam-User: X-Rspamd-Queue-Id: AA08F12002F X-Rspamd-Server: rspam02 X-HE-Tag: 1723538498-132332 X-HE-Meta: U2FsdGVkX18odWHM1f7GySOgpp/6MxVdKMQdI2VB1RniVRbNQX3k5Jzfz+uHfK+fdrEwEHP0G1HTwyVRnhmwzXdLFm7pXwRAsR3LzVcyJVnJE7O2wa+EnN/lQN17YWmTJ5Cs0sdtA0SXrwzM/Jh4XeiyxlwhQqcHsafALjGoLosmwLUqKmEvAhzyIapeSZHegIlpr1sChZWx3qZ785/m4pgZGQ2su3BV3+bPNnihkuY2KFOmz6lN+oL7/f0i0b7F8uwfVBe1wydD6Cb7fuDho9W0rJ6XBoz+PHqlKtZ+YzEiHqjFf+QWbzmzVZAcYM5bNfCKBmM43dt79fhgSz9mw1vTf88lnKVZb2dzIEB0dLt3k7LideXhsh/u8ZpnHuXJZyvdZGo62sTYH6wetaM64ePC02MLwfG96SyvRXebJeWKvFfKslwJy+32D9PUCHWgpGtR2VZMtbI8qPTByouF5TjksKu8DnNC9sL4twaHJADnTMsWM44Z21PLBib0XiOffvhvqohtnuR14al6LZeIbJJAESzduMeCfEoKzAtqFOi2zJwgvEevo0kb0bdmy33gvixvPvJk//W25kgTwgLXI5eAiKyDxUSiiFZ+EPP+Bx6kyG1KwjpY0OTNWPYJ1oN9Kj0+fBzgkEPeVKpFE/IwTHce+bWuXCX0ZyXmmP5yMPEoGfTDJnalBhTh1/1UcEvCdSGsVo69q7EXtccQkMbNc5yzU1nAD64zzPqtKuEUeYyDg4MiGfWYNJJCkOjjsPOnslSDx7+KbE5K8rNOEsrWU1bA0VGQHxdlBlZynSmhKnNhZL2yUcECaEnycQsaz0pScONVbBNIOO5uesTXZz84g7F/JB7G+cUHDwM+FSUChh9qqGsBpqxsdm0hmcEgn3xuK0zU2dpRXolCWGpLh9GsGmPCuFSOn6FqeLH86u4IhnyFHw4j58ZkqLx+btW/9WjggSiOfcycrWyktvL8ULQ rYaemWNE IQ0uvO7dnZ2Ex0fV3MJi+48Z2smem4u8gf39+NqTWYuxN2oRG/SX6bIdZefuXeKjQ+Nn5ntZSh+dK3BcPDclqxwieV3J9fUk19RsRZ+Xk/QWUedE5SqcmOwHLviN3BDPZRYfaSjyPmcdUz+mfn4useA1gH4T34sd3K2De4xrg8eEOfYiKfXAeSnG545bF73QDgrDru/2ulDLOH5Rnw4Rg1I91N5UK6dvlpQ6vRcinb2wS7D+Vn98fKyf7M5GXq38Wu62xXD+uRS4iR4DKThAPd/mCa8cw1i4Yu6dMSRzhuW9c4GxNWDfboQ2dtlFa8jFZQc/98weGLP6Ao0H036TSDCw0ULSwXQ9ZnZAPmCCyvbvnussSJ6q3lzrfQ0gnYxIpk7A/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Replace get_first/next_page func series and kmap_atomic to new helper, no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 69305ddc0e81..b07c14552276 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -942,16 +942,16 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) { unsigned int freeobj = 1; unsigned long off = 0; - struct page *page = get_first_page(zspage); + struct zpdesc *zpdesc = get_first_zpdesc(zspage); - while (page) { - struct page *next_page; + while (zpdesc) { + struct zpdesc *next_zpdesc; struct link_free *link; void *vaddr; - set_first_obj_offset(page, off); + set_first_obj_offset(zpdesc_page(zpdesc), off); - vaddr = kmap_atomic(page); + vaddr = zpdesc_kmap_atomic(zpdesc); link = (struct link_free *)vaddr + off / sizeof(*link); while ((off += class->size) < PAGE_SIZE) { @@ -964,8 +964,8 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) * page, which must point to the first object on the next * page (if present) */ - next_page = get_next_page(page); - if (next_page) { + next_zpdesc = get_next_zpdesc(zpdesc); + if (next_zpdesc) { link->next = freeobj++ << OBJ_TAG_BITS; } else { /* @@ -975,7 +975,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) link->next = -1UL << OBJ_TAG_BITS; } kunmap_atomic(vaddr); - page = next_page; + zpdesc = next_zpdesc; off %= PAGE_SIZE; } From patchwork Tue Aug 13 08:45:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761558 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79417C52D7B for ; Tue, 13 Aug 2024 08:41:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 03BFB6B00AB; Tue, 13 Aug 2024 04:41:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F041C6B00AD; Tue, 13 Aug 2024 04:41:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D56016B00AC; Tue, 13 Aug 2024 04:41:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B0E4E6B00AA for ; Tue, 13 Aug 2024 04:41:44 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 39958140693 for ; Tue, 13 Aug 2024 08:41:44 +0000 (UTC) X-FDA: 82446578928.01.9DC456F Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf22.hostedemail.com (Postfix) with ESMTP id 9FB79C000D for ; Tue, 13 Aug 2024 08:41:42 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Zn4AO2PP; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf22.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538424; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8ctuRfyWnTA65JNahQV20ejpSulzJ95NrLnSE4x0zYg=; b=k1wGKub5ZMdiaw1+4j5XPewvz/cnYnJ8ycmyLBCX0D/0AUMAOAGwCrrbJ3S24e34KSUd2w YOWSSkckhQJYj4EltXztNzkPWbm+lec+PCU73Vv60gxWdhhZLIqtHaXFhf6jtWSxhPMPqI EWXohoyySF+9KszPbQzCwIbNhe/MzRQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538424; a=rsa-sha256; cv=none; b=DTxTc383j4+dLNxFSlFNSDGQkyLIO9gZYs+tOF63U2TVSuyzJ6/ylWm76obzoDw78hqCfM ddbsSWzzLWrJq7pF3TbqKxS00WQ9EkoTIuAYWAsgoSqPGysyQ6ELtAy49v6pBsUXEO6I9e 54gn5EApuLLm6Ka87swFC1LhwjpbVYc= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Zn4AO2PP; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf22.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id DA2B7615AA; Tue, 13 Aug 2024 08:41:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6896CC4AF0B; Tue, 13 Aug 2024 08:41:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538501; bh=TLwbltxTVpT2lgiPJc6Jxl8phogKw6tUJfKuP0Et5M4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Zn4AO2PPsEJcQ4S/HgpTW9guymMeO2zo3b9e8FX/6yLaPpDw0yvUCSZEF1oFBDj2V ee4zE0Yh6pulxHPa96xkoQA8A38ttfDQppDSn3RBMZez9+OUE2NO1KlMRwQjgRWSfC MW25K0Kbmj/zZZO12e6xpBLe8W8zJA6dBZvuw6mmzVnXgdXxtQ+7LXnPPqAJFP9z8G IaZuTq09bkCVYWjqEmZZLFL5NbsJJEnOzGWh4c7AFa6Af5SoD6Yego+/BzJ2D0mO8w 6j7jE3ZKKt/fXyxXpy/uhyjt47zFMj3n8FY+7qL5SI8jqBzoxQRKYHJ8QAxwY7xUe6 LnSzxgNYWLTpg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 09/21] mm/zsmalloc: convert obj_to_page() and zs_free() to use zpdesc Date: Tue, 13 Aug 2024 16:45:55 +0800 Message-ID: <20240813084611.4122571-10-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 9FB79C000D X-Stat-Signature: jzi194fdyzhrxqintujuchgcbexya34f X-Rspam-User: X-HE-Tag: 1723538502-164609 X-HE-Meta: U2FsdGVkX19EXTAKJXWg7OqDH1MxL6hTSvgOzrNbLUIsMyTTslYQ4s+UUGXsqP3I4BJfC+q6CukHBhh0lodf3+0DzsmPUFPjghd3CVAevgB6SLrmxBl6LvcPdjm9p3U6ZLmX2BC0emTb/nmFVXy+rrLnT/n3pSc7jUR3FwgHVYXxSlVhWjtIuS+XfobazY5+xpUmPmqafANfnkE3yd4rQBJI9PUDNA4/4eB20+q/FbSxxgJUm6exwKdcrsFZPbE+OOgVZhiewDM8hKciiNzbWb3rYtkC6BU5FSEY3iVCX2ot0f/TncQIqB2W3yiXtfyuwcNC7pXVTZdqsp21UtCl1z/URnj9q/eoZcLd3aBUb+cH8Beq56HP9/B50VnqAIJyi9lVi/hRYRKdYD6LZLvNLeFuHE5mwrDldBEK6FQJSR5y/pEHFpl9pvnJ3qComK8Ugj3latWExHprawXkcaEJpMNrnNHdQL98oYs97upNu8U6LPsp6A8HIZwqY3L1cf1CTVKlMXbyKTLGqMhwo/Zk4dSn879wJGdOoJIAe9K/uyzrw0yez/yk8VIf95ejcQ6q/UROf+/HRAqcNDSeEQqvlsvjU3/dd8woIH+C3dh8yLd/5aPjHLQWnxEsTvPyaCJbyk7i/Ll98hQoNYHJLOtb2odizYlab0Nrd4wg956UMXa7ULCYui1ig7t9WvkGiWswD9IGX0gxFMvMYUgUODtf4rkBNsFxu2WolLzoYXxZ63vnwTfqH6V7vIEPYykb/DQNrNRQ/SHUV5I/UlMkDYbO7z97gvKviAKbTlDAQhguVy2/ySDH6PrwqybuS47UnFsCArccoMnRILpE9OPIi2402UKKdKMckHynOHhOoISR+FQ9E/wP2qcIntfAdfzxJzrKev1e2mCbCeUVnk0umntXxFHWiWNKSbTIJRdMJxycUFiNsWzuTMBJUyU4a8pULGDH7SxEoG1ma+RkCVO8gWN i3JIDZv1 NlUbmbPDI81FaWrdL3//OBsHcEA2OUIE57nVNqmze6Pj0UTEPx8/BeFBZLA/JH7tOHmY1u6L16tic+bCj7lMSpmc/WmbOwtKf7v+VMFae+7vLl95yymKyWINyEcXvx5bjjGTW3G7fRmf9xgKIuOfiRdk2VyY0Cd6mEKpxd/G4mAPKDw5ezTUT5vSgAL3L95nOABDz0UM+zyjrfo/OenXm05AkhZoPogX3H4iQYxUPdBM0Wo/eQf8bznzt6Mu3Fv1B0rTzuZL1Hp1cnndUb9SguQauFW+QRwGkcNuhRURk/friLgyEd7Mhl5vNi1Dhw/dR+NMtOxDrCmpoFqkAXFKIM7pbIpRfdC5QLLiUkSNhSibOWxT1kb2J7GKDuaLv1YpmcBX6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Rename obj_to_page() to obj_to_zpdesc() and also convert it and its user zs_free() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index b07c14552276..cb90defd3c0a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -814,9 +814,9 @@ static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, *obj_idx = (obj & OBJ_INDEX_MASK); } -static void obj_to_page(unsigned long obj, struct page **page) +static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) { - *page = pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc = pfn_zpdesc(obj >> OBJ_INDEX_BITS); } /** @@ -1490,7 +1490,7 @@ static void obj_free(int class_size, unsigned long obj) void zs_free(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long obj; struct size_class *class; int fullness; @@ -1504,8 +1504,8 @@ void zs_free(struct zs_pool *pool, unsigned long handle) */ read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); - obj_to_page(obj, &f_page); - zspage = get_zspage(f_page); + obj_to_zpdesc(obj, &f_zpdesc); + zspage = get_zspage(zpdesc_page(f_zpdesc)); class = zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); From patchwork Tue Aug 13 08:45:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17011C52D7C for ; Tue, 13 Aug 2024 08:41:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A4ABD6B00B0; Tue, 13 Aug 2024 04:41:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9AC9A6B00AD; Tue, 13 Aug 2024 04:41:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7FF1F6B00B0; Tue, 13 Aug 2024 04:41:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5B3956B00AD for ; Tue, 13 Aug 2024 04:41:52 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id D6C721206CE for ; Tue, 13 Aug 2024 08:41:51 +0000 (UTC) X-FDA: 82446579222.11.AFF553D Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf20.hostedemail.com (Postfix) with ESMTP id 9C5191C0025 for ; Tue, 13 Aug 2024 08:41:49 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=dOOxP1Wv; spf=pass (imf20.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538439; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vU9lypoqlsum1LdSoQWmMsAo23xxIBWq3aYARm1N1tc=; b=HzfdHp/nYomsh+irk71UvlJOtjspc3DYketE1SOCEq3B4DLvPJbSftpHVD8issvtJM4CZE /wa2nyn3XmwY432RLi9MOr01avxDc5dEYlKRhAiR1hKKHTY60TdGDOTAVeOsQDyy9eS22u wvxRJbw4d4b3R7/kPh9tAq8T9b6VXTk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538439; a=rsa-sha256; cv=none; b=EgVPhOZK55dxon8qK/Gecun0yPYzQGlzJ4Cw82syhGFnhCuHgRmyc6774Bo/YELGQRbCev egI0yqj8tHjDJGlXgHmjZgBxzsvSPhqUBEiBnL6Bvz7d9wxl6RqPU4K1W/HOmzs50tjnP4 k92IH6Oy68MN7GpxJQplrKwcOFdZru8= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=dOOxP1Wv; spf=pass (imf20.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id B17B0CE1235; Tue, 13 Aug 2024 08:41:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 45719C4AF10; Tue, 13 Aug 2024 08:41:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538505; bh=UmY8mz0Mo/B8gMn4RPUMsn8q8kByucvcQFpc8ROaoN8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dOOxP1WvR4nlRS/PNgkaS23vVd55SRWG+qIRE209IfYLyLsqGZjR7PklKPuEYICYJ XeL5/VZs+2k81LiQUCMDbY4hJfhw4fIletndPnqrB4P0GXkBkofSmpzqnXYOO081Vr 3SsdAS8ceOfC1hyNu2WdynXMx5o2fHHL46sbzY8uTYIY+G2F3A8bGMOAObdCoZUTPj C9KAAb+2OfoGY1Y25Imog1SooutwDOnDEry17xuJOhHxDogxrQXQUpB1DLISTIjOhT YiZI9A/IdZ7bhzYUmI36JbUZqh/Bs9jLalJ5FRfHyT+W2uibxDS6T1OFyvVi1gHpmq ZRXdWlzPjK9Sw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 10/21] mm/zsmalloc: add zpdesc_is_isolated()/zpdesc_zone() helper for zs_page_migrate() Date: Tue, 13 Aug 2024 16:45:56 +0800 Message-ID: <20240813084611.4122571-11-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 9C5191C0025 X-Stat-Signature: trreaqw4gq8r73m9i939zo7hzjiiz6ua X-HE-Tag: 1723538509-809587 X-HE-Meta: U2FsdGVkX18Is247OO+GB83TX1IeTFpgCvR9ekcd8nL+j426CvbvOX2mATua8pdK/64UlWcjwCQP8d/MsjBJ9bI2F14rLTagxPEr9bv2ApWaw9kgAj/wiBqkGugb//ay90NC13Faf+RaPRm5/SEKItOVwrmb//kwAzRcU4XCf1TxJ/bwiv/HwRDl2ZB19wILhpnro/gNtoXa/bjq8RajBe1UsHBOXmor97xDHpBBEk7Z5uj0MtgNE8Gso9LPxL+krbIRTgkluY3kP5e2vsdbXGdTv1QU1hyHf5sHQp8G2CiWI0dDKR555u+ozDHMM/NtANZAC6GDfxW3gfa2DaFLB9PlhpiWxwq7GU6Ir6Jqp9IDKq4r/1vjDgvkw8M/b2tZ80HSemT4Iqyx+L5iEG4dReObUfjeCD08Vlv2ipU2E3Pcz2AuhaZQX3slXA49LcquXEs1dbZSMOTZK1eeYnWhpqxjuCy7jnETjlWs7+B6ppAY0X8Avdv2XRdlkwAJ2+xAw/apuM9SmM6/jk3j5dAVC+oOPNUJzzhOmR2Pf/Bc26V+OsC1LUs8rBePcFQeDRTRqrptAVwES4zdcyyHckXwe3bWFFDQG/4KX2UBkva8knqxqvoUhEU+M58S7jwApw3th2vndgx6j9eaUWmwFdkXTZebcLhs01wdK7toVde+Aa8KsczfeKf3L/i5QDrJ7wiq2tPZeMOw6ZX1irbuoFF6xPBKiEpXroqOnYUSR2dhw49UUy8EObD8sK3oUeDvtQ9sLXNDrHLxkLOjBNLVodEp6lTBT2kt9pdp9GQACEhqwL97kFsMkQY3lpq3JwXebpyeDLuGtlB+y7C1k3m7EIIPSXpvELMPFyueqQMErel/J0y1gFonsVF+NLN/BStcnzHZ593gHqC4dMkGpV+Jj9AygVoP71XtRvoSJUbPdzIem4EX62cLgBhXMwM9qNY+4Y5DSsjp3Zoeo4Zdz+Pc30B jdVOZxxk A7iBMvTSmEPEyMa1p1hnt0vok2vaUnSBXbiE5tBZDzbUxYYpNUCSJ5s9Jnic9+Vypsdzds9pmxJabvv5mqTtR+q7Fzwrvh+5wHSEt0OgQUN0YyQ1SxRVwCMPnAn6vDy0nPrf4R411VWtAIZ12ALd5lweBLQEjN9YpO2PRlV6gG/tpEuwRDn/S7qIPW7PSz3HqtsONVUFfJjy7rHFikji8BBK7ExlTfWR5x9CbNaO3YUvsm3CubeNclar2bJBZSql3c9v/8PAPZr10YJro1jlb55Mjbc5oAdJt8WBrBPzcT/YOX/4YQ2FUaQDU+H9DKTz0cXGCzlNgdxzAVH/hL7lIw4EQgVMKlV+MzhS094vovZs9bpTRiM3HjqSdtA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To convert page to zpdesc in zs_page_migrate(), we added zpdesc_is_isolated()/zpdesc_zone() helpers. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 11 +++++++++++ mm/zsmalloc.c | 30 ++++++++++++++++-------------- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 3a65a7d494b7..4b42d8517fcb 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -114,4 +114,15 @@ static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, { __SetPageMovable(zpdesc_page(zpdesc), mops); } + +static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) +{ + return PageIsolated(zpdesc_page(zpdesc)); +} + +static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) +{ + return page_zone(zpdesc_page(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index cb90defd3c0a..65c4252412b3 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1824,19 +1824,21 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct size_class *class; struct zspage *zspage; struct zpdesc *dummy; + struct zpdesc *newzpdesc = page_zpdesc(newpage); + struct zpdesc *zpdesc = page_zpdesc(page); void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; unsigned long old_obj, new_obj; unsigned int obj_idx; - VM_BUG_ON_PAGE(!PageIsolated(page), page); + VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(newpage); + __SetPageZsmalloc(zpdesc_page(newzpdesc)); /* The page is locked, so this pointer must remain valid */ - zspage = get_zspage(page); + zspage = get_zspage(zpdesc_page(zpdesc)); pool = zspage->pool; /* @@ -1853,30 +1855,30 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); - offset = get_first_obj_offset(page); - s_addr = kmap_atomic(page); + offset = get_first_obj_offset(zpdesc_page(zpdesc)); + s_addr = zpdesc_kmap_atomic(zpdesc); /* * Here, any user cannot access all objects in the zspage so let's move. */ - d_addr = kmap_atomic(newpage); + d_addr = zpdesc_kmap_atomic(newzpdesc); copy_page(d_addr, s_addr); kunmap_atomic(d_addr); for (addr = s_addr + offset; addr < s_addr + PAGE_SIZE; addr += class->size) { - if (obj_allocated(page_zpdesc(page), addr, &handle)) { + if (obj_allocated(zpdesc, addr, &handle)) { old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj = (unsigned long)location_to_obj(newpage, + new_obj = (unsigned long)location_to_obj(zpdesc_page(newzpdesc), obj_idx); record_obj(handle, new_obj); } } kunmap_atomic(s_addr); - replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); + replace_sub_page(class, zspage, newzpdesc, zpdesc); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. @@ -1885,14 +1887,14 @@ static int zs_page_migrate(struct page *newpage, struct page *page, spin_unlock(&class->lock); migrate_write_unlock(zspage); - get_page(newpage); - if (page_zone(newpage) != page_zone(page)) { - dec_zone_page_state(page, NR_ZSPAGES); - inc_zone_page_state(newpage, NR_ZSPAGES); + zpdesc_get(newzpdesc); + if (zpdesc_zone(newzpdesc) != zpdesc_zone(zpdesc)) { + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_inc_zone_page_state(newzpdesc); } reset_page(page); - put_page(page); + zpdesc_put(zpdesc); return MIGRATEPAGE_SUCCESS; } From patchwork Tue Aug 13 08:45:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761560 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C56BC52D7B for ; Tue, 13 Aug 2024 08:41:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C7C7D6B00AD; Tue, 13 Aug 2024 04:41:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BAE776B00AE; Tue, 13 Aug 2024 04:41:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 937D36B00AF; Tue, 13 Aug 2024 04:41:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 625FA6B00AE for ; Tue, 13 Aug 2024 04:41:52 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 1B7E71406C2 for ; Tue, 13 Aug 2024 08:41:52 +0000 (UTC) X-FDA: 82446579264.01.6C63857 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf21.hostedemail.com (Postfix) with ESMTP id 80DAB1C000D for ; Tue, 13 Aug 2024 08:41:50 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Xo7gB3Bq; spf=pass (imf21.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538440; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=z25iHKcWSinSDIeAECI3R1jU/7muqXwjZWaq6uW8gek=; b=nAQmnYt0qIARP3en707q5TbStIVr9waY5N7ZUUvw2uNo3gqenesmqnF2flVKA/yWCCjVYN xRP/V7VPIZDOPQMVJDzdvWRsINAuQvD99gQcnC2c7IYZ+P0Q/8/P8xXFnoixKZubOXjUFr vblRI0Eg1uQ14bSPgy5dwYgwweW+1yQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538440; a=rsa-sha256; cv=none; b=63n1CJkzml2z77dS1sBNH+NLKTBhZLPonbpQeoyvkwuOA3UkuVNfGjPw65u/OyUcXX2uvZ Ik3FxhyigGhmQm9YeLw77sdXuqPQVmJ8o+zK08GVVQT6OjgqMxtcO5O0lX+42pnmVp6bEb G54F2iGDApC61DD4kX/S/DB+VLlShlI= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Xo7gB3Bq; spf=pass (imf21.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id BF16E61057; Tue, 13 Aug 2024 08:41:49 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4D8E3C4AF12; Tue, 13 Aug 2024 08:41:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538509; bh=43XgPIuMdTYjompc4vGGiVQL8qZrEOEuMCnAM7TFR7U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Xo7gB3BqFWVM4It6qj2lRPdZgFjGIcIeJ/86oumVRtJApctRnsQGjwaEQPKdxMvdd MIjd+B5OVxXlzcGCXoZEk19wwJ3ilpyb4FBBmyoWbikJ/vlYyjgxOR2qyRs2xsJ9jh iXar1Zrpgk0vDVxw1cUMmGUgHjZbTZk5J+utPy+6t3/0MCdFiLgdWnUEKw2G4UJRry IN2pWUO+/6I0YChVjnKMQJyZTj3WiSG9RlNiPrlV7UFTJG92WEm4irz8zk14pmPVYS vkJQj3Rarc8ig0DecFxTs6bgMD/6IU+9OxCfzbfLdQztTGMrRIddglPt1yIUMHeNsk 5GoTdMIMtC6/A== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 11/21] mm/zsmalloc: rename reset_page to reset_zpdesc and use zpdesc in it Date: Tue, 13 Aug 2024 16:45:57 +0800 Message-ID: <20240813084611.4122571-12-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 80DAB1C000D X-Stat-Signature: xadz4nu4iq9m3rrkrgpuyinmkd1a4x6a X-HE-Tag: 1723538510-832638 X-HE-Meta: U2FsdGVkX1/qtTRcQW14BVF+4/nGzmA4swGWOo+v4g49MCLp+hc4KkhQ90QxNA2/QDLfT/QSpDytkCPY5EexXFLLvGsghsesAjpoiE+5PkggtoNUL8n9wW+dwkaU5kqPIdQMtyyft2yrYFm27W1/rYnMwxQX//30iZNlfZJ9Ds1GnKsbUTIVfPiwrlpMCmZ7/qmynaaA7b+qKDawdtv0lWBl+o+ae1wOi8DTqKAVNpek85xeEDGds1OmkFsQ5pTClD0KVaVmUQOSt5aBt7mhYVahz7EoEZc2KFDwfvqNPW37/D9zpXTqKcK3q9KJUeZAyhQIGp4gwgBppspI5KgrqJtVg75VFayXw0N0STIsvUKKXSP2vw8Ljba4qhRgo7i4olbTsMQXe4X0U7rB9xaVNNMYgxZi1u64IugPGNGVt6uMID7ynVNYpcGGo38yJV2mXLTW7FPRI4VDE4ahFNIhn/6a+qqsdp9fDqiudCXZT42iyfR0IQcTRspFF/eeX5Kq3HLYSWBcZcSCYaKAVINQuTU+UqtLFE2Vl3IXXVhUEMuuc2kvxX/WM74pOwTxusDYwc6oazDVt5yX/SHAjNrWVzB+mjJm2scOt8frnDss4dX5DjhT1E2/duWpYXCFC7WE4irB+HmJdzFK35Y74mdGfIOekJWvTsDXnVDun9DYU1ARuNLgM5hmU26CW9tuTV6UEveucz1Q11tqoLODCs4mUZMfQu2dyR8M6GWpMJxLoQfT+ZGodxUNUnJYivxOiirAY7lQDVABbSdlJjZ42Y0OqwW/WTd+9JMtIDpgGw0qYHlBR/POYMa4QXR+wy12ZAxISVFYsvumgwV2J2NIZsQwI3cX72PuPV67Vr/1IYJYdNxM7RJ8uAXVaQkQGKA+3vqLJALZ18/FLh3fH0RhXb4sPayMFrjxVPQ/NXNH27GwaSxSoWbW9IEwTznMePACZuUW/TGvwpFTT4J9b4eD8Xp KHMrwveJ SDIdfhnmvVNABEmMWPL38rvBU0m4XmzP7AuzS+U/v2XOBEs4Q9cucLFz8ThSzkjPEVFgERZqNt2GY5xZYTHvWTDdAiUSVCNh1SKK0YzE8kID0IWMkFbDXDabAeNcaMpLA7WEc/SYwuDrICCOf6gjO8oPdaR5Rn/864bivB3pTrLo8mNSJ48bHE8m4zuJYlnUPXdcCP6F/+/6U/sT7LLsp0wBRQ7SgSBwmrWUY4+wQgsaA1SMLoX+yM7Xy8xdj6FmfJRGvUduEaI/xxxQvbavchrq+G7rkBkadmb87ILPS1cIBJYfr5Rvpsjmbt2abFimHwEKRX7Zabd33w/zEOMtJczTP+fUmWhzm0Kc1IeqdsrL0kyw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi zpdesc.zspage matches with page.private, zpdesc.next matches with page.index. They will be reset in reset_page() wich is called prior to free base pages of a zspage. Use zpdesc to replace page struct and rename it to reset_zpdesc(), few page helper still left since they are used too widely. Signed-off-by: Alex Shi --- mm/zsmalloc.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 65c4252412b3..6c5dccbc9a60 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -859,12 +859,14 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, return true; } -static void reset_page(struct page *page) +static void reset_zpdesc(struct zpdesc *zpdesc) { + struct page *page = zpdesc_page(zpdesc); + __ClearPageMovable(page); ClearPagePrivate(page); - set_page_private(page, 0); - page->index = 0; + zpdesc->zspage = NULL; + zpdesc->next = NULL; reset_first_obj_offset(page); __ClearPageZsmalloc(page); } @@ -904,7 +906,7 @@ static void __free_zspage(struct zs_pool *pool, struct size_class *class, do { VM_BUG_ON_PAGE(!PageLocked(page), page); next = get_next_page(page); - reset_page(page); + reset_zpdesc(page_zpdesc(page)); unlock_page(page); dec_zone_page_state(page, NR_ZSPAGES); put_page(page); @@ -1893,7 +1895,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, zpdesc_inc_zone_page_state(newzpdesc); } - reset_page(page); + reset_zpdesc(zpdesc); zpdesc_put(zpdesc); return MIGRATEPAGE_SUCCESS; From patchwork Tue Aug 13 08:45:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2282C52D7B for ; Tue, 13 Aug 2024 08:41:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 989D36B00AF; Tue, 13 Aug 2024 04:41:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9107D6B00B1; Tue, 13 Aug 2024 04:41:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 78AA96B00B2; Tue, 13 Aug 2024 04:41:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 5648A6B00AF for ; Tue, 13 Aug 2024 04:41:56 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0DDA5806BE for ; Tue, 13 Aug 2024 08:41:56 +0000 (UTC) X-FDA: 82446579432.16.7B0CAC6 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf30.hostedemail.com (Postfix) with ESMTP id 65D8180012 for ; Tue, 13 Aug 2024 08:41:54 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Ng8rlXIL; spf=pass (imf30.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538443; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+l31cbHDmQQnmUIem/Tt9RW9I4dStO9PjAQ3mgIerxY=; b=M9ztxuIs0ILPwqpkKsXFH3/fmeSiqq9zNMbZPBqDafzzcxBIHIDOYtSvu7G8ckrzLYRPn9 3TQGzer9u67Vni3VeBwyGO6XTksdsrdWEKItkyjYmXwT8WYlIlRZBk1oVntDblk3nYjZjv i9l/W49urYIlePDOAmz87xl7d7a1FDM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538443; a=rsa-sha256; cv=none; b=TKJMt+Y/86K96hXZraAOYtIprXpKaxfxoOiCVq/u9N85muogl9M9d8gOcyDrZfEXvDt0N2 RdD/Z5Y/mKWB2wzcnUaI3U4FGeybvxov2fIN67NI6an/ix+gBbrPsYzjIhzVyp4VrYisVj eQmQUOb4ugsuEQKeAmAxOw5OdMcmIRs= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Ng8rlXIL; spf=pass (imf30.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 9D89B615A5; Tue, 13 Aug 2024 08:41:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2AA27C4AF11; Tue, 13 Aug 2024 08:41:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538513; bh=Zf9TjxFFbO1e13JTKQj4PfBu8BRbAcD0oZRJJHyNdF8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ng8rlXILf+oInKW3qAfGb1XtZwQc0EivxacQrteO99zmio5Op2vaSm9cKsH+s1glz hJ0rJrDKlc0/qijL9tJ01zDCu1RpN1zV2AEN9qj11s9Zx10q5OPDKnwo3duHiTiPNx vKSDhEysjCzTbVyFhzVB73nFhg0avnJsnwNCvZDzwTxe8BfEcu0mqXawP/+s19lgK1 phB0C4MjtZvYdtZ79SPDYXjiQjHUc068HvT8fcgQLniUWBeoVsyl6RpGZVF6TwGvSR hW3TDaMc3aKQLK+52Loibwk8NS3uYXPbVig0vN29xtUkM18bRFwOoBXm9j6PKwhxz4 /Hx8TbYDOrQnw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 12/21] mm/zsmalloc: convert __free_zspage() to use zdsesc Date: Tue, 13 Aug 2024 16:45:58 +0800 Message-ID: <20240813084611.4122571-13-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: pw1dj5g68s5px5c649re8dyd88fwmmh8 X-Rspamd-Queue-Id: 65D8180012 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1723538514-444826 X-HE-Meta: U2FsdGVkX18gWSm0F6AWs7dd4KaqOCDUs2s3sZzol8nE1HI1RPej7TqYd1qc7ArWqxgwdgrrWQlgHrMQkdgRsqzfm1R1h1fVDnvclpjDTtnAaejcElgSFurQFLi+k4AFqBNy1pSOMtHqmuAdXMEQJbv+H66XHN6O32U7PGj1yRDiKSUrZAbjZC0hk7J2IrcD4CaGCckEcEAO8p6b8QqFCMR3/5tKrbcKk6q6jQQAZbQ6XsutPgHH2ruscfHJaw+7xmareeVpyjXwdgiMkWfYQZFpcXY5lLJgYxlne3yGrcQRhI/de9HkewUj/94BfscqS4fw8zgynXZL9wJFZ072nydX0XHUUm4+X4Uxy41I7pbx3CW7hueYNDs/9YtC/irQI9/E9ssrD9TKLRhelSnUJ6JUcQo4aUG2PfcYujGcmC6pEY5gA7KWy3V0qNp7CTNmbhFLUWROxBQaYDzZQb305SG2HtQDHnuKtUre9T+wMUSEeWrv9rYceTyfzpwvhjL/GBXBaLXhdcafUBaZdsFwNbmHnyLlkZYFEcZ0hq/VVDCaXHdE1g3YghwDDqk2yyPw0/Fxtw2dAZpzr8hZ12d4zE+5yCfKHnDxVy2fAOYrlssaGTkOg+q1A2xQu0T+CTNO5nkd8B7/MQ7+UEoZFNTGQCRYBZBtiSl+gd8eUK8Wo4YvF7Nd6r/y0gY0nV1cXFDrekcbDuj2Jyz6pmly9mFfBwjIhSbNN6pE6ls4UJhAMkmw2UW3pvaDXhtajoDVfT8Fa3y+oCAnWEs55rN1IprMfzFXBcu9isISGeyFdllnQ/pfjvd0VVEdpdoQP5FkmHRqmUjcCVTfzRNAaMpjOS7g56te25CCyduC88dhMkKuENG+UMjeE37YynS6wYCmVs2MPwicjlQFXfPzjlY9Lp56uSplM1KZ7KaGS97UnB4OmObUGXNmN7b6M2GZtBb7BQZV0ZxBhTGpX/Hdb72JBRR d/Z0Gzur OXKifg8s4eA6wgEE+B6V+5d6OtZghcjvL/91e0jx94U/ISiTAER0PBVW0EFmXi2P96UQ2zVF2XAPtUS56G9ta/fmMzz1eQ91JTD0Bac1Cu+onM7H2D7PdjM7QF6m7mDecgrePwB/WMYkak4fu9UoDJa/xT8V9VBzfKsacoB+ra+lqZr6omOYY8Eg5rF8+a8STyxGXsRlWu2FPYbJBb5OwjjEWn1E+skGaQY3fEFQLSegfoctRYcL6ArWgA18pHnqIuo7w5rwE+ruvEX5tZLyRqEqJaoaRImDL2kHciFtc3om268dDLzst15fGU0zZ71hGF51uDlXgEFtKjd1bl9kIt6Xvvc/og/0u0J4LidFDsOaQ/5GojqnwPY6b2No4LyOuZ3fZ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Introduce zpdesc_is_locked() and convert __free_zspage() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 4 ++++ mm/zsmalloc.c | 20 ++++++++++---------- 2 files changed, 14 insertions(+), 10 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 4b42d8517fcb..a1834d36ccfc 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -125,4 +125,8 @@ static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) return page_zone(zpdesc_page(zpdesc)); } +static inline bool zpdesc_is_locked(struct zpdesc *zpdesc) +{ + return PageLocked(zpdesc_page(zpdesc)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 6c5dccbc9a60..7b3344c226a0 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -895,23 +895,23 @@ static int trylock_zspage(struct zspage *zspage) static void __free_zspage(struct zs_pool *pool, struct size_class *class, struct zspage *zspage) { - struct page *page, *next; + struct zpdesc *zpdesc, *next; assert_spin_locked(&class->lock); VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness != ZS_INUSE_RATIO_0); - next = page = get_first_page(zspage); + next = zpdesc = get_first_zpdesc(zspage); do { - VM_BUG_ON_PAGE(!PageLocked(page), page); - next = get_next_page(page); - reset_zpdesc(page_zpdesc(page)); - unlock_page(page); - dec_zone_page_state(page, NR_ZSPAGES); - put_page(page); - page = next; - } while (page != NULL); + VM_BUG_ON_PAGE(!zpdesc_is_locked(zpdesc), zpdesc_page(zpdesc)); + next = get_next_zpdesc(zpdesc); + reset_zpdesc(zpdesc); + zpdesc_unlock(zpdesc); + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_put(zpdesc); + zpdesc = next; + } while (zpdesc != NULL); cache_free_zspage(pool, zspage); From patchwork Tue Aug 13 08:45:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761562 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7AB8C52D7B for ; Tue, 13 Aug 2024 08:42:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 591246B00B3; Tue, 13 Aug 2024 04:42:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 51E236B00B4; Tue, 13 Aug 2024 04:42:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3928F6B00B5; Tue, 13 Aug 2024 04:42:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 12D0E6B00B3 for ; Tue, 13 Aug 2024 04:42:03 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id C2B34A74A9 for ; Tue, 13 Aug 2024 08:42:02 +0000 (UTC) X-FDA: 82446579684.02.65438D2 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf29.hostedemail.com (Postfix) with ESMTP id C9909120009 for ; Tue, 13 Aug 2024 08:42:00 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=SyU1F0CU; spf=pass (imf29.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538451; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wvcMpJhOWVpL4p36y+TteGibialMwhqjG3mnLTxXOsc=; b=HbTcepD+iSFWDlMev4py6d6q0qNhkgObgvq8/ZZ1cSKssjsy/uwPvHP0H6jrYjZW40HTRT 4xRMIlZrp66w4mrGBzfmyjXqjD9IH4pfCVDdYuzlO4lOgOSFo09uK2mom1pcbAMw+GxxLA yh/kEavBSHP+19yy0WPjw3Tc1xiByUI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538451; a=rsa-sha256; cv=none; b=ahUMem0VZ0uISD41v9dStJnLJ/M42HoD6WgVZmePVtDYeXQG3GUsno15kkhiAHN7Wx6Szj Lt87Jv96QYwtL0bYA7W2Un/W6H0mjVcpjyIcfnWKE0PZ6F5bRSFeHiTFGZI5ajQjgBBjh6 YmX2jHk+HMumjlskwEBY9lXM3o3pRAE= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=SyU1F0CU; spf=pass (imf29.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id DE359CE1234; Tue, 13 Aug 2024 08:41:57 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 094C4C4AF09; Tue, 13 Aug 2024 08:41:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538517; bh=P2h4TL0al2Tdq16vEvHZlxJuPRha9tB1mdH4uW7GsvU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SyU1F0CUC5fJJ+b0xwqG/Ht51CluyUnYwwigzprVYJCkZsEq4VEoBHyVwwQ2VHi+8 3QoMrvaINRWhJ/usOIFbXtV2ZefP60g2/PcvhoM6VIgi9e9J5p0EcYK7sudn+/ET3Z BiXsmhroLMiO0q5npsqO+h9hJRPRkv/trQP27fIIiGhqhZJt+2OdqFtovOSWnue1zN yc4GNEfoV6BOTJicX27tXNKpte65+7eeh9T6ZKi1C+gw/Wt9/mMOG5Qx6CYVTu3ouo 1qWLwB/0gyxRxR3udXWKOGCr8+5kajpmf1mzvCscAOe45l1aezR3oeHw+vkUDjAd7d Vb9d+lE8wwL/Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 13/21] mm/zsmalloc: convert location_to_obj() to take zpdesc Date: Tue, 13 Aug 2024 16:45:59 +0800 Message-ID: <20240813084611.4122571-14-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: C9909120009 X-Stat-Signature: 78nuj6yp4po5uw8kpz64zczm64954np9 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1723538520-418094 X-HE-Meta: U2FsdGVkX1+MpQzbnDg/fmI/DI9TsjMyo1z2XtO3+giKB48NQU5HsEc679LG+wYlOuLvlrpwcrZxtFuuzJNOuHux7W2JpKVx9SyuU5K/EDTEBnRcJvb6SILyhUvi2dJA7W+FSYQFvkz5N0A+UpWY8E/oB4ejou24tPnqjROcW7myiSljwBY56/3KAVQ2vRw7covtTmFrQPDkUTXCBZKIUyiuGtQ7GeuiLoXNhLJQbZJMTdmveSGaQZvg3SsHPP58abkLMp6stvUqIXOo51iQtE7cSe8/Wp1UEW/eZjjOS9bu+Cd8wUyXm8JWr2JtUBimkjkQU6JtuEWNgLy0OKO6QP0CgfdiCYgskrXo6JhR8DXEI7UI6GDxPYPe79XNyp6LegqEh7LkFOKGQOhYuJQwJI6ARpeCViFSBJXj2QvfSoMus/6A50hDfMa7UrEGC5n5KWFKQPOjLJRYlfOZ9Iu9qeH5f4iX7LjrLfsrIMhnYH7C5RNmJ4SU37FKIV3qnX3+2+EYFg7D+7EsLQvItuSSrsvERd8Fa3i0dQRamDpL6+wr35RBapCZlqgQZSjXVtB3ssqDtru8dAQYIolmLN0m7CWz6YwzHLdDG4xCmn43yV5fUSwVUkFe6i1VFSxEPwwt6tGypCeq6hocvAC4R25S8OjnXcXY+Ka2jW1hDL/sV0kSZzwusYCjF44nu8bp526x3WViaNwaRjy+JeP4fKvLSQk9wvoOAan61mq3yJ9SsKdBdKUtU957Gwx9dpXQ30HUlqU+Z6jLDLJMZsOaBy3fIxqMGnIfAs/QETb7HotxUdFd/5/5PATsDWa/ghcYETirjm9xQKZ86s3oLNg8Sh7VxaTCrY13+KV5myJZHazHHoR6dJqmxyQJlvRJxHaSfGPTiMIwibYdcF4CQNyd8L6hYQvAwTyudcOTCcU06MkMnkk9sMYtMjwS9ZPcsLG2+vK/thWxTXHVWaAsWbNrC15 cB+JsQHW HGoJpSLxy9VnpCcBHEAb9DrQbjQWay4cNK51gBvSDjr0XSoox/Y8hLazGVlADgvf4uAPUMx2xkFSxS/4Bogc2IqeOpAlN8dbM0m5ZArULlgxvjj1tvcWQHh+/QlKL01grp/BSxocqc9q+PYlG2vzR+tmrZPBaMJUodyL37pz3wdJ7y8TUMJSWiRq99Ff5aQota76NhfbnF4D6r3F4kuN0DzJdHmbuDviD7+x72hYeuRhqrWK/u0UPqChUyl4UlVt4xbISVI87d46coGwm38A+D5GhDIZD6Bdm0CkzutUVciKRvYLhijB+QMN8/3CDoAdqHEmfRIet6GtAZFeKWBT004gv8J1DebdccdCHxrTNJKZ1DfXtxLeCUJ0nQw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> As all users of location_to_obj() now use zpdesc, convert location_to_obj() to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7b3344c226a0..9218f1e6e8ef 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -820,15 +820,15 @@ static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) } /** - * location_to_obj - get obj value encoded from (, ) - * @page: page object resides in zspage + * location_to_obj - get obj value encoded from (, ) + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static unsigned long location_to_obj(struct page *page, unsigned int obj_idx) +static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int obj_idx) { unsigned long obj; - obj = page_to_pfn(page) << OBJ_INDEX_BITS; + obj = zpdesc_pfn(zpdesc) << OBJ_INDEX_BITS; obj |= obj_idx & OBJ_INDEX_MASK; return obj; @@ -1386,7 +1386,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); - obj = location_to_obj(zpdesc_page(m_zpdesc), obj); + obj = location_to_obj(m_zpdesc, obj); record_obj(handle, obj); return obj; @@ -1873,8 +1873,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj = (unsigned long)location_to_obj(zpdesc_page(newzpdesc), - obj_idx); + new_obj = (unsigned long)location_to_obj(newzpdesc, obj_idx); record_obj(handle, new_obj); } } From patchwork Tue Aug 13 08:46:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761563 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9185C52D7C for ; Tue, 13 Aug 2024 08:42:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 828296B00B4; Tue, 13 Aug 2024 04:42:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7B2D66B00B5; Tue, 13 Aug 2024 04:42:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 519256B00B6; Tue, 13 Aug 2024 04:42:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 31ACC6B00B4 for ; Tue, 13 Aug 2024 04:42:04 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id DC5DA40712 for ; Tue, 13 Aug 2024 08:42:03 +0000 (UTC) X-FDA: 82446579726.28.679D5D5 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf23.hostedemail.com (Postfix) with ESMTP id 41AE9140026 for ; Tue, 13 Aug 2024 08:42:02 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=FD7LH1Td; spf=pass (imf23.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538466; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=R1Tc/xK+L9PQjZoUv+mESG9WEY9SW/PgJ93dHMXkRYM=; b=1L+rErdRhLZZa4FhLj9kSaPXvBQ2iIqFar3+tYLzHi2bggIcYYEWZbbqnjqjJYa60B0+lM mJq3z6R6lECBj8XHyyQ5qKoN4zany3aj6dJoz/KuJdlJnNHDqO/G9VlMDb41Q+5YTF1RPW dH3dgSRC4QOkOCNX6pwm9uyTatLj2Q0= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=FD7LH1Td; spf=pass (imf23.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538466; a=rsa-sha256; cv=none; b=O0DpO+VPRGpRNo4NG8jGZQO5KtBAy1ShoOHFtp7E0t+bauk0SB/oqonTXnogVFUpVw/Uq8 wv0CwDEzDyli0XdPs6Bq/zvVms+5Vcq0eSxSL7CiBIFzZ5Y502ah5Grsr4jdEjCM/3VcaZ rx3VMXBshX7Crjgb0sX+s82nZ7N4UZ0= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 855F4615A2; Tue, 13 Aug 2024 08:42:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 12BF9C4AF10; Tue, 13 Aug 2024 08:41:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538521; bh=lwqsbsNLYr/NAY4uxgyMtSwAZaoCij/qukX2cmqm4rY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FD7LH1TdysUcP3n1SbAE6auP5bdtAyVHXf3rUWPZVQR1EC39qS51lMmMomKTs3fvy 7fsB8mSnAG/hgU65T4GMmYCDPKayYBgRSRtWVmfvdU5gEtZA+kPGlUaeTzdQRp8vjf 4U7tvJjBF5qFb9kSP0xZrNyJTFcF8lSvOl0NmegGoWZBVevdSVEHorHaBIh5yFkCr4 3ob6BNnneg8KHjTpnx2o2raeE/wob3UTFXVqrK0UqM7kJtjrkHE12Xj9jXvQhYoeyQ ktd8xeRN0S6JfPDRmiWXuDQ0JLZs24fIZ42/mgUTS4NExhZ8tTjn+QBkifl5Mj3Nam btW7bLc+cfArw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 14/21] mm/zsmalloc: convert migrate_zspage() to use zpdesc Date: Tue, 13 Aug 2024 16:46:00 +0800 Message-ID: <20240813084611.4122571-15-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: ddj3xr5766zew4rhxkeq1p1cu7gsosg8 X-Rspam-User: X-Rspamd-Queue-Id: 41AE9140026 X-Rspamd-Server: rspam02 X-HE-Tag: 1723538522-693964 X-HE-Meta: U2FsdGVkX1+3rELWuXxiSd+0z89mhWS4s6U6G71jjVwX48VSiDXqimaeE7yB0m3zZeJ38KtMfuYjmUCZ0xsstWuFq/kOuwXDfcAF3VM7Z9sn3cefHzgwez5Rt3BlhEInFlrNssiJ3TX33hEgYJaHs3dcF3IsH82M/zeRfy0fCHQK5OMJuKg4LPctS8h9tqBAKO3w8/kiRwYydlHWcfyFoQLL72ltFlNiQ96Ohf/1VDCdjHDhXMv0fOKevYL5U923hTW6IWS8ic0VSfLveRcq+se+aJFE04dS6wGzkQmitgygfr+R0xNnHvjduvLVmjjGmbFYvEcguwOCKqdfBVyTfTtC8JeAUCGG4p7qKGwI5nruu5zuBcF5bg8z1JpSXyGQHmudyqO/4nxbQ/32AC2yNZ7dTc3wDoaKWiQ50373NMPOHlO5XskKwug7vYXQm8cLjqTe210hbKAMZ/ZK2hH+h+CH/ei3yN4VrQ7a8pk2UQMtirAnODtuwHcsSgS3jXr5D8WNsw1/nQs9Vu8Ujk/+UhzWTrmJh133PZgCnWThd8pCNxH2IgnwmVgoC4DmyNS3Bi5arUni26YODJ9v03E/3GBo6fnTtqt3EcOObhxxIhqbUD4TkaCe7o6K7PSWUEdG3bn/Dibxrd9dIBdUqI+e46I93rnQWd6NmAfw1BOGumOtOG+gEr9x73YZL9TbKkzsWDkzFW09bqKHqUwze4LvK4s2qdwpD0KlFoE+/BT61zxE9rHNCHpipadICDPnNjNF/7U6R+1UyolB467qwVcuC04EuhAeuGGsbIteD2eNbRt53V51mp2v+Y8cSj0/ozQaZeStuDLH16heGbD5BIuzZd5DxONoNrR0QZpuf6kTO6mDLD20TX/J8G/TKGpNeOAepWe9xwZ+GubNFlqXp0CYMmuglXdJQq/jAs+oUhyZmTWLbcZQCAhKIGztY87Mql7N9EJVV5uEDmNEApk1eTV aQkFTixL QujdrfLBpWp3w4j8Jo/1KhBWgaU8Yl0ELXGRzNmdQ7zG7GL4iFGN4kQ17O2Ed2LlJq1kg8b0NYxi7/hmMtuADnXWBBPjHvl/zzOnan6sdC0JT2xk+bb5F3R1f/hfXG2cBgMTr/JAoWdqTwe4dx7KIRtQXXdadkDLQOFQhbapYSltOT/4jhg/R90GI2iukmDDLVH4vLC+8t31t0qzJZUMc1IoN3SNtbCfkt/maROdrbMLqca0xqc6h6v88wW0r34YgdU4RUrg4ul4yWChimagfDkNKhMoglKYEMnqIB7Oi/vrqQJ2L8kBRfWtRk1Q4AcEk6BWXpG3Z82x79UJ4+bEJKGBVdIDHaeUWMogUYhcvYnfuRGRyTEVIYahjt39UX1AgwFGu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first/next_page. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 9218f1e6e8ef..f93af6e10c3a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1630,14 +1630,14 @@ static void migrate_zspage(struct zs_pool *pool, struct zspage *src_zspage, unsigned long used_obj, free_obj; unsigned long handle; int obj_idx = 0; - struct page *s_page = get_first_page(src_zspage); + struct zpdesc *s_zpdesc = get_first_zpdesc(src_zspage); struct size_class *class = pool->size_class[src_zspage->class]; while (1) { - handle = find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); + handle = find_alloced_obj(class, s_zpdesc, &obj_idx); if (!handle) { - s_page = get_next_page(s_page); - if (!s_page) + s_zpdesc = get_next_zpdesc(s_zpdesc); + if (!s_zpdesc) break; obj_idx = 0; continue; From patchwork Tue Aug 13 08:46:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761564 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29328C52D7B for ; Tue, 13 Aug 2024 08:42:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B36436B00B7; Tue, 13 Aug 2024 04:42:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AC0896B00B8; Tue, 13 Aug 2024 04:42:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8E9D26B00B9; Tue, 13 Aug 2024 04:42:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6C4986B00B7 for ; Tue, 13 Aug 2024 04:42:11 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2B1E61C30EB for ; Tue, 13 Aug 2024 08:42:11 +0000 (UTC) X-FDA: 82446580062.25.6063A7C Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf08.hostedemail.com (Postfix) with ESMTP id E54F2160008 for ; Tue, 13 Aug 2024 08:42:08 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=D0tJ5d2Y; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf08.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538472; a=rsa-sha256; cv=none; b=u6he7Ws13kVf+xNLjRGHfIzb8vziVG+9yTMzNwOolM9MHVHw4QaQ4J8ou7gz/PeZgaNq9g 545QukBlbilL37JdXEb+41Q99xFiE4xLMdg2flBjMuCI47UNizR3eOJaFNZvqe2oPinSOD FSwPPrnxMqZDw9d5s1w8IIFDokErkcE= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=D0tJ5d2Y; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf08.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538472; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=x2bsOUDxDA0avGMNOI+ZqM1BCFVQ4Kz7WTFkvpU5/aI=; b=mXr7op/VPbbEbbd17u3f8mmedh73ZDbX11RrVEWnwe4DZHaOXp8AAwJmDMrr3pq7mjFVWK Did+zBw3mLzkw5pBa+2EfQCYax57Qbfm0LEZ0Jyy/X9oWY6b5pip3JWaB+PKUkak7hG9iZ tx8hHCmzQMMHXRWFyyjdzalwiuvLKvQ= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 488B3CE122F; Tue, 13 Aug 2024 08:42:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E40DCC4AF16; Tue, 13 Aug 2024 08:42:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538525; bh=tL/aSxtJHYmt2JA8n2u9RLnt0i9TICyqYsVXNrqh3nU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=D0tJ5d2Ykx0EcLRUEijuPnSiFdQLUU+KiE+TEkgtbafVTWnhrhpjOOGIrKgUD+c1P nWObHJs0C3iwN/7tMgmGpNy89peRAfNOizZEYiOy6KG0gDzXJFn7hm3uhifphw5MqJ HpLxzjmtjt1nX7g1JiLabCtATwm1VUdkHHigB9bxgc3mDSZU1X8Iiwhm3iJOp0U3Og TZQDncbP6fbBm1G+6g3EGSdeyhoHL166jXO92b579SmM0/4viqqMgA4EXFUSis7uEd LUaTOvfVj/9xJVb+GTjsTyke/3ZEeF3t2nH4qXnAK6+sVBkbQnTxhYSP8Vmjcu3O4I Dyl83gBdiNbSQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 15/21] mm/zsmalloc: convert get_zspage() to take zpdesc Date: Tue, 13 Aug 2024 16:46:01 +0800 Message-ID: <20240813084611.4122571-16-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: E54F2160008 X-Stat-Signature: 4nyd37gadfnfu6bw6ntfys1stjqpf8h4 X-Rspam-User: X-HE-Tag: 1723538528-649552 X-HE-Meta: U2FsdGVkX191+YnB634MG4vVDID9i9in0PqngJNLwOX87rw/wAgLMnZ6F/GUeszQO0mQ5C2gRgZyM54oXqqg3SOCFftgbvnbINB234P6obCdFOSn4CPVvbd68NUBOrzzJPcSGcV9OLVJgSM7TRiJIhbHtJ8mmX2NHYSbkAOpW14n1OcQ3V+dUKAEKauE8oCutZeC741E+Mcq7lJI5hJQL7KfHELpNBUwwRlw4MyAXj2cpsIXN8UKfcjYv8HDCm3aYMOtSha/0DBL3NhETT+un5Ldv1mnxxZEJtV/6D0IXtmjpBIuhjBhNlFyzI4TPsjKrZGsT30nldJrYNWTQTLCvZ+W6GSaIBFjGYxvrwmXU92hI76+67p0DPXAMDqzl7jfbPy05aZXr2/MlEuB74GDQsB2G9EXFqlxL8fjc7j9pu9d5ZLF9gQvCavS/Vwj+GPBCUiRnf9aBFcFksrJ5nRpMPPWd5TDXNTEl1hcL4HNUf4ugKXSuWah9bKmOGLtLl7BMMqfJ3dgorNIB1QzGOtFx57mDo8FVCyUAdQmHXmpEM1DzgcqKf9emxhreEphq9IdMHndP05Ska9eBn8jG4jmByZuWb0iaX+mqi/5/R7jV3t4XxcMAXQeKuno0aohztK/O4CVJAQzPH7P8IzAQNo8ia4Jg7ISK3omjyOsSsJuu5eri0RHG0P1mTjplE9vXc+OhZoMkG3zkCeV6aKKM0bTE4s/sxLsdS87wNarKsRgWzMbdJ7qS7I/wUCULI9GVHvswkN/ReI9TvCtkhwCwtFltLmXKoVOp5FziC+CoOzmVRBToPOGfsCb1f2v8nm8zS7fiK6lDeCUI4es9QdYCGXA3yVMb8FhopT/ixYEK890bXP0LyZPCjkAtNR+V0UMc7f/lJzJHkI2DaDFKqkBMZhCpXmZ2R1+yuqD+rN6IEVykGV2jtecnKcIflLSIyOhnImaItCsl91VqGBphQDqC7c F77Q/60e gSDqUqkQ5t3Juxetki+maTKeG4+0+MyBjdjwkIvFKvk0FDV+AcgJ2mbH3FVCsWePeQd01gTsoK32naBNAzXOs4VgYERmus+Vlg4/C45AzFfY5c2lpzdlNrr3MT5+z7C4ZxS1P48uCY24kKo2z1JFosFG1NZaI1sk3+OlcHPITsCG2FAF5JFO+ZyUUN7TPAeb+0BxvL3ZKCW90TUZqMK2LtF3DQKPbQzAOkspyxqSALuULd09nVHgh+o5mJyuN6+Xn3A905vp3mmW3I2cHvq1oexiCT/z6rLlW1eChN0pYUS9fhi4b2x0kz+GgbLAC0Y2SxLeKW+fDjd+i5kKwcCzfWs3ZIrDw5l+/3j0M0aqFuKRtqomTvjDoRsmi+w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Now that all users except get_next_page() (which will be removed in later patch) use zpdesc, convert get_zspage() to take zpdesc instead of page. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index f93af6e10c3a..5e96ff2ee0d4 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -773,9 +773,9 @@ static int fix_fullness_group(struct size_class *class, struct zspage *zspage) return newfg; } -static struct zspage *get_zspage(struct page *page) +static struct zspage *get_zspage(struct zpdesc *zpdesc) { - struct zspage *zspage = (struct zspage *)page_private(page); + struct zspage *zspage = zpdesc->zspage; BUG_ON(zspage->magic != ZSPAGE_MAGIC); return zspage; @@ -783,7 +783,7 @@ static struct zspage *get_zspage(struct page *page) static struct page *get_next_page(struct page *page) { - struct zspage *zspage = get_zspage(page); + struct zspage *zspage = get_zspage(page_zpdesc(page)); if (unlikely(ZsHugePage(zspage))) return NULL; @@ -793,7 +793,7 @@ static struct page *get_next_page(struct page *page) static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { - struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage = get_zspage(zpdesc); if (unlikely(ZsHugePage(zspage))) return NULL; @@ -843,7 +843,7 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage = get_zspage(zpdesc); if (unlikely(ZsHugePage(zspage))) { VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); @@ -1259,7 +1259,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1309,7 +1309,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1473,7 +1473,7 @@ static void obj_free(int class_size, unsigned long obj) obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); - zspage = get_zspage(zpdesc_page(f_zpdesc)); + zspage = get_zspage(f_zpdesc); vaddr = zpdesc_kmap_atomic(f_zpdesc); link = (struct link_free *)(vaddr + f_offset); @@ -1507,7 +1507,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle) read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); obj_to_zpdesc(obj, &f_zpdesc); - zspage = get_zspage(zpdesc_page(f_zpdesc)); + zspage = get_zspage(f_zpdesc); class = zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); @@ -1840,7 +1840,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, __SetPageZsmalloc(zpdesc_page(newzpdesc)); /* The page is locked, so this pointer must remain valid */ - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); pool = zspage->pool; /* From patchwork Tue Aug 13 08:46:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36CFDC52D7B for ; Tue, 13 Aug 2024 08:42:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BF96F6B00B9; Tue, 13 Aug 2024 04:42:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B82416B00BA; Tue, 13 Aug 2024 04:42:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A22DB6B00BB; Tue, 13 Aug 2024 04:42:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 787EB6B00B9 for ; Tue, 13 Aug 2024 04:42:15 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 39E16A069D for ; Tue, 13 Aug 2024 08:42:15 +0000 (UTC) X-FDA: 82446580230.15.C352BB3 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf17.hostedemail.com (Postfix) with ESMTP id F315D40016 for ; Tue, 13 Aug 2024 08:42:12 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=JgfTSlvZ; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf17.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538522; a=rsa-sha256; cv=none; b=mblkfmrdt9yACE9un+miM/RagdnmGaQD3/MNLMiquKIfSI4iUnBhS8TdmwiJHJJ5ziz+45 cjYNrXopqqh8xdBEm/g6PhvCY8vdgVsMEIItmX31am0vvTIWijb7MvE3D5IQP46WDCvVs7 QQJYliyZJ8NeBns7LKyZ44loqBoLP6U= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=JgfTSlvZ; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf17.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538522; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=N43L6InCiBMwZBnzz7kfNDs3dEtiPSvTUSj7c7OvkXc=; b=ftC0S3TYvbkSlnO3dtAkBM3Nx4pL22yDp/KE2ihtaD03IKjYaCYdGb4QXKJNxwY1aP+9j0 JiJAHgsuo58QO4UGKrF8bL0HOcCSnDCgkg0416KhF1ZDJyq8SQBr1job/ekfotxeCSUX+K C4QDky+3GoXtx7S6n6vvr84Xx89QD3Y= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 2C566CE123D; Tue, 13 Aug 2024 08:42:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EE86FC4AF13; Tue, 13 Aug 2024 08:42:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538529; bh=gQaMHcDtxoBXgRT3p7vOtyKNFbzrfkZ7CKreedzRtjs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JgfTSlvZRX5rj4F+je0IgQVoumPPB1HFUsceRotVPVhoikowdDEW0P2qObUTYI398 gR1/EIMADa8MqSXujMKyHg/EExmLFsocPWJg6SoEbAOwiulDmKDAQOclW/XRzGG0uC KpWTOdWx2HgFPAywG6qxV7VN5PH3agkB+nina86PjV5/5qTIA7ebSafJN3QNkipVW5 AUs2ctwy+2+ZBzAftUKxISXseq9T18Wg5QL6U8S+DU1CjGcZJJOMb1wab8R8Mn1GEb IzCYbHJRKIqxQVx1HPqmCnGxr5TG8ZZr16pysTaSatTJE36v7R6CxlKYcdWuIyKVIE HNnVoo5DzUppA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 16/21] mm/zsmalloc: convert SetZsPageMovable and remove unused funcs Date: Tue, 13 Aug 2024 16:46:02 +0800 Message-ID: <20240813084611.4122571-17-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: F315D40016 X-Rspamd-Server: rspam01 X-Stat-Signature: cf3xfqufnbweoothrd86jumy46mrjn3w X-HE-Tag: 1723538532-673725 X-HE-Meta: U2FsdGVkX1/jDSwSEA9mD4axtWsCZfR4X8616fhARl/T5N8/FNy/UuHsicPubppPimdXhSs65XTvpv4VBm+t581NTnIG3WSKJdP3wxlBjNFKg0PZK0fHxMEvGIYxf5gMw7Vv5KuoAAA3le+t3WSIqS0VJK2Yhm603385IWZA2ctB7zWmIlzZFP1XWQBRsozkDSwT7clu2iY8XkbBhGQphPAh+EXYP0+BGEwvTfDlCqGGL3WfYGFQI1nvxBv8FeI99CqB8MCc2CL02Fq2IW6/BWyLn4PzzIuQLxA0i02VDiabPt0J0TN7mbduPG8yVCH1FWX4bTGisBdpE2ZmRXQOT+lJjwj6affHCK42TFEH545/5ZCfTarzmNUZUcrVSSygY9kBiRSurF1BBS9rYjnfwT6prxssuHeWHgrVEhDSBk9RjrdjmxZq3Gb7q3Jk16P4N5lGXStL2iYuGLQAYaMoYhPAjv71LY4U8m3UgsyOss8pGCtCdhmOIu3I31BuVxTyOIO8Pyriih0RKEcuZvQZ2qcG5wGY7GEiqBwKWkNjzyA6GmqZbSGuQgorOxnJbQipULBKqVOkQHZ+IwH9TAqCOj4J4eYBSJPHCv+YJoCQ6aFcih9ac/tzMoZ72K8uTNee+RCfK+FazrP6zlnpogYruvEncxEtP7OvoaZHXUX/yoirf0UBP1lw8pMNJ6RifO04lhf/FbIoZSN9nevFbmfbFmo78jPlMzw4ECJmVRyFd7VH1pQaw2rrwwsTUmlRgPVzzrB8DDk8VxS2vibF0JsfsoUaZ6vkybgvm9efs378tqmdnXWbDLu/lna3dgRlSq5bOs92RnnbGxPp+eFxRxu1hsLMXVzu0gj/pb7DYtCHOc1yjvaYtSy0nKTRO7aR3P2GNxePEDJEMenRItkIbIR3fPCMlhCjEy6TeKQqmWpzOGVd9ZRsbHHlORB+6BsyvsEuYCcPpaD+VE5UKeypBAw 3pYSp938 LJjQAUYo2LLLjVKAp+esp1HVTFJl21l+KrqvkS0cIdbzyAGkYSakyzbY+grufj9xkN6l1MQ4ApINK9j2MRxRrQRSi8XAD+CUCQifFw+0q/sUWwX/5U53ogycFQLPiO21Xo44h3dcF/jHVn38wPiwfEVFhwLE9q5gSrglbsjA5i7bMU3G5/TvxIlW/PDq4ZKhpBDsSNQKwG2q38LvKH+aiAGthf4MexZGcf4HwmjiAk9g0lzpoQTUSelQi/RnOxFwC91lZga8XKM77a1l/6pBb/7LsDpOhNzLWwlpmCxsGVsGQgaHABDjPFyHx+zWZF9+Etfq54tIgAJzKG3Q= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Convert SetZsPageMovable() to use zpdesc, and then remove unused funcs: get_next_page()/get_first_page()/is_first_page(). Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 33 +++++---------------------------- 1 file changed, 5 insertions(+), 28 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5e96ff2ee0d4..e2b203db192a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -462,11 +462,6 @@ static DEFINE_PER_CPU(struct mapping_area, zs_map_area) = { .lock = INIT_LOCAL_LOCK(lock), }; -static __maybe_unused int is_first_page(struct page *page) -{ - return PagePrivate(page); -} - static inline bool is_first_zpdesc(struct zpdesc *zpdesc) { return PagePrivate(zpdesc_page(zpdesc)); @@ -483,14 +478,6 @@ static inline void mod_zspage_inuse(struct zspage *zspage, int val) zspage->inuse += val; } -static inline struct page *get_first_page(struct zspage *zspage) -{ - struct page *first_page = zpdesc_page(zspage->first_zpdesc); - - VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); - return first_page; -} - static struct zpdesc *get_first_zpdesc(struct zspage *zspage) { struct zpdesc *first_zpdesc = zspage->first_zpdesc; @@ -781,16 +768,6 @@ static struct zspage *get_zspage(struct zpdesc *zpdesc) return zspage; } -static struct page *get_next_page(struct page *page) -{ - struct zspage *zspage = get_zspage(page_zpdesc(page)); - - if (unlikely(ZsHugePage(zspage))) - return NULL; - - return (struct page *)page->index; -} - static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { struct zspage *zspage = get_zspage(zpdesc); @@ -1964,13 +1941,13 @@ static void init_deferred_free(struct zs_pool *pool) static void SetZsPageMovable(struct zs_pool *pool, struct zspage *zspage) { - struct page *page = get_first_page(zspage); + struct zpdesc *zpdesc = get_first_zpdesc(zspage); do { - WARN_ON(!trylock_page(page)); - __SetPageMovable(page, &zsmalloc_mops); - unlock_page(page); - } while ((page = get_next_page(page)) != NULL); + WARN_ON(!zpdesc_trylock(zpdesc)); + __zpdesc_set_movable(zpdesc, &zsmalloc_mops); + zpdesc_unlock(zpdesc); + } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); } #else static inline void zs_flush_migration(struct zs_pool *pool) { } From patchwork Tue Aug 13 08:46:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761566 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E01DC52D7C for ; Tue, 13 Aug 2024 08:42:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C70F76B00BA; Tue, 13 Aug 2024 04:42:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BF9906B00BB; Tue, 13 Aug 2024 04:42:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A74DD6B00BC; Tue, 13 Aug 2024 04:42:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 83EF16B00BA for ; Tue, 13 Aug 2024 04:42:16 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0F2011406D4 for ; Tue, 13 Aug 2024 08:42:16 +0000 (UTC) X-FDA: 82446580272.17.AC8EEB6 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf04.hostedemail.com (Postfix) with ESMTP id 611B440007 for ; Tue, 13 Aug 2024 08:42:14 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=cyXENoc4; spf=pass (imf04.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538522; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wV7BSr/1QairbT3fg8+MgBmQkzh01Sx27nNz+bgrgRA=; b=DyoH5TIPeYh16Wne7/fMWCYvjiWnQO9vzm374+K9Pg4Mg2q3GyZCMsCx/h/Mg2EOfvrOFS VGLFOYVgoe/LIp4xPraJYDZW1qpgunwPIWg5xAhaa7gW/GXoThUMHcU48lCSuyfqp/MgWp Ypv3YcHioSqwGPDRIUFR1Hs0h7ooWw4= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=cyXENoc4; spf=pass (imf04.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538522; a=rsa-sha256; cv=none; b=dF+8FKxRgWc7AHaP7JanQmd+X8SpY/9XkXgxo8jyKNpoaFVFIKkALI/Ny2SAbn1pY5wCAM cjP3N8DblW7UlIOwvaMG+3wL9Up8rg3Rqe0ybquCgonf/hT24eOI5Bctb5WwYaiApNXNkk N/QBdZOcegAfB1Jtn5QdbGRNGwcK+bc= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 9A6CB615A6; Tue, 13 Aug 2024 08:42:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F3D94C4AF09; Tue, 13 Aug 2024 08:42:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538533; bh=8TsEc1mQDjgsj8r155q+1srlMOAnmBBOJU38nCFETQE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cyXENoc4P62M2wfhgRRIdhledFsy/PPHOdILmE+RD6XzAahA3OzA3VF9DM6zOSCoy u1k5uThAaRqIGvfMJb3qwlxKDH5Bc+jo/ELrtOcius+pSxrZu+unAkJohA+TEjsomK Q9ugK5Af0stkeShlVpT7BbLXOsslLhd2x80E336Te/5DhiYu4aKBmX/nCYEFTpkeuJ ZGWQv62YyFDgnMRa1mnUu7X2rkdLlxqjuG+p0kUVj1S1T9k3GPAssIrfEgYe78oyib IWEIkNs5axTKVNgVzYx7J00DrOdE1P8HL+qy/HVN2sz4Nl6Sdwmq4j96BmCBTeY2wJ qVDdhTuO1dZAw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 17/21] mm/zsmalloc: convert get/set_first_obj_offset() to take zpdesc Date: Tue, 13 Aug 2024 16:46:03 +0800 Message-ID: <20240813084611.4122571-18-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 5u48a5hjk33uka3h67687izoft4eh8cw X-Rspamd-Queue-Id: 611B440007 X-Rspamd-Server: rspam11 X-HE-Tag: 1723538534-367161 X-HE-Meta: U2FsdGVkX1+QGsGl0G4vRt6tEwW3EqUdWHw1yOqGisV1G3MCdpPeivUhrjhEX6f5MlOfkXVPZVOmDkQVX+iL6xW9BEYMobDahTAgEBv1jTmq0vZH1bqGoYMn4/WTQ4QhsHErzCOE1etsUR+NRrWxp/e/J/5ILhEAvGavx6lPi8jNRorQfXrlXKBnLXfISFF0QMx74SdA0aON7PcNojauSvqDoCeG8Jcg0zwDSavh2KZU3GXcUOfYoMePu8YY+SI6CD+eoOyy2Ljp4MOQwATWBkGRYP4SvL6i7G/De8QXks/o0lF9wg0IV54PJmfPUHKnScdW/DzmBFJRMVvDEK1h2jfYvhJotLKFELcHmr2Ez+FyaA9N2J5r3iBHlwQQMkXxtzqQ8oUg2MlrZHrXDhSEdKCsmNT5IKBVaLdXyT6p975NWPeoift+JqX0QedNWn67+Xc85z4UH2pCVnmSUhb4zIuXXp6ooBlTQjkPCnscf5yqi9hkebGSey8+uT0ckYMH5OabovER51Vys80Ab/ws32Zfzo8YQSyVbYiLo26w56lEIfwBqTqJuAmMdGYktljnXuJ6HHcIkyFuosmhCDgU7qJYjXBVKI+0GbCfcOlUGBxh9KBWqyPNUJJD4f7yB2vhYcOwrPPeTP9RwaiIjNDDxTBHMMwh0FbOJYfjNbJ/KDvuU4XF2r8gFBb+ki/9j6ncNnGtobVmaKRshUQ+qhMAM1ioTp/cwlxeQdSvxFaAwZNIJEVreFvmc6jOAKkDf/Isq6oPGNM0gJQrDRdWsnO8FAnxKFpRHJrh6lIgbQxvi6M2Enl6ScmwGqAzwl6/FKCmVUdl8Pb5G0uT57imbx0hR8eE2jV2qThkZkS4Zhi4bzDSjURtxcd+9jiXKppTBNSQqnigzk2Vc0mfPtiYtz2looE8vcy9piBs3v66jeSL6b33MkBAEqCytba12pqh0ZOKUPUurcXArwvbtzwUjd5 LjW+ioKS s2qguHhDixbR6k/tyWobWtF0aGULPFR3+d8eFfq5SZx9lmocFQOa7hpu2tsROhwXXxNxhzvQnldoG3c/WIojF/8Gbn/LrwGsAehcZj+NfZvaIVXN4hmaBDWsiiPKy/3Uns2jyv2RHmBEFrtacdP/rsrN/KaDwNhekh4xwhi5YaCgrDQcswdsK2GGyfh0CaBbaqlEUvy/eNIwId46t8aZJ1g0RDP0r319E0DTof3Myf/ODy7ObT7fmi6/RzEY9oFBxg+XgdePQ2X8WKKCgKeUocnvgrzpEctBcC88DiQdPCn5zkxKi+kFuAYeFblZ2VJhUjN8Ip5luZJwJ9apXEfuukRJS6bSsFfygiomBcx7yMzwCMExKNYbArToLr8tanH1LUw76 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Now that all users of get/set_first_obj_offset() are converted to use zpdesc, convert them to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e2b203db192a..8553542edacb 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -488,26 +488,26 @@ static struct zpdesc *get_first_zpdesc(struct zspage *zspage) #define FIRST_OBJ_PAGE_TYPE_MASK 0xffff -static inline void reset_first_obj_offset(struct page *page) +static inline void reset_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - page->page_type |= FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + zpdesc->first_obj_offset |= FIRST_OBJ_PAGE_TYPE_MASK; } -static inline unsigned int get_first_obj_offset(struct page *page) +static inline unsigned int get_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - return page->page_type & FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + return zpdesc->first_obj_offset & FIRST_OBJ_PAGE_TYPE_MASK; } -static inline void set_first_obj_offset(struct page *page, unsigned int offset) +static inline void set_first_obj_offset(struct zpdesc *zpdesc, unsigned int offset) { /* With 16 bit available, we can support offsets into 64 KiB pages. */ BUILD_BUG_ON(PAGE_SIZE > SZ_64K); - VM_WARN_ON_ONCE(!PageZsmalloc(page)); + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); VM_WARN_ON_ONCE(offset & ~FIRST_OBJ_PAGE_TYPE_MASK); - page->page_type &= ~FIRST_OBJ_PAGE_TYPE_MASK; - page->page_type |= offset & FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset &= ~FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset |= offset & FIRST_OBJ_PAGE_TYPE_MASK; } static inline unsigned int get_freeobj(struct zspage *zspage) @@ -844,7 +844,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) ClearPagePrivate(page); zpdesc->zspage = NULL; zpdesc->next = NULL; - reset_first_obj_offset(page); + reset_first_obj_offset(zpdesc); __ClearPageZsmalloc(page); } @@ -928,7 +928,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) struct link_free *link; void *vaddr; - set_first_obj_offset(zpdesc_page(zpdesc), off); + set_first_obj_offset(zpdesc, off); vaddr = zpdesc_kmap_atomic(zpdesc); link = (struct link_free *)vaddr + off / sizeof(*link); @@ -1583,7 +1583,7 @@ static unsigned long find_alloced_obj(struct size_class *class, unsigned long handle = 0; void *addr = zpdesc_kmap_atomic(zpdesc); - offset = get_first_obj_offset(zpdesc_page(zpdesc)); + offset = get_first_obj_offset(zpdesc); offset += class->size * index; while (offset < PAGE_SIZE) { @@ -1778,8 +1778,8 @@ static void replace_sub_page(struct size_class *class, struct zspage *zspage, } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); create_page_chain(class, zspage, zpdescs); - first_obj_offset = get_first_obj_offset(zpdesc_page(oldzpdesc)); - set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); + first_obj_offset = get_first_obj_offset(oldzpdesc); + set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) newzpdesc->handle = oldzpdesc->handle; __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); @@ -1834,7 +1834,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); - offset = get_first_obj_offset(zpdesc_page(zpdesc)); + offset = get_first_obj_offset(zpdesc); s_addr = zpdesc_kmap_atomic(zpdesc); /* From patchwork Tue Aug 13 08:46:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E99DC52D7C for ; Tue, 13 Aug 2024 08:42:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2AAE76B00BD; Tue, 13 Aug 2024 04:42:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 233086B00BE; Tue, 13 Aug 2024 04:42:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0AE646B00BF; Tue, 13 Aug 2024 04:42:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id EF3EF6B00BD for ; Tue, 13 Aug 2024 04:42:22 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A4F0B406F5 for ; Tue, 13 Aug 2024 08:42:22 +0000 (UTC) X-FDA: 82446580524.04.7C20EB7 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf17.hostedemail.com (Postfix) with ESMTP id A70E740024 for ; Tue, 13 Aug 2024 08:42:20 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mxpHjX6D; spf=pass (imf17.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538506; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=D5AsasRwZQtsJ8WKNjhf3ld5eMYx0ChSPMHeS3q+gqs=; b=x2eylKXCkuNNUM53LtEE0itezp9NDHm11jbuztQe0+IeoHxL/UM5KjOB3t++88kmcmAkUs ISpvqHAA25p1CANkiHOJh4ujq6Y4P6Ia4yViohWMWOFUmtljdUhkavfH5bEI/25cOfdgTy ttIQPm3+UAqfVbedCzB6tK08eSn4iec= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mxpHjX6D; spf=pass (imf17.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538506; a=rsa-sha256; cv=none; b=4USO1Z6yo1leeN1OeR48wOLmHO2VC/sW2OjRazLS37DhX4/pzaaFCKXuUv+7vQqCq2V1qT xNHHiCl0I/4Y+MJujzZc/gsThDXvYEKqAW65BS9n/9B2fEe3xMRLfudAoY+hBdHvyepEX3 Lw5+lasy2+ccIJKl+bkvyRbKJNfqSzY= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 1BAC2CE1232; Tue, 13 Aug 2024 08:42:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 053CBC4AF15; Tue, 13 Aug 2024 08:42:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538537; bh=xc1UgoycHzcMSLrT87O9Jysr++5t0k6ZYOyK9RWDCgc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mxpHjX6Dipw7WJGW9x/CzrdZjjx3J2swcfh5GtuduL50FV7HjHZSe+ro4MjvMCUyA PcMTPHKXGAN19nP01URDq5t9Iau8uj9OiNhIq5TxCQZY2yn6OSUljkeCv2ZtSh4LTj u04TdFn20fcNRN0p5vZ+wuShi2vcaTbze1tOo/JWqVvoHy8OyGilw1ZzeTE1cSskNZ 4JHIFSL7oKHyZIeEv5GStb+5Rn3G3hQdbzyFDPCE5QIgtRD1iQFYBOSbujz//injjQ G8i7trnT5ZET9efzMHq8kwGTqu7jjEMeVLgB09lYALGXQLbWfO/eCg9vElqgQcPykN cD7JvS6W96uRA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 18/21] mm/zsmalloc: introduce __zpdesc_clear_movable Date: Tue, 13 Aug 2024 16:46:04 +0800 Message-ID: <20240813084611.4122571-19-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: A70E740024 X-Stat-Signature: zpxbdfjgoscdy31e4wc5rd18ugr34srr X-HE-Tag: 1723538540-481771 X-HE-Meta: U2FsdGVkX1/O4VzOftTky+1oCSlt3yWW2VaO9ngxDWAnPaGakHeMYI3u2G9lgGnBOjPZU2/Rye9soYjFklGiyCh5YDLURN2TPcbhQZ6V30szOe5wpsR/jLd4v9HRgaNy8uALdN/RXmeTp1yn2Ui42Mwj/qOdun1+uK11PxVs59O7e79hr+uyZD1AzlPWCt5ZynvIhvTNrb4qrAwg57RXfxxQiHuUqV0u+Yft79r3dJnnM3Ow25KG0iCB7dN07tzBFRAp0+QLTWTHsZMqmUZx0a+tucsdAWSm8LAxH2t0ctVweBI6A1FzwZEzXZHy1DYGPuVL24kRdU8IwUGXEL4F99wzTOY63uKzh3Sqam03RLe7W4W/wObqJkZeoMwqHRxWLZ5ZYM2zU5GuE8RpLmCD6RYzOeJkapoEWp9a/0q5Sx3it7XHSItwgRVjVsmF2pv/3eOStFOamHG3sLOllFvzZdpQsn5meSQD1T3Y/GSSs6kvD3xkQmb9hrW5GdopaF6QMxKZ33AQN6zZX+oBq4ZpGnLGMud20eFJ1lrQ+28j7aCMI/7IPT1SaKl4YTLRCE6iZ4ZJHIE8Oc6c5PSbsGxbaRQnib4FTMgl4kZeL2mbGKPI68CyHqdpO4stFYyWWClvdujhRDeqJR+B/YwhHH+kvXYhZqVvf59WyPVIp1qpT2gdkw1X5MDhRcFJ8tB0VBO/RmZPBH214c1r2LUcCy2TFRaGiSY4TWk35PRuKxTVp2KG9jssDxTcOyfhj7gDfv9CJ5rNYXK9zbP5lqHdL/hItkRDzXqlsccEyxROf/pAK9xs7wLo19Gzc0EF3OJ7SzGglJp7EDOGBxuPX2ZBIMvhafjM8AMnVcaGre28QLxPJeXRqUBP6rInL6P6xol/V5hHWFkh3t1sQVvS3Nn6zd8zOXWNyXpSS3Eg7kkXtRWnC9GMKVpOEVBo6xSuB5d0JMlOpTRHPEJa4u9GGekv9X/ kA+Rcyv7 sHjM00Ldwemt8QJckH6V7BVnrlQMrc3Du37FGr7a7yr/JUxy4CpHHoHMm5qHzadOgB2ph/zFq9/Vn/HBZZlh2dNFEV7eNcu969Eso6GfxsIO/QsZsjp/HWqaqIvGHgmJxAiGMQ/lwwoPT3IW8IG76TWTVuWzt/Hvn/XQ8XpWmv/hFwtKPCLdr8cf//fJdP+QjV5YM2BGpSxWZ0Vqj2k2JDy/tsAKyeqkTUdMYKK1WHSSEmRFv7OvT1lWTSqfWgYj4f717W7CjqvABOA1m0LoTkqn1j7KyNkK0AGjD5qbOiukq7XDrbxAh+yXjlDnI6bf9YpGP8aL7aCPD+Fz7cVGl/lzxRZAs5T1bT6m1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Add a helper __zpdesc_clear_movable() for __ClearPageMovable(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index a1834d36ccfc..747a2d410a35 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -115,6 +115,11 @@ static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, __SetPageMovable(zpdesc_page(zpdesc), mops); } +static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) +{ + __ClearPageMovable(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 8553542edacb..64b9ea011111 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -840,7 +840,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) { struct page *page = zpdesc_page(zpdesc); - __ClearPageMovable(page); + __zpdesc_clear_movable(zpdesc); ClearPagePrivate(page); zpdesc->zspage = NULL; zpdesc->next = NULL; From patchwork Tue Aug 13 08:46:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761568 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 538C7C52D7B for ; Tue, 13 Aug 2024 08:42:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8500B6B00BE; Tue, 13 Aug 2024 04:42:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7D8356B00BF; Tue, 13 Aug 2024 04:42:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 651E46B00C1; Tue, 13 Aug 2024 04:42:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2FF036B00BF for ; Tue, 13 Aug 2024 04:42:24 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E4B701C30D8 for ; Tue, 13 Aug 2024 08:42:23 +0000 (UTC) X-FDA: 82446580566.13.878A730 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf22.hostedemail.com (Postfix) with ESMTP id 42FC2C0013 for ; Tue, 13 Aug 2024 08:42:22 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=e6kjY4QB; spf=pass (imf22.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538471; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dTXvl5YBfUlmT0//8duifZ2I6YpaXJ1Po/eYt0e/ytY=; b=VtcKUMFAFkZElT7xXALsa9obwD06txTqI/CJmZISMsEUUn/oK657NfB44G9dLNjmvww/an Y3mZqvyC0V8GfYLXEZWzwPnj2D3v7Ik7z1u/wSq7MlrvxrnDIhRr8m4ZX6u0dggJzRBjet j8qyvUtoetYIFUzO/eYs1QDMh2COJRw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538471; a=rsa-sha256; cv=none; b=UYxh4c6W6BA8R/h15AySdJAEPFoSfO4psX8DVvvae44LGP2vK/wQZ3T2CkV2GqsJ67xE/i znS+8onbGuQ73SptdpdiJdbvF7pwL8f2jmaHTACQjmkYuEI9L1xlaL+bHO9hHwAQNGMIlJ wRgqa1qekxgztw+JhdtHB8YyNlIUu0g= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=e6kjY4QB; spf=pass (imf22.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 79255615A6; Tue, 13 Aug 2024 08:42:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D4173C4AF0B; Tue, 13 Aug 2024 08:42:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538541; bh=7RDgs8VMI/6+eF2Z36b9xbjlJF/pcQYAzN7TZVLEquw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=e6kjY4QBWwfbNWiHe8VElP9KCnrcfr8ExLDYLTSGXAyZUKoHjuxSHsysbcFQW3sNz uJOJTUslbpGJu3jmOoWVmgIAI7VIY/0WRpaO/7Imdj3ZxKk0jNedzIdBKEcvY83QS9 YY9frlOlRS6WTCJCdPM6OB/PEnbl13D0XWfGM1Y8xZvUWK03GC3pmj/TKRyuK1gziy 8l38bTjL3SNjTgdTJmSzABhQZadMZ5IlqQy5ybd4c4HkOqJ7kNnzf/l71n8EdBaMeO 3HPTf+qZgh21Pf9MA4I3Qsixy8tr89Dq0GHnmIOqIZ049Ah98ulIdocJHoRVakaR8i p9USBCGEmOmCQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 19/21] mm/zsmalloc: introduce __zpdesc_clear/set_zsmalloc() Date: Tue, 13 Aug 2024 16:46:05 +0800 Message-ID: <20240813084611.4122571-20-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: 6ikeibdbtkzpp3efuh4uq7mohek43b4b X-Rspamd-Queue-Id: 42FC2C0013 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1723538542-876044 X-HE-Meta: U2FsdGVkX1+6LfBUb++dlSmJIcxVyjs6DVbmICd3DkPzlnRD1xpJ4eE/5BSP7KXeJ3TlVEfi2HPj2oi0ny3DLgMU2MAgmpwzbsh785n1+i5IsHzTaTWhBnl41wEpr3py3zdHz7nD7t/R8p4anv3LrkZB4el/KUvUZhvxNgojDSbbsS/qK6LkTWRK03AYtHrRviYyZW9l7NEaI/piC8cvkcY+WK59uBxInlWgPeDT/l1svQxgPjoA6ardq2NTfs5z983wacpWyIDMyImlFlXyUYs3YhhjenAdCBdHlI9EChwt4dtJOgjzbkFDeThPf4NmzCvF1z7M7EQV7WmxKE349xR1PQxkR8WFdXDxiaP77MfZ5U6txAgH5atzvxrKDBUVngl7sz6/NxaeJn+nik3bSt/oJdbKthu/ebHHfyVkz2A1WxoGpKNtzROuSYTb3YIk8Cpl6jXPvnkYv+aRrcTceMeFp7v1clIJd2LrCWDzzFX1IdCPACgyFclXxAk+1cXNbIpBeqzCJv/r+LuJOdosuWgLVmIKDC0qR1xFZCcFAB8++hSm6/HssbFKausmcCOyGdZ6Dw/fnf9N4HNqv5SgxcrPTOVXNz5ZWuY8bUmqU+Fh51sPTqA/JjdnHX4C66IXjwSu1yDfX6VUyiejqDF1oA+KxkrICiBe218gRtXzu5mD6soMDrqQXu1CQEjtfqOq5nfWz2B2N69uu/KlITDt5rbEUIeXeZF7Axts1XCTVDQgCfl6/3UnfmaMrtWfcwUG7Ekxvz3dUJffP0iM+BiXs80eywBF/JL7Y1T/z9nGleRrhuyJFNbPR9hTw8za0Q7mGqb1CQdX/mYX2Yhu2AKIjY2IxCQNBx/1cbCj0zggfpnHXAl/H1B62JexbAPWli+HRKea+yi46Ap3iNeQwqG5eajAbag/jzoxp/MB98FZ7afa9fPHHryMl4W6kGqXFx98YkAMgY3FwoC2zs6TSG3 RZeC0FJ1 BAG1cgWV7X205zuxacpPCPsgvpGb6D2Fkms9UqhLfTXAktaHg2KESErtGnSJzwNOn9I5iTCYzPy6rn4XhEdhduxZzH0BV9sbSiyHCwgckaOstQ9ZYx1MMCjq5AtE1leTNsfDOAf7W0M0y8B9IqkXuOJXsm3A8ov8fe5UwwgoEJbQElkpVaiWmiJeehOh56qW+GIfPl48O4ewlLq/+FaTACAinMZpEWk07BeTsXoZniODFDFI/O2y34jOm92zPHjQhADUSt56Xv4ogSTYJNrQV0wrdydwXONfuH3X4rFXIs2G2BhoZyUhk75ouWVwG1HhBW7lCNQtpPPxOIU7aDjtUlgAFUXHP9UTbLQYcRFev/pPX6xc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Add helper __zpdesc_clear_zsmalloc() for __ClearPageZsmalloc(), __zpdesc_set_zsmalloc() for __SetPageZsmalloc(), and use them in callers. Signed-off-by: Alex Shi --- mm/zpdesc.h | 10 ++++++++++ mm/zsmalloc.c | 8 ++++---- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 747a2d410a35..33f599081281 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -120,6 +120,16 @@ static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) __ClearPageMovable(zpdesc_page(zpdesc)); } +static inline void __zpdesc_set_zsmalloc(struct zpdesc *zpdesc) +{ + __SetPageZsmalloc(zpdesc_page(zpdesc)); +} + +static inline void __zpdesc_clear_zsmalloc(struct zpdesc *zpdesc) +{ + __ClearPageZsmalloc(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 64b9ea011111..6e7cd5acf5e5 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -845,7 +845,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) zpdesc->zspage = NULL; zpdesc->next = NULL; reset_first_obj_offset(zpdesc); - __ClearPageZsmalloc(page); + __zpdesc_clear_zsmalloc(zpdesc); } static int trylock_zspage(struct zspage *zspage) @@ -1018,13 +1018,13 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, if (!zpdesc) { while (--i >= 0) { zpdesc_dec_zone_page_state(zpdescs[i]); - __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + __zpdesc_clear_zsmalloc(zpdescs[i]); free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(zpdesc_page(zpdesc)); + __zpdesc_set_zsmalloc(zpdesc); zpdesc_inc_zone_page_state(zpdesc); zpdescs[i] = zpdesc; @@ -1814,7 +1814,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(zpdesc_page(newzpdesc)); + __zpdesc_set_zsmalloc(newzpdesc); /* The page is locked, so this pointer must remain valid */ zspage = get_zspage(zpdesc); From patchwork Tue Aug 13 08:46:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761569 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6EA9C52D7C for ; Tue, 13 Aug 2024 08:42:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5B9AC6B0089; Tue, 13 Aug 2024 04:42:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F1A06B00BF; Tue, 13 Aug 2024 04:42:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 395D16B00C2; Tue, 13 Aug 2024 04:42:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 15F606B0089 for ; Tue, 13 Aug 2024 04:42:28 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id BF4094071C for ; Tue, 13 Aug 2024 08:42:27 +0000 (UTC) X-FDA: 82446580734.22.C964827 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf25.hostedemail.com (Postfix) with ESMTP id 2B676A0028 for ; Tue, 13 Aug 2024 08:42:25 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mejI+7CX; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf25.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538467; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Gdw27DXKth1M/PuiOaBe0oIyfvVPsznflJoc+OFbnwA=; b=mXUH8QsEP3NDsMwQseeTzJUC5J4b2qEMbd+GRsbU7bGUS9V/AeP+6EYKr61BSZl/dLGT5I Si+bmwR90+y6jXEcjya1Ma7SYI6wkhOv9ce1p/YH3ZdFypXlh4X9X68nvLw7cv1RdEJh38 lov7xJ5OojIgIcSrX+rqTch6bWMPtxc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538467; a=rsa-sha256; cv=none; b=0lGrGmeevhqnVw6aTMENwVNedPDjCLnqHUSphAXlYIb5hXVA7RFqdtQTx1F2mI9Ryr15aO RMCT50vG99Xz/Pu1DpvCh4w2lvrWnBz12iRRNl2I56KUNpIncqGOwXYfZz9oKl28DXd1LZ CHHOJpnmLPULe0heYjrKzoBc61w9YEI= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mejI+7CX; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf25.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 535A6615A2; Tue, 13 Aug 2024 08:42:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D7BD2C4AF11; Tue, 13 Aug 2024 08:42:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538545; bh=GGradnuK41+byBnHyefAqZD3LeHddFsDEO4CJxk+f0E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mejI+7CXbaNN6QrWv+gyGHDhIeialjnXVUEkoTuuLiEMeYDjaTjw2VDn0SOFfEsh+ vkOy9tq9upX8uw+PGZwrRA7TL02hSe/SSLQu9ho+WBZ/gGhSAi0deOsMzgTWJj0qFK bAPIKaZJNX23wmCL/wjJgJyg+taUjPfBiJicIxiIJnRybb3u38WhJyPrZjzcKFdfnY dJhLNNOuTQDiHAN8zzMxGCNabpnbvFHoiTF8PrCj8dp/ZbafQ/mgFOcm1LRBCL0w4N TAkESIteUK13GxF+JSJWpscWFt5ewHxF8BP4wjSjo24OLkNaugLTprwReAlL22HdVS 6lhrf5QpTGRjQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 20/21] mm/zsmalloc: introduce zpdesc_clear_first() helper Date: Tue, 13 Aug 2024 16:46:06 +0800 Message-ID: <20240813084611.4122571-21-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 2B676A0028 X-Stat-Signature: j57h3gmzeigwm88db17f6dokm4mwonoy X-Rspam-User: X-HE-Tag: 1723538545-359740 X-HE-Meta: U2FsdGVkX1/DOqbyNeVKEVBae2nbcRxXAOQbHGkCeTvNcJyGvXyzj11oppdkw6JA3PqJexC5rV2wGDp7ELasq3g6Sb3wLWK1m8MBU1sIBQdUrsZ4Kej6rgpXU9kzNqITpCIzmSbIk1m6+OsC9idQAWIdN1gIeYT3+H0Dpq+gVbzIXava5b5nL4aF2hNKLWUjJn/0hvOtXpwqFH09WsFfNO/rzlP0/rLn5ZJWvbNz/pLNExZij3FUlUfrksRoE34EEVR5wdScAMZB60W1m+KE6Ojx7hdfG55b2AH8i8xi7rl0PxWL98O0TsuQahfWOQU1mfbT7pqjgpJQGnzKF9K7bXCp5G/eLkoaYMa5Q3ECP20TQR91ztjn7Pax6rQRAr8/F3GvygUoU/aCsjHevbxl+gjRVEM6Vkh+8QPNxqHFaiDVyT4ji+d/wApYoOyaas79c+RkreUbiXGNcZvdzRfxBhG6eWIhiULxs4x/x6jce2gQ7QrnuQTiBZKGqXCzsIw9irCKfwTDr9L9fdNaU1kekleeDmSVG/zRPpJGlZ73Ew6QvIHXUMADogHwOGrJqpsoxuTsee/dAy1RISxKNSU+FDzuW8ahTyJ+DfzSKvIpnK32xGZ9uDlcQMnHOPRDFBP31GdMvruBTtjOIKeQm99MVgwHBbPlDF2OmX5P8GpBOv6v/BoOKX9fElRMRr8g2QC/yBPaw9V4V3rJn6No64qpLPKmpsvz0a64v9z3Tlhh/RRhv8U624XGIZ8efkLpkThmlMYf3KoKKxbPMm1o8hr4lG2XnIqj/0s88GCixYG8/GGBqLdj9qqWoctqGHYl6Ps9qImAD62mo3RP0Z744ecfG2voD0KfE77icjCWK2J9gP8Fde2RpwK/yFdoCE/BU2idBgcho3VKK0KH1N9HSvm1/ibaUSKL+USl48JwzctqtQf9ijhcq8OC9eWjZkCDdJxrVv88Bs1NowIWjLdqQBF AskHodGX vidXVfpkBaRB1H0Uk/MsqPUBVB7lAR4D5XU25oHwSeJ3u83p2Imq9DRyx+EWEOcjjhsgm54q4OAa1wgn5PvPCzhlIHNB6YDon8eeDHAhjF1/9eTTon6qfkK/cMJQ8FF23Ps++F3/ml3mcQTSxG6HngXPR4uis10eSUXHNe26sd1Eon/fIQslfQlvIPYQhf0T8kpcyqkP8V27WCanyllY82eAJ2xakfrEA0ffYuAUOJQiMZFGe++tHmO6JstRhZU9vBgnEBD0fVTDXwIjQ0Et32mfJd/qaO8zhod+9spULJ/gYr7oYly9Fs9WGzKRVObMUV8XdvKnEOmUQrxq1xFH9kjg/vUUHw97O6LjM X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Like the zpdesc_set_first(), introduce zpdesc_clear_first() helper for ClearPagePrivate(), then clean up a 'struct page' usage in reset_zpdesc(). Signed-off-by: Alex Shi --- mm/zsmalloc.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 6e7cd5acf5e5..3dbe6bfa656b 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -253,6 +253,11 @@ static inline void zpdesc_set_first(struct zpdesc *zpdesc) SetPagePrivate(zpdesc_page(zpdesc)); } +static inline void zpdesc_clear_first(struct zpdesc *zpdesc) +{ + ClearPagePrivate(zpdesc_page(zpdesc)); +} + static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) { inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); @@ -838,10 +843,8 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, static void reset_zpdesc(struct zpdesc *zpdesc) { - struct page *page = zpdesc_page(zpdesc); - __zpdesc_clear_movable(zpdesc); - ClearPagePrivate(page); + zpdesc_clear_first(zpdesc); zpdesc->zspage = NULL; zpdesc->next = NULL; reset_first_obj_offset(zpdesc); From patchwork Tue Aug 13 08:46:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13761570 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21739C52D7B for ; Tue, 13 Aug 2024 08:42:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9FDC36B00C3; Tue, 13 Aug 2024 04:42:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9AC698D0003; Tue, 13 Aug 2024 04:42:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8262D8D0001; Tue, 13 Aug 2024 04:42:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 6285B6B00C3 for ; Tue, 13 Aug 2024 04:42:35 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 26B1AA06A3 for ; Tue, 13 Aug 2024 08:42:35 +0000 (UTC) X-FDA: 82446581070.17.5EEB86D Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf12.hostedemail.com (Postfix) with ESMTP id EAE8D40018 for ; Tue, 13 Aug 2024 08:42:32 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Nmh3whJ7; spf=pass (imf12.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723538460; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+l1XQTB6Updd1BLYoUSDckX2L2bDz03eAO6ZtUzDD7g=; b=RJvYA9eO8LlpeyrfzaQ508RRD/mMoZvqtGSP7ayTRyGmzvlvv0f/0eIxiUTTookgd+/xQg MPjgTp3r5t+N0hs3z68K4Pz54lP4nmOsaP/opmCCjFJNPNbxsrak4Q1/OtnQRf289Y8D1e s0zi4qqH2jdty8KezK06YgDiaLX/wU8= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Nmh3whJ7; spf=pass (imf12.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723538460; a=rsa-sha256; cv=none; b=weC26//VwgFBNCBAAGB0iUQTuhlqkFCVx5uoSPURAHET7tPow7CqUfZlpEfLqv2rvOejwb hwGNwMFLtAxGZCNL/ISdV2rk8Q3wCLWe8xPJCwmTglbMoxzKpQGcAUfeMcMQ43FdtV6cP0 g3sLG/WBs0BMyKE64B0Yr/FpvK6kSTE= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 2F097CE1234; Tue, 13 Aug 2024 08:42:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B22CAC4AF0B; Tue, 13 Aug 2024 08:42:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723538549; bh=LliY088wtfrFrkEgAuCs8QB3X7nNhs8HeH2Nz6Y9mEw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Nmh3whJ7Km/PF9o1d6vjECtU9Q80lnjNCgYP9PH9/NyPaAXEUHXt67dUXagF2kxJN iRDLDAo3ER+9SiJxE5n9vGjNdT/ystypfHXN7rz2cDxps/B76wJ1GXiO9x7Svsqary uIeHzEY7e575DxDpwD7wIG5T4JVhCNm2AuRWmHrw4z+gMy9A7dSCRxFv7E1XjDJFyl 4aJmKyjQ4Miw0W6rZon7W/bWTIIPlWusKE7wnMSAoqCIsVfv2RBunbaMWUR2kJNKLl G5Ckm23NitU1OBGjeMAD202eP27uIhCa7dduav/25zMWGJX2zZuJJuyj0w8f4xmKiU yrOb2BecslR9g== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v6 21/21] mm/zsmalloc: update comments for page->zpdesc changes Date: Tue, 13 Aug 2024 16:46:07 +0800 Message-ID: <20240813084611.4122571-22-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084611.4122571-1-alexs@kernel.org> References: <20240813084611.4122571-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: EAE8D40018 X-Stat-Signature: h7a4w1tj6rhkzkyyzfx79f4wmx5gwefx X-Rspam-User: X-HE-Tag: 1723538552-867904 X-HE-Meta: U2FsdGVkX18jzmMXwhWkkmKegqq8pTTWAb6Chi/8tzdgGL/7LBZdloVbo1Ogb/+8ugFF21s++5tTuH3hV6uz8kX0gGiV1+b6TUDPemZg68R54ihYzjyPPFcoc/ptD/tNe9KcOHvg6zlsqOKhNBv1ppNaSJM2pAWaixAWtIy3lzFnjE/WNCa8FwSfsNLKM0Tv3Basv9D60/JOhizE7b5vK0/RYId1z4IJGv2HrDixy66908y+XB7kGA725cUD6YyuIMR9gPSS6NPSUlzyZfXAN27GFYrv66NQQXmadUAzqOZ9H6d5xpIADEB8/3fJhsNhJ4loKbqMVbhA09iYxH/GBcnoBXAje2dcKwtaI+sDvfYHdYj8xwk/NMN6fwZWc8Lmd56cOWGTVTkvFT8vmPLqOXx09CMe/TcmxmyjoYNyIHtODkoKwXLTFcoZvebHNvsuTtK6rXyaw7yoTgWkTXybR83YzfE1eYoQ/mZJTtxsUi4vCwJ4Y9SXpeMfumhdRJ9tUtotcKmE6NWL+h0VHPb3psBozpYntAOK1Pb9zz+fdJjNC2HiegMhlzjtQgzW24FFUNTfy252RNerFE4PUG8FuPR7H99zzaOKjG2M+QYexvcUROSV54XJTFtBhdYRnAplK/8D8Nz2X9ZWu8VpDkhKXX2+VdSaLWaTjDBjQygU3h/CbHj627QN+U0TSpnHaVM81KL40Q+LNq7f1UrXDshZ7VNRT9w/BvsZ8BsftZ9E038CPanKqL9R9wYEk+LNOV+qgb6NHdIhzk6duMTe9RlV0uGZmp+vE3HFGpoUttXlDnnOQL7osENzpTlYIXD+W7CZU0kE/hRLLoo4C/l7r2RcubwdAd5txZ/Rr/h5m6/KEYQERMagQfbJVJGSpmmB/mUU+5KchdFye3S9ujbxoJefKdftbx+Zt+A0spOJ2zQfy/ABK8RUnqcFEBybP+FoHugBPY53uFV3flB4DaEp+8+ 3e9UMyJg NQpNtleS9zYKJfBS1nmz+wfhboZ4zVr7cle6myQ75xhMUGFSBZZG1g978aEuUx6NKjo+PfT1AGC0TWsIQl4i/xtn6D8hb4l4zRISYcucCR3unvsjov3uIhzBED5kgfwNFDGDpgQrGm1y+Fxu0cP9dCHofbqAaQuFpCHRNJqz21dZNblMacO52TEWC0nnsRHKz/IFlELsGQ0/x06ObjhMGXTF8Z9bcRCqunSuQEEx6yVH8i5TDDm1Aw2uzZS2jFE4z9xTMrZXnETq3BO/87/cnx6KpkvIt/thpfwsWa9hD03ejp1m12S0E/7xEUg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi After the page to zpdesc conversion, there still left few comments or function named with page not zpdesc, let's update the comments and rename function create_page_chain() as create_zpdesc_chain(). Signed-off-by: Alex Shi --- mm/zsmalloc.c | 47 ++++++++++++++++++++++++++--------------------- 1 file changed, 26 insertions(+), 21 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 3dbe6bfa656b..37619f4b074b 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -17,14 +17,16 @@ * * Usage of struct zpdesc fields: * zpdesc->zspage: points to zspage - * zpdesc->next: links together all component pages of a zspage + * zpdesc->next: links together all component zpdescs of a zspage * For the huge page, this is always 0, so we use this field * to store handle. * zpdesc->first_obj_offset: PG_zsmalloc, lower 16 bit locate the first * object offset in a subpage of a zspage * * Usage of struct zpdesc(page) flags: - * PG_private: identifies the first component page + * PG_private: identifies the first component zpdesc + * PG_lock: lock all component zpdescs for a zspage free, serialize with + * migration */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt @@ -191,7 +193,10 @@ struct size_class { */ int size; int objs_per_zspage; - /* Number of PAGE_SIZE sized pages to combine to form a 'zspage' */ + /* + * Number of PAGE_SIZE sized zpdescs/pages to combine to + * form a 'zspage' + */ int pages_per_zspage; unsigned int index; @@ -907,7 +912,7 @@ static void free_zspage(struct zs_pool *pool, struct size_class *class, /* * Since zs_free couldn't be sleepable, this function cannot call - * lock_page. The page locks trylock_zspage got will be released + * lock_page. The zpdesc locks trylock_zspage got will be released * by __free_zspage. */ if (!trylock_zspage(zspage)) { @@ -964,7 +969,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) set_freeobj(zspage, 0); } -static void create_page_chain(struct size_class *class, struct zspage *zspage, +static void create_zpdesc_chain(struct size_class *class, struct zspage *zspage, struct zpdesc *zpdescs[]) { int i; @@ -973,9 +978,9 @@ static void create_page_chain(struct size_class *class, struct zspage *zspage, int nr_zpdescs = class->pages_per_zspage; /* - * Allocate individual pages and link them together as: - * 1. all pages are linked together using zpdesc->next - * 2. each sub-page point to zspage using zpdesc->zspage + * Allocate individual zpdescs and link them together as: + * 1. all zpdescs are linked together using zpdesc->next + * 2. each sub-zpdesc point to zspage using zpdesc->zspage * * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). @@ -1033,7 +1038,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, zpdescs[i] = zpdesc; } - create_page_chain(class, zspage, zpdescs); + create_zpdesc_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool = pool; zspage->class = class->index; @@ -1360,7 +1365,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, /* record handle in the header of allocated chunk */ link->handle = handle | OBJ_ALLOCATED_TAG; else - /* record handle to page->index */ + /* record handle to zpdesc->handle */ zspage->first_zpdesc->handle = handle | OBJ_ALLOCATED_TAG; kunmap_atomic(vaddr); @@ -1693,19 +1698,19 @@ static int putback_zspage(struct size_class *class, struct zspage *zspage) #ifdef CONFIG_COMPACTION /* * To prevent zspage destroy during migration, zspage freeing should - * hold locks of all pages in the zspage. + * hold locks of all component zpdesc in the zspage. */ static void lock_zspage(struct zspage *zspage) { struct zpdesc *curr_zpdesc, *zpdesc; /* - * Pages we haven't locked yet can be migrated off the list while we're + * Zpdesc we haven't locked yet can be migrated off the list while we're * trying to lock them, so we need to be careful and only attempt to - * lock each page under migrate_read_lock(). Otherwise, the page we lock - * may no longer belong to the zspage. This means that we may wait for - * the wrong page to unlock, so we must take a reference to the page - * prior to waiting for it to unlock outside migrate_read_lock(). + * lock each zpdesc under migrate_read_lock(). Otherwise, the zpdesc we + * lock may no longer belong to the zspage. This means that we may wait + * for the wrong zpdesc to unlock, so we must take a reference to the + * zpdesc prior to waiting for it to unlock outside migrate_read_lock(). */ while (1) { migrate_read_lock(zspage); @@ -1780,7 +1785,7 @@ static void replace_sub_page(struct size_class *class, struct zspage *zspage, idx++; } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); - create_page_chain(class, zspage, zpdescs); + create_zpdesc_chain(class, zspage, zpdescs); first_obj_offset = get_first_obj_offset(oldzpdesc); set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) @@ -1791,8 +1796,8 @@ static void replace_sub_page(struct size_class *class, struct zspage *zspage, static bool zs_page_isolate(struct page *page, isolate_mode_t mode) { /* - * Page is locked so zspage couldn't be destroyed. For detail, look at - * lock_zspage in free_zspage. + * Page/zpdesc is locked so zspage couldn't be destroyed. For detail, + * look at lock_zspage in free_zspage. */ VM_BUG_ON_PAGE(PageIsolated(page), page); @@ -1819,7 +1824,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* We're committed, tell the world that this is a Zsmalloc page. */ __zpdesc_set_zsmalloc(newzpdesc); - /* The page is locked, so this pointer must remain valid */ + /* The zpdesc/page is locked, so this pointer must remain valid */ zspage = get_zspage(zpdesc); pool = zspage->pool; @@ -1892,7 +1897,7 @@ static const struct movable_operations zsmalloc_mops = { }; /* - * Caller should hold page_lock of all pages in the zspage + * Caller should hold zpdesc locks of all in the zspage * In here, we cannot use zspage meta data. */ static void async_free_zspage(struct work_struct *work)