From patchwork Mon Jul 8 06:33:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726190 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06C45C3271E for ; Mon, 8 Jul 2024 06:28:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0DD5B6B0098; Mon, 8 Jul 2024 02:28:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 08F1B6B0099; Mon, 8 Jul 2024 02:28:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6F5D6B009A; Mon, 8 Jul 2024 02:28:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id CC3AE6B0098 for ; Mon, 8 Jul 2024 02:28:52 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 7C1E1A1168 for ; Mon, 8 Jul 2024 06:28:52 +0000 (UTC) X-FDA: 82315607304.03.F65A9DE Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf03.hostedemail.com (Postfix) with ESMTP id 3B38D20014 for ; Mon, 8 Jul 2024 06:28:49 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="iCMJo9/v"; spf=pass (imf03.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420101; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MqmHqiGVOjk5tqr/pWwNbPensb3fB2wbEZ4KMjtWBKU=; b=ogYxKfGp6LQPFfrRao5fnD2+zgamltWs6upvMXZDSyQcxfJU+ZMxVgc9cWd1Typ6zojn// VZwEW5VMLQZS0gsuvO8oy82+t3WejaDkV5rulEcJRZcLknRCFN5fOHkyT6ohh3LjGg0Pcw lOReDaGC3MqJegLbwO2Jkw/0jn498UI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420101; a=rsa-sha256; cv=none; b=jQ4YGqXQhNE72PF0s8ynO1lfatv8HVpILyJQV52k2pMwDkqekft6THmLbF6jk4Iuv5Y5xR VWKbkawFwRzvuPUJ20VGxjlOiZGezqYWQ5NkgIQkGKhS3Ao+NtsJ4Hmw0A3visJFjyWt1p 0/etXN8b/4VyRWbD3/Jg+mUzuVJO6OQ= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="iCMJo9/v"; spf=pass (imf03.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id E9029CE0A89; Mon, 8 Jul 2024 06:28:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2B319C116B1; Mon, 8 Jul 2024 06:28:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420125; bh=QwfKZsyWKxmeoIlS8kXArrgVhPn5VQ2gqaeTqGCWsTw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iCMJo9/vSPJSaeCJLkrFmQWg/h52GHkcjcL/5WtAlAGBH2Z4f7gCFaUwqdR2huO8c Hwmd96q7j0coqLb4+LdNtXrNHxHdguBuIYmjcIhrtNwx13cWhtxmySF5Qrz/+kUbxx 7TzYzdagLRPIChRZyMH0KvB7i5rlieVZ2Ru7AG1kQHyKqV9i6t4CIGb6XfjX8oFJ5Y J/TDVncXA3cM0o6WbE680pKILmn1EN73ArAVKswe8XIV6yt6Px/OwQejHwmv8tNvwH FJnQyCW7Vx36BZdU0k8z3rko9+zlTh6ZqXsDTpwqXafp80NJ8VnP+r0C/UxT+b0ZGD wYjHHXaqvBdIg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 01/20] mm/zsmalloc: add zpdesc memory descriptor for zswap.zpool Date: Mon, 8 Jul 2024 14:33:22 +0800 Message-ID: <20240708063344.1096626-2-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: 5o93ds96kxcnpxnm4hx69pi8qub6snyz X-Rspamd-Queue-Id: 3B38D20014 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1720420129-585344 X-HE-Meta: U2FsdGVkX18Wzjpl0RctvD0WdBaNH3Ol0afomLRbpxPckaSzdsipeI8rOpgRah1eclfuc71qQeDtCDNhtmjKUeg/6seO6IKDMXUbMx09m9nNBGFQ2QkEzO6cahZ6mq75Sb9A6vRKvEMC8OV/9pCtMz7wVpF7RGhP+ZfmJmKBT300JxGgNt+DN6Lso3+bh89yzj0MTOF+upZlcb01vX5ogA9Mwlp3CkySMeXSQTsCOmdBka3tPB7+MM6V3CWb1ooi0eXPt9+q1O0WhE7hrAizjn1CXtktO+9LVARFny4w/FMrH6O/31erZqNxLpNpjImQnv4TMfej/3uQdHuvwXMbIlLNSV6aJlH35TF0qU4J5MK9mUqR3jtSeVNyDbiM8xIjtN6uLf4/HXAORH3IMDqUck5JxPG+Ixhbds7iExTtIWISdDzs6wRVDzsQb/1c36+xaPyS8RJCXtnH72KlI6bLKzv/hKBHxg+Pc+9zlbfGsBipd5c2QSn1rhyZ0crkGr8qUwC8QMnHbHMErxZvX73RsR/rlknhc4ECTs47m40Ls2a6gZywzy4rGMPV7smwWWC8slZoZIMBxtSNKSphACJVDpi/MzKe5tX3R0fUBSsQydFMHC92zEVg4Eu5laWnNTupWohT44JrCANaG0TKXgmNQgbasVAX9q5i6hPL5zco6krw/gp5vm6wU1K0wLhEPM7RXl1ITPxl4PCTMzu5ODUN/KSTIRJtj71KZn7RWXnD7LD6T749/y0hM4sumxBO/4l8ahABWmEbb8vCMYqXKOJVFmo9iu3G3LXg7hROGB243wzLVuD5SBk7kZK3rZWnu+pgo+CkKCrggjwq2I/N7Wvf3/GLJOwSH7YwUHbz+f0Bqv6yu+JCy7gwStiuqW8bukmMKawZxNR3U7r3D+TkIO7HND1PIwOirFkKw4nEK3rAxuJtd6Nxf5aOhjJG7mSOOQcNrVC+jZ5zsp0j07ewxK6 t6r7kmJB ocap7nSd4ZBMIgKZJ2/lA2nTZdHwB9L7PkPq/8XajUQ2hcBmNJQyp7gDgx/d0ym5n/euSRaY/kcWuFu+9s+XrpOc61lRc0J0hJx3AhjrjleWzMnnkui2afs8hCGvMIwmAF1aeon9YlIO2/kPVfYXm3Bkt9HjoI4T4P5BcEgonqTsEnaSE79HJQLbbzf/3hp4NXXqBziDKUL/BH95AL6ZDxOVEw+zPT/6t09NYsm46GH0IKbkfDOTtzdaccw8LMLJ1vrFnpPHaLrAJLlhscbvAST6b1J40Z5yV7TPiAZrnBddUz2QrLQwzWxdPBZGJt4cCsy/cYi0MApYpEC8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi (Tencent) The 1st patch introduces new memory decriptor zpdesc and rename zspage.first_page to zspage.first_zpdesc, no functional change. We removed PG_owner_priv_1 since it was moved to zspage after commit a41ec880aa7b ("zsmalloc: move huge compressed obj from page to zspage"). And keep the memcg_data member, since as Yosry pointed out: "When the pages are freed, put_page() -> folio_put() -> __folio_put() will call mem_cgroup_uncharge(). The latter will call folio_memcg() (which reads folio->memcg_data) to figure out if uncharging needs to be done. There are also other similar code paths that will check folio->memcg_data. It is currently expected to be present for all folios. So until we have custom code paths per-folio type for allocation/freeing/etc, we need to keep folio->memcg_data present and properly initialized." Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi (Tencent) --- mm/zpdesc.h | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++ mm/zsmalloc.c | 21 ++++++++-------- 2 files changed, 76 insertions(+), 11 deletions(-) create mode 100644 mm/zpdesc.h diff --git a/mm/zpdesc.h b/mm/zpdesc.h new file mode 100644 index 000000000000..2dbef231f616 --- /dev/null +++ b/mm/zpdesc.h @@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* zpdesc.h: zswap.zpool memory descriptor + * + * Written by Alex Shi (Tencent) + * Hyeonggon Yoo <42.hyeyoo@gmail.com> + */ +#ifndef __MM_ZPDESC_H__ +#define __MM_ZPDESC_H__ + +/* + * struct zpdesc - Memory descriptor for zpool memory, now is for zsmalloc + * @flags: Page flags, PG_private: identifies the first component page + * @lru: Indirectly used by page migration + * @mops: Used by page migration + * @next: Next zpdesc in a zspage in zsmalloc zpool + * @handle: For huge zspage in zsmalloc zpool + * @zspage: Pointer to zspage in zsmalloc + * @memcg_data: Memory Control Group data. + * + * This struct overlays struct page for now. Do not modify without a good + * understanding of the issues. + */ +struct zpdesc { + unsigned long flags; + struct list_head lru; + struct movable_operations *mops; + union { + /* Next zpdescs in a zspage in zsmalloc zpool */ + struct zpdesc *next; + /* For huge zspage in zsmalloc zpool */ + unsigned long handle; + }; + struct zspage *zspage; + unsigned long _zp_pad_1; +#ifdef CONFIG_MEMCG + unsigned long memcg_data; +#endif +}; +#define ZPDESC_MATCH(pg, zp) \ + static_assert(offsetof(struct page, pg) == offsetof(struct zpdesc, zp)) + +ZPDESC_MATCH(flags, flags); +ZPDESC_MATCH(lru, lru); +ZPDESC_MATCH(mapping, mops); +ZPDESC_MATCH(index, next); +ZPDESC_MATCH(index, handle); +ZPDESC_MATCH(private, zspage); +#ifdef CONFIG_MEMCG +ZPDESC_MATCH(memcg_data, memcg_data); +#endif +#undef ZPDESC_MATCH +static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); + +#define zpdesc_page(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct page *)(zp), \ + struct zpdesc *: (struct page *)(zp))) + +#define zpdesc_folio(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct folio *)(zp), \ + struct zpdesc *: (struct folio *)(zp))) + +#define page_zpdesc(p) (_Generic((p), \ + const struct page *: (const struct zpdesc *)(p), \ + struct page *: (struct zpdesc *)(p))) + +#endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5d6581ab7c07..a532851025f9 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -13,20 +13,18 @@ /* * Following is how we use various fields and flags of underlying - * struct page(s) to form a zspage. + * struct zpdesc(page) to form a zspage. * - * Usage of struct page fields: - * page->private: points to zspage - * page->index: links together all component pages of a zspage + * Usage of struct zpdesc fields: + * zpdesc->zspage: points to zspage + * zpdesc->next: links together all component pages of a zspage * For the huge page, this is always 0, so we use this field * to store handle. * page->page_type: PG_zsmalloc, lower 16 bit locate the first object * offset in a subpage of a zspage * - * Usage of struct page flags: + * Usage of struct zpdesc(page) flags: * PG_private: identifies the first component page - * PG_owner_priv_1: identifies the huge component page - * */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt @@ -64,6 +62,7 @@ #include #include #include +#include "zpdesc.h" #define ZSPAGE_MAGIC 0x58 @@ -253,7 +252,7 @@ struct zspage { }; unsigned int inuse; unsigned int freeobj; - struct page *first_page; + struct zpdesc *first_zpdesc; struct list_head list; /* fullness list */ struct zs_pool *pool; rwlock_t lock; @@ -448,7 +447,7 @@ static inline void mod_zspage_inuse(struct zspage *zspage, int val) static inline struct page *get_first_page(struct zspage *zspage) { - struct page *first_page = zspage->first_page; + struct page *first_page = zpdesc_page(zspage->first_zpdesc); VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); return first_page; @@ -948,7 +947,7 @@ static void create_page_chain(struct size_class *class, struct zspage *zspage, set_page_private(page, (unsigned long)zspage); page->index = 0; if (i == 0) { - zspage->first_page = page; + zspage->first_zpdesc = page_zpdesc(page); SetPagePrivate(page); if (unlikely(class->objs_per_zspage == 1 && class->pages_per_zspage == 1)) @@ -1324,7 +1323,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, link->handle = handle | OBJ_ALLOCATED_TAG; else /* record handle to page->index */ - zspage->first_page->index = handle | OBJ_ALLOCATED_TAG; + zspage->first_zpdesc->handle = handle | OBJ_ALLOCATED_TAG; kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); From patchwork Mon Jul 8 06:33:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1507BC3DA42 for ; Mon, 8 Jul 2024 06:28:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A79A86B0096; Mon, 8 Jul 2024 02:28:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A29F26B0098; Mon, 8 Jul 2024 02:28:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8F1B36B0099; Mon, 8 Jul 2024 02:28:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 716B96B0096 for ; Mon, 8 Jul 2024 02:28:52 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id E15C9811BB for ; Mon, 8 Jul 2024 06:28:51 +0000 (UTC) X-FDA: 82315607262.03.136B7D2 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf01.hostedemail.com (Postfix) with ESMTP id 3A0694000D for ; Mon, 8 Jul 2024 06:28:50 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=kpT8bdSg; spf=pass (imf01.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420116; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gvGb8PrTXpPglDqMeVvJFhDAzrZyeHYoETDvh74GHm0=; b=EBBLa6KejH6XSY3t52ss+Y8cQO74qfAdKYHCmT/Z6ZJMMGXCsELhLcUfdWix1wdeQVeGqL yXoYLb8/P4QbSMyX/HxQgz0M006cgAfK2zcsvk6TJD1mwUSBByOwR8SelPtPWSXQpsIWuc ddQOTYvLLwvjcsaLdT31QLB4vjPllao= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=kpT8bdSg; spf=pass (imf01.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420116; a=rsa-sha256; cv=none; b=eppO9FYoT0KkA8hfktzuDGPNDRVRopbz7IWo2JdvguzX0V/mnlNURX0ns+FHz4VmNUojgm tdu1zALsLV3pYinbdvm0YeX4jGWSo9uPLpX8aDfoOlwdLeHNiket/lpSiIY1VyTm4GKz7G WLxKAiaUNMrwyPc56C+TVHo1eJNt8Ro= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 2D4BE60AEA; Mon, 8 Jul 2024 06:28:49 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DF041C4AF0D; Mon, 8 Jul 2024 06:28:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420129; bh=4H8b1UkHtxXxZUO1ViFEqBmxLyLsVYUHjKLF8FeKQ/8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kpT8bdSgT3bmJpQR4uQG2X4CW6Kx4jj3I+s0aC4vc9tFaJ2B68jJshgCE37tGoa5s 1CMTQ7jKY4sI9dbb1JydFGhBLT58ChokB7oWgM4nTNnG5pW4ZvRr4mPXTTVFCqH4SM 42Q4qfgkCHZmv8r8rXGFYMPzdDe8SnnwmEU6ya/yEGxrCNRKv/Xm1OGDtunKYiBzsI ULYN+GTR22haM6FRpvpVRyAAWMLglJZdTZ9SaYpdFHK+sK4XYgwRIk0y319HE13Wgm THOoR1bVe7ACwR/zn/gTiY6XvbopHfVbrXFA9RpfOvXLFHXR4EK31WpKiqwvXKpkcG Ye2YagdhzAvgw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 02/20] mm/zsmalloc: use zpdesc in trylock_zspage/lock_zspage Date: Mon, 8 Jul 2024 14:33:23 +0800 Message-ID: <20240708063344.1096626-3-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 3A0694000D X-Stat-Signature: mop1crndfujzyne7znrp59tr7xtt9foq X-HE-Tag: 1720420130-949171 X-HE-Meta: U2FsdGVkX19jbdcAM1hvPNjSsLRZgcwe7OvyfWcu7OSSVGaxO5pxjzZQOArnDg2Pwh62eyKSQSvHmwFSOu5VZebmr5GbDbvRVIslNrqUoMFKCaZhJKcc/fxAneT6+wS5QB13zfYXFhU3c/6zBSBmOYwL5dTGGwp0ubw+hnjX+d8YthWhvWoglWcx95HidkXKmRD/+gdXXSYhQIXu5sbfdMhOrzO1lxRgBuV7uFkRkIMa5ZNz86RbgnFLlDAc7NgmKCscAMuS8RxaKJY60XwOnA7O/42JKJDFgJLIbugXbvTrBlK1SDwK7B9f7JeEsE1jSwGu1sKyz8B7lqwiVZ4aobzDOZ4g4CJzE6STbwq2IyW/+Y5wbyse9Oha/JNrEC1bL9VX2yvE4+v3B1QCx29P9GFsPUV/fgIO2+W4es4Z0vF/E6zu793+DXjPkvb/dZUICMuJ2cPNDo8idpX+3VPuzo6qBpati+lafro1+R64zYGqj/FDkFlkPcOphWjMqqQIQZfvxtc3VUW4Ue4UtbtQojV1TFvL2OVLRcTKuGRIITKz72RS7eITsuzPi5vysIELYNepe+MzUwMS85mEq+K2CK8Zmv5ViMln+nfefeZGCKr6QBiJ1Vy4/7vEMXHzsYCyYu5EyPTtQ3ExIOEYC2/NgaXi2rJfA5V0ZkJJQzIlRd/R2QVboyWLRuelRc7y1w0W0nRI86pHrdT3WHt8e286Dg22uI8z3Bg/RZSH0M+NTMD5HUP46oPq0ZI8Be4/l5Tfkg4m45aqkA6GVT9zXKF8vzdGvJHfGDlCF5g9J0bB2rXrxDj+ugk8WqJC9tEXsi27LjSCKCdKnoZ30RYgLxTC+VAZaKH+l1Aq8BM1Eb/hQjOL8giRJPPNiTMxH1C64ySMJ5plJ+6FtrNuyTEpTemxmBSJxx4y1ktEEhhHa1Jc12sfm1MJWkUUah+186yG/vYXCexGsKGqPvs2ZfBh1p3 MFqyqCT7 MIw9FBzVDWDZCx4NRLSpehyfTYSMP39jivvHAQN2K0UL5ePoVMu1qfLW27019aIM8DM7z8h5dZhkFi+EofoHxLT0YEqL4BjFHHixFASGNbhb4KbaeokDZrVbqplO7/Pr7BMlzRt4UOGdOVLRgzgOpwL43hyB+hGk+xvu6TkroJACGu99Hq7Y5ZATfhssVxMwWi8rQHyOHJ7tAyLxpg4N1DgPZvKe7pa2y0QgZ62Vb0RM2vL/KC2/iLbgGAR9NPQTfJ5HcrQmqGVgCyrzuPrZyVSC59zDp9fPS+IhNM20NOoUKGXCbM5NbNuapLXvLsID+iC5D2tL4O1BLeePuhUeRtgZ5dRV3Kizj3C9FInNTEssT09eXj7OYofyRp55+4mLld+Dy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi (Tencent) To use zpdesc in trylock_zspage/lock_zspage funcs, we add couple of helpers: zpdesc_lock/zpdesc_unlock/zpdesc_trylock/zpdesc_wait_locked and zpdesc_get/zpdesc_put for this purpose. Here we use the folio series func in guts for 2 reasons, one zswap.zpool only get single page, and use folio could save some compound_head checking; two, folio_put could bypass devmap checking that we don't need. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi (Tencent) --- mm/zpdesc.h | 30 ++++++++++++++++++++++++ mm/zsmalloc.c | 64 ++++++++++++++++++++++++++++++++++----------------- 2 files changed, 73 insertions(+), 21 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 2dbef231f616..3b04197cec9d 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -63,4 +63,34 @@ static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); const struct page *: (const struct zpdesc *)(p), \ struct page *: (struct zpdesc *)(p))) +static inline void zpdesc_lock(struct zpdesc *zpdesc) +{ + folio_lock(zpdesc_folio(zpdesc)); +} + +static inline bool zpdesc_trylock(struct zpdesc *zpdesc) +{ + return folio_trylock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_unlock(struct zpdesc *zpdesc) +{ + folio_unlock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_wait_locked(struct zpdesc *zpdesc) +{ + folio_wait_locked(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_get(struct zpdesc *zpdesc) +{ + folio_get(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_put(struct zpdesc *zpdesc) +{ + folio_put(zpdesc_folio(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index a532851025f9..243677a9c6d2 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -433,13 +433,17 @@ static __maybe_unused int is_first_page(struct page *page) return PagePrivate(page); } +static int is_first_zpdesc(struct zpdesc *zpdesc) +{ + return PagePrivate(zpdesc_page(zpdesc)); +} + /* Protected by class->lock */ static inline int get_zspage_inuse(struct zspage *zspage) { return zspage->inuse; } - static inline void mod_zspage_inuse(struct zspage *zspage, int val) { zspage->inuse += val; @@ -453,6 +457,14 @@ static inline struct page *get_first_page(struct zspage *zspage) return first_page; } +static struct zpdesc *get_first_zpdesc(struct zspage *zspage) +{ + struct zpdesc *first_zpdesc = zspage->first_zpdesc; + + VM_BUG_ON_PAGE(!is_first_zpdesc(first_zpdesc), zpdesc_page(first_zpdesc)); + return first_zpdesc; +} + #define FIRST_OBJ_PAGE_TYPE_MASK 0xffff static inline void reset_first_obj_offset(struct page *page) @@ -745,6 +757,16 @@ static struct page *get_next_page(struct page *page) return (struct page *)page->index; } +static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) +{ + struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + + if (unlikely(ZsHugePage(zspage))) + return NULL; + + return zpdesc->next; +} + /** * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value @@ -815,11 +837,11 @@ static void reset_page(struct page *page) static int trylock_zspage(struct zspage *zspage) { - struct page *cursor, *fail; + struct zpdesc *cursor, *fail; - for (cursor = get_first_page(zspage); cursor != NULL; cursor = - get_next_page(cursor)) { - if (!trylock_page(cursor)) { + for (cursor = get_first_zpdesc(zspage); cursor != NULL; cursor = + get_next_zpdesc(cursor)) { + if (!zpdesc_trylock(cursor)) { fail = cursor; goto unlock; } @@ -827,9 +849,9 @@ static int trylock_zspage(struct zspage *zspage) return 1; unlock: - for (cursor = get_first_page(zspage); cursor != fail; cursor = - get_next_page(cursor)) - unlock_page(cursor); + for (cursor = get_first_zpdesc(zspage); cursor != fail; cursor = + get_next_zpdesc(cursor)) + zpdesc_unlock(cursor); return 0; } @@ -1658,7 +1680,7 @@ static int putback_zspage(struct size_class *class, struct zspage *zspage) */ static void lock_zspage(struct zspage *zspage) { - struct page *curr_page, *page; + struct zpdesc *curr_zpdesc, *zpdesc; /* * Pages we haven't locked yet can be migrated off the list while we're @@ -1670,24 +1692,24 @@ static void lock_zspage(struct zspage *zspage) */ while (1) { migrate_read_lock(zspage); - page = get_first_page(zspage); - if (trylock_page(page)) + zpdesc = get_first_zpdesc(zspage); + if (zpdesc_trylock(zpdesc)) break; - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); } - curr_page = page; - while ((page = get_next_page(curr_page))) { - if (trylock_page(page)) { - curr_page = page; + curr_zpdesc = zpdesc; + while ((zpdesc = get_next_zpdesc(curr_zpdesc))) { + if (zpdesc_trylock(zpdesc)) { + curr_zpdesc = zpdesc; } else { - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); migrate_read_lock(zspage); } } From patchwork Mon Jul 8 06:33:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36A8BC3DA42 for ; Mon, 8 Jul 2024 06:28:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C29C36B009A; Mon, 8 Jul 2024 02:28:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B89F56B009B; Mon, 8 Jul 2024 02:28:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A7AC36B009C; Mon, 8 Jul 2024 02:28:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 8956C6B009A for ; Mon, 8 Jul 2024 02:28:55 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 392E81C219B for ; Mon, 8 Jul 2024 06:28:55 +0000 (UTC) X-FDA: 82315607430.11.B255D48 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf15.hostedemail.com (Postfix) with ESMTP id 887BEA000B for ; Mon, 8 Jul 2024 06:28:53 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=sIFBrxAh; spf=pass (imf15.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420105; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=La+n7rGIuGdFgnpyRwIwda8lqibLhQ76NX4rjz+EkHc=; b=F7BdwZYvCZYxkPvoXXalQ+sjEOywszq8GPo9HjJ9IZ8rtwrEMkTwr77np9Re+XCQJh71yr +BNb1I6unLDYJrLijIoOSy8JY8wwIDGXHFGjAlETMVwETTa7iAoFs84Ih0DcyOkMxBFzXi jv/1+fuNiAFD8e7Iv8ytxteGwBHHITU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420105; a=rsa-sha256; cv=none; b=asOqkvkhc3+f5aQPpQaW5N//0eNVVTAlOYUNxMje5LWNEwrr2B4+caOQjSFwvB3HhhUm4z Ok6Qvhd4JLEZSxOjIePerg6KNVQPhPguX0QQe47Xap61DsN19GWQNWRz3zVCJ027K2nIOV 8jfLymjOJL3ZsD+k9MCK4ixUZYCgp1c= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=sIFBrxAh; spf=pass (imf15.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id BEB0660ACD; Mon, 8 Jul 2024 06:28:52 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9EC9AC116B1; Mon, 8 Jul 2024 06:28:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420132; bh=tmhg+UoGuxLgiU/lnh0EuqzBHj5JgCLnTFMLR6vDXTI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sIFBrxAh+/uE4yv8Zd7MjpS9ccK07cFXKmfpP6Zay+VljtAK9V1ULuV2jXwDkIedx V715o8OO3VX4wzi3m7mMJYZwfQ+fFFf4QqDMjdcW76M781nHLVnMdsWMQb7d9NaqW7 63O+xYhqtBjdx76BJ9dY662CFuFOfHPjOZV81/eUEa5g6yuL+st800m6rL/1u+wORn oVGzoxHA5x+orY+sjaOiWnp3opLoh388Qd+uKplMd6+eVHwHHbWp8cPq5+mzY4eK0w 5I98WraI3cj1UYwMg59uY3taov4j525VdhWGHymwEupsZOqr1iHiN88zcvjLqBSmoe nEpSVZXJv0MNw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 03/20] mm/zsmalloc: convert __zs_map_object/__zs_unmap_object to use zpdesc Date: Mon, 8 Jul 2024 14:33:24 +0800 Message-ID: <20240708063344.1096626-4-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 887BEA000B X-Stat-Signature: tzfjb4ydt8cf3zyd5aizs1enoz1yphyw X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1720420133-666699 X-HE-Meta: U2FsdGVkX19mp4RDkWF5Z2CWSYscNfK1FZAT/KOuBbNbaxOSjmEYAz5dukqAuMx0j909UU3FotpUyTY+XpCpinoELqeKgKLSfXzBWF8yISnzhDZJqY9cJlJRJz+b4CDNbPqsvsc5HNr33C1D91Kqp6VSH2ZqmKA1I88RP7kVlcMjzWmZNJoAd/5Ga1p84XdN1uflw7h19Bs870TlHajbrFr4lUGg+rj+Qv5WT81j015VcxeX1pxJWTbVOCcv7ZgSG1OS1+XOBnSVotA1jzBojb085nBer7vbmfFRem16k6ZIyolnt9Palt0vtNugZhIFT2iFVx/J07NgreGvgWb+udIFC++XvwhHMBZCRZfLYIa9wkCZiQLZC6VG41c+zgi+SoyymtvgFrHKwunQULpBDis6FB8g7eyT87TONhZVgPm8Sy70UfhOiE2vpLdfaNwyhxVGOBySiVFyXOtjm2rdXYaQMiyRi52Lvw8DM/pqeHO3Va0ONsueiaMwtu9U0o41CpciY15Ex9U51xGNrBseAJviWVgW0bldBe3ZTVaUlhtJ0avat6QIJNrrXZTlnb67HKaQQNElHkEAdEwOFUeU8H/5fCqSUwSyHpZDEfrM94y+ivqZW6giJthEfAyOp130V1iVXyZpFtpLldSw+50A9PjjUeWBhMMBJjy0U9zlBa1PkTIkejUEhFAORugx0inQUXnRGGoxTp+Wv0s2yacdX3bzqjMZb4p4gDZvZUKBdMacuaHaYbLFyJSm28M/WqXct4pWRY+LglbBClyb8urXxNCNrctC0Y5Ytpq/3GrVuh2RHlhfkN0mM1xLn7BUqaT8nJF4CWtzrOsjkmpfg3DFvv/hTSsVNWrWtMGitO5016PnQto94y8Z7CfZ0YGCb9n+TTPyQ/DbEbboRMYPOH7xP/rDpUpTvcNxK0yagVrPXbFEZ1UAiLt48n4/+pcZ9orntrfBZ8uoBHPY4Qzmpd4 WaA12V6w QEK5zJpFgWRemy6pVIzRTeXjz7PpKi+UyPRLv1Q96+wlfHSv8R3SlXyGDxVCFlNQa3FyruxKtX446J00CzUzeeu3W+nzH5O2fQSEpah3ONhPNALHAAJo/sjz5rRaAYzjHpigffaXMqA5h1qRJgKLdjxlL7fSskNlLSyjo9UOkNgXUSZcENAyosfoARAFVZ/hjYa8YKULCr0JpSqMTxOGlq+gLTBmkV4xd5WQBKec72bgYg+cpjIs4CGQlsMfP8/EseCf+iPwDRRjRL1542GCJQs8HM/P0GwTf6kN7Bizo1YhM6d+wOUivPidJ4WDc4OObgN2D6WjKvU7DbY4AgEAcuyk2qGFoAIBVQeLur+zmnF/kUboMQzlzqB73XdMqPqSLvtj2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> These two functions take pointer to an array of struct page. Introduce zpdesc_kmap_atomic() and make __zs_{map,unmap}_object() take pointer to an array of zpdesc instead of page. Add silly type casting when calling them. Casting will be removed late. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi (Tencent) --- mm/zsmalloc.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 243677a9c6d2..68a39c233d34 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -243,6 +243,11 @@ struct zs_pool { atomic_t compaction_in_progress; }; +static inline void *zpdesc_kmap_atomic(struct zpdesc *zpdesc) +{ + return kmap_atomic(zpdesc_page(zpdesc)); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -1061,7 +1066,7 @@ static inline void __zs_cpu_down(struct mapping_area *area) } static void *__zs_map_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1078,10 +1083,10 @@ static void *__zs_map_object(struct mapping_area *area, sizes[1] = size - sizes[0]; /* copy object to per-cpu buffer */ - addr = kmap_atomic(pages[0]); + addr = zpdesc_kmap_atomic(zpdescs[0]); memcpy(buf, addr + off, sizes[0]); kunmap_atomic(addr); - addr = kmap_atomic(pages[1]); + addr = zpdesc_kmap_atomic(zpdescs[1]); memcpy(buf + sizes[0], addr, sizes[1]); kunmap_atomic(addr); out: @@ -1089,7 +1094,7 @@ static void *__zs_map_object(struct mapping_area *area, } static void __zs_unmap_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1108,10 +1113,10 @@ static void __zs_unmap_object(struct mapping_area *area, sizes[1] = size - sizes[0]; /* copy per-cpu buffer to object */ - addr = kmap_atomic(pages[0]); + addr = zpdesc_kmap_atomic(zpdescs[0]); memcpy(addr + off, buf, sizes[0]); kunmap_atomic(addr); - addr = kmap_atomic(pages[1]); + addr = zpdesc_kmap_atomic(zpdescs[1]); memcpy(addr, buf + sizes[0], sizes[1]); kunmap_atomic(addr); @@ -1252,7 +1257,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, pages[1] = get_next_page(page); BUG_ON(!pages[1]); - ret = __zs_map_object(area, pages, off, class->size); + ret = __zs_map_object(area, (struct zpdesc **)pages, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret += ZS_HANDLE_SIZE; @@ -1287,7 +1292,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) pages[1] = get_next_page(page); BUG_ON(!pages[1]); - __zs_unmap_object(area, pages, off, class->size); + __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); } local_unlock(&zs_map_area.lock); From patchwork Mon Jul 8 06:33:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726192 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24C00C3271E for ; Mon, 8 Jul 2024 06:29:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AA3446B009D; Mon, 8 Jul 2024 02:29:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A53346B009E; Mon, 8 Jul 2024 02:29:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8CB3A6B009F; Mon, 8 Jul 2024 02:29:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 6E5596B009D for ; Mon, 8 Jul 2024 02:29:02 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 31B35140874 for ; Mon, 8 Jul 2024 06:29:02 +0000 (UTC) X-FDA: 82315607724.18.BAF9266 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf09.hostedemail.com (Postfix) with ESMTP id E6CF3140017 for ; Mon, 8 Jul 2024 06:28:59 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Bs2Uv9IC; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420126; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7uzdzqM6ON93bhIv0ePH2hYz+dV873VySOwQ95FWc0s=; b=f6d/RbDfc+YtOsGR6B59UsyP9NtG3GA6/0iZZ1mktjmaC11DciVICvuNk1CeF2Cc6gtVP4 nvQG8QTlY7yyooq/teBb8JtN/fnnSWUiz+Qc9HVU7infetk0b9JHxWqgR4jRoFR1mF7hEl aG4i4/Abmbd1kNEv2HRg+vWoz8XqhQw= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Bs2Uv9IC; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420126; a=rsa-sha256; cv=none; b=VNvDUhYv3vtvRXIKx/VmYZPpIY4BOVu1xucX4UQUSXYORKkH1l8TYzKpaajk++g0cs6SNw W0C6gkLyPnKu/2Kg6cyx34TD/n7N+CUd/yQI8tlexKpPZEDuXtnycu22pW5Fm8ddhEAC0K PF45JuX0So3lENKJM6m68CzrUNw3YM8= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 0541ECE0A89; Mon, 8 Jul 2024 06:28:57 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3A694C4AF0B; Mon, 8 Jul 2024 06:28:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420136; bh=83jcf8H2ELpZcgLjgz32nYRKXID3GsZDqOi1CXzndR8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Bs2Uv9ICcXS8Sew+dSWNGK1eSMFRpY70r/3FWCgrZS6JN0tiH0+a5nxvKN3Yl+YtJ gE6Yz+5wSlHjGep1v2dYdKBlMunZqqBAcDd/GvG4acDD0vI6Wh6K8moqosV4r+F101 swVXn7hn2qMzhAygustSXm0WSTAGmTgHatJjzn5dtzR2Xz8f3cmGuh78nTxuRQAtSR wjFMhlS7jRFJS2gaRYSZEWKnYaraHCvpp+LwayujCyA0Vdiw5RWRlgNOGpBn98TUwj vE+qHsptRfzPeVDJXYFkB94b5EiWwl1ZPQvgc6vnBTsk1dG/FJYPxo+LpP8AkKwCIB guhs8FMg1J+Hw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 04/20] mm/zsmalloc: add and use pfn/zpdesc seeking funcs Date: Mon, 8 Jul 2024 14:33:25 +0800 Message-ID: <20240708063344.1096626-5-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: E6CF3140017 X-Stat-Signature: 3ikob8793yju5sewps8jhz8ung7n9974 X-HE-Tag: 1720420139-721843 X-HE-Meta: U2FsdGVkX181GPbTIzYP8Ooe390USAC1aKbfAxiXm0j00TioiD9bcgrCk0wSyI8yn2/gaqSfeS11ijzeLf+UjNkcpb8axj+uoGKkVb130Wx49AfaHRPLblBau58q7ofr6/qS7WNpj9PT5SsdVEi5Ynd150Sxt3ul1aRG4B2OM1kfjfqc+PFLzAvJ3UqwHRpuSvU7ddpnlPdJdP5WMi1HwzlUnnbXyJ5xod66stnr07zQI5zUOFwDO74eYhCZSszXpEMjcee8r1QlWMmr0PR8qr6A/U2k7oyuJVyfVF1PjojO8GhHyHfleo70oe9qSE6yLH0ciAXhEIRVv2Uytw2hnHPRbbMseQXK5Dbs9CFX7dVIIYNhQj0hr8q5jPnhranqhJNM1kRhP3+KQkZuTO2eQJHyT3y5j54GzYPUr2WPXn2FBao+g3N3Ztq7589F+t+X1kzjWwPDg51l88Q4WSJzVHioQl1EvXzpvxYVk4zbr8CkNpK/WuW1gKmydcvfiZo83bsrofdAX8AQEkZPG0QlTgHcioJnGjIUzWWLu51IeFNcfaiM0maPcWcXEfmSBoEeqASIj7YQ8wWVt+Gbt/gN/dgD7/i2Dlj/00hNHlOPKZaEGfwxxIL2gs2SvJwdLl5g19k6y6BTBDSkc/uaQOpn3tei4+X0RE8FR8iFJd6nLJt9Vd9e1kHwDr0ttGys+aDyWtrgc6L7MzWsSOLGv7NmytQrmXz5Jzd5nYrxwdIOG48UlEi1sJY038X77lxiOINuzuU58dxfOx8RRY56K+KEmbyKQOPKei1yihs7CqVmLNC7G3X1KXIXR+7zzEREJfWFe5ZcOkj4YF7JuWEL9EkXT24IBaeMegaHMDxNXQHe+vfbGqTZPb3/4coOVR8gnKA1DuYOhuyqq0ZyiEaX1i1o2nUiCm9IjGshCYbc4alWqWhZqDjaovWYjwnvxUuKHjhv8XIH6M+NZCIHaywBE8b P++D3VIN xSyg2sBAqjc8RxnScMU/SP2awWn1lrFkz4KlzYfbEQg7D2qNZT1cnKXOmZlS5IOhAifyO+jq8vQMpd5NtrfTvqj5PNxXkuMgaXC8CyBb9befN5L31Z4Wn76B85K25UiMrBpdQqd1+hCgnxmVt0t0SfPwX9TYKDHOLhB06UihCKraYx5i8cUJw5loAqHuhn29Toaz53XgC03WZSpazUTsKaf0gGMWpWC1LlH2S3s16Rv87O7YZFV5UcN1uWrWTrziEL2OgRjAxXganAUy1AxGCZqWVLkActu/tOWSeg/XFB/SB1gqkjGqWcR/jhAUs5Yn8YVXOj49LDkMwAHI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Add pfn_zpdesc conversion, convert obj_to_location() to take zpdesc and also convert its users to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi (Tencent) --- mm/zpdesc.h | 9 +++++++ mm/zsmalloc.c | 75 ++++++++++++++++++++++++++------------------------- 2 files changed, 47 insertions(+), 37 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 3b04197cec9d..79ec40b03956 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -93,4 +93,13 @@ static inline void zpdesc_put(struct zpdesc *zpdesc) folio_put(zpdesc_folio(zpdesc)); } +static inline unsigned long zpdesc_pfn(struct zpdesc *zpdesc) +{ + return page_to_pfn(zpdesc_page(zpdesc)); +} + +static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) +{ + return page_zpdesc(pfn_to_page(pfn)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 68a39c233d34..149fe2b332cb 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -773,15 +773,15 @@ static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) } /** - * obj_to_location - get (, ) from encoded object value + * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value - * @page: page object resides in zspage + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static void obj_to_location(unsigned long obj, struct page **page, +static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, unsigned int *obj_idx) { - *page = pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc = pfn_zpdesc(obj >> OBJ_INDEX_BITS); *obj_idx = (obj & OBJ_INDEX_MASK); } @@ -1208,13 +1208,13 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, enum zs_mapmode mm) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; struct size_class *class; struct mapping_area *area; - struct page *pages[2]; + struct zpdesc *zpdescs[2]; void *ret; /* @@ -1227,8 +1227,8 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, /* It guarantees it can get zspage from handle safely */ read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage = get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc_page(zpdesc)); /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1247,17 +1247,17 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, area->vm_mm = mm; if (off + class->size <= PAGE_SIZE) { /* this object is contained entirely within a page */ - area->vm_addr = kmap_atomic(page); + area->vm_addr = zpdesc_kmap_atomic(zpdesc); ret = area->vm_addr + off; goto out; } /* this object spans two pages */ - pages[0] = page; - pages[1] = get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] = zpdesc; + zpdescs[1] = get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); - ret = __zs_map_object(area, (struct zpdesc **)pages, off, class->size); + ret = __zs_map_object(area, zpdescs, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret += ZS_HANDLE_SIZE; @@ -1269,7 +1269,7 @@ EXPORT_SYMBOL_GPL(zs_map_object); void zs_unmap_object(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; @@ -1277,8 +1277,8 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) struct mapping_area *area; obj = handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage = get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc_page(zpdesc)); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1286,13 +1286,13 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) if (off + class->size <= PAGE_SIZE) kunmap_atomic(area->vm_addr); else { - struct page *pages[2]; + struct zpdesc *zpdescs[2]; - pages[0] = page; - pages[1] = get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] = zpdesc; + zpdescs[1] = get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); - __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); + __zs_unmap_object(area, zpdescs, off, class->size); } local_unlock(&zs_map_area.lock); @@ -1434,23 +1434,24 @@ static void obj_free(int class_size, unsigned long obj) { struct link_free *link; struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long f_offset; unsigned int f_objidx; void *vaddr; - obj_to_location(obj, &f_page, &f_objidx); + + obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); - zspage = get_zspage(f_page); + zspage = get_zspage(zpdesc_page(f_zpdesc)); - vaddr = kmap_atomic(f_page); + vaddr = zpdesc_kmap_atomic(f_zpdesc); link = (struct link_free *)(vaddr + f_offset); /* Insert this object in containing zspage's freelist */ if (likely(!ZsHugePage(zspage))) link->next = get_freeobj(zspage) << OBJ_TAG_BITS; else - f_page->index = 0; + f_zpdesc->next = NULL; set_freeobj(zspage, f_objidx); kunmap_atomic(vaddr); @@ -1495,7 +1496,7 @@ EXPORT_SYMBOL_GPL(zs_free); static void zs_object_copy(struct size_class *class, unsigned long dst, unsigned long src) { - struct page *s_page, *d_page; + struct zpdesc *s_zpdesc, *d_zpdesc; unsigned int s_objidx, d_objidx; unsigned long s_off, d_off; void *s_addr, *d_addr; @@ -1504,8 +1505,8 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, s_size = d_size = class->size; - obj_to_location(src, &s_page, &s_objidx); - obj_to_location(dst, &d_page, &d_objidx); + obj_to_location(src, &s_zpdesc, &s_objidx); + obj_to_location(dst, &d_zpdesc, &d_objidx); s_off = offset_in_page(class->size * s_objidx); d_off = offset_in_page(class->size * d_objidx); @@ -1516,8 +1517,8 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, if (d_off + class->size > PAGE_SIZE) d_size = PAGE_SIZE - d_off; - s_addr = kmap_atomic(s_page); - d_addr = kmap_atomic(d_page); + s_addr = zpdesc_kmap_atomic(s_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); while (1) { size = min(s_size, d_size); @@ -1542,17 +1543,17 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, if (s_off >= PAGE_SIZE) { kunmap_atomic(d_addr); kunmap_atomic(s_addr); - s_page = get_next_page(s_page); - s_addr = kmap_atomic(s_page); - d_addr = kmap_atomic(d_page); + s_zpdesc = get_next_zpdesc(s_zpdesc); + s_addr = zpdesc_kmap_atomic(s_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); s_size = class->size - written; s_off = 0; } if (d_off >= PAGE_SIZE) { kunmap_atomic(d_addr); - d_page = get_next_page(d_page); - d_addr = kmap_atomic(d_page); + d_zpdesc = get_next_zpdesc(d_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); d_size = class->size - written; d_off = 0; } @@ -1791,7 +1792,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct zs_pool *pool; struct size_class *class; struct zspage *zspage; - struct page *dummy; + struct zpdesc *dummy; void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; From patchwork Mon Jul 8 06:33:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1737C3DA45 for ; Mon, 8 Jul 2024 06:29:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 01A4A6B009E; Mon, 8 Jul 2024 02:29:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F0C066B009F; Mon, 8 Jul 2024 02:29:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DAD8E6B00A0; Mon, 8 Jul 2024 02:29:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B458B6B009F for ; Mon, 8 Jul 2024 02:29:02 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 7B24A1A10B6 for ; Mon, 8 Jul 2024 06:29:02 +0000 (UTC) X-FDA: 82315607724.23.0D34EF0 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf22.hostedemail.com (Postfix) with ESMTP id DF34DC0010 for ; Mon, 8 Jul 2024 06:29:00 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=e+9GU7+O; spf=pass (imf22.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420118; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=l5UquMvigMJ3jlequ1KKubsljHfVWFhIyFeAFPJfZAs=; b=A7OC8bICYQd9TjxDt7/LdQBzQvHZ9P+4mj0OemzhZlX8sD2ouDGe22ky2UHineMHUr7Dts TaOsejSNlp0YIoptijlhckxusYUfPsxWDL2wOYv3/icy01cw8Ftm5M6NaW8pG2SubXPqIv vDX9JXDwn2AsEgvZcexerg0FCnTvqh8= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=e+9GU7+O; spf=pass (imf22.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420118; a=rsa-sha256; cv=none; b=2POiMb5jxkZlDDYCbC8ayQ1GKfBV1rnXMnbr6Ce98dmIPVo3OoHab0TDP+7ckter1kldG0 OOoSMS5XkAVbj/WsbIR4R3sBXLwCFiXhBJxvrEYNxwtmxPnFG/baY3yrMVYDQzd78FMtLi KKyX/vu9Wyr/u1APtKCf+qbU9ESOmO4= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 134EB60A78; Mon, 8 Jul 2024 06:29:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EC9ABC116B1; Mon, 8 Jul 2024 06:28:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420139; bh=Kaf63ckMzfd8JfdGDWbfHAhpNFgdzACIKJu7j9bsBGA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=e+9GU7+OFriAgyhewgBuS4MpJ13g2HtbEpGw2kuHmhxZGAFHfZrMhm5wfrLZt6eSu MF2wyNfgAdjMs9E8qq5ZSCV9PMJAr4/A1OaqAx8uKJabfBUQR38cugW/jaeyMk42rK RQMEpRHjPzDEPwQrXf+OZ6N9tpRgoJRDv++LCDOZxvLEdvf+FMPtlmENm8JmLvLO6a +/RnpebZBStyMYM1IHwbCJKZpxkRFUUEpCMF/DK5JHekR4mjFG/ANV615+EmvR0SLR OXmgjPJq0yh2NRtIZKeP5dQYB1AQ/luVVGcKmwcAKJT4Lg43bXs4RbfPa+gouK3IQm ul+eMziT36sgg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 05/20] mm/zsmalloc: convert obj_malloc() to use zpdesc Date: Mon, 8 Jul 2024 14:33:26 +0800 Message-ID: <20240708063344.1096626-6-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: fnfih6igbahr1qpcbiikrcnxfrxbwfyf X-Rspam-User: X-Rspamd-Queue-Id: DF34DC0010 X-Rspamd-Server: rspam02 X-HE-Tag: 1720420140-697336 X-HE-Meta: U2FsdGVkX1+OAtRWSTUr8hCi6aJdpby96UZ9wW1hgr5Hr2f7P7ybPj8KkUBYxj50CBk1k1lCW4g81RDLe+zjG5tb4WbjwnkirbwwnIVp0Jjia0tCVM8/NJ0FiuAJ+Ld7P60VtrBNDlmpTTqqu+HGZ8EgPF9BRqjKajjqPJU8uWjDezQ0BxUnIT4y+VO+Bqi5tTYaqpobgbSEWQ3ItZof1ZmOT34QWwdwSfc3oUrGKEqPnCs5eD+3d/My891CiMPxJIYcnbEurzBjVaHun5QnEajup3wxHuEIsxK5S4zIShnOdEH39Wmku+AVksbAvAYi+jhpddA5lbRFiEZn8+6NWALDWYtc4V2+ji2026uaJWzXWpQQlhn+7gaeaub9Rce4U1iGLNd4PeJmlP+Mh/1wCBrGD01DRTx9c/rtkF+pb/Id/iZVbpgKqPPfVoBqqthu2GinIIJjltu5w+vbdS5XiHlRHnm3QKkpaMxuKQuq9fNMkXrn7KxRPqZOttl4nH1fRWIro7WFEYOtzCwEyBg+Q1j1CuGppjycBLITySI9C64twHzwPEl0ULqjEJbqMUJbnlesgPs9T8LRinVKPa73sUrbua61WLXmisddP8gojRhArg/ieGmkJnrwp+65kMFEbuuPDXhDbCVNFkT26Ix9P+A1TUdb0+BD/R3jMCrkayVBmu1IzA2/dHXg/qfw+gSp8eJb4QvUBgqgIIp3SB3SO/dcW5Kxrd7AlpUxLrRFT2JumBk4xY8sMoK6FXFarJKox9VUv0yo6pPNZEQUyS7VIMhoilxOyiAuaO2AooEqOIzfQxb1/pf6kuhWrV4QJ9enRSwDwMxhqxZdvm9LeXu1IoYtdxdcxSLwsAzzPk8DIqtTmMdFIFarGbPxE10wN3qIU1eOkcyqDYTCe6hYdsc3H1U5IA3Cz5SpAToJrK8sLrZwxBKYPR4Byi0l1HuQciKtiQehWctfXnE1IzKVQS2 nxA/1O0n 1v/X5OMldjKJYePTagfiKdjMTIg124SOWayTQl4ItOywo1/3ANmyQLxRfc8rov/ce2j0m9Uig0zLRwsxe6OP+3AH/w5b7nWWiezyMQeVMEu68JfCTef220elWgyMsX1xxTqEDwGbmJt0gtniaVJs6S4UZpxX4uV9FZeAyLqnvnL9fGHki45xvMIbR2KQOtFWs7iSG2Ny39o5DwJdshstlosZclOrj9erbHRl/9CCapXw62h08sKz5wz/K3lKqyrqUJpCsqnihX8qsord6/wJfWPq8SkVnOVQUPML/Q3KRXGVyA0dbkWvKL4n/pLSyDtt26tWytcYpgiOAtxRNHZzAz9NLWskoJwo7wJd4HQbwifKc4Lft5Eb3c0noNvumfH0CL3lS X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first_page/get_next_page. no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi (Tencent) --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 149fe2b332cb..bbc165cb587d 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1322,12 +1322,12 @@ EXPORT_SYMBOL_GPL(zs_huge_class_size); static unsigned long obj_malloc(struct zs_pool *pool, struct zspage *zspage, unsigned long handle) { - int i, nr_page, offset; + int i, nr_zpdesc, offset; unsigned long obj; struct link_free *link; struct size_class *class; - struct page *m_page; + struct zpdesc *m_zpdesc; unsigned long m_offset; void *vaddr; @@ -1335,14 +1335,14 @@ static unsigned long obj_malloc(struct zs_pool *pool, obj = get_freeobj(zspage); offset = obj * class->size; - nr_page = offset >> PAGE_SHIFT; + nr_zpdesc = offset >> PAGE_SHIFT; m_offset = offset_in_page(offset); - m_page = get_first_page(zspage); + m_zpdesc = get_first_zpdesc(zspage); - for (i = 0; i < nr_page; i++) - m_page = get_next_page(m_page); + for (i = 0; i < nr_zpdesc; i++) + m_zpdesc = get_next_zpdesc(m_zpdesc); - vaddr = kmap_atomic(m_page); + vaddr = zpdesc_kmap_atomic(m_zpdesc); link = (struct link_free *)vaddr + m_offset / sizeof(*link); set_freeobj(zspage, link->next >> OBJ_TAG_BITS); if (likely(!ZsHugePage(zspage))) @@ -1355,7 +1355,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); - obj = location_to_obj(m_page, obj); + obj = location_to_obj(zpdesc_page(m_zpdesc), obj); record_obj(handle, obj); return obj; From patchwork Mon Jul 8 06:33:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726194 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94E0CC3DA42 for ; Mon, 8 Jul 2024 06:29:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 28F496B00A1; Mon, 8 Jul 2024 02:29:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23E186B00A2; Mon, 8 Jul 2024 02:29:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B6846B00A3; Mon, 8 Jul 2024 02:29:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id DB8556B00A1 for ; Mon, 8 Jul 2024 02:29:09 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id A292A121137 for ; Mon, 8 Jul 2024 06:29:09 +0000 (UTC) X-FDA: 82315608018.01.B3AEE0A Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf26.hostedemail.com (Postfix) with ESMTP id 523F614001F for ; Mon, 8 Jul 2024 06:29:06 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=iFPjsLTe; spf=pass (imf26.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420118; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=W1He/r4hhQFSeokZC2RLbg062zbXW7Tzoh0YJXomhAo=; b=BED/yhhoC0FApGzE2dqUBVmcBzXj3tZOr1AH7o9e6PzY9PP/v+DV1C76yajmTVPuQ8B4VM WFxPmjSTlq8Lt9ZCHU/7u4ynA9p0jfZzS4l/430DCdQXT83djW7ytZD72mpfRFDUQxcMOT OazVlzlEGB9CRJQvrCLWPV8Vi3dW/bU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420118; a=rsa-sha256; cv=none; b=wY+9Rjy+yj+uko1wGgphcfTTnujeiDnoGs1gh2ZpdlWe1zBwRKAtOXM/eOqvXRGvu89XHt X0Yp7J3jkDAah1i0WmXdq9LFJtjD0alTutkpd0GTWf2dyeWLZHs1CGCOkKSP3zE4EMdE6Q mKVk1FiwjtDT2hyUfC7E7oTjV9QrjQ8= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=iFPjsLTe; spf=pass (imf26.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 527BDCE0A57; Mon, 8 Jul 2024 06:29:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 85057C4AF0D; Mon, 8 Jul 2024 06:29:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420143; bh=wQxGtmbL1aK1VwnBgVGS8tWob34VVxGd8gh+B3iTQwc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iFPjsLTezAT14viZJrWtHow972VaopLMBb8d3TjRdKFke5FbWCPT2+C5MZvc50sws SdN4cefEBvuK4j7c5szip31wVMQetv+8wvaxKWGHgWXkF7uXCSVdH2LAOkpAY1fiBd kqwXVTS4G+22/QZM+hSywN6/rBnYYRQbgnUrAVuvKNo8mucHvrgNnMeWWothY6XpIC fmshppA9dnJBTFqzO6yl1GlmgTYZvw+T4Q1+i7uz5dvZAkJTWw5+jucyJh7WJcK7d2 vyxh/8hCvC2GuTlXHECl5wussQuJi27BpBslTQUjnLwbm7JADBR8BBQlWHr8PK/ZRS NYDv6k1PMMX6Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 06/20] mm/zsmalloc: convert create_page_chain() and its users to use zpdesc Date: Mon, 8 Jul 2024 14:33:27 +0800 Message-ID: <20240708063344.1096626-7-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: sbxi8f4qhf6madbm5tj8qcnhyr8jazjb X-Rspamd-Queue-Id: 523F614001F X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1720420146-868057 X-HE-Meta: U2FsdGVkX190Q5CUkjnPS4GcnGiCGy7WhaUVj+rC27F3klvtzhXCRO4pr6Ys2xgpGObf8GhjtVZs6s/IQgiBpYZHY9C6gA6UK7uRYCfu5BxCkwbadZOza6BdT4WqzOjhT+2ipW9/MUcUdD3w2pYVBQZ43FaNw0lFHIaLtm5IMsChLsC84uDvqbRzDooh+ny5vV35lljynqBzGWCn1alTP+6On7xNzdBuOvjbKw1TBuXw9ivmXsTqKBI7xd8IKLLIuL4HWZynzM7rpGkctnGiXumuyrwvc6ORTXUrZbRGf71XBd6O9nD+7L84URp63RdB6cboxTKjJcLH3Je2BNPzntjhvidwGQ8GZ/4HP0zzYyaW98ECl5NFZtDsJb0s932RJ9/o78j93ecECj47HVO2vLGaWpg93BZ6jMnG9l0nCkqrlh5xKEJBnPd9jUxYYN70tkNf41+0W1cNXnbUS9weUok2R1JPvKr6xTssnSD+abpK8Zg0+r7oUWGzrYDbNeVjIs+hWX2RVeiLxMo65f6dHqTBuIW854NUjf501ZNWlZbX5j3QB5XXqRJj9H0nt1IPpKvzR+ZnqtqrgJKT2rBaAWb7d8pH2OX91RfFTarl3CfT4SU6H8aMFKC9FqpOPkwnRjog7f8Pmyw/odY+tie3rhkPtk5g6AawyBN1dfxxp0/tFaLBandP0CW+objH+64iIt8Q1dZiyPt48BdP4jp1fbYR4GmAOx6smtffLPN1yYE4pMnEthLZOEsM//KOGuqY9rA+HMgOJHPYzTWYN9qbqv18ejoLFHLZw7c07gotYFJViwWgcA+RVZR85J0y1c0c+E12QJvv36Is4JEBWBUQbqH4p56WORnvthnrhYUWJiM8Ds5YqiJZFq0q0j9HEZjBGXFt3h+SShtvpH8jo6PzUc/IfN/GLqC6x9t4fSy4NBcdGrjxrRyu5I8k+BK3fU6IRK0Md0MfISTvtlBDsxH rU5QJu+e /W+grl4BvwygaoBc+C6oBdcmqFzcZoAbwPX1WkOokznk9qMWe34vAva01zsohZB0hn578inzYPUpmapSbCEE4cjYV4+nwJuob5ZdaWKATM2vIlmeA8iOQadW3kw/LmavORD0oSFhAWHwfLouzivRpHU6678kXFErWZS3Izttc3AT4FFj/hqW+RcsL+rXigjes9lTTqgNKTH5Xqgiwrq/Os7UbEZrHe/tzHVp6ygnOyp6jt7XlW96BXrqHrdfLwAqHPtm1gV+oYcYCYN2l7T9NHccPgWnD4S+Zht9Xzzd/M1AwBHKJTvnt04RU4/KAOrjkhBkk5lDyTDNOb8LA8BS2iMbgQ198BsQHmaUOfnmYX5QuLulkyl6Yn72Fog== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi (Tencent) Introduce a few helper functions for conversion to convert create_page_chain() to use zpdesc, then use zpdesc in replace_sub_page() too. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi (Tencent) --- mm/zpdesc.h | 6 +++ mm/zsmalloc.c | 115 +++++++++++++++++++++++++++++++++----------------- 2 files changed, 82 insertions(+), 39 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 79ec40b03956..2293453f5d57 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -102,4 +102,10 @@ static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) { return page_zpdesc(pfn_to_page(pfn)); } + +static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, + const struct movable_operations *mops) +{ + __SetPageMovable(zpdesc_page(zpdesc), mops); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index bbc165cb587d..a8f390beeab8 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -248,6 +248,41 @@ static inline void *zpdesc_kmap_atomic(struct zpdesc *zpdesc) return kmap_atomic(zpdesc_page(zpdesc)); } +static inline void zpdesc_set_zspage(struct zpdesc *zpdesc, + struct zspage *zspage) +{ + zpdesc->zspage = zspage; +} + +static inline void zpdesc_set_first(struct zpdesc *zpdesc) +{ + SetPagePrivate(zpdesc_page(zpdesc)); +} + +static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) +{ + inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline void zpdesc_dec_zone_page_state(struct zpdesc *zpdesc) +{ + dec_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline struct zpdesc *alloc_zpdesc(gfp_t gfp) +{ + struct page *page = alloc_page(gfp); + + return page_zpdesc(page); +} + +static inline void free_zpdesc(struct zpdesc *zpdesc) +{ + struct page *page = zpdesc_page(zpdesc); + + __free_page(page); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -954,35 +989,35 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) } static void create_page_chain(struct size_class *class, struct zspage *zspage, - struct page *pages[]) + struct zpdesc *zpdescs[]) { int i; - struct page *page; - struct page *prev_page = NULL; - int nr_pages = class->pages_per_zspage; + struct zpdesc *zpdesc; + struct zpdesc *prev_zpdesc = NULL; + int nr_zpdescs = class->pages_per_zspage; /* * Allocate individual pages and link them together as: - * 1. all pages are linked together using page->index - * 2. each sub-page point to zspage using page->private + * 1. all pages are linked together using zpdesc->next + * 2. each sub-page point to zspage using zpdesc->zspage * - * we set PG_private to identify the first page (i.e. no other sub-page + * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). */ - for (i = 0; i < nr_pages; i++) { - page = pages[i]; - set_page_private(page, (unsigned long)zspage); - page->index = 0; + for (i = 0; i < nr_zpdescs; i++) { + zpdesc = zpdescs[i]; + zpdesc_set_zspage(zpdesc, zspage); + zpdesc->next = NULL; if (i == 0) { - zspage->first_zpdesc = page_zpdesc(page); - SetPagePrivate(page); + zspage->first_zpdesc = zpdesc; + zpdesc_set_first(zpdesc); if (unlikely(class->objs_per_zspage == 1 && class->pages_per_zspage == 1)) SetZsHugePage(zspage); } else { - prev_page->index = (unsigned long)page; + prev_zpdesc->next = zpdesc; } - prev_page = page; + prev_zpdesc = zpdesc; } } @@ -994,7 +1029,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, gfp_t gfp) { int i; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE]; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE]; struct zspage *zspage = cache_alloc_zspage(pool, gfp); if (!zspage) @@ -1004,25 +1039,25 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, migrate_lock_init(zspage); for (i = 0; i < class->pages_per_zspage; i++) { - struct page *page; + struct zpdesc *zpdesc; - page = alloc_page(gfp); - if (!page) { + zpdesc = alloc_zpdesc(gfp); + if (!zpdesc) { while (--i >= 0) { - dec_zone_page_state(pages[i], NR_ZSPAGES); - __ClearPageZsmalloc(pages[i]); - __free_page(pages[i]); + zpdesc_dec_zone_page_state(zpdescs[i]); + __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(page); + __SetPageZsmalloc(zpdesc_page(zpdesc)); - inc_zone_page_state(page, NR_ZSPAGES); - pages[i] = page; + zpdesc_inc_zone_page_state(zpdesc); + zpdescs[i] = zpdesc; } - create_page_chain(class, zspage, pages); + create_page_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool = pool; zspage->class = class->index; @@ -1753,26 +1788,28 @@ static void migrate_write_unlock(struct zspage *zspage) static const struct movable_operations zsmalloc_mops; static void replace_sub_page(struct size_class *class, struct zspage *zspage, - struct page *newpage, struct page *oldpage) + struct zpdesc *newzpdesc, struct zpdesc *oldzpdesc) { - struct page *page; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE] = {NULL, }; + struct zpdesc *zpdesc; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE] = {NULL, }; + unsigned int first_obj_offset; int idx = 0; - page = get_first_page(zspage); + zpdesc = get_first_zpdesc(zspage); do { - if (page == oldpage) - pages[idx] = newpage; + if (zpdesc == oldzpdesc) + zpdescs[idx] = newzpdesc; else - pages[idx] = page; + zpdescs[idx] = zpdesc; idx++; - } while ((page = get_next_page(page)) != NULL); + } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); - create_page_chain(class, zspage, pages); - set_first_obj_offset(newpage, get_first_obj_offset(oldpage)); + create_page_chain(class, zspage, zpdescs); + first_obj_offset = get_first_obj_offset(zpdesc_page(oldzpdesc)); + set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); if (unlikely(ZsHugePage(zspage))) - newpage->index = oldpage->index; - __SetPageMovable(newpage, &zsmalloc_mops); + newzpdesc->handle = oldzpdesc->handle; + __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); } static bool zs_page_isolate(struct page *page, isolate_mode_t mode) @@ -1845,7 +1882,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, } kunmap_atomic(s_addr); - replace_sub_page(class, zspage, newpage, page); + replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. From patchwork Mon Jul 8 06:33:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70B47C3271E for ; Mon, 8 Jul 2024 06:29:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1EF916B00A2; Mon, 8 Jul 2024 02:29:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 12A176B00A3; Mon, 8 Jul 2024 02:29:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EE52F6B00A4; Mon, 8 Jul 2024 02:29:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CD0A86B00A2 for ; Mon, 8 Jul 2024 02:29:10 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 4452BC0ECB for ; Mon, 8 Jul 2024 06:29:10 +0000 (UTC) X-FDA: 82315608060.20.21674E0 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf17.hostedemail.com (Postfix) with ESMTP id A7D9040008 for ; Mon, 8 Jul 2024 06:29:08 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=oZUatPhA; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf17.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420125; a=rsa-sha256; cv=none; b=VR1+yiPIpoRVImsMF4trOCqP0x23UHjnC7KLIJXNvVvPRGf+J7lFytKInaNa/XAFEw4dnr CW7ugl8qKmKCgDgApbrt1gPnC0z1cOtlgJpSqRvcj64+LCQHzqGZRZ7d6RkHP8j960pr4s AUUscXTeaoDNSrff3ZaccvAei4MhUbM= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=oZUatPhA; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf17.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420125; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Vht4Mu2UzYx5Ihggdky1n1LO1j8nZdajLePSwRaN2xo=; b=dkUM7BEXjFSaeV7v4e/49WEviPl16Wnwrzb/vISs8Qi9ELCOaRd0DnnKkFt4uzp+FdwhdS Qipfw526JakqWQN7ThQQ6rZ3Bx5EOTMumIXvN94O2DwYPiXuCcfmqUCn5JegzDP6oRAkqI /+PEEo0QIBCETcyTfW8YJn2k3ASkLPQ= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id E629F60A78; Mon, 8 Jul 2024 06:29:07 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 47754C116B1; Mon, 8 Jul 2024 06:29:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420147; bh=eUVr/nevUiypHdkqs90LAQGcxdPsPYteX43+BHTjXcE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oZUatPhAeDrGrCo7sj3uta4gUIoHLfsfvRqLImOTiu9Ew6pEEgEFIOq0FSp+n5nwr QUQtfQqHNsuPdOaitMA8AzGvFRLInyN24C77Tk7y0e98ECidpaZy2xPt6WcuarP7L5 TBOyX1bZOTr7g5ZtY4U7xHoKTOGv9CJ0Ii9hB6TgLH4X+XjfqtBiqYubR/UrpapQHY bvCJQvBU/k1baFZ5PsaTYfxcJGwbs7QtuI7RelgsDIjmtrGKew2vbFGQE177XxhgdU bkpQ0aVFkhdhzwcff+LWaITeqGvljAUNPo8L8o/7dFg4yBfq34RKmo2O+m8jrmccwF WyP+xYNxqjE0g== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 07/20] mm/zsmalloc: convert obj_allocated() and related helpers to use zpdesc Date: Mon, 8 Jul 2024 14:33:28 +0800 Message-ID: <20240708063344.1096626-8-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: A7D9040008 X-Stat-Signature: jtgy6u89iqj8gpxpuzdazymc1nrryjqb X-Rspam-User: X-HE-Tag: 1720420148-410079 X-HE-Meta: U2FsdGVkX1+m4+Q0LodWYFH4ts5Tk21FSGKFBXU1372It3lusBvA3GrLOzwzjb3wykttpJ/m8OzjrKlujn8wvaaBulpbvPPbwA0i5VUUoiSNvIyUkpKBGQ0aVJ4mgIQ7ZafpTwY7Q3CuOwYABuqK6qjIBXTmN52kPQn7kOrU0prnXHoXYm1MYJqEkfPd5p9gu0sNQvUf2V+rRmNxyAbZ+UXJkCz3NAhXufNJg2xaYYkhn0eFJwtkOjBfog8pZ18RfQOrOqN8rYaPbQwAFVQicDZLSX5WVhKx6fu99Bv4LUPEAgJqQw02rXUMJC+mhVvzjKYE+xFvIrgfRuxBKh3ZOu/+zhLzsy820v1VPTitFiA57kg2jjuXmQ74A2Hil41NxsHd3PUM95otqCHbJzH4L00DU8FnhFKmjvhQWIhi/0Ot6YVvr9+ko53Q8Y5qL05bN01MVyE22ar+yvFk1ocwWEH1IDGAyL41Rhi9Tq8Rz+7kwEGQ2rX8iHEe3RDb9FRiflb+7b4/4x0D0bFphwAMEpfcC4I/Vc912bQvL83infJP65Rsxl9sasGDZdNd9hdNOjlcAl0zPBDQFUiIs3umKcul9jBRcCdk5ajllofOeQgVfEs3ASm3jtXVKAJt9DjIK59qU+SKNNCAS9/puco9vTYARfW9s59OoTkGGxRHg7WH2jOGtPgPRdCwqaW9icBErlq8zvY67iYUykQCGjSsVx/MFBFIwl2lnNxkElA6841vPDk3DK9dY5etYwbBP+3uqz+2gb2fCpQlK2ZsPR1+YGhPR4jIxPp+o4904U0Ync1SCcDYH3jerxNHx6/bNrzdOfxpIuysqQuJ7y7Dj/rlh3vFQPl3AEMhFV8u/VUA/rcZm2XmFrpEawWuh+uWWjr0xwYRW+ZOgqjxvC04L/leeRALVprPTsiu+iGrlTTdG2CTjQiML9Le/hvLGC9qoGgfc349EPzRTGTaOBSByRf NCFrAcN0 YXxSQ5Q+hWdW+bSHA6OQTAWatE/u3TYxp/8AsJgRcWcqhQmKBYN7defCZHgA2/Uot8O+mfAqPgJH1Flx6jXiKSD4n1MW5boUBEXLMboCtSWWQnzt08PYe8xMv28vSI8ds1XtORJJw7nrI2bH8cVd9Av3c2QBOmBI9umVbfr8XjYqVhTp1tH4Iv8tU9RtSfJu3/42YbD6yETnj8wQMUto6p1TCycjzI9WMzqLSOONBZwhSXKmq9XCKkjteFfUFrEg/+/cl8SUE67Qiv2op7JzJBxx5SOll9tQQ0XWa1HFo7ol3tPymFzSvTFPd+nvGkLQo9UFSbTMt121aH/4SaSL3L90lYcRdWRyO0HXhDMJ/iVztgiSNQdGXnKzTMzyeyNeO4Sbu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Convert obj_allocated(), and related helpers to take zpdesc. Also make its callers to cast (struct page *) to (struct zpdesc *) when calling them. The users will be converted gradually as there are many. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi (Tencent) --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index a8f390beeab8..29b9fa5baa46 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -845,15 +845,15 @@ static unsigned long handle_to_obj(unsigned long handle) return *(unsigned long *)handle; } -static inline bool obj_allocated(struct page *page, void *obj, +static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage = get_zspage(page); + struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); if (unlikely(ZsHugePage(zspage))) { - VM_BUG_ON_PAGE(!is_first_page(page), page); - handle = page->index; + VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); + handle = zpdesc->handle; } else handle = *(unsigned long *)obj; @@ -1603,18 +1603,18 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, * return handle. */ static unsigned long find_alloced_obj(struct size_class *class, - struct page *page, int *obj_idx) + struct zpdesc *zpdesc, int *obj_idx) { unsigned int offset; int index = *obj_idx; unsigned long handle = 0; - void *addr = kmap_atomic(page); + void *addr = zpdesc_kmap_atomic(zpdesc); - offset = get_first_obj_offset(page); + offset = get_first_obj_offset(zpdesc_page(zpdesc)); offset += class->size * index; while (offset < PAGE_SIZE) { - if (obj_allocated(page, addr + offset, &handle)) + if (obj_allocated(zpdesc, addr + offset, &handle)) break; offset += class->size; @@ -1638,7 +1638,7 @@ static void migrate_zspage(struct zs_pool *pool, struct zspage *src_zspage, struct size_class *class = pool->size_class[src_zspage->class]; while (1) { - handle = find_alloced_obj(class, s_page, &obj_idx); + handle = find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); if (!handle) { s_page = get_next_page(s_page); if (!s_page) @@ -1871,7 +1871,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, for (addr = s_addr + offset; addr < s_addr + PAGE_SIZE; addr += class->size) { - if (obj_allocated(page, addr, &handle)) { + if (obj_allocated(page_zpdesc(page), addr, &handle)) { old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); From patchwork Mon Jul 8 06:33:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726196 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 882C6C3271E for ; Mon, 8 Jul 2024 06:29:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E25D66B00A4; Mon, 8 Jul 2024 02:29:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DD4F36B00A5; Mon, 8 Jul 2024 02:29:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C774A6B00A6; Mon, 8 Jul 2024 02:29:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A6FE86B00A4 for ; Mon, 8 Jul 2024 02:29:14 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3220A1610E8 for ; Mon, 8 Jul 2024 06:29:14 +0000 (UTC) X-FDA: 82315608228.12.382AA49 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf23.hostedemail.com (Postfix) with ESMTP id 6F9B814000D for ; Mon, 8 Jul 2024 06:29:12 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Udx6ZKiI; spf=pass (imf23.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420138; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wVyqKA+nApm/aW8DpcUSLYRZbT7zL/VqgGna7L1YR1I=; b=GBs7DrYzt20ylZwk4rf5qyePxBNispKomqZ2VYsjIoVO8p4Voh0K6fDQdYXtEVx61QMj/f I7PH1SsTWmnKiKWp9mHd2KE4PIF3DptZI7Thp41wqh0Lx3TwbU9K+OIF0zwUywUpzju55K PWjbN+y4zYjl+oj0zeggZ1mkKpgtiOY= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Udx6ZKiI; spf=pass (imf23.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420138; a=rsa-sha256; cv=none; b=LTWYF0zwkAOoSRz0LzUMA/snlcnSa1Ymlx1oHr4Mvr329NSEoWzgJIx6hC0yR00lKRtNQr y2F4ZR/J/oFwpkI3pGLXvneZRIfA+1I3yp5FdLwpv8w3juW1jE8L5sX3H1F3ZJKWFRVqBD 8GG2PA1R9ADoBtegxEXIz5kuu5k8LFA= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id A2F1860ACD; Mon, 8 Jul 2024 06:29:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 61E6FC116B1; Mon, 8 Jul 2024 06:29:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420151; bh=kTQuh9UlyfPp1whk32zz/TIJv5bm40Slp4CbPrYPxLg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Udx6ZKiI6UdDrWncO84TIziVamqJ5fINWm6oO/wLz3/XChs2OG1+FQ3/2CXFk5bsB +xnt0gGqyIejEqOGH8mFMn3jHKC05UXg68HadyIlWmoZJevSRe1mMUblw0apD2CvLE VVDiiNoSjd3uqsz3OlEEfcUkLSZ9kpfZB1DNcQ5zm1oZJOptjMhdj7m84Q8I/TYLUA Uyghf4PZ+BvaS3mhbU6a5VGy9FfQEJAtExYAJoU6t+X34kjcmzv8Z3mYH4eXeu+65d Qot4m1jNXsFwaryY0Q6swUwKwKZdsvF0NOycLP9VHpe5l9TAuhtbK8/5ogoE++r8jW umZhKDt6S3Gog== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 08/20] mm/zsmalloc: convert init_zspage() to use zpdesc Date: Mon, 8 Jul 2024 14:33:29 +0800 Message-ID: <20240708063344.1096626-9-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 6F9B814000D X-Stat-Signature: wj13e3ddxkesm9hpgi17ribmgheo6ha1 X-HE-Tag: 1720420152-612989 X-HE-Meta: U2FsdGVkX18s+qajTgaOUpM0J7bB/t50zKFAWtXQ3roa/WW5eg+Rsv2qtoBn4I3Ng3f0nQXo+pweC4/IjdQKceeE8rINCMyNC8E+P1/b22Tm/wWhZTEjR4L48OAtp+miIpgyCt/7YS1SNK12D4nHqHkMNcshb2vCBRoWoopPz85IIDLnlk2eRjulgSWdyoGLBhPMIde7EO1cmbbUv9l4AIVpV0qsVha2OoREyuiYDUA7lAhFp6AYs9VrKc+H9Q8Wa9N62+1+STew+ut0kGbdgMrTFFjkXMTjamplSa3msg8+6JC5ePkrp0nDhrS6UrFMy7NnZK/9v+QuJRLrOxFl4Utklc4q2WB10Sn2jmqpG7L9N5F1yb8cX4YDdV5bZvcbIsZ1fw1VW75nBvfMigrad480JGRpRc2OhIhKE1I+UIIG58/v/soJcvegTNGZmDCZI7WgErCJhZZdTeI7Ev0ie/syjgH61QeCGQrbIEllsab9BnUKKDqjO9DuZoz1CrBHpRMBqtA+Yr/Krpx8fissF2oG1B4LrIjy1aQ+/nTxb6pF/+JEqdPYkC17mx88p12G96dO2YFCH5MEAVOLM67819+MDCakWDFkpRlDhIP4CLAkk252wK9rKEYCJn6QwFYR5jMEEdCa9RpV7A6QwaJZfH3UO1JOqMt5Opw1C3+ZIaNesiEWk2k0JVa919EezLNVrhLtlzr7NLJT/tOXOEZzp6dKn9wVwOC3TtWL4iH24dNM2otkMYpi3pySrxUxvTxtOi5hxvv0eRZvnA/S+1QMZ8nUmL55FfeHcQUYWZ8J4AmMP0AITtbDCcjPVnODuQS0t0zgbqdmRcFMhaqQPpJFTyFi5WHa9ByEbYGjRTa9r5Fq1xOzHH439bgMqkWCRN/LSgzIHCckn/Y9o/z5RHZZ/YKPESxcIoZPguMDY3L3JiSQZWGvPR8COE/x/Cgyg8zS447lFIGMGW7d9b1b81a IZfrbHCk K8/+hV1gl5VD1QcJhvfvhHfTtmozSjS8+7AdProDE2qBvOwJtWM9xrSopk9rFIN5QA0vQ4zSYgIabEhrA+PCp+tDG54MsFlu09kaeqkh10P6bpmuijUiFowFo+qkvBfIHGH5tES89XPXHEsbRgF937hLVmUN+ANcIaX5Cyf7UV01I5P2EKgLbXSnFaZtCD/ABBM4tt5szXLbIcQ96ig63KOK8lyLt62tHRoFE+PFzzqxYO/hgyYfRcbebf4C+WOyo+QA/4LygSTMTogRO7NLzKZXugf7e+YB1LX2XwuMY7DYxBQxtLxjSLk84oDnoJ2OkZO7hWw+fvQYALcyi2JPf6tNQwZMxSdgZHMuY16pcgv3HL9488Ecc1iphW5Dskas7wYsm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Replace get_first/next_page func series and kmap_atomic to new helper, no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi (Tencent) --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 29b9fa5baa46..d3558f3f8bc3 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -948,16 +948,16 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) { unsigned int freeobj = 1; unsigned long off = 0; - struct page *page = get_first_page(zspage); + struct zpdesc *zpdesc = get_first_zpdesc(zspage); - while (page) { - struct page *next_page; + while (zpdesc) { + struct zpdesc *next_zpdesc; struct link_free *link; void *vaddr; - set_first_obj_offset(page, off); + set_first_obj_offset(zpdesc_page(zpdesc), off); - vaddr = kmap_atomic(page); + vaddr = zpdesc_kmap_atomic(zpdesc); link = (struct link_free *)vaddr + off / sizeof(*link); while ((off += class->size) < PAGE_SIZE) { @@ -970,8 +970,8 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) * page, which must point to the first object on the next * page (if present) */ - next_page = get_next_page(page); - if (next_page) { + next_zpdesc = get_next_zpdesc(zpdesc); + if (next_zpdesc) { link->next = freeobj++ << OBJ_TAG_BITS; } else { /* @@ -981,7 +981,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) link->next = -1UL << OBJ_TAG_BITS; } kunmap_atomic(vaddr); - page = next_page; + zpdesc = next_zpdesc; off %= PAGE_SIZE; } From patchwork Mon Jul 8 06:33:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726197 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D09C1C3271E for ; Mon, 8 Jul 2024 06:29:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 686376B00A6; Mon, 8 Jul 2024 02:29:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 60CBB6B00A7; Mon, 8 Jul 2024 02:29:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4AD5A6B00A8; Mon, 8 Jul 2024 02:29:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 26CE36B00A6 for ; Mon, 8 Jul 2024 02:29:18 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D5A3AC10EA for ; Mon, 8 Jul 2024 06:29:17 +0000 (UTC) X-FDA: 82315608354.09.4430BEE Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf14.hostedemail.com (Postfix) with ESMTP id 2DD52100017 for ; Mon, 8 Jul 2024 06:29:15 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=bj79AIzd; spf=pass (imf14.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420128; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=phQ8tz7yXBkNlTZGC9m4+DenCUupj5avJ6HqjXj/9bQ=; b=MKHRPbdgYyN16v2goG7UhSEOXpc1rCTG/Y9hdh/jFA6BUxLBO0B3IHoCo31/8BmYhPjBxm 509u+DeyVvGERvn/ZuWsVARzesI/6lWVxzWwN9rmJRxwCHOrR7b9PIz1uD6N4pBKolXlVx O98YpPsj4JvqHOcFJ4qw0URnasgOKWE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420128; a=rsa-sha256; cv=none; b=YKZKcIDtsbrhYTpSxnJU3whD7E7ekkMHYmu3cvZj/diLJ0hg+0KbC6vKXK1jBKIlEdb3vL OIxthULVrw839OOBzSUx7+qIkLDVD+Y5C5gwm3POq03lI+wKQRbM187D9VRwJvp/oSKCBX X1W6rDVJW1rJ1qDK89u8l7fsyFngVjU= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=bj79AIzd; spf=pass (imf14.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 6553E60AEA; Mon, 8 Jul 2024 06:29:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 23D9DC116B1; Mon, 8 Jul 2024 06:29:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420155; bh=+/6M40wiCxzKTfpFU8y734xJ8A1hop+0lKYvujpVbRg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bj79AIzd5Z96XkWJssrQs+SgHiVAhDvkvP7hybFxHBlia0CX41bCz2dfAe1FEvavy ucW6BDXw013iA/lgkUiD5DgFHRea6EFDOX9+8HM2uM9TPPJNuJ1VH1g5XqEj6bfFTw 9mHe15jm4warZWmrhLIn6yMxXXdVaTxcTksSp1MlPWPMoBlnxjIEXt48H/+sYbZVF7 GCslgTjIOKhj83dyvlBFsD3uBjPddg4JuIZZAjASa4DLkpp5j746AIFjUg7QXG474a 9/l1EE3zbQ1sWJVejTgqd9VqAr9xEOKn1aYKN6KBo+pSfT2RLMFmBjoVb9jsRauqSC rznbzr55ZCVBg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 09/20] mm/zsmalloc: convert obj_to_page() and zs_free() to use zpdesc Date: Mon, 8 Jul 2024 14:33:30 +0800 Message-ID: <20240708063344.1096626-10-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 2DD52100017 X-Stat-Signature: 7xuwhrtp64zhxzqshbhen75fypn1zzk6 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1720420155-620377 X-HE-Meta: U2FsdGVkX1/zM3aE2W4LT/WNcM1hF5C5gb7fBG1jUdy1ZaiZh1181uos5zD7PRkj3QS9USrjK0qHgADQWHE5cY/2VTGhv8+EVGuCILlej/snUgwXxA8mFM4tUlgOVM9/vzjCUqf2KzPbSB7yqi+/uNWi/TnuH22bAmo5DyUGkHrePEmUqpD+WyATmyNY1yPFqiQtgYB9STaGuv9oC6JU4wsOz1lLWPb5CLOA24SHkq8+k5VtonNibnSNr1eZEbOW3FKFS+D1J+Wd2dBtq4PgexIFWJf1vCtbfvURWLVFJEVfdec9C2LGhZ9hjjLeOQdNLpGrcKIUDjJAlpdWLHyxvs3NwrJOmtBHNas99fpzxiC3ZcVDZUVg8nmil1lCz29gD+bvHjOoRI3cKVJvsH2JUxRDh+i8LZMzfawJF7OUF9ibjRuFP5Tt/G32Xkk4wHvDBy5/fwbe6MREEEfaOHPw95HmyyR72akVbSjORHaQSzoPCMS1qUuOyIAVi0Rf43+Vhf/+QHw+hyjpDBXYYd+v1VhrJEzxms1HbTahKgT5YlxOPkTtLVuW5HhYIdhK/9BNw4G5Ty5TmoDtICiMffqIFMVW6StE7bU1KjTZPLu6P0JG4ohbnaiNIQs0wKB06hmHGLs7eadeHnfSBkgd+SjtFM+Tk9Xze4gQLuIK4Yxwo72QzWmFsT7zjUKsMPXaQtzA61xufAZ0l7gy1j2q0lpWFn+PrwZ+elVvzlBxJd8MvYxkhSU1Kijjue7gutUEtGnZnGt4YMI4Q1+Ht3wfZjcsEFxVRtfEJQdT0x1uIkMiWe4ycvgj8J2cv1FtnZ/SI0ephaP94fCSRam6qYGLCJjVb2huEfGQN4G3hfZP53OXsX86XjTBb6unZCR+hWMNdtLSNkSZGCXelagz7EhKJi7HMi7xHHZlgAR0RkaXYStmPY66hY/bgneMo38U4Z3OLaID44Vn3+LAOuEAImSST8S KNXzOvbH pyep9ptNjwonOs/hB1B17NxeQdI1i46YYodCZe1J6rsVrQU6bakOQe/7Ac9tSoDEPO3VFsl7nbITGUmbKIebJ9svXmpsTO3UeOaBfKGSgmfCyQ0p7NhrZZjRdwK+uv6FQYCGdAAG+9gK8leB7/ksuxfSWNsexXhQo8yP99l2SkylSsCpOFPHs7JfKBWSUxuDr7dcDgQPfLVGqJ6W+0u+KOfAm9uaCiFo/ktCM5MPmKR25deNHHt7Q4mutrnbD3JMUZObNFP172LQeqB6hSL6yPOvwQZ7ZEMKPSvHL2bniP5BgXxgyeWjc5M+W5ZQIbbo6C1K2ggc1nwCuoCRLKqH0zABpzl4AsQGCJamZSKxLK9pK6zZUWOxb+Eb8CAzVWPIuh1m/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Rename obj_to_page() to obj_to_zpdesc() and also convert it and its user zs_free() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi (Tencent) --- mm/zsmalloc.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index d3558f3f8bc3..7aa4a4acaec9 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -820,9 +820,9 @@ static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, *obj_idx = (obj & OBJ_INDEX_MASK); } -static void obj_to_page(unsigned long obj, struct page **page) +static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) { - *page = pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc = pfn_zpdesc(obj >> OBJ_INDEX_BITS); } /** @@ -1496,7 +1496,7 @@ static void obj_free(int class_size, unsigned long obj) void zs_free(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long obj; struct size_class *class; int fullness; @@ -1510,8 +1510,8 @@ void zs_free(struct zs_pool *pool, unsigned long handle) */ read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); - obj_to_page(obj, &f_page); - zspage = get_zspage(f_page); + obj_to_zpdesc(obj, &f_zpdesc); + zspage = get_zspage(zpdesc_page(f_zpdesc)); class = zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); From patchwork Mon Jul 8 06:33:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726198 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F27DC3271E for ; Mon, 8 Jul 2024 06:29:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0B7ED6B00A8; Mon, 8 Jul 2024 02:29:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0410D6B00A9; Mon, 8 Jul 2024 02:29:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DFBBF6B00AA; Mon, 8 Jul 2024 02:29:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BFB0E6B00A8 for ; Mon, 8 Jul 2024 02:29:21 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 7D6F7A3172 for ; Mon, 8 Jul 2024 06:29:21 +0000 (UTC) X-FDA: 82315608522.21.6D8FE16 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf12.hostedemail.com (Postfix) with ESMTP id D6EDA4000F for ; Mon, 8 Jul 2024 06:29:19 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=hebeQ3k0; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf12.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420127; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tp79eLWX50qbmgcw+JNCh0sTs81YDiRvHuTLRpwVokc=; b=qqEdrgkZacW2cn7vmKvpPkzIBQJJVM4cTy0Gw1CUdJcj8adpTdwvjpcQTvI2TEYw7EwveI dPo0ki15pM7677PQLi4yVAtz47PUhTeIEi/iqf+pS6f9nB1gxT13WLTOMRwEYHnw3e7Pv4 vus41R63Ygg8xZhkaA1BGv8bAV6xQ3Q= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420127; a=rsa-sha256; cv=none; b=wBfeps8v3qmhw7BUMRutUfpYCminAEdiiUDvxEYkaKIUPd5nqBf38ef9Fxr0mTUw5bPELF 8BjS+MIxU2SsF7pwC9dDSQYwA1x+O5EJdn72J3TstNbIXVVYCO2JuLU4DoSD20bFQ2gAPM rcNJTzylCvzmtZzFVBz/9PoOhgR5W/I= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=hebeQ3k0; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf12.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 1D05E60AD9; Mon, 8 Jul 2024 06:29:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D6E61C116B1; Mon, 8 Jul 2024 06:29:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420158; bh=eLEoZgV0ypx4IcCw8JtJdFgQAqXajv6Dsx6/oMszAA4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hebeQ3k0Iltjd3ERyywf5cF9ODSAm8MNeYFZ8QxgekeMq8VG+m6vhC9W32CO4ABH+ VNQ5WsibBUHoCie0KvD+wxpe1WwWSm8Mz8PAfB/bZT2Sf2qGd6xr+Q9NIGJ/m62uwG o3gZug61V9v0A0bPozSxKy3eoUDzZ8U/L9/FYbbPgp2F+wsCcYJJ8qqBYoHl1nEXZY iI5i+1DBM8BbNOgDhV6ZehdXQXE80fsySTLO/uiZ8Sb6krWbmGzBEH6jr98MU30fHI vbfw3bUjscXkgBXIjC5Ar39fojb32Wmqggul321fu6Sr4qllQHUwE+bEWRbsjTo/jz exGZ62Z3odasg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 10/20] mm/zsmalloc: add zpdesc_is_isolated/zpdesc_zone helper for zs_page_migrate Date: Mon, 8 Jul 2024 14:33:31 +0800 Message-ID: <20240708063344.1096626-11-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: D6EDA4000F X-Stat-Signature: qintinb9yg64toui6ztuzy7jxmq9icgu X-Rspam-User: X-HE-Tag: 1720420159-622186 X-HE-Meta: U2FsdGVkX1/uvZAGQ+4eIrr0J+2bApgYdn6DUKOAgcxdewdy1L2IhqQ/RHo7AkBTnRw/y0WrNaNHF6hAyaavxljWJfVeyZeUPYqYIy6qRwkjehYyuGdR8VUDK/P4rV/fvADEsVky9eKzvmFyVniK77pAGDGuqXhcGcfYcIw3vlpK40Qef6EJ4g0AEE7Lcuw6k+D7I5PXwhFLNO6X2Pb0K3OwOTm2LXIBZyAYbMxzdIiZO+DeC09yyE3xVOTfumoaDYPknthDOuCU77ZUbGwv4Fu03m8wvY8HwYZMu5Ei5U33DyTllmot8dRo0M4hvLfkSFjTBHPUQKPj5EPKdIfCTPjmfiHWpB17eZjIitoBoB+y4natpqy08zCdl28bXjEVwSVWzFlX3XJdjvleam11zvpJ2o3P7ByjC7TZA8dznH8betMtdv9BhkwxC3ZuNuqwwCI/jt2KMuyR9ZQr44HzouAALK93jGNqCtGth7GVGSs08smE46CeUD6FCvr5FP2yicD4F9m2FRRQ3+a/KvRV+1PhFOxW+DMvs8cq4Fz+onw2cA15p7Za1sZdA14HgzrGiAKKzNzHD+0xZTLWvrqfGE1NQj+w2euWXk51dr2QpFyn7N14hjJlVgG9CJPSyCNy805tVvsw/6gGNBxD/e16YiIhhEE3Oc+nN3pDXmV4B8aOx5H9oVe/idU1k8k/+YM9+NxLZ2wGvy9O4hEKNel6d5XVxhZG7EOq8dNCE2zl82QrV+LqYg3+g5erZSPXzGrlYWoYZEoWgFPEL08JsSyHILLj1ZPAOl9Lz+Cqya5+ulAU34bFXMszIYK2bjbPzld/g4/+Az++z0bWVC2Ed5Lumbp3gASjLFEd6cD7Gl00PFm04Fylffvz8P1L7rgbPU6SlFP2bJxlcA/J1LS5dFl8hy/jVuYUEKxOswFqi/fsOPKAo3gUO1yEJKxm5c7T1WR1n7kzzSwCBX9Eb8wHq43 yJt3MUVX ebsBaXXORGZ4Xg1wDhngRbv/Z9tUDUBnGJz6IKiyUOq9eq8yvEcw7m93yDNy6z3eeqb/DHs8xndsenaLxDlXaAtOA5wmsFp6OSf9sFQVN5qhrLE8FlgUGPwkaPNl2InNWeRwRipgEsF0pPHRhy8K97o1+zYvATIw7HzL3Cda/76iDbhMoJhwu8JsKRJijk1S1jLVcpf3RkKR7JwHuX3cgoa6bVeCcPSgcqQ8OZZY4GgvOpErbFOSWs9eSUhd7xWH0u/6HJXLKMA0a9eIGJunYpzLwIPzkanlMNRWfreHPndCLF0oHTnAY6E3AKTM1gUmsCsmKXV77zSwGdp9Xe6yzM6wl6LZx9XT8qmbNM1ZNsebpVRZ2zRhFhrpYQpNHGeD2aUQG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To convert page to zpdesc in zs_page_migrate(), we added zpdesc_is_isolated/zpdesc_zone helpers. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi (Tencent) --- mm/zpdesc.h | 11 +++++++++++ mm/zsmalloc.c | 30 ++++++++++++++++-------------- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 2293453f5d57..ad04c8337cae 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -108,4 +108,15 @@ static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, { __SetPageMovable(zpdesc_page(zpdesc), mops); } + +static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) +{ + return PageIsolated(zpdesc_page(zpdesc)); +} + +static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) +{ + return page_zone(zpdesc_page(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7aa4a4acaec9..9bc9b14187ed 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1830,19 +1830,21 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct size_class *class; struct zspage *zspage; struct zpdesc *dummy; + struct zpdesc *newzpdesc = page_zpdesc(newpage); + struct zpdesc *zpdesc = page_zpdesc(page); void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; unsigned long old_obj, new_obj; unsigned int obj_idx; - VM_BUG_ON_PAGE(!PageIsolated(page), page); + VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(newpage); + __SetPageZsmalloc(zpdesc_page(newzpdesc)); /* The page is locked, so this pointer must remain valid */ - zspage = get_zspage(page); + zspage = get_zspage(zpdesc_page(zpdesc)); pool = zspage->pool; /* @@ -1859,30 +1861,30 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); - offset = get_first_obj_offset(page); - s_addr = kmap_atomic(page); + offset = get_first_obj_offset(zpdesc_page(zpdesc)); + s_addr = zpdesc_kmap_atomic(zpdesc); /* * Here, any user cannot access all objects in the zspage so let's move. */ - d_addr = kmap_atomic(newpage); + d_addr = zpdesc_kmap_atomic(newzpdesc); copy_page(d_addr, s_addr); kunmap_atomic(d_addr); for (addr = s_addr + offset; addr < s_addr + PAGE_SIZE; addr += class->size) { - if (obj_allocated(page_zpdesc(page), addr, &handle)) { + if (obj_allocated(zpdesc, addr, &handle)) { old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj = (unsigned long)location_to_obj(newpage, + new_obj = (unsigned long)location_to_obj(zpdesc_page(newzpdesc), obj_idx); record_obj(handle, new_obj); } } kunmap_atomic(s_addr); - replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); + replace_sub_page(class, zspage, newzpdesc, zpdesc); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. @@ -1891,14 +1893,14 @@ static int zs_page_migrate(struct page *newpage, struct page *page, spin_unlock(&class->lock); migrate_write_unlock(zspage); - get_page(newpage); - if (page_zone(newpage) != page_zone(page)) { - dec_zone_page_state(page, NR_ZSPAGES); - inc_zone_page_state(newpage, NR_ZSPAGES); + zpdesc_get(newzpdesc); + if (zpdesc_zone(newzpdesc) != zpdesc_zone(zpdesc)) { + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_inc_zone_page_state(newzpdesc); } reset_page(page); - put_page(page); + zpdesc_put(zpdesc); return MIGRATEPAGE_SUCCESS; } From patchwork Mon Jul 8 06:33:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726199 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A5C0C3271E for ; Mon, 8 Jul 2024 06:29:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ACC326B00AA; Mon, 8 Jul 2024 02:29:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A796A6B00AB; Mon, 8 Jul 2024 02:29:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 941CB6B00AC; Mon, 8 Jul 2024 02:29:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 73C126B00AA for ; Mon, 8 Jul 2024 02:29:25 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 2AD85C112F for ; Mon, 8 Jul 2024 06:29:25 +0000 (UTC) X-FDA: 82315608690.09.1291DFB Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf12.hostedemail.com (Postfix) with ESMTP id 8F61840006 for ; Mon, 8 Jul 2024 06:29:23 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=HA0Ua3KN; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf12.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420131; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=02jNdsEZ0aezxhuFGmcStssVo+4mij05629XzD8iNVI=; b=CQ31TRLzhvw5S3/d5Y+1INh6OTHZF3Fsd6BPuWtQ2t3lSctgCA4IqQ4+WgBzNfpkgknKlT eQYY1TOZxxvR1MXXx3D1TFGiex16DiQnd0UfPr3b7uDUnaZoABiuuzf6AxfmOEKkGJGzj0 u9CiBcebuYXrqzXKzSbUs0H6xcCYHqU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420131; a=rsa-sha256; cv=none; b=N4ISfUP3BzsCCHXqCB0c6FkLRN8QimP5yuZXdwmHDjwlLv8UxeWrF0ojxbuiPyvbGQ8sNF oBgbekQAW2FMeEaXzOeVrXucIXaUG5zMilhqmFoPTLFpIVIFL5PBWIbseYDqaspB8Sjqhb X/9DKuR716D6unhTsiGWLiICzVOBriI= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=HA0Ua3KN; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf12.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id CD17360ADE; Mon, 8 Jul 2024 06:29:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 70850C116B1; Mon, 8 Jul 2024 06:29:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420162; bh=78ef+c2vUld0iffCBKj3cSVjaJfSet6R8UzdeRHd7pU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HA0Ua3KNiyLRSlvuk8iE+wvrkOCsWZzp0NhLUjf5s2/M8n++cHNi8RMTSGlC5wHEi ga/OjuScMR/XI5u8/qk7ckMACpo/8r2jtib0KRymUrBa73A5JOs5PbPHtMiENWQkNy QOu4Lx0gn6RJy2fFwwquEray1BrfIRfvhthbzt8ZR6j20/xDY9vX2VTSKs0+G2bfTX 7nl5N/ADiWZtJ0N82eBiTBBzWhTrr+C3DTl+vxqFhB5GqjpdZXcYkXFZ4x+4kdLCiZ 8YaAWhVw7ch+c7m+UhwS5zWToaQnWLCaCPvCPKJtR6CF5CUyoNhQpwoJbIQfLstmLI /YL5eKNTw0BKQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 11/20] mm/zsmalloc: rename reset_page to reset_zpdesc and use zpdesc in it Date: Mon, 8 Jul 2024 14:33:32 +0800 Message-ID: <20240708063344.1096626-12-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 8F61840006 X-Stat-Signature: 1w764t6b1qywzi3zw68qz8uo65ms3zdx X-Rspam-User: X-HE-Tag: 1720420163-606344 X-HE-Meta: U2FsdGVkX19WQHdJirFpp1mqtOl2f+nQj13PLl/Ne+8xrqSRa3zLBWSvr9eoazDzKGkHRw6SI5l4tdfW6Eg5U3KbBMxORApXA4eoUOu7W0UgC9gql3jk5o3AtmI8ZXfBwJLp+REwkYM17sobZLf9xrha3e1ynUzseu/E6tMFUIOJaxD8kPy+XujOrxUkWPIWafiQipjOCec3jbCuUocLBRGmkIx+SlNNr3lDV/Qj6VKs9jMO68wtNUewRZWnnppKPHcVCUtdE9PtDwrUuZ1PY0hTpCp1Lfdj6FxiMNnmCZmew6pIjvOhfArl+OCoqmgammxfnZfL8bDE7pA15ix9zPRpZT0k7+vvDSz98Ffws+UbQ39Anwh8RcmnJfPWnhTDysjtdorn/hwjeQto4cxnXrpwYUVEB3L0mQuwZB/1EDwRxdblixHSaZ3UiWqiFIqa8zrKMKFu2SsysgBicnb5+IkTGs0YEj7dOiJBnWPW9CmyEZgrF26eHRu5CVYPRJQgb+/Oh8pS5VQHfGW7PBHQMOaCUmCt6SxjJoEzYwDzS5FW53ahsTX6tdaNxVU9FeS0TuzK2CNHJgzf37I/QSsdSoZGDThtuOtMXvBY4vDhzIBXP8YBd3NuRIqWDMBBYcu60b6LvuFRwIjYfeXiSg7eY0Yg9RXCzN9n2X84WWRCRf/ogRMEb1FA2VdTbfhDWg/23S5a3y72sYERTnhMmVuDxLI+XmUkPFEUQ9AIFYb4Sg2msf6z+YDT/sy6PxZBTLk0KOinsnYNyaGq5Jyo/Iq3yWLmlMoPfi/kOsPGhyOmHiVbLl+dBGUGXg/8mJ3grooa/USGFGLUfD4pbLXz3Ye3FfAD5j2a++4M0XqEwgqogDgOC4wotIuYetH2KzyYisNnvvDnpF3yOdfC/GmGpvVIGmBBxFjemZuwdMnLOHYpK9ysnhdfsXbMxo5nlHasgO6bk3//1tNeoXpLF9UoxIY cUuZ8XFo Ij/xhCZngk8T6PnDBnreNskE3ZVSI6Zk5pbM1wtqBYC6lCHA2vTd2eob8Q3qtbvhEmBYnOdeytCPSoNzW17isyVyQYnsnYz5xq1ZCzzgXLfOuCM8uj9K2DzBJr4c0XKP8NZXA/65PK32nqU3t2GGjV2uhqeIiGtXfiPWyuIU8o0rcPBVnU48odhQPN2yL+zFoTf16jauVmOVVnj80AGzpZ4njG2uHk7MGvO7hl7bmmaiaH6jCPxB7+W7p+PVs2n1PbtTokVdomIrtFZWvxOQGsHNcow46SgjGklRfsElQz1lEhgTVvTW1IsTs3qLW8cFVol76oTMEoSdPE9LBtg2tzkNxFytgypNkhmrGuAhQ/d/CDVE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi (Tencent) zpdesc.zspage matches with page.private, zpdesc.next matches with page.index. They will be reset in reset_page() wich is called prior to free base pages of a zspage. Use zpdesc to replace page struct and rename it to reset_zpdesc(), few page helper still left since they are used too widely. Signed-off-by: Alex Shi (Tencent) --- mm/zsmalloc.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 9bc9b14187ed..6d1971836391 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -865,12 +865,14 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, return true; } -static void reset_page(struct page *page) +static void reset_zpdesc(struct zpdesc *zpdesc) { + struct page *page = zpdesc_page(zpdesc); + __ClearPageMovable(page); ClearPagePrivate(page); - set_page_private(page, 0); - page->index = 0; + zpdesc->zspage = NULL; + zpdesc->next = NULL; reset_first_obj_offset(page); __ClearPageZsmalloc(page); } @@ -910,7 +912,7 @@ static void __free_zspage(struct zs_pool *pool, struct size_class *class, do { VM_BUG_ON_PAGE(!PageLocked(page), page); next = get_next_page(page); - reset_page(page); + reset_zpdesc(page_zpdesc(page)); unlock_page(page); dec_zone_page_state(page, NR_ZSPAGES); put_page(page); @@ -1899,7 +1901,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, zpdesc_inc_zone_page_state(newzpdesc); } - reset_page(page); + reset_zpdesc(zpdesc); zpdesc_put(zpdesc); return MIGRATEPAGE_SUCCESS; From patchwork Mon Jul 8 06:33:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726200 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59A77C3271E for ; Mon, 8 Jul 2024 06:29:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E74386B00AC; Mon, 8 Jul 2024 02:29:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E22A46B00AD; Mon, 8 Jul 2024 02:29:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CEBC16B00AE; Mon, 8 Jul 2024 02:29:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id AFBEC6B00AC for ; Mon, 8 Jul 2024 02:29:31 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 536E014111B for ; Mon, 8 Jul 2024 06:29:31 +0000 (UTC) X-FDA: 82315608942.27.9523F86 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf08.hostedemail.com (Postfix) with ESMTP id 53D59160002 for ; Mon, 8 Jul 2024 06:29:28 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=RiUIttnE; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf08.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420137; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=V6ERS0tQTmFArjz0DBTTuSqJLGZAXlVcoV29bs+X2Eg=; b=PjbziEcj4WUuPFpngcaZPQwhCVmTmGomxYdkTYs6E/0/T+4AaXp0d1dZPBYAmqqUmbAMQy Ws1Wx3rMEFOG/GozCpP1tZrzu712gyobTGdeoXKqnkmdHNQlHIFphOR1SFreb4EdUYOo3w 9dhQUTLJt81i2NBpdgwhsFdbqkhYWuw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420137; a=rsa-sha256; cv=none; b=tJpmn1sIyuZeN9mLqC7ay+0S1Xo0zXFyMARXyqOIv7KTVz+IIKQt5cpTxMRiGcZ+zp301s jaZ9u7lcxpE8qPUifhES0UQ3n084YtHd6NSWAhsLdcgK8xICfJWd9qW113z0UdR0S43Feg yIKB3Rfj8cO3gz2Kijz+6RiQrCVYrAM= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=RiUIttnE; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf08.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 9B39BCE0AB8; Mon, 8 Jul 2024 06:29:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2D450C4AF0B; Mon, 8 Jul 2024 06:29:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420166; bh=F6gyGa/Nhcaa49tqL1AcCvvFaZR6EdIliZgtM0OjhK4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RiUIttnE915B5POlt/rGkjRDxwMkMUiIsyhchqZid2hdw6qQfozwkJ9cRUtOfaZWY ecSVJ4QQb/fdFBE9zfzyTA2BZQ/RZsYI6z/9adNmf/+XhmJydHF+/B49U4BOPli6FZ i+H2xFep0Wh7TcTsawSQ0/5F+Km/FzU4iOKH5y+BJhwbFrL+YhRlq7VBl8TY9/DR6+ 9B9+DFLrjkjS9AcDH5+JUEVesyTatZ396Mq4zYeXPyscrToCEr2YgnPtcKHsDEslgD MVG6bEWLnGK/BxKoGIZwYSo/4qLqon08Zeykg32nXyrja7dsffK90F6QWlZSv8eCUu qhdiNIl+Xz0Ug== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 12/20] mm/zsmalloc: convert __free_zspage() to use zdsesc Date: Mon, 8 Jul 2024 14:33:33 +0800 Message-ID: <20240708063344.1096626-13-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 53D59160002 X-Stat-Signature: jrjdkcmtiuj916riji57obpigzhaosdx X-Rspam-User: X-HE-Tag: 1720420168-822695 X-HE-Meta: U2FsdGVkX18v4OxGMXkCd+UT/RZEc3aJ5u1jTOE+LDre3zNKmi/qqFmCKWN/VuNmZRiH6mafFOVxCGHc7iM6dQEw1cW1JYUY6chwypUikPjRP7LZOAkEzDA9cY2JAjABheFSU7W7kpQ9LHVBjV0WaKue8zBJJLsrUxg40HETImJgzaUfmpFXBDeXoh0NfbSDzM925v9jYUcu4LOXTV7jZj70d7wuIvFoGSEnd5G/NYZkQNiIP1775wD4tyd81x9LFzWwSJPN18PTyMQTNeCK+zIBxG9Va+zy7C7HOLEacp0F78yb1PhzCuUEy2JGmCfwMmlKT1Osv6PaTIsT0bIgT9OFtaMqgn0WL6FM2cUa6h88agirlXougpY0qI7LZwsKJsFDUTp0/k0e7sunJ/06PwJ9SVRgzw0a22g1ImN1tS84EZs0ya8XyJXgnfe1keDbSITw0awLwuxobQFdqIRdj5Iky//7tUYSD0AyZ25IlSEUEkfr+uU+gFdKU7uyCN3vlfEpRZbI3ffPr1pzC/SrEVCxFe+c14d/RsYxcuveTzI0Zj0CXJKuYrZQYjzYZSLtTeA2Qr8fNeVGCGqe5BPApwCeOJPox/WW5R19fQ+Oqyw5eZGSuhiXilv9EXCzBjRu27+Au+wwj8AQWUPAhDRNyZWLTfs2nI0OecmS5c+EP76hk/EXqq1LAOey2/MMGv+/kkDKcrHDsd0/L3jn70bcIx5INCa166g9vGkdpcIXs6okBuczzBhv+sWjN/gfB/k74jEk2rylWQ7YoDB9OkmpqgQRdgKQ3GqvBOEQayqeUoIggTOZew66YiVTz1vxGC4Rv0QBXC/CmAWegDN7sgbrY6alTUU3ME0qPHPYmRTX+40IFUQ6iSu7gH5pI1UPkI3MpAV1Fq2YL3VU0OrGYJ0W+0037T9dt0c2TKnJego5IjB5ug2fUVh6wO+/zzeE0U6xdxo/7s2X0OAmDPca5N7 z8Yaebum OjHP0ux9zDoYOjgHyWLkIpsxYzCeY38CaQE1ZkrXxfEPp39+7yFFVCoqUC69bOjRlJaJJL9VzsnGygM9dLO/GiiLiRq4/BY/boUJ1NNmWwxvGE30hHUhynVkGwaCN2LN1egQzc/LRAfF2v3+zPi97jP293OIr5Px8sfRgFPXB7DXRHVYw9UDHcfuwFH/85NSOqKgaaguTuSeb5eLllE4Ke07m8uallLZBesjlG5PdQs6RvrFYHZy4kYNQarvBDTP6LWVQ1iSUgPp7XqO2FuHvh8wWwK0TQfjUIfeO0UVQqBd9tKuvUU0lThu5KenVmPQ8fHUBqvEcX5SW+nJhEpjYYNzcEEh6cDgIHQSoJh8WbeTB7oySiP0g9P+4fMdK0zPzWqWv X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Introduce zpdesc_is_locked() and convert __free_zspage() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi (Tencent) --- mm/zpdesc.h | 4 ++++ mm/zsmalloc.c | 20 ++++++++++---------- 2 files changed, 14 insertions(+), 10 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index ad04c8337cae..72c8c072b4c8 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -119,4 +119,8 @@ static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) return page_zone(zpdesc_page(zpdesc)); } +static inline bool zpdesc_is_locked(struct zpdesc *zpdesc) +{ + return PageLocked(zpdesc_page(zpdesc)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 6d1971836391..68fdea7b6e0d 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -901,23 +901,23 @@ static int trylock_zspage(struct zspage *zspage) static void __free_zspage(struct zs_pool *pool, struct size_class *class, struct zspage *zspage) { - struct page *page, *next; + struct zpdesc *zpdesc, *next; assert_spin_locked(&class->lock); VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness != ZS_INUSE_RATIO_0); - next = page = get_first_page(zspage); + next = zpdesc = get_first_zpdesc(zspage); do { - VM_BUG_ON_PAGE(!PageLocked(page), page); - next = get_next_page(page); - reset_zpdesc(page_zpdesc(page)); - unlock_page(page); - dec_zone_page_state(page, NR_ZSPAGES); - put_page(page); - page = next; - } while (page != NULL); + VM_BUG_ON_PAGE(!zpdesc_is_locked(zpdesc), zpdesc_page(zpdesc)); + next = get_next_zpdesc(zpdesc); + reset_zpdesc(zpdesc); + zpdesc_unlock(zpdesc); + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_put(zpdesc); + zpdesc = next; + } while (zpdesc != NULL); cache_free_zspage(pool, zspage); From patchwork Mon Jul 8 06:33:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0A96C3DA42 for ; Mon, 8 Jul 2024 06:29:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5C0686B00AE; Mon, 8 Jul 2024 02:29:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 570096B00AF; Mon, 8 Jul 2024 02:29:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 437CE6B00B0; Mon, 8 Jul 2024 02:29:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 227106B00AE for ; Mon, 8 Jul 2024 02:29:36 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id CFA7A120F47 for ; Mon, 8 Jul 2024 06:29:35 +0000 (UTC) X-FDA: 82315609110.12.E5F1E25 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf15.hostedemail.com (Postfix) with ESMTP id 99B1EA0023 for ; Mon, 8 Jul 2024 06:29:33 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=SVk5iNSo; spf=pass (imf15.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420160; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4tJHBOc18GmiecRtdLoq7MoV4LMDbclFzcv0ZoPmA08=; b=yhJi5GoAEDd+w1lnhCFW5wJ/R3B/yJ+MohQiRzbYEFAYR5d+vzxKXfE6UgoPxAoIzSwFIv 4mCiMCxKvUDx+5bWk2nAnUil72TaaBSzxeplAIv9S0i/1VK63Kcs7fLc2wxc0sESuOjnQc ediEaxuhF78dZsg+EuQp4WTT7b6ZtiI= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=SVk5iNSo; spf=pass (imf15.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420160; a=rsa-sha256; cv=none; b=NBqmBZlfT6pfcZVnUMbeR5ydgJ2Ed6SWahby8eKZgqYTF+uTFOvTBDvpA4RBw4SENaPrEz G34yMRglpY9upKTsLzkqH568G+050QRw1gnlIBNb9Gy3neo3VDOhT+06br0oE494D0SGkU 01wtup5DrYTYOl8VX2lbXEMeXjY7AVY= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id A9AE1CE0A58; Mon, 8 Jul 2024 06:29:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E1C84C116B1; Mon, 8 Jul 2024 06:29:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420170; bh=FoPFFhZjgJ7k8SPuHkOnh4ZOuMsrgriGi5An3x0tRys=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SVk5iNSoWEypU6s06xUAyZFEaiQ1Lg9SaOx3SHhS8HIM3gOa6jnzHjXuVpb7C+u+G CUS6gTMu1aJOHfYMh1aiodRaOueC1INYKRIwcbJuEHMDRzbhoYUUdn+dw/UhRWFQPb YOqyBMdjtfNuVZCvBM094GknA+ub5pPohmKxG+mvZ+Nm6SKp7IV/SH+zPDLRreMD2G sucCviNkh1ywa/cuflg9Ocj6Jy66eePgmkIiHVrIOVmO5I4t6DiBpMje8pB+6w7hz4 FOMaRLVBR7AfGWwpT4+oLOQkdenToTM2w5UgRWJw1jw7MRLjGW75nLcW44/UPvFNgM KHaM1Hk6JxHRw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 13/20] mm/zsmalloc: convert location_to_obj() to take zpdesc Date: Mon, 8 Jul 2024 14:33:34 +0800 Message-ID: <20240708063344.1096626-14-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 99B1EA0023 X-Stat-Signature: 98c4sz4y4nx6nxogohg9jz5w6ekw66g7 X-HE-Tag: 1720420173-673197 X-HE-Meta: U2FsdGVkX19Ov5berM7Rbw5pJ2HQbwfo/9p7JaXx7jvKDX7m9+pWmjcpu2Dyly8TWMWx5IEAYq8C8kr8GbSl0qGfHNCWKCw/JmkyITNwhV40ZdO1NfiW/CrTJnhtvA2lTMlCipzD7LaPa6Ak2qBkQ1CZvGuDMcW9T8z3922cUVEqaq6FU1qcR/mdWpXttNj2yNbUM+hbMpSYkif0zxNusHD8bP6hzQQp5sWBvt9uTZOpXBrCPYrKWmSajW4zILwImEp8wYSTZPDaJnbBNI9MHHeLd32SEflO6YP9YwO52UgeFj98YsZKNEV/LdjZDmsoLFYbnYEBb8lxkp1f+4+hyZPTQV2/pE1+qzUT1N2tRwERGSYs31k5u1KeygFo6FpjrmKtSPU6S2Jjxcx/uiCrlm8KHpsOUNezVqmlYs48qldlmO4ItIejZnqdDPjsjS0fIYtkjUX8+13SSkx9KGGnFmbcN0g++f9khZtiRA3S/ogaDcLGXXdSFLX5IbzDPSswOjqeh1HmvaThktXlI4mN3nHNLcBfL/QG2tHRsjVqDv2k4sW/dJCjGHfybgieObF0H4JqZrAA8Z7DZ3+WLyT7fTCW7hkfGxL3cfxbHcFBivmdAEvr4bosgtMMp2vPlL29+0HGo8AS/fqoZPdNPq2HfWH3lED3coTSsDNU49YuNxns5MwHvVWNHxvystvM7HxONO1nX4vgredcJ/eyLmXMW8PCbYnG2O1JG/wmHeQtQXUYShmlzSwO8DQltfBbyuCFBS0gZOO78EURP7YpvIguGqwsjTxW6PaA3ENatvcEjsONlw9DdshRmnGLRDsju5LK/DQgsupRSXzDeBQ0MlDDjhFCB9/Nk1hz7akea7tlfROT1Oxdp7gakYWGkqECZ1PefCYk+JWRhSzVY+lO8F97WPnzTziBeRO2+Z2bgtcOo+7UJgvURGgckBzDSQIXV5DBx53kKk/kuvF/+LlNrBp 5e1C0Htz jPlp+9kHUbjPfpleY0LevlOghxyNbzIDL1/Pj2ZOQZzkdemidX1DRfF13UFnKNBFiD18L2fQ4tR/+Dgbls7ePNpz6wBHCRiKxaqTjW1a1OC/PfXkEzr2lyHkAk5OG070/ZPFjOsZHElBTebj4zsj3gAJ+6FmZy6Ye00qRlmraQr8VZUlSW/gmgfAa+v+IDf7hzOpLouZ+LgwJsLyO5I/mGnUgtWXdcw4rOJjCUpYs6EtKIC5WI9/S43zLCA5g5jre0D9TupMMkqYenqQFya+kSo8pi41gtkHw/pTtBbaNvVTsWRr19agqVSsyjSOUWE9fuhVXTaIxx5ewE4g58U1gUCHXTn391/FXhjXok2r+7gS4ch3hdLT1jf661Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> As all users of location_to_obj() now use zpdesc, convert location_to_obj() to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi (Tencent) --- mm/zsmalloc.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 68fdea7b6e0d..e291c7319485 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -826,15 +826,15 @@ static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) } /** - * location_to_obj - get obj value encoded from (, ) - * @page: page object resides in zspage + * location_to_obj - get obj value encoded from (, ) + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static unsigned long location_to_obj(struct page *page, unsigned int obj_idx) +static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int obj_idx) { unsigned long obj; - obj = page_to_pfn(page) << OBJ_INDEX_BITS; + obj = zpdesc_pfn(zpdesc) << OBJ_INDEX_BITS; obj |= obj_idx & OBJ_INDEX_MASK; return obj; @@ -1392,7 +1392,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); - obj = location_to_obj(zpdesc_page(m_zpdesc), obj); + obj = location_to_obj(m_zpdesc, obj); record_obj(handle, obj); return obj; @@ -1879,8 +1879,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj = (unsigned long)location_to_obj(zpdesc_page(newzpdesc), - obj_idx); + new_obj = (unsigned long)location_to_obj(newzpdesc, obj_idx); record_obj(handle, new_obj); } } From patchwork Mon Jul 8 06:33:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726202 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10683C3271E for ; Mon, 8 Jul 2024 06:29:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 91AA36B00B0; Mon, 8 Jul 2024 02:29:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8CBBB6B00B1; Mon, 8 Jul 2024 02:29:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 76BE16B00B2; Mon, 8 Jul 2024 02:29:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 532106B00B0 for ; Mon, 8 Jul 2024 02:29:39 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id CD59B140874 for ; Mon, 8 Jul 2024 06:29:38 +0000 (UTC) X-FDA: 82315609236.11.36C8A3E Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf05.hostedemail.com (Postfix) with ESMTP id DA3E7100014 for ; Mon, 8 Jul 2024 06:29:36 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=UJRGhy3n; spf=pass (imf05.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420154; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wqxxSvqCMIZ21aSVFDr+uBI0lr6IEYEhrBmWF4x5Kfw=; b=biTR1zZHKBsbrQditnzXwJMHjG+MAqnxzkAD5QaumXCFOAE1wafue2xSW3UYbp1vy10Yop ptZVL6e360dvVSiw+emBbv02ygrX/XfDSCH8uoja6QCudP0JYR4yIhsrXL3PJ5MJ5kzBQz 1Aij5plwXJLVTuEA4AsC7pSuPLB7n2w= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=UJRGhy3n; spf=pass (imf05.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420154; a=rsa-sha256; cv=none; b=Du93ozA1sHZetFE3B0TEpZqdl9Xu3zwji/XuI0Lic6oRRhznRY23Z9l6r65uWV49dnkkb4 PZ6bchxtOpkNQGlVCG7f3srsERZ+oFDQXysrrg8f40tNBnw7+lp372pULU/T5tRtMKDRHQ G8As3PYWIoPUc4qw3ZoUCqhanvoARbM= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id E4F70CE0A7C; Mon, 8 Jul 2024 06:29:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9F7BAC4AF0B; Mon, 8 Jul 2024 06:29:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420173; bh=cbT48olUQwlsCFwwUYnjfHlBbFy2rEQMOoWsy01+W3Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UJRGhy3np6draIrQ5dcJhaA8p3BR4ME3fuTKdWducweIJsmTBd+uSKYPdZll3y7rm BxNOBa57Zm1zxbKwTabeuj6D09i+Zx1Uqgc0/aMswDBCkx3BTVd/V86FadDBFg1jSX BwKjViQcGzHORnrteaoEWLQOTQNPKxr/BeqB/dbTV5U0VQTG/0EXsIWSJ2BmctSbcS 2viqi+gbSiBA8ugd6tbWwy5+9acn1fm262K3J2Ks5WpmB6jzbUYQUrPsTs9RMuSHtF oCsf2LjiyQPjkrP1tVBJhIa6h36V1RZwq/GgYIqENIYg8Da1WzohE7MWLj8O9AMB8z hNL8CYaHOa2kw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 14/20] mm/zsmalloc: convert migrate_zspage() to use zpdesc Date: Mon, 8 Jul 2024 14:33:35 +0800 Message-ID: <20240708063344.1096626-15-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: miskzgm69m13zywzpkudu4fdrw7x8fmd X-Rspam-User: X-Rspamd-Queue-Id: DA3E7100014 X-Rspamd-Server: rspam02 X-HE-Tag: 1720420176-758036 X-HE-Meta: U2FsdGVkX1/wriH3SSc+1xf9FGpNwLLxiENrTQF0j3YlCEbjEHFKOWhiqXBBOISQtKqoMdz6/a3f1rVdTaZwuUYaYQPf2i2uEY3ipBQ3aLSNTpigdQubAjRAfszZ+wumdEbA4Qco49ZGaw0CeRgK8iPeaLu2d3l+GBgrvK2DLaPrs/R7JI7GhDTxraISYqqdok6iyXp75TpaTuEHth9m2DC8dYIe8gGMeHMANGJEk3Tjhgsqs5ltageFoKiGg1ozROlk1ecj6SKVyWaRhtrhDB52Wshcg8kUunHdV+GQhc46nPercFWeKFadtFcIXOFo213qMx/Pp1kiGEWVi7VEVVonMcHtMvTPJ7+9bsJfKkHp70hiv03VYnhab/jBtkaZTpToTKZaMugp/gqf3/Hd70NtqjvX1nRPSWdsiBFZaizu3W8YcyPsw2f8aqAD7oibfeFE1c96BF+hwu4mCHEDzir1Pe5klQUytwkX1pIQIOCJkpdU+bvksqDz/nQLcRsgWnJ/5+g3R+GsAJrJIKOswaBV5yVuwTK6PXRgk6EYs4YopCHPkNWR1a5VT18ymLUEOy/H3+TpWXcfyHmvnH2U7xAZksiYSbLCi0kpQ32xALucAe8JWST6d8/vFu0uYhrkE+XiQPFLssbqzr/WN6VzYTrf2ha2/989t1hsCu50lhTMT403ahmz4Ln08hP276r/mtE1G8y1DOyodf8W9KPKT7O9BrGiz+VS42QEFGwEoZMUYJm9UdV6z7zzA6yg7NFIAuv27Ig+9XVVwyHvnC7Z08/wrNBY7mL6pLCky7XZNZqbEeI5f/cyYqdGBJIKDC+p5X1vlg9TuOHin1NYnLQlPxZXr3ADWDGVmqjWjUS9qjV7C5oo+xuBwEo7xJipGzE9lDv5739NHMCJ0kTFIv3Rcp1vluIhR+1/b4uA/XH5TVXUPKHwjRVoQ756471hDW7AU9TG7JdwHESVEbar7Oz VaPrLXLK pNgA8XzZ7osv61xJS4bzhu5ZYabiL45bhCygtmuBsL06SMHPClBWVElxXCk0ZKo747Ak2/CkAta8XT3zxRuMkEphgTQ9VozxpCOGv3NHiBHA6aGLZ2p6Fg+DeNHFdrxpVn7UJww/zxo0TUdu46Mr3dG3GcfBTHnWLVjuRUVJG3ayWLy+QNr31dKJEzBvc7XzTUK+yDrjSmE0ilsbCtM1EgMPipaSTzXHMCdKDFmq+qL9lhiRvev1InzutIr5gD68qXi6+bdECuqSIw6kb8uG7Z9eubJgSTXTny9oyhjKOKg/deoP41elxExP068uBGxmByuDBiQkCfpQILaSyLTARwAphE/UBvZHCrNtYZFVJ5upsBf0VTrBRI7T11g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first/next_page. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi (Tencent) --- mm/zsmalloc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e291c7319485..7d039b0c66db 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1636,14 +1636,14 @@ static void migrate_zspage(struct zs_pool *pool, struct zspage *src_zspage, unsigned long used_obj, free_obj; unsigned long handle; int obj_idx = 0; - struct page *s_page = get_first_page(src_zspage); + struct zpdesc *s_zpdesc = get_first_zpdesc(src_zspage); struct size_class *class = pool->size_class[src_zspage->class]; while (1) { - handle = find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); + handle = find_alloced_obj(class, s_zpdesc, &obj_idx); if (!handle) { - s_page = get_next_page(s_page); - if (!s_page) + s_zpdesc = get_next_zpdesc(s_zpdesc); + if (!s_zpdesc) break; obj_idx = 0; continue; From patchwork Mon Jul 8 06:33:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726203 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD09AC3271E for ; Mon, 8 Jul 2024 06:29:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 749A66B00B2; Mon, 8 Jul 2024 02:29:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6F8376B00B3; Mon, 8 Jul 2024 02:29:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 59A0D6B00B4; Mon, 8 Jul 2024 02:29:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 386416B00B2 for ; Mon, 8 Jul 2024 02:29:43 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E08EA140E9D for ; Mon, 8 Jul 2024 06:29:42 +0000 (UTC) X-FDA: 82315609404.16.E78CC98 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf06.hostedemail.com (Postfix) with ESMTP id 99E6F180008 for ; Mon, 8 Jul 2024 06:29:40 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=YL80aVHo; spf=pass (imf06.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420166; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1DflF/ItTzyTCM0gRNFR0nSyEfyNKXkOXyCf3/nI6x0=; b=x+b6GC3EsEn1SWXDHYgz7usuteSfJmhw7cfvsWwTbyPv9zK3yagOEtMewwYmJI0zeQSsOZ IWbOGCmCXUoURDbr82TfOigJ5VOnSYQJkUa0qC/1ykRdsHhycY4GeoZqdLOXsphxnFOTHj foVrTg2mubqm5c9fNoxV+d8ZUIIKTB8= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=YL80aVHo; spf=pass (imf06.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420166; a=rsa-sha256; cv=none; b=iTT8R/vzFOD8uZ7Zk8Pq4GATczlHwRwhcTbjl/yZhXPaXMc6sIKR+/MdaMwbOB/Uvnbtu7 qp3FSYT/kSEbO9aQI8HbfQvVPqHEocDg4NgSLcxswnLsJAbHtMjnsa+N/mqAoJS4sKM1DC N7zWPlhkSJRtfSc7em6+a6GP/6IsJho= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id F35D5CE0AAF; Mon, 8 Jul 2024 06:29:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 365C6C116B1; Mon, 8 Jul 2024 06:29:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420177; bh=7eF5F3tLj5M84TMn26AMq8BpyTBKqnCtBSnMwvbth1I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YL80aVHoDjsPpQiEFbQTkgxQsZ584pywl8w3bPOSXFSqwYmZXgdEwogm6GaWDr7Jz IL40gwAKS4R5HieLcR2aixdW1ZSNWXe2OK3CXRIKSoqf6zPhWdbVbWekFTy2yUNuAM qPrnz32NbPsJ3fKvenqrjhmV08ZQio/iPNunbTc3ItLjwdwL7FM7J9vSZcOQxmzurn 6tByRNjMwTTOFDhCdCinryOYpSJDfCLxfgx27Q4A75DaxcaCPsMDUkvPp76JmkCXIH 1/sC7c74gsr+OeCq7Vh8ij43eR84+rzJcYe23EQgfrfR8uwEZSnKwA2xlTFVEXYTcO ebeTPLAH4yTSw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 15/20] mm/zsmalloc: convert get_zspage() to take zpdesc Date: Mon, 8 Jul 2024 14:33:36 +0800 Message-ID: <20240708063344.1096626-16-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 99E6F180008 X-Stat-Signature: f5zqxsngdbxbdc57mpqr7rus4teor6ck X-Rspam-User: X-HE-Tag: 1720420180-404161 X-HE-Meta: U2FsdGVkX1+IO2xwOe9c5PMk93p42wEbPCXxGMo/1D9teG1YM0qd88tja4gUhhU5K1cKE8WS0dABT+1UM7F0Q7rtQGMV7J1RiHzAAVy0wjpOo5/uqasuXduZNj7kX4J2La/tyFgYgW1gk6Ckr7tAAzJvenGEUqwBIRrXA5YhgZyH+/4yXXz9hs94nWP0de8JYnAv2ZOwo4CcaqfkoglTqdRu0uv2F5geli2Hzq7ZViWop9ekdn6JgGzwAxNkj5a/PxlSV0KskpwPotWR2VvQyn6nOQE0JxBwOdFDET3zsW5E1YW6fmL4+0RZIHPGbG9TUtlMPcBmbIexbbWAGSBM/jc6nvtbMrahjJbA1HFkZgbA8KAJNN7wt+FEfVGqW6HtdwLCTRgdGCnHMPa/AIwqXq8jJ0XQrldAVDlC+1jme9LCSXoqXmcwxsVJEvXqJFklZ3xNokPSOwmrP9phQiE1DYhtYbumI97Vtc5NXthHCGswyNKC5cDVHSF9LqkDrAZ44WY7F799AY8DwBfClmS5eOyrDYKrR1NIkKhxzolhPn17s7vtuE89JCq2ooGl+bGQdv4HxmzESdofu5OFnBMcynGhOen1InBT1d3LIk9cKr+uiGRTNb+a/bur2nO54NY7JjW+enCucKKjPA0EGr7YRO1keIZZkMyymLQfJWl5jhd5De1Apo+LWesI8Su9rPJpMjO4d5cu0brfkqWPvEPD79+k3soUqz3ohqt7qncdxMHMczNEZ4KLoiXTt84Ks+Dx/gb3J6/+ZOGVxq3FslpPXrX0zoxamUGpbSI0G0oWlvpLLZEBqcSKxFqsMbWWVKfWiVLXJupWX82mgpk3fN1gWZ6YmA3mKO368UniEdt3UmwQi5ZCjCpdjsSS3laWQ32ZuoXokDeN2ckNe6NThQRh61bK/OeiIstbWpjhLIpDTC8ZbAviYWP7GPPBNIQx2eSylB9v68uMR34YaYfOyac QNzqNeKB emBsQ7TfhBfuLSk6jv+ipbayksbiPrTMEDHj07R9f+Sjb6nmAZA+VC0Cyvj4zTU5r7WaHapRy4dpIo1FitjU3BExsRDMVIkvir22k2MwlEiPP+vkeKfEs6P9ZnlfV4k3YuekC+2n0x2qr9F3BHl7Nd9vw5EXTslhIeLafvu9hpBjmliLx+2ai9d5u7r5yNUYRMFyVccpjZjc3tCR6j9n7h2wwPRCcj48oxBsObDEiKw5Jp+04yfXoPMGo7RWGwFJzU0NThplksIpd/ObiVI/kEloQGyBcRUpR1ieom0ZMLZ9obBVeO7R0HAhvfwtTI5q/V/Ldi+Aw0Td3ArdgQfzfXBppkP5m9NgdWvb3Xf9/vUj6BjbWqB+f2ixuRg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Now that all users except get_next_page() (which will be removed in later patch) use zpdesc, convert get_zspage() to take zpdesc instead of page. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi (Tencent) --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7d039b0c66db..458ad696b473 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -779,9 +779,9 @@ static int fix_fullness_group(struct size_class *class, struct zspage *zspage) return newfg; } -static struct zspage *get_zspage(struct page *page) +static struct zspage *get_zspage(struct zpdesc *zpdesc) { - struct zspage *zspage = (struct zspage *)page_private(page); + struct zspage *zspage = zpdesc->zspage; BUG_ON(zspage->magic != ZSPAGE_MAGIC); return zspage; @@ -789,7 +789,7 @@ static struct zspage *get_zspage(struct page *page) static struct page *get_next_page(struct page *page) { - struct zspage *zspage = get_zspage(page); + struct zspage *zspage = get_zspage(page_zpdesc(page)); if (unlikely(ZsHugePage(zspage))) return NULL; @@ -799,7 +799,7 @@ static struct page *get_next_page(struct page *page) static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { - struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage = get_zspage(zpdesc); if (unlikely(ZsHugePage(zspage))) return NULL; @@ -849,7 +849,7 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage = get_zspage(zpdesc); if (unlikely(ZsHugePage(zspage))) { VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); @@ -1265,7 +1265,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1315,7 +1315,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1479,7 +1479,7 @@ static void obj_free(int class_size, unsigned long obj) obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); - zspage = get_zspage(zpdesc_page(f_zpdesc)); + zspage = get_zspage(f_zpdesc); vaddr = zpdesc_kmap_atomic(f_zpdesc); link = (struct link_free *)(vaddr + f_offset); @@ -1513,7 +1513,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle) read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); obj_to_zpdesc(obj, &f_zpdesc); - zspage = get_zspage(zpdesc_page(f_zpdesc)); + zspage = get_zspage(f_zpdesc); class = zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); @@ -1846,7 +1846,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, __SetPageZsmalloc(zpdesc_page(newzpdesc)); /* The page is locked, so this pointer must remain valid */ - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); pool = zspage->pool; /* From patchwork Mon Jul 8 06:33:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726204 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A05ABC3DA45 for ; Mon, 8 Jul 2024 06:29:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 413AE6B00B3; Mon, 8 Jul 2024 02:29:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3C6B66B00B4; Mon, 8 Jul 2024 02:29:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0BBE06B00B5; Mon, 8 Jul 2024 02:29:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E3A346B00B3 for ; Mon, 8 Jul 2024 02:29:43 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id A76DD1C1D47 for ; Mon, 8 Jul 2024 06:29:43 +0000 (UTC) X-FDA: 82315609446.11.2081CBF Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf01.hostedemail.com (Postfix) with ESMTP id 126F34001B for ; Mon, 8 Jul 2024 06:29:41 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=PLZIVCAc; spf=pass (imf01.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420167; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QvNipKPteTSiibEbCGzU4M4s8UfTiRuyWVvwWM9yy7c=; b=bzK39HlnJJxL6qEkYpoV3Pa1ULqWaBxQet4dakRsWkqIY2bHFKaELJ6oGwIDz4Pk57mMWl OcdvzeTQFI2on0l2LNOM8ce8A8scOuUxNnqXtzSUF0Bkg5f2ZYrifphyz6WzwrAb8DJNam PG2mC4tk/+Fbx7YTfqFtTI5WUXxk/Kg= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=PLZIVCAc; spf=pass (imf01.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420167; a=rsa-sha256; cv=none; b=kqik0hG1WVmvoW/g01MbIt80KOAdUbcf33TCHsTykmeb69rphNjruqeY0uBknqQjlyHSW0 bQ1hGcIe+jwmdJ4gjFtV0cE/si1eJWTRicaKA9VkRxHaxVqWkWHk2vmGIRRdmSdA8RbP7C TzaUZawNwbEeYxG8ztsjhDkv2MmZYZA= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 5387360AD9; Mon, 8 Jul 2024 06:29:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E712BC4AF0C; Mon, 8 Jul 2024 06:29:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420181; bh=TaGzviLe6NGcMCDjFOvP7sDnEJpuSU1XTpwZAcepYDo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PLZIVCAcwi0LP6n9+0RZBSXSigP2dfRTuOADXKXuWbgBCN0En57eYC5RSgs/1KtfS DVIzAun5YHyUuQoBUGzwuDYGBBdNZwNvpM+rRFJ6VWSjPUG5CpnTMp5A1yDpyc7vD2 WP+uhvZep47BARC5c9jpTlVqf0y7Kz7D+Yq3L4MDQ86P1RuxQXIXTqAIQcgXDvnzwg UtAPWYZA9ek8HVwniRE5UU8dKYGfnTrePnKl8DCQDiV1Udgp7vk+n46IaoL/PkmIeF Z5YaezCR7ig33vTKcfrPcYHW37YL2UrMeTFdY3VPVBc3XwPwXe7fFLu9rM4tcZy960 NbRxZc64HmFyA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 16/20] mm/zsmalloc: convert SetZsPageMovable and remove unused funcs Date: Mon, 8 Jul 2024 14:33:37 +0800 Message-ID: <20240708063344.1096626-17-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 126F34001B X-Stat-Signature: 4fx3okot4bumodxj9hhf5e8nugs4zpub X-Rspam-User: X-HE-Tag: 1720420181-451242 X-HE-Meta: U2FsdGVkX1/KSKyph2SDXxUfqiI/gs2bvnhezQyBxkzTEB3bDYDAqEKkZvOzEk2s2BhJZVUxMNv6NP1QHmUpwkDuJky+PwejaQVzFV9LNakZW8h70t+gI8PooPKu274y/H2tK8EpXgojFL2hFO6CxnhLQV5B/yOy/icd4gWPkWO+Z7Qy4faEvEnYhwAwIlj352R7OBGISkE4jhcbYCwC3R2gaaRQ/yV/4T4l0z+Z6CLijbo57f09smC+eNG1KDukmC+Ulrb4X+2PILre2oH7PADfJsTxcNsAfKW2USF9E0PEmxdamlIvbSoZ/M1Hhn3BZfQIFqsGpdNVqzX2OLWAcpb0+7xpJ6n/pnPQwOwFQWSgWS8O6ekHVoMK/cSgu1UZYjDcN8BTaCMPZTI+MK5ukpqQOS09Q9GCuOSvDwK+K1HxaRwQEtzloKmlfg/pSL2AoBBFZ6of/oa5wM0DvjDEgoORCQ8Z67te9zFpDuK4nuQdywjc5zvNT+pkUiDWqX5IeL0o6LHRyP0F01K/+d/gA7RKrOQJDhCSz7jGICzk/yj4OCkw/IaqYJVSD84K9rNMnIhFrTKUQ7iWuKPb85Z35GPbbHlaVs0Ir3wBJ9VzW1Hvhhr9qMpd9kkRRf9LtPausnI0OhIy9OFRUpb/gSCAEda4CX0TyMYqn7aSIIPGrOrn8i+xPViRADlmCBACTVrJL8M7QxTq7NfXVxiKA/lw2C4pK384OgHwxVLZafc73h04MbbeO2P0c4DhjTeBELvmdpBiXdWCBpWrK7f1vzMNoILE9/R6aCGaEp4ZNaPjxx78UOOxoPvOe3Zqs6JEEwqtRanICTtLGrFpHHhoeBBLuTz+D3urGhu9Jy4QZZCi8NrKi3YIDhA6WyS0lR65smDm+JP/hNihVowTgM2dTb7cSHz7GA9PFRKg+rTHyEH5T/KYhm61iebTohJUhR4pheb4Q0i82UyARypGqurjDho Xg1Fe+Nf wilfClYTSMJqkD11U/RLwWQ2W9Oop9T47GN8q6euNSwvAmDQMc37tNRcu+wvsyU/Kz5LBpQJdTqcc24UqpGvn0lKH6KkpaTtIRJ3T8FgMPBMOvGe9GBYojLLsbwzaxW3UhIuM5EBsFxHrlE0m7+cb7MS5dC1NJFBlyuczrhTeIBH1Gii6J/0ujztDKE52QzrXNqvv2YV3mEr+go7D4esodwVZowb0PlyO7LcCXwXtWbpv7g8G7dgfmDjZD/YqWihSjtl0V0Bl6VgAKqsa8tsH5Cte/021DpVYirmU+S0towR7ap0LhccR8QDMNvnJNE1ZH0eNkFi1M6g6T7k= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi (Tencent) Convert SetZsPageMovable() to use zpdesc, and then remove unused funcs:get_next_page/get_first_page/is_first_page. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi (Tencent) --- mm/zsmalloc.c | 33 +++++---------------------------- 1 file changed, 5 insertions(+), 28 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 458ad696b473..8b713ac03902 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -468,11 +468,6 @@ static DEFINE_PER_CPU(struct mapping_area, zs_map_area) = { .lock = INIT_LOCAL_LOCK(lock), }; -static __maybe_unused int is_first_page(struct page *page) -{ - return PagePrivate(page); -} - static int is_first_zpdesc(struct zpdesc *zpdesc) { return PagePrivate(zpdesc_page(zpdesc)); @@ -489,14 +484,6 @@ static inline void mod_zspage_inuse(struct zspage *zspage, int val) zspage->inuse += val; } -static inline struct page *get_first_page(struct zspage *zspage) -{ - struct page *first_page = zpdesc_page(zspage->first_zpdesc); - - VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); - return first_page; -} - static struct zpdesc *get_first_zpdesc(struct zspage *zspage) { struct zpdesc *first_zpdesc = zspage->first_zpdesc; @@ -787,16 +774,6 @@ static struct zspage *get_zspage(struct zpdesc *zpdesc) return zspage; } -static struct page *get_next_page(struct page *page) -{ - struct zspage *zspage = get_zspage(page_zpdesc(page)); - - if (unlikely(ZsHugePage(zspage))) - return NULL; - - return (struct page *)page->index; -} - static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { struct zspage *zspage = get_zspage(zpdesc); @@ -1970,13 +1947,13 @@ static void init_deferred_free(struct zs_pool *pool) static void SetZsPageMovable(struct zs_pool *pool, struct zspage *zspage) { - struct page *page = get_first_page(zspage); + struct zpdesc *zpdesc = get_first_zpdesc(zspage); do { - WARN_ON(!trylock_page(page)); - __SetPageMovable(page, &zsmalloc_mops); - unlock_page(page); - } while ((page = get_next_page(page)) != NULL); + WARN_ON(!zpdesc_trylock(zpdesc)); + __zpdesc_set_movable(zpdesc, &zsmalloc_mops); + zpdesc_unlock(zpdesc); + } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); } #else static inline void zs_flush_migration(struct zs_pool *pool) { } From patchwork Mon Jul 8 06:33:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726205 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 401BBC3DA42 for ; Mon, 8 Jul 2024 06:29:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C2CA06B00B7; Mon, 8 Jul 2024 02:29:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BD8CC6B00B8; Mon, 8 Jul 2024 02:29:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A7B046B00B9; Mon, 8 Jul 2024 02:29:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 85D9D6B00B7 for ; Mon, 8 Jul 2024 02:29:50 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 4CFE81A10A4 for ; Mon, 8 Jul 2024 06:29:50 +0000 (UTC) X-FDA: 82315609740.20.DA62448 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf07.hostedemail.com (Postfix) with ESMTP id F3AD54001B for ; Mon, 8 Jul 2024 06:29:47 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=H3TVIOi8; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf07.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420156; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2X3iHMqFI+jq+3USO/y0cSHCGA/h5vCQ47OZlN/JjM4=; b=pMqks5VzfPpb5rGn+2WlQjw0QAMmAh7oc9QjfOkcJAGZGgBCB5Svxp/EWRE5I7Ql0nWwDY t7H/32d1XgMgE4VKXRdwe3lpFquaXn0AQLOkA7l9Pa3HFByWpznf4KobN3DaBy0Q6wde+B 9F7CpX7DA+pLqlyDKRjhJXXVUw8rPDY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420156; a=rsa-sha256; cv=none; b=LevQInwQc/+aiefC6vN/ZsDwX2Y+a38pYY+M5cH99fyIoita+uG49KtPoKr69pkSD6mLNG GC3GfbbBvWFxo4gJHaqvEaJn+Zlk2aQV8AHOD4rCwRGizjJnBDUHIAY5T9mInt2zcnx6Wi ghn+tkjxmgN98lp694/DDZ4rYnLl8/A= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=H3TVIOi8; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf07.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 705E7CE0A7C; Mon, 8 Jul 2024 06:29:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A7497C116B1; Mon, 8 Jul 2024 06:29:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420184; bh=Y64RHT8UUVO7Yuad89btOx9mw5qKYY1Q7Nr/pGMgLzg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H3TVIOi8tn+/Jpf1JSx2+yuyzZOtImlkUm5t0LnzYklpni4kdSFiNHrqpQOXZqOL6 eKTBzQL6JVvFtsElAu7G5prq398JdgHze+6UeKvCmxHlMaC6BpNtTzo0tlkPZi5e6u DXlNgwNKMHm0UQBzrcZX6BPaN3TTk5eMyjGw7qRf9HJC8/bqtDsza7aNuxJB5E98Kh cVBOvKC38kiKD4gMr4YS7nAAFe17otobJQn5FNGbjOzVh8XnFWdWFnTy2QLvYSiqu7 +b4+8NbWTwh9n/uuXoC10fhHNQkguOvxg8q07JSihLZaF7N3nVxyDqDLJTP6G62bBx ROe9sU1M31ITQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 17/20] mm/zsmalloc: convert get/set_first_obj_offset() to take zpdesc Date: Mon, 8 Jul 2024 14:33:38 +0800 Message-ID: <20240708063344.1096626-18-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: F3AD54001B X-Stat-Signature: xcxc5x3nemgtoq9uspwmd7ht5zmpoc6z X-Rspam-User: X-HE-Tag: 1720420187-753619 X-HE-Meta: U2FsdGVkX1+lVhjMXq2fEggd4Mf3RVQsPPv3xIIXqJEJtuEcSr+mDPD4wP65k0eJU6uBruN3AI8RYijTCk5ZihOeeNRv2NZZSr/xOyXguSAEpILMIyGqMPmk7PsbEpap7Jr5Foj5uVRIeKTFiiXXiRmGBgnHEP50/MIiuMOmzC5Qkts5VdCdmBT8kxgzsfK7YKQfRjZuXZZGMQJOW4mShVBHwahXPFP0taCg8TVlRk+biaulG/G9G6/6VN8p4tNhtcuyczMrKjm+lxFDz8oS7fFx1UZ3qAR9Fo1kYp7LirolEoGImBeJwt3xKyezq9SUDVs5/rFYcaHt3wzL7KD7Nun6lhQ0/ujO9ekFfSFUBIOrgKiJcnLa1IbCfL9ThPFu4MH1Yqitm5AkjQ+cM7UVAB92vcHdX0d8iMSoKJ//3qWZ2IOFmad1xA5LNFRcQsjtJAULTjVungHkC6h5xph0Ut/dXNUNC+sgnM5pWGz5dFb6opM7WaYI+XccR7spXhi0yE0laDMCu5jyRYYQ16Z0lwTOqCtUUL4/rj/HiD+mS5nzq5vQRoYjQDw1OZy/A0xz/dTEPpzMWZREO3C2+CBrAqjtawiVr6ZQqXwXzOxb2PbEwA420d5a1XGv4zE2Dv73O6g95UnhbWy5A+wjtmdWkkzSvNJGz+ynoTaTAAif9q8q/eQjGIkuWdP4K/Cr28NMemcf6BTBNL/+6nF0+v+mb32a5ask6/1FEf2y9R9YfSh/naEyhw1EuyTomFTDKRKvUp/YIQPJSUOEoij7BuE34RqTBLKdvQKcb+I6Yl9PvCxhRuXrzWh1IXktFXB1ZwThy3GmdIBhb8xIfJdQU01bRvM+grDaKrFT5fcdxiy4WPU5D9ODRi2Uv91apHntaNkpGmXodY89nvETZnrs+iaZhl5g+UhlNJsqkVkeIWCmS2IQ9ZUnxsWvxF0Unpur+hBtpS7gFWKRtFcPEvE1YnO uh+rPtqc jDlX6dM3UYgBXOxmOCQyKWuUjIkwYp47MLznZmMw97sEEw3sY7PvjdUEufGjliaedYWHJx7/34rN7IIH1/KRReKl0jamdISBKJTBeNky8+pPqTeekyEgLwdJD9PrfFnuhcvEXR3DQ059LqgDlf1aiJit9lFYL+oTbZAxfwoZq0gwY4VvBPGmRVaSSW4HtJ3ZZQr7Ow3+kBuUB2BGMzT3qsfHZWabi5XSrSksCabHOhaWtF6O+9pu5fXNVsCb4i3iUQ8QGeMClp99pFcUBJ8c3nQlmqWdJFsDwXLBFH9cVRafPTLB5Z4olW3ntgejd3//tqtApLc9+ZjEQAAxt3teiLg7cZUV5PwiPpnkET5uo/ongeseOjecXKMiom1jIzs6YLA1+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi (Tencent) Now that all users of get/set_first_obj_offset() are converted to use zpdesc, convert them to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi (Tencent) --- mm/zpdesc.h | 7 ++++++- mm/zsmalloc.c | 36 ++++++++++++++++++------------------ 2 files changed, 24 insertions(+), 19 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 72c8c072b4c8..f64e813f4847 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -15,6 +15,8 @@ * @next: Next zpdesc in a zspage in zsmalloc zpool * @handle: For huge zspage in zsmalloc zpool * @zspage: Pointer to zspage in zsmalloc + * @first_obj_offset: First object offset in zsmalloc zpool + * @_refcount: Indirectly use by page migration * @memcg_data: Memory Control Group data. * * This struct overlays struct page for now. Do not modify without a good @@ -31,7 +33,8 @@ struct zpdesc { unsigned long handle; }; struct zspage *zspage; - unsigned long _zp_pad_1; + unsigned int first_obj_offset; + atomic_t _refcount; #ifdef CONFIG_MEMCG unsigned long memcg_data; #endif @@ -45,6 +48,8 @@ ZPDESC_MATCH(mapping, mops); ZPDESC_MATCH(index, next); ZPDESC_MATCH(index, handle); ZPDESC_MATCH(private, zspage); +ZPDESC_MATCH(page_type, first_obj_offset); +ZPDESC_MATCH(_refcount, _refcount); #ifdef CONFIG_MEMCG ZPDESC_MATCH(memcg_data, memcg_data); #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 8b713ac03902..bb8b5f13a966 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -20,8 +20,8 @@ * zpdesc->next: links together all component pages of a zspage * For the huge page, this is always 0, so we use this field * to store handle. - * page->page_type: PG_zsmalloc, lower 16 bit locate the first object - * offset in a subpage of a zspage + * zpdesc->first_obj_offset: PG_zsmalloc, lower 16 bit locate the first + * object offset in a subpage of a zspage * * Usage of struct zpdesc(page) flags: * PG_private: identifies the first component page @@ -494,26 +494,26 @@ static struct zpdesc *get_first_zpdesc(struct zspage *zspage) #define FIRST_OBJ_PAGE_TYPE_MASK 0xffff -static inline void reset_first_obj_offset(struct page *page) +static inline void reset_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - page->page_type |= FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + zpdesc->first_obj_offset |= FIRST_OBJ_PAGE_TYPE_MASK; } -static inline unsigned int get_first_obj_offset(struct page *page) +static inline unsigned int get_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - return page->page_type & FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + return zpdesc->first_obj_offset & FIRST_OBJ_PAGE_TYPE_MASK; } -static inline void set_first_obj_offset(struct page *page, unsigned int offset) +static inline void set_first_obj_offset(struct zpdesc *zpdesc, unsigned int offset) { /* With 16 bit available, we can support offsets into 64 KiB pages. */ BUILD_BUG_ON(PAGE_SIZE > SZ_64K); - VM_WARN_ON_ONCE(!PageZsmalloc(page)); + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); VM_WARN_ON_ONCE(offset & ~FIRST_OBJ_PAGE_TYPE_MASK); - page->page_type &= ~FIRST_OBJ_PAGE_TYPE_MASK; - page->page_type |= offset & FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset &= ~FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset |= offset & FIRST_OBJ_PAGE_TYPE_MASK; } static inline unsigned int get_freeobj(struct zspage *zspage) @@ -850,7 +850,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) ClearPagePrivate(page); zpdesc->zspage = NULL; zpdesc->next = NULL; - reset_first_obj_offset(page); + reset_first_obj_offset(zpdesc); __ClearPageZsmalloc(page); } @@ -934,7 +934,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) struct link_free *link; void *vaddr; - set_first_obj_offset(zpdesc_page(zpdesc), off); + set_first_obj_offset(zpdesc, off); vaddr = zpdesc_kmap_atomic(zpdesc); link = (struct link_free *)vaddr + off / sizeof(*link); @@ -1589,7 +1589,7 @@ static unsigned long find_alloced_obj(struct size_class *class, unsigned long handle = 0; void *addr = zpdesc_kmap_atomic(zpdesc); - offset = get_first_obj_offset(zpdesc_page(zpdesc)); + offset = get_first_obj_offset(zpdesc); offset += class->size * index; while (offset < PAGE_SIZE) { @@ -1784,8 +1784,8 @@ static void replace_sub_page(struct size_class *class, struct zspage *zspage, } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); create_page_chain(class, zspage, zpdescs); - first_obj_offset = get_first_obj_offset(zpdesc_page(oldzpdesc)); - set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); + first_obj_offset = get_first_obj_offset(oldzpdesc); + set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) newzpdesc->handle = oldzpdesc->handle; __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); @@ -1840,7 +1840,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); - offset = get_first_obj_offset(zpdesc_page(zpdesc)); + offset = get_first_obj_offset(zpdesc); s_addr = zpdesc_kmap_atomic(zpdesc); /* From patchwork Mon Jul 8 06:33:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726206 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3131FC3271E for ; Mon, 8 Jul 2024 06:29:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 880016B00B8; Mon, 8 Jul 2024 02:29:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8314D6B00B9; Mon, 8 Jul 2024 02:29:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6A9E66B00BA; Mon, 8 Jul 2024 02:29:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 45CD86B00B8 for ; Mon, 8 Jul 2024 02:29:51 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 036E1A1150 for ; Mon, 8 Jul 2024 06:29:50 +0000 (UTC) X-FDA: 82315609782.01.120D9E1 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf08.hostedemail.com (Postfix) with ESMTP id 67ACA16001F for ; Mon, 8 Jul 2024 06:29:49 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jWOtxdXd; spf=pass (imf08.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420174; a=rsa-sha256; cv=none; b=RPUk7UXrmNuX93eL+E+kfAJnNxycGYK5Y6H9upWPEfYys/lGiokLjauplblVdvKSItHVHJ b90fDL6XUqShffmUg4ZyWhhgA+Gpkp/BAc5Rh5Va/r4zinXeH8C8y2d3sWFLpkqHLv5og+ UQ4UwEiKDNB7tuTBRm465VzrY6Sx56E= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jWOtxdXd; spf=pass (imf08.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420174; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xRpGx7pbvCfhSG5wQGxOYh2B5tfxNNOrim9iy+8VdlU=; b=K48m3knzy7gTp27tWe9O5nl31Dp+SX9BDDSiwMY3PQ8cGzaOgdstx9MfqyILm3Rf4hlVM8 iYhXBe9vW+GNWam0DSEgDracgw7koALkPgnrjAlfJ3sBsJe5IIfL75/18ykHF8PDobWtVn bDzqPrWrA1wdWRTno6/FVV3SHQPRg7o= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 9F9ED60ACD; Mon, 8 Jul 2024 06:29:48 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 66EF4C4AF0C; Mon, 8 Jul 2024 06:29:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420188; bh=H24xLCJMLoMGxtoL02JioPie/ckhzBt773ZzPhU2CSg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jWOtxdXdW3pAl2WmrOj1bOnrw+gHOrlhH3zXwNH4eP7cA0yNru3LwOhi4tzn80Rbx cVD76ITdy6l5ySTmIFusfFSQBHDl8J2VB4jbf3A54xhBGhKvlLpl8LWFsbDN6yqiiW s2n8BudgTiO2TsxYnXK3NQY9OpoBTNB5iGwrGOdRxzmGT8ouZKbT+91GBd49d5aNq4 l09v7kThk+LyOa6yi3Rt5tTRG8o5GcSjP6xGzOshEy/RdkCcZhw0faZV7UOxRLyCzt dixnOcM1H9x7wG3WYr7joJy86clgn16ANgHuJ4rzG9xZ0sTJu/7N4idkcziq+vkXnp VaalbA+QoqWpg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 18/20] mm/zsmalloc: introduce __zpdesc_clear_movable Date: Mon, 8 Jul 2024 14:33:39 +0800 Message-ID: <20240708063344.1096626-19-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: y97kmg453fnfazxg9p8rykefm8pgjpc6 X-Rspamd-Queue-Id: 67ACA16001F X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1720420189-908729 X-HE-Meta: U2FsdGVkX198TLNAj5GmznuOfglxQPxgXyY6ZQMgGScPRd0PUdz1aDD6qoOSSK6RZJWm8/9h4Q9A0m9/ff9Pl7UNyTh0a5YjPXlB9kq+75lPi6FnJ6cTJgBBLbvq5TKfsXJPKtlN7ra10OzXQ/Y5ng+O9x5p15DbeGHDKOlClI35NRgTPoG6oEVPK9kFW81YuxV5f0wYD+3RCxXzS8G1v2KGm3w0PzLA2XG+72C13VvUaoFzSP02d29bvYfg+hP635rE7jV2rFoxX+B3b+fGrbem73MsNkd1J0lcUxI9j2d6xRps1DgHa++2j7FoIWAArJSK5WBj5nNV4ATlvmM83BSkk2piNXBg4iUdwqS/kbIA+nSZNjQtW9mojJTpBe0xQhyqHRfwnJL0oVRH7CNFeYb8CcSP71fxGXQXKgFxDlhV6OOZj1z6mieegzSRIc3lHqf5VMoqPbddo7x6JiULmq64x/T80zvD8aUFepy8EhyJ55rlwTy/YjtbQmBwLrGJOaSXAew1Lf3nN+j1egnmBi+eSJxgRZ1xezms4J8VfkwipU8kx36XhpoMe6taQ8DKCelytr7lQwx+OJZ4teje6K/u/R07FuQ1D7x3Dn68xhOuIkAW7zzUiQ4uCSsWc6BcYzJYiZsgrko3TVLZoOm9d4LR4E9qCju3T88NiKIM2mAtdQV8v3J6SUbkmpQsbr3BiMgaX2wwc/JBPdJCbCh6sd4txiTKbQVDdCpjT772/5VeU7oivTkwMJngGNiPHcoOVVLJ8e20Ar1uaipHCDCU2pDaBoTdFcC6agNu8y3Q4IN1sTKmjPTCj3P9P9MV+NeQSn4v7Gqe+kEnoVmcTuuGfUWFBkYgPI/9mOOIqTPlkueVXzAmVxAqW6ZwsDc9nAnMDH5xHyrZ8PRP/rSUKM4AualgDO8T54ZG8wPGUOA2cdBSK099rVvtb+boZdurqyIb4hhbsI+rbm6PfNdXYLt 4RxpnBJG 3gVgtFMNMuZkB43lm0kCjkqWwvCC3gjJku6xsUwF7tRjLJHRNGbCttEXczrnuAXVM51I7NlIWiaavCHQtOYW5S4mzaeraXW1RoAjjNMopoTYUu+z8F0yUEjRpbrV0voF82NBgXzC0nXzf/wBHj+BPXfHRwo6QUBlBWmUqPYRWnD4gENkJpykk/DRXMkDNYZtCImYoi0pIuh8ViE1ZusVq7v59yAOzRlElm7BTojb16OdH1XN1OYAmuGtTcWvadx1/ApFOmo5R5eN4eOiZYLJBh+YsWndFVaYXxy1HKXlkKR4HjwrQKAkrK6DIr/eb7UqKq/CxIWjJWga1Ru9rPXiUeH+lZBooQXRZzpwy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi (Tencent) Add a helper __zpdesc_clear_movable() for __ClearPageMovable(), and use it in callers to make code clear. Signed-off-by: Alex Shi (Tencent) --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index f64e813f4847..5db4fbe2d139 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -114,6 +114,11 @@ static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, __SetPageMovable(zpdesc_page(zpdesc), mops); } +static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) +{ + __ClearPageMovable(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index bb8b5f13a966..e1d3ad50538c 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -846,7 +846,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) { struct page *page = zpdesc_page(zpdesc); - __ClearPageMovable(page); + __zpdesc_clear_movable(zpdesc); ClearPagePrivate(page); zpdesc->zspage = NULL; zpdesc->next = NULL; From patchwork Mon Jul 8 06:33:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49ECCC3DA42 for ; Mon, 8 Jul 2024 06:29:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CC5F76B00BA; Mon, 8 Jul 2024 02:29:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C505A6B00BB; Mon, 8 Jul 2024 02:29:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ACA3A6B00BC; Mon, 8 Jul 2024 02:29:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 7ED4E6B00BA for ; Mon, 8 Jul 2024 02:29:57 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 29647C112C for ; Mon, 8 Jul 2024 06:29:57 +0000 (UTC) X-FDA: 82315610034.24.D01BEC7 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf16.hostedemail.com (Postfix) with ESMTP id 2EB3018001C for ; Mon, 8 Jul 2024 06:29:54 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=UEHuelEs; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf16.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420171; a=rsa-sha256; cv=none; b=4OOTRCd83sqyO4ep9h5amLT3cATIGafVQa2eq6wulIxOMXcoWjIP3S8Qv4IeuMPKqYd3/a dSvu9Zm8pWrplJ47MCimtjRMibY+sMU+efLDzJ7uEKM3KJfkzxMG4iIkqzf/ObK6FsR+Ey 569rdoM248YlRT2JUfZo6BrO3C/Cgos= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=UEHuelEs; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf16.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420171; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MCgdwtxG6pnLG8vlAl+Sjd4oA+Jy4eLtS6tY87doaJg=; b=Q0CuKBAtJfxtWcwRxFS54Xkjj0WlQIAAVvmFg8s4rb8keyy1GkLaoujDDA72mcpoRR3Z5x npHj+ewXJ8pfcioaD6jfyga+tM4Q9FyIM/DFubcRkTBjCuMTV27kt2IJt48OL9gTbVgv2R qum75PtvD1M4fqBq1cXxPN+MtteG8kQ= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 9514BCE0A57; Mon, 8 Jul 2024 06:29:52 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F38EDC4AF0C; Mon, 8 Jul 2024 06:29:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420191; bh=ld5kbfQznGqvmPXTBRxo+Gb6MyvpG3bwUJO7MMXVjOQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UEHuelEsMNDLfNwnCkG7vPJYg9IVTB41xkci7s51NiPFoDJgxr1CUaRji/cVlqlNA vYdN8sXnViZNe3kSp0lsazF7+z8vLY4xWfdHqBLMGbC9cmI4uqUdMGYDo3TQBRCOan bINlkUPVQMuXuhIZ+Nmq7VOu7niPAgJ43p5dnvdDeoiuXWitnqDwZtAzukSp15x9ZK cWnFn0H6I99oVMAEI4Rc5Q3DCx5/rPZKTYyg1s6wKXmbZf96IvA0lyCPzp1T9ncpt5 0eZQBV6bEE+3Jborz9xlh5OA8pkf9vfMZbQks6eVJYhboRqe0qKCucYlARPqFpn8/R Gv4XpWfGxrqZQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 19/20] mm/zsmalloc: introduce __zpdesc_clear_zsmalloc Date: Mon, 8 Jul 2024 14:33:40 +0800 Message-ID: <20240708063344.1096626-20-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 2EB3018001C X-Stat-Signature: 18yno8irn6k9uhkfmtanwm9mp83tmjco X-Rspam-User: X-HE-Tag: 1720420194-727428 X-HE-Meta: U2FsdGVkX19/w4PozThD8lCQOQ1y0YOY+OoYzZP/Z5UGxDU+GsdKqzx/xKH3F0vnAYD8AhnrFYNI/TJdHSjuESJCZiNxz6eKXYrfNAlItkEqI5tfza2M/m42XIiY4axwx7CFtt/Z2AuKKAlAA6Y7piKwAqv0j4qAD4+fX55VsmaOZ0h/1mH4g5C5D86SeuWJlMOCqnu2cVcp+9Sgz1bKE63lFBqWkvd1zJApVM9cwoszXaRGDGOoZgnNq1tTj3XDoSTJKAa7K0oNLDfVAwnzcAus5YvgHsdysRh/bTnPfUxqBPsJJbq49bDrZOJjSvB+YxePj01T879ZG9L0ED/bHzR8381aTlvMB5TKnEwKqadiICqgv0czEeiYy7wVxgVdjSGsB38CAjGkMFK3Ld6MdsPVM+/cbcs9WK11ay3iiXqV7lRwZmwtDL3kvp3ChpRTJP5S1YCLgX4IsDNULJoO6HI0/sqFiqZovo+BnUnyDDKaHvDTcNkWqzqovdTD2LYDdzh+vD340S8JtJxunJ0KR7W4pl45s1U2CRdRFn7OZbQjhLid88IlDf5ONhRgEvJSD96vJAfwXf5NbZCUwxrst0vf6mtJizfE7hODYVUvSPrWNziRcIUO5R9snwB0Nwnz1lLYTfEgwkeCSAB9fIks1QvkKAmXaE3tcMJY/qW+dlMFIRDs3rC6/DcAAXbKNSUz2/b7ldt2Yikh+V9DMtAZH5fkse3PxGawkJd/lng6hHVcsCi172meRIlWzP+wcEL3+5npBi+XqLKnbIkrM9EYOUlCFDwuQTSvyyBtqIOnqRJdgvbBjrtBS94WwhTEqOd9spzDbgtUGFKWEv8Q/xa/09QW1/m4w0XEbVDGRsd3Lu+8e/rP91WSTYb2BzlxEOlIc51zh5UZRRW0Y12+mdGfgAIZ2e2O+klok9XY9YXipuEEMYTO8N1vSZQLV5KXgoW+tq+h67AgZmXTTSO04Ik g7L6XMaR IxVu5iLo6+I5X0VT/MvcwZuUIf15ZzAjUBdoJp4J87A0vjcQzqIpFALjlqsmspp3QiINhyeh+voAcDNeo1WSZ7nfpfsRI0bPNTKrybs6Q/pVptqY7bvpOHKL2TiD9ktdguHL1hFEKEHZLytUjY1jdE5sHPZNPIG375Q1eHxKfGzXnX79moMk0oD968xO13G3Mg1oqED3sMoCmW51l15zSjJg/sNNt85yo6JiM+2gSatRiMRr6Zc/ctcJSCCCzu2NWRttC/ZM9kJFE5IKLSNDxliEqXGkq0Ok6XVF+kKAdvMxnx1C4k++oZzt54DKkU4SWspGCDO9nMiiAM64v9B/V4OYZ1y5IyqCxbaE2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi (Tencent) Add a helper __zpdesc_clear_zsmalloc() for __ClearPageZsmalloc(), and use it in callers to make code clear. Signed-off-by: Alex Shi (Tencent) --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 5db4fbe2d139..05def4d45265 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -119,6 +119,11 @@ static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) __ClearPageMovable(zpdesc_page(zpdesc)); } +static inline void __zpdesc_clear_zsmalloc(struct zpdesc *zpdesc) +{ + __ClearPageZsmalloc(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e1d3ad50538c..d88602fb0233 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -851,7 +851,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) zpdesc->zspage = NULL; zpdesc->next = NULL; reset_first_obj_offset(zpdesc); - __ClearPageZsmalloc(page); + __zpdesc_clear_zsmalloc(zpdesc); } static int trylock_zspage(struct zspage *zspage) @@ -1024,7 +1024,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, if (!zpdesc) { while (--i >= 0) { zpdesc_dec_zone_page_state(zpdescs[i]); - __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + __zpdesc_clear_zsmalloc(zpdescs[i]); free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); From patchwork Mon Jul 8 06:33:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13726208 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E591C3271E for ; Mon, 8 Jul 2024 06:30:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EC6276B00BB; Mon, 8 Jul 2024 02:29:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DFC6F6B00BE; Mon, 8 Jul 2024 02:29:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD9956B00BD; Mon, 8 Jul 2024 02:29:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 8F15E6B00BB for ; Mon, 8 Jul 2024 02:29:58 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 4AA20C112C for ; Mon, 8 Jul 2024 06:29:58 +0000 (UTC) X-FDA: 82315610076.26.A14FAAA Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf16.hostedemail.com (Postfix) with ESMTP id 9A6C518000F for ; Mon, 8 Jul 2024 06:29:56 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=P9pIQxRc; spf=pass (imf16.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720420168; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wsxt4rNTbw9Dagf/zy9fIyYFtO9bbv9nnBup6MaVTck=; b=rN2lraOsm2TNE17ubAzKR44fqNZ7G73FZKUAlHzL/c5bAViPd8ay6AOwVvdgY05F/RAxvF /s3Pt27Y5oIMxDm48PdmYy1VuJvlImJzl7wUGzRe6/9nz+2ZMB6ZBLfJdpQojQ+G9/FH3J NO4Y9rhDxfYqLkJLpTP9zRUuwRDOWPQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720420168; a=rsa-sha256; cv=none; b=Z0QzKK7PIB5ROd3wD1EmDZdrtc81RhnDlsgSt20jfZMEdRhk4m3CQUhFvh9LV6yr9nMOUt oQEqNWDaAqkjQNz7dNlKjmqlp50Oby3bkYm7p+5DauP8vWvqb9VU8vw6oI9WGB5A9rKmEn K3DcepfBJrMJ4ehC7hyhZgoE4lAAeZc= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=P9pIQxRc; spf=pass (imf16.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id CB28360AC1; Mon, 8 Jul 2024 06:29:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 89CF3C4AF0B; Mon, 8 Jul 2024 06:29:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720420195; bh=YePXlWlhu0t9cQ2FhNb4Tu9YraM33q9YNNO84M2/1AU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=P9pIQxRc74qXODLyeYpN9Usf+mKhKNyEtYaG/uShlHPiP4pI4bAMJ0ZxPksRq+auS Qq89KVFjK+cS/UBpysNLdcOw2E2hcw1sCMvZl3KUc+HOJbs49yZs8V9xBzD7CQpPXk xrf1GXo8qGKrN8Mfj5BxDOcq96OdiaGcpT+j9Z+Vx0xpJbuPHuGsusNl5njig/hpaW a2NyztZknNxisBfbFjaz0C56dqMEDimRCGr7nI1JjNqJV+zZNtkz4BQqyS4XDUDCt+ XUshBq06XfyGRAvEqCFVF5NOrrRoaTNdPTkLKW0edIVF8ssJm5jovq14e7nkoO7ZPB O0r04fevGt45g== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v3 20/20] mm/zsmalloc: introduce __zpdesc_set_zsmalloc() Date: Mon, 8 Jul 2024 14:33:41 +0800 Message-ID: <20240708063344.1096626-21-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240708063344.1096626-1-alexs@kernel.org> References: <20240708063344.1096626-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 9A6C518000F X-Stat-Signature: 8ohgdk1kbtcmbmwnx49u5sbff5nar5k8 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1720420196-318136 X-HE-Meta: U2FsdGVkX1+pc+KThKtNv7KI8jeOtdMUrJ7tPPhV8eD5I3YjWruUFHbS0x5use2qIhsNila8CdcEEicelLhlP3oS25tGPCkggMzG0vq5S7aWwPEmiLLkUzdw4Ut1XtnU6lJzTrCsL0wfT8APNjc+HXT62Sm1XCh4aGOzekcd3ovGv2bc0bWAwuvRD82KDj+8l8tU1RKma5Jb2J+2jDnEntQ3BmEnNRct0dFBGgK8pZRYpSW3pqmLYV6DAkm3s6KwT66TrCfePV660UXGrBmuASe/1bfsUJts695p+41jm2eREKDgRVPbapw4UHDlry2s4valoBcr8CRmbWuxJA/3KB/wR6b36KJZgNpG4uHLuyEn2JnECjEskGKQ2MaqstIpUgoE+dVQPtdvlcuhjGc0z+zh9dK5fdNVGijkf3UaWwv12wNbfqzTp5xGmhNZ9ivAfAyW/aYEfAZXDDqg2WSRtZeSJJ1SKe95wEvly50hU/z/4wiz8Z0Ln4Rk7Apvvjn/ZJ9Tw1Rxy6iEQszDDMBnJ0+sXoOpQMZ18OBwzwy9ZgwdW5R/oFtVWcL6x4djkYXbyRQ971C82l1u5LOko7w8Z3pd+gSDmajTcGwQGcbtMh+iNhHaBhZ5+X8aQL1EBakla0GjX+EolnmgeliaN+p5OhG2wHxwPBpRXVGzNh0VD1RFF/yB/ArWVi7cLdP5fGBBqLqkXmf2fc6gcNDVCdKyv5+OzvHPkj10bRRKC6RBO6vSH7m2ow+yyJ5z6JfKNdTlldjSgS3wGvfPz94SmfpFcOkfMnT/CNRd+H5dUQaSOtc2LyO2LWkuQ0mnW3LEX3ZfUwNQ5JX30eKtMCp90/IgAv2Bha6WaQtwPOgSwUnBuE6UIo88tOblJBK1th0GrUs9XsKkUWSNR6y/eoHBFatrXDt1P+YQngD8KcKBfUvF+8tCul7L/5XF6Y02UwNZn/qEn4IpvrK1fjM9SOmzcOu msPrj5df Dc2a9wVO/kyTNuNCBal2gebaby3az4L3YMud+nGIC3O2fl47zrcdsabVguTPKvB67FYj9K+K3HFjr1D4TcwUxEKJPSrpAdvNoSxwCudh3rf+p8RQScJVNiUwVubSTSkAtqV64ZrgD1F+wOzSw1URrMLJHWWokQyq5ZzTJme2u6FdYUoTkPvlfmQTlnL0Ku1PmTwhMx+EgEvROzK7XzYYI4S8aQoadGVFJTs7Aq4M1mnsHHGt+ygpoqpNEjS9TmM2ywiBGEqmxhTVcUi1WEGVwzv632hGT4BNSK9PN/eLi+fkwzGKLxmQLQTz9uHXvCC1T8s8RKbLBaXFvb9jeDMdDNtXDQ1EEStjq75dI X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi (Tencent) Add a helper __zpdesc_set_zsmalloc() for __SetPageZsmalloc(), and use it in callers to make code clear. Signed-off-by: Alex Shi (Tencent) --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 05def4d45265..06371ce60cd1 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -119,6 +119,11 @@ static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) __ClearPageMovable(zpdesc_page(zpdesc)); } +static inline void __zpdesc_set_zsmalloc(struct zpdesc *zpdesc) +{ + __SetPageZsmalloc(zpdesc_page(zpdesc)); +} + static inline void __zpdesc_clear_zsmalloc(struct zpdesc *zpdesc) { __ClearPageZsmalloc(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index d88602fb0233..7f8e02df4e3e 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1030,7 +1030,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(zpdesc_page(zpdesc)); + __zpdesc_set_zsmalloc(zpdesc); zpdesc_inc_zone_page_state(zpdesc); zpdescs[i] = zpdesc; @@ -1820,7 +1820,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(zpdesc_page(newzpdesc)); + __zpdesc_set_zsmalloc(newzpdesc); /* The page is locked, so this pointer must remain valid */ zspage = get_zspage(zpdesc);