From patchwork Fri Jun 28 03:11:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715388 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17AAEC30658 for ; Fri, 28 Jun 2024 03:07:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6926B6B009A; Thu, 27 Jun 2024 23:07:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 63E996B009B; Thu, 27 Jun 2024 23:07:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 490256B009C; Thu, 27 Jun 2024 23:07:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 261FC6B009B for ; Thu, 27 Jun 2024 23:07:00 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id D4962161493 for ; Fri, 28 Jun 2024 03:06:59 +0000 (UTC) X-FDA: 82278810558.27.48633E1 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf17.hostedemail.com (Postfix) with ESMTP id 8D58740013 for ; Fri, 28 Jun 2024 03:06:57 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=C87rd9lC; spf=pass (imf17.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544004; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pliznEXsoNx12JLUTI50q2lt5Ms5yprwYwyCL/gWicQ=; b=34JgNnkGxhPCFVwoX2ztOgDZzPzqnihG0uiwWV9bqSMoR7qJp2FWee607CrYsZij/g/NH8 Hxf8Y7zUqpT9ZJxFTpWt/pR+bkknoUC6jLNOIxxSH2a6UtrP/+sxT2664ngSYdGsuXO4GZ 4DnEMXUWgzgULtoNIV9DOiih11mcjFc= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=C87rd9lC; spf=pass (imf17.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544004; a=rsa-sha256; cv=none; b=aS7r8X/Dim+ZIF2GLX0SnICDIckMCrJsGRP/zXBE/BEDsu9FtVJlVwZV7y1G6L61sEvbi5 5jG0pusmbxic6FIephgJriuVN6Hhx4HhD+sDutBojndfFpnv/ZQaMacCkCBa3kX14WAQAt tuyovIx/FPmx2jbucCAckq+bZQ5EX6w= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 317ACCE3396; Fri, 28 Jun 2024 03:06:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 01F3FC4AF09; Fri, 28 Jun 2024 03:06:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544012; bh=Cr0rk228Nf4Ejmcf123tMGr+wVgBtKk7ChztlR6oqcc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=C87rd9lCAIEm3gmgN1HsqdNpdHVQnk+7Q4Nv5dOer+u/gx8+vllp84o4qmsNwEaZg +Fe1YLW2saEdNkRrbeqC8q7l72FvR+c0OCa5xkz+05L0MavtAPM7hr/UDfktBh/QSb oJjlWdO597SbcB/MMgnDSkhzMxI2B6jMBPx9yG08XhP6Go5Ay+7NS4e/baBvqUc1sO eFslcQG/0/9QdUnqkjnahncWxOulT4eXMZVUN0hMXgwqy8SzeqVX0Aj7BXIoByXD3F WgfD365aGDEcOL1toHZ63a0fAY+j4Rit+f8tKukzw1F/O63OwIGXMV+jk/g1251juN woROqUiGIiSlQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 01/20] mm/zsmalloc: add zpdesc memory descriptor for zswap.zpool Date: Fri, 28 Jun 2024 11:11:16 +0800 Message-ID: <20240628031138.429622-2-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: 7bfygdptskwex137tntkbankdwgxfo65 X-Rspam-User: X-Rspamd-Queue-Id: 8D58740013 X-Rspamd-Server: rspam02 X-HE-Tag: 1719544017-767013 X-HE-Meta: U2FsdGVkX18EfclkF/XKpPD7/Otnfgei/lVx4O6Er8ZPwe8lO/smvwb8oz9gWJfdD9fx1i+zpW5CNmF2YrMlRln1LhgHDXmZsUlYzmka1otsqTKSwSbo/KJ3gnSm/ZNgKGEXoTIxCQH9AFgivRUjzK7ERrO4s30DIA7HUY7RfNz9+spjH8rrrwYn7LwrxaQYg+uWYC1buv2dEsYpDNkwvaUSml4nzpAOiYT+jCNPaMjTZg0J+/MGDAl8GWFJafRLGgeDEiM41RkT3/2EwtlKJ6Sdyb5DCEUdGaaMKlhI/1D/XKVvRyDy9+n9AHIZMCJRIxGQO2Qg/sxQ8Cv80/pMZ0/s1wW7352nd+dgD15W1qREioKDz7QyCusRWf18Qt+FpYCdATI1W8cgxnMKF+2C4/0+EnvXCADYy+tYNDeo3WFL8QJjZqFpyVT6dioMnwdB+DRvXjR8TprJHw95JrP4jqrq+Wdy/1gKwOIcf4fU7SE9WtsFogBVpO1US7hO8riGOaoPTF7RmR2sAdv7nPOGngrfOrsEj4T8uts7SDpMXTzLrnDz86LN9Vm/Y2GkVdfOcn7S6YZorBHabUUSdCyCTg5sAYSYeX/o3Im5eQLGpxhaf1U1PHkBk4G6ArRuaqYFapiNLqcGE/9tA6DXT0AWxyYOI7mdPPX63P6fiy/vx4chBXbIdP1jKTCN7Wqf8PJx6pNp5B/kJs7IgzoMkRZFYO1MpwvTB145XAjcxtvH+vYzvJ2yImlvsYXDffSMCoVgXHXknKBCXPRBps2wTUL5vEknOecMsxvUBL0bf53xHY/qz2cD+LTf6W6Kk7cqNx1Fr/2YIb15cNcFGYYluwOuZ/FgHZV74GO+AyJHUoAcMGwblQBeMg5gFNgn1ZEL9rg1ugWxV2qlbFKOogNxeAWyc1dxxTffs4QuND69n53ylFvT8IbdTjkVGle/+ByDFy9C8sfBA1NWT9wEdmSIyAT 9qPuaifq xIjA7NSd3A3l2f5kdNgemDalGshn+7OT43gcyoOUaKsZhZspfRVge9QUkQdKVpZqNDPDQGGw0b3liBAktVUBwaJkP9zAqQDfEdmYm62g3PQrMXs6j4/i5WkU0LSbIf3Ad9o8tOi0KByY+c7jFhScCB+2r6kMaXvtiLzRLkXT4CNTwQ9TXT8uaXogOyGVLXzD36lsk1jO/grzAbfeI+C1KljDG1Re6nDST4i8XmfNfXOSMJi6rbNY7SKUwTx2dC55cunwPfcAweHGpOC1q0d3ZrMpFLLtQFH6QGA9XbzwJLMptd8qPakC+leWvvMgQhZpeRjWjdtA3DjqX7kk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi The 1st patch introduces new memory decriptor zpdesc and rename zspage.first_page to zspage.first_zpdesc, no functional change. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++ mm/zsmalloc.c | 19 ++++++++--------- 2 files changed, 66 insertions(+), 9 deletions(-) create mode 100644 mm/zpdesc.h diff --git a/mm/zpdesc.h b/mm/zpdesc.h new file mode 100644 index 000000000000..a1ab5ebaa936 --- /dev/null +++ b/mm/zpdesc.h @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* zpdesc.h: zswap.zpool memory descriptor + * + * Written by Alex Shi + * Hyeonggon Yoo <42.hyeyoo@gmail.com> + */ +#ifndef __MM_ZPDESC_H__ +#define __MM_ZPDESC_H__ + +/* + * struct zpdesc - Memory descriptor for zpool memory, now is for zsmalloc + * @flags: Page flags, PG_private: identifies the first component page + * @lru: Indirected used by page migration + * @next: Next zpdesc in a zspage in zsmalloc zpool + * @handle: For huge zspage in zsmalloc zpool + * @zspage: Pointer to zspage in zsmalloc + * + * This struct overlays struct page for now. Do not modify without a good + * understanding of the issues. + */ +struct zpdesc { + unsigned long flags; + struct list_head lru; + unsigned long _zp_pad_1; + union { + /* Next zpdescs in a zspage in zsmalloc zpool */ + struct zpdesc *next; + /* For huge zspage in zsmalloc zpool */ + unsigned long handle; + }; + struct zspage *zspage; +}; +#define ZPDESC_MATCH(pg, zp) \ + static_assert(offsetof(struct page, pg) == offsetof(struct zpdesc, zp)) + +ZPDESC_MATCH(flags, flags); +ZPDESC_MATCH(lru, lru); +ZPDESC_MATCH(index, next); +ZPDESC_MATCH(index, handle); +ZPDESC_MATCH(private, zspage); +#undef ZPDESC_MATCH +static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); + +#define zpdesc_page(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct page *)(zp), \ + struct zpdesc *: (struct page *)(zp))) + +#define zpdesc_folio(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct folio *)(zp), \ + struct zpdesc *: (struct folio *)(zp))) + +#define page_zpdesc(p) (_Generic((p), \ + const struct page *: (const struct zpdesc *)(p), \ + struct page *: (struct zpdesc *)(p))) + +#endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index fec1a39e5bbe..67bb80b7413a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -13,17 +13,17 @@ /* * Following is how we use various fields and flags of underlying - * struct page(s) to form a zspage. + * struct zpdesc(page) to form a zspage. * - * Usage of struct page fields: - * page->private: points to zspage - * page->index: links together all component pages of a zspage + * Usage of struct zpdesc fields: + * zpdesc->zspage: points to zspage + * zpdesc->next: links together all component pages of a zspage * For the huge page, this is always 0, so we use this field * to store handle. * page->page_type: PG_zsmalloc, lower 16 bit locate the first object * offset in a subpage of a zspage * - * Usage of struct page flags: + * Usage of struct zpdesc(page) flags: * PG_private: identifies the first component page * PG_owner_priv_1: identifies the huge component page * @@ -64,6 +64,7 @@ #include #include #include +#include "zpdesc.h" #define ZSPAGE_MAGIC 0x58 @@ -253,7 +254,7 @@ struct zspage { }; unsigned int inuse; unsigned int freeobj; - struct page *first_page; + struct zpdesc *first_zpdesc; struct list_head list; /* fullness list */ struct zs_pool *pool; rwlock_t lock; @@ -448,7 +449,7 @@ static inline void mod_zspage_inuse(struct zspage *zspage, int val) static inline struct page *get_first_page(struct zspage *zspage) { - struct page *first_page = zspage->first_page; + struct page *first_page = zpdesc_page(zspage->first_zpdesc); VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); return first_page; @@ -948,7 +949,7 @@ static void create_page_chain(struct size_class *class, struct zspage *zspage, set_page_private(page, (unsigned long)zspage); page->index = 0; if (i == 0) { - zspage->first_page = page; + zspage->first_zpdesc = page_zpdesc(page); SetPagePrivate(page); if (unlikely(class->objs_per_zspage == 1 && class->pages_per_zspage == 1)) @@ -1325,7 +1326,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, link->handle = handle; else /* record handle to page->index */ - zspage->first_page->index = handle; + zspage->first_zpdesc->handle = handle; kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); From patchwork Fri Jun 28 03:11:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B941C2BD09 for ; Fri, 28 Jun 2024 03:07:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1116D6B0099; Thu, 27 Jun 2024 23:07:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C0046B009A; Thu, 27 Jun 2024 23:07:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EA26B6B009B; Thu, 27 Jun 2024 23:06:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id CAD7A6B0099 for ; Thu, 27 Jun 2024 23:06:59 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 8191B1C27D0 for ; Fri, 28 Jun 2024 03:06:59 +0000 (UTC) X-FDA: 82278810558.14.921E5B5 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf21.hostedemail.com (Postfix) with ESMTP id C4BD91C0008 for ; Fri, 28 Jun 2024 03:06:57 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=uxYPxwOw; spf=pass (imf21.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544000; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hLk/LA9ZIbUceyqGEON0eZxJ2K6QMdqjnRcs/r3mo0M=; b=Hnz4uVSXQKG6M3Zme8heSEHG8WcQw2aDeGc265WzMvJAAnJD6E9t1P7SrTeewJ7qOYuGWJ fac+93eD570GV2F4bAkDoJ4rtIJZ+THMj7PtGjkhcjzVQkMBkfmMfpl8+mOaSWWwpd4e+6 UWsS2TTguQQ1j72S5rN1sM364Ds9LEY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544000; a=rsa-sha256; cv=none; b=WrLtiLOXTDnRz6zrns5eGevf3KC4CWzT2oREQnQ8ktaRGGunoYMcNUChgaVIu9KyOCpTgS ljlX103wyiCoW0NprhjkhpcateO+1+q6deQF1n1KQXa1MnUJPRwv3gf7tl9xZSGclHXJdC afd8EsZTcxN/yw0LnxbxtLCC+nKR/Lw= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=uxYPxwOw; spf=pass (imf21.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id E5E006205A; Fri, 28 Jun 2024 03:06:56 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 89772C2BBFC; Fri, 28 Jun 2024 03:06:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544016; bh=wX8wNBVCAeYaipWwjXbga4FkUGc1GqAIKlVyc5psJK0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uxYPxwOwKZkbAWcRCcj/faaq1syYlYM8BWU58yRNfmwar7N4AMMqbCcR61gQJ3hmJ nB9GYWciKXfEiUS4NLgEoiQquUcjB/0zyoiAKOBO8+xHDR4G1NEdIJ+wzy6h/WKB6H CLp9kQKd6Nbc+hxoc8kVP2YhPDyc6geIASE7dPiNVOMTMETM/IOUEOHBiw4hZ7rRVu 65oZiddBo5ncFNhCp1hfrtQm0GC4UA4uaVNCI03ksluRAywiDglD098AbmGckA0pU3 FHTvWD16NC64U9SYB4VXsPbRUDHmAuFRN4VfeFfhLySAXPbaYWFPevnO5EHDrJ8et0 7d+6X6G15pkQg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 02/20] mm/zsmalloc: use zpdesc in trylock_zspage/lock_zspage Date: Fri, 28 Jun 2024 11:11:17 +0800 Message-ID: <20240628031138.429622-3-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: C4BD91C0008 X-Stat-Signature: qsa8acsgphttazhdzuexgaz1muhjegzc X-HE-Tag: 1719544017-617130 X-HE-Meta: U2FsdGVkX1/Bz8EDbXHYSKtLYn0K0Z8OTT6jgffoncG21fcU6Mapk4Wg+iUEFZJN2wB85DGCnFiHE5dsKP1Wfge/v+RYm+GYSdckxLxgvv8fxBL1pk4CA6m6PBmLDF4CPg/sDRbltCpZ5DQULc3LEZw6g2Ze4BYEkJItHd8PVwuT9/JcjFASk4oQuZ69/2kXJqEGqNnnu+B4f4FcsZBgRO2XADCnKcqxXElRCj7ZSYIRvgaQqh58xKq+Sywa5K0vDfq0Y+9r1tFyAQf1Fg3ABgjs36RcvO/9sgnbo2yVCfBpzHjU8mTisr4+ig8Y4D1GQo0vqjaCycqpZq0NRqkjCSBjT44N226SSzF55wYZIcMP0LxAJhD34SbgnT14wWEe8W2nLLVE6o7CMUkfTtJ3ei0+URYdA4cPw7Bs76GXan1K0CKRnIhd4/L1EEZCItrNxZMT7U5mj7h5jQK1IFziczaGu6uTWz0dv/YDWvtXNXrz3Jqp0Bzh7H/wEYc7+z/rXlgag1CSTEGUAQxxqBaGePjHX+XZ44s+hqyqlAwzZ10zm28QXbktJKxyLGEMulADSsxZ4LpmukGoCdCUl095Ht/dnSdK7Lw+jz+uZGeV66KBHmYlYdJJkU2YWT3GjBH4NGIiZ1/JZO+2BSSQJ7wq7VYvdMrxBNQbOii02xP5VCGWoSynezFcFRuH9f9sHvDiYo69bX9HyDSP8brzvAvs6Dqi55cQr3fxENGdq9mD1AUzEV5qMF0jrP/MQNpCpjcwnlxp8Sb3XQ+ugiih7p4oxPI9x2X9OI5cGxowGeDhGt2Xl0/4dXIbV8Rz2jNlylnYvpiFqq3w+xOXxG3zdWY3Qj6ncTT65kQsuqOnRF9NHXtO9X+K5+dFZ73Hpj4oPtHPFgP23Bq5GrfdUVTG9Ka+Y6MBZNLYg8+aVHx66UMGdh5zWDcHmmAxyYbL90cQHnjKplAEIar2Q2CJnQSGrEp QtBZcquJ 8fQSIqHD/nUHeWgGnwhTAJUfGiJXyBuBnCutho8sWLZK7+LMskqtvZEStCImEvoYA7bmUoJpvpffiYriqWvytgQiciq2+w3AUVo5SXb2Bqr9qZaepbwypJR4qnm4E6dGH14W/0d8qZOFd/wJ+cA2kn1iiNDeEKLzQW6vQN9o1+9snRPz10Hj5UPNzP7RjfJUPnDmRevxUaEc2htG3nHkocD+dP7WFhU+qHPeaKA1Psc9JRxcFIWNm/4xokaRkWF1HfgcY2dfWHKjX20Z+Dc8jsndZn4hP38Gb03a2BLaBsV8bst3K8ZWs1EbjM7TLRfcuUcmog1lTc3gVoZTKW8dEhhpcCxk3z9DiGt4MpscDqixnBv3aInoIgu77tY+2oEce19PP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi To use zpdesc in trylock_zspage/lock_zspage funcs, we add couple of helpers: zpdesc_lock/zpdesc_unlock/zpdesc_trylock/zpdesc_wait_locked and zpdesc_get/zpdesc_put for this purpose. Here we use the folio series func in guts for 2 reasons, one zswap.zpool only get single page, and use folio could save some compound_head checking; two, folio_put could bypass devmap checking that we don't need. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 30 ++++++++++++++++++++++++ mm/zsmalloc.c | 64 ++++++++++++++++++++++++++++++++++----------------- 2 files changed, 73 insertions(+), 21 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index a1ab5ebaa936..fd95277843d5 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -53,4 +53,34 @@ static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); const struct page *: (const struct zpdesc *)(p), \ struct page *: (struct zpdesc *)(p))) +static inline void zpdesc_lock(struct zpdesc *zpdesc) +{ + folio_lock(zpdesc_folio(zpdesc)); +} + +static inline bool zpdesc_trylock(struct zpdesc *zpdesc) +{ + return folio_trylock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_unlock(struct zpdesc *zpdesc) +{ + folio_unlock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_wait_locked(struct zpdesc *zpdesc) +{ + folio_wait_locked(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_get(struct zpdesc *zpdesc) +{ + folio_get(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_put(struct zpdesc *zpdesc) +{ + folio_put(zpdesc_folio(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 67bb80b7413a..9835121109d1 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -435,13 +435,17 @@ static __maybe_unused int is_first_page(struct page *page) return PagePrivate(page); } +static int is_first_zpdesc(struct zpdesc *zpdesc) +{ + return PagePrivate(zpdesc_page(zpdesc)); +} + /* Protected by class->lock */ static inline int get_zspage_inuse(struct zspage *zspage) { return zspage->inuse; } - static inline void mod_zspage_inuse(struct zspage *zspage, int val) { zspage->inuse += val; @@ -455,6 +459,14 @@ static inline struct page *get_first_page(struct zspage *zspage) return first_page; } +static struct zpdesc *get_first_zpdesc(struct zspage *zspage) +{ + struct zpdesc *first_zpdesc = zspage->first_zpdesc; + + VM_BUG_ON_PAGE(!is_first_zpdesc(first_zpdesc), zpdesc_page(first_zpdesc)); + return first_zpdesc; +} + #define FIRST_OBJ_PAGE_TYPE_MASK 0xffff static inline void reset_first_obj_offset(struct page *page) @@ -747,6 +759,16 @@ static struct page *get_next_page(struct page *page) return (struct page *)page->index; } +static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) +{ + struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + + if (unlikely(ZsHugePage(zspage))) + return NULL; + + return zpdesc->next; +} + /** * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value @@ -817,11 +839,11 @@ static void reset_page(struct page *page) static int trylock_zspage(struct zspage *zspage) { - struct page *cursor, *fail; + struct zpdesc *cursor, *fail; - for (cursor = get_first_page(zspage); cursor != NULL; cursor = - get_next_page(cursor)) { - if (!trylock_page(cursor)) { + for (cursor = get_first_zpdesc(zspage); cursor != NULL; cursor = + get_next_zpdesc(cursor)) { + if (!zpdesc_trylock(cursor)) { fail = cursor; goto unlock; } @@ -829,9 +851,9 @@ static int trylock_zspage(struct zspage *zspage) return 1; unlock: - for (cursor = get_first_page(zspage); cursor != fail; cursor = - get_next_page(cursor)) - unlock_page(cursor); + for (cursor = get_first_zpdesc(zspage); cursor != fail; cursor = + get_next_zpdesc(cursor)) + zpdesc_unlock(cursor); return 0; } @@ -1663,7 +1685,7 @@ static int putback_zspage(struct size_class *class, struct zspage *zspage) */ static void lock_zspage(struct zspage *zspage) { - struct page *curr_page, *page; + struct zpdesc *curr_zpdesc, *zpdesc; /* * Pages we haven't locked yet can be migrated off the list while we're @@ -1675,24 +1697,24 @@ static void lock_zspage(struct zspage *zspage) */ while (1) { migrate_read_lock(zspage); - page = get_first_page(zspage); - if (trylock_page(page)) + zpdesc = get_first_zpdesc(zspage); + if (zpdesc_trylock(zpdesc)) break; - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); } - curr_page = page; - while ((page = get_next_page(curr_page))) { - if (trylock_page(page)) { - curr_page = page; + curr_zpdesc = zpdesc; + while ((zpdesc = get_next_zpdesc(curr_zpdesc))) { + if (zpdesc_trylock(zpdesc)) { + curr_zpdesc = zpdesc; } else { - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); migrate_read_lock(zspage); } } From patchwork Fri Jun 28 03:11:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715389 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2CAFC2BD09 for ; Fri, 28 Jun 2024 03:07:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0C6ED6B0089; Thu, 27 Jun 2024 23:07:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 04BDA6B0095; Thu, 27 Jun 2024 23:07:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE13C6B009C; Thu, 27 Jun 2024 23:07:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B740F6B0089 for ; Thu, 27 Jun 2024 23:07:03 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3271416148B for ; Fri, 28 Jun 2024 03:07:03 +0000 (UTC) X-FDA: 82278810726.11.81663D2 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf23.hostedemail.com (Postfix) with ESMTP id 84F63140009 for ; Fri, 28 Jun 2024 03:07:01 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="kePCV/TI"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf23.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544001; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=g7cGUq+URoNvvexaaMQLwI3OYNEW0J8WQAWgi8R3hs0=; b=YYqKen1j2o4L65mLWVeR9t3PjPNL2kr/ppWi2wyV/YvBMOKp2tPxIoJjpq9BMhHvHpHYok E/XYVKWP+QScIo4a4s8aDTP5ZhhzYpVbS6NovdhUsJaQK5T3hVLNOurnNMSFIoFUkTTJd1 rxOW/sCDxE68dsVMqLXwUeVwRwquQwM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544001; a=rsa-sha256; cv=none; b=T9tQqpZpYTv0Oa5Cc51sNcrHM8HPgG/L9JHCRBlY2Ay8FYLoxBTC89gZK6Mn5ElpuPrz9/ e2+AdO3lqo/QY68PLk9Kq/+Il/7U0TtwQS76zknCq8k5y7raWsN0DT9JJMYIOvqfW0+Mm4 gKv4pmpQj+ylFqtW2IdhXnRYAz1YRb8= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="kePCV/TI"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf23.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id A1C3762055; Fri, 28 Jun 2024 03:07:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 47CB1C2BBFC; Fri, 28 Jun 2024 03:06:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544020; bh=la1BOuF9dyPKXa6VzFValIggtH+1hQli1SJEv8vNjvY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kePCV/TI59oEiDU6RiCVWdhIVIIKK6xMrJRCLPWy/EAVyv+bgOGbbgE/wlA1MOr7d rQzOIbgaBwAOnvtUtf19hEHYCrJ4dovLCdTK8438MXdWmSM58iKRvGF9DbM3lXfOYp rKGIs0YnIzt4nnDUsyin6MVfxZaBbsF+kCD+jDs9TxrG+UIVVJ7vYbIlnowNDwFzVO 3js5naNvOwhjn7Am/WMKBsjRm83kLOsGAJuAjxrSQQiy+GQDMucwfZSIA9QICY4h8P 0uTp0tQREmNquy0fJ0kNTYxSLm+8YiI3moo28X2+FqrPCZLhgupx7dSpIyxsczlkRE 349W7Hg/LeVMA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 03/20] mm/zsmalloc: convert __zs_map_object/__zs_unmap_object to use zpdesc Date: Fri, 28 Jun 2024 11:11:18 +0800 Message-ID: <20240628031138.429622-4-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 84F63140009 X-Stat-Signature: wn6quqowpq9jdxoaaqere1z9ozxhbjxs X-Rspam-User: X-HE-Tag: 1719544021-998137 X-HE-Meta: U2FsdGVkX18ctT1fHEv+vVda99X5aRu4lSQMEBZaRT4Pp0SvKnOtnt31xOvJCSQfXL+p3vUfNupccwKKD9L5Nbj/37ZJok1qAXYuYyG23XzDDo3X/c8uKoP6MjB5Fh7+YZCImOVKZf6SHzMMd0UPlRA+b+mkSjbKUtvQhfRqe6hTLNjJFzD0/3p6/imQmSHHhsTSzUFGaf/BpTsY8JunhkFyfPq8AwHXwy5k7K1cdmzwV+wlfbYLjrhgFNhu/c5g8fLGHNXku+TeHe6XQyqjCWKDBJpJlT2jag/zeoKa+pckWOfDYBAfgoiN9SDnj7biP0OAbNfOi5aP62e2CWmnIK9UqiebUMcY/7qjPO/tP2p7esp8zCNA+pAvJMXyxHjfgZFFvldZcJV7b15dxnMJjwYalXCYVk306ixHst3NsALLhIn5JoLPEcxCsKPZ7oYtjqJYcqUDUMix4A9ZoL3rVjvTHenVXCmq6Lu4mzE+ynjXy65vJXn5HODdXNxSD1fIrN7BWlY2KhHRb0pC7buwisr9LF3RYHNoLBJrvwkH92EQ+/O4Ye2MRuLiRU5G8NyDhXnnOnBq0theL20H1HDFdMYwypgzZPHQNfnOXS+fITNj/qBnjraJwuSInBElLkKJNJJ4aKIyhjX9qMJaE/oqeyHRcwf+tc+ZrrtO911I2Grmm2D1DiKiCRKPtZACOYfuTod4p+rJMknYqTUQXkmN+nHk7U7slzLUsvmW30jmwKQKu7nHXgFF/QGoR8CGyM2yPBjuNU05y9YChaX3WlyltZ2f/+845jB90xgYp0qLTJbUAchpewzqPVoFDE4jn7DTQQKJQdvBluBEHaeBOqDfrzDM/OgSF0kvg8GnpOWAV2IlbFRw/cLGrhqIAdXQjmd8KELXdHUpqLb+q3NObEA+xJPN/YXX8XbpB69ba/lPMMX5s7SQIC0Sflms9MyFQ0FOI0N6Zl1CyCkA1A+AwMq poHIuP2y mQ4IdZMORstpwvFFL2IEILOxJhvAzWZl/zPonDbJJcsXIN9fHFSZa7Oe6/p04fSJXQ99Th9mOaEQ/tAmumK+d/QnW9HpARQkNnCAdI+Z10O1Dlg34qUATn1FN9fK0UyFLleBkggLXVL8mQBg+dbcbjhu8jgYEBb9j9QAe6ILC3xPBWQVP53HmfBv2Ft3j3SOhE+LuPycqMXHyjMyI+NpWxuF37jpmtraQ2jbCietssCaTz2q6zoFV5wLlklmid3YBr9iCvQfa5ttW9yg8A9RagQLuMEfPgkWMjZ2i4q2LC6qMtc53eTTD326BvAJOkvwbJKW8MUwc61s1SEFKIK4bl0LG7WcfqhuVXD1TyQHr/re3yCYPmDfPEUMqTzmZqc4ak2AO X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> These two functions take pointer to an array of struct page. Introduce zpdesc_kmap_atomic() and make __zs_{map,unmap}_object() take pointer to an array of zpdesc instead of page. Add silly type casting when calling them. Casting will be removed late. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 9835121109d1..cedd3dfb9124 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -245,6 +245,11 @@ struct zs_pool { atomic_t compaction_in_progress; }; +static inline void *zpdesc_kmap_atomic(struct zpdesc *zpdesc) +{ + return kmap_atomic(zpdesc_page(zpdesc)); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -1063,7 +1068,7 @@ static inline void __zs_cpu_down(struct mapping_area *area) } static void *__zs_map_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1080,10 +1085,10 @@ static void *__zs_map_object(struct mapping_area *area, sizes[1] = size - sizes[0]; /* copy object to per-cpu buffer */ - addr = kmap_atomic(pages[0]); + addr = zpdesc_kmap_atomic(zpdescs[0]); memcpy(buf, addr + off, sizes[0]); kunmap_atomic(addr); - addr = kmap_atomic(pages[1]); + addr = zpdesc_kmap_atomic(zpdescs[1]); memcpy(buf + sizes[0], addr, sizes[1]); kunmap_atomic(addr); out: @@ -1091,7 +1096,7 @@ static void *__zs_map_object(struct mapping_area *area, } static void __zs_unmap_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1110,10 +1115,10 @@ static void __zs_unmap_object(struct mapping_area *area, sizes[1] = size - sizes[0]; /* copy per-cpu buffer to object */ - addr = kmap_atomic(pages[0]); + addr = zpdesc_kmap_atomic(zpdescs[0]); memcpy(addr + off, buf, sizes[0]); kunmap_atomic(addr); - addr = kmap_atomic(pages[1]); + addr = zpdesc_kmap_atomic(zpdescs[1]); memcpy(addr, buf + sizes[0], sizes[1]); kunmap_atomic(addr); @@ -1254,7 +1259,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, pages[1] = get_next_page(page); BUG_ON(!pages[1]); - ret = __zs_map_object(area, pages, off, class->size); + ret = __zs_map_object(area, (struct zpdesc **)pages, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret += ZS_HANDLE_SIZE; @@ -1289,7 +1294,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) pages[1] = get_next_page(page); BUG_ON(!pages[1]); - __zs_unmap_object(area, pages, off, class->size); + __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); } local_unlock(&zs_map_area.lock); From patchwork Fri Jun 28 03:11:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715390 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DEC9DC30653 for ; Fri, 28 Jun 2024 03:07:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3F62C6B009F; Thu, 27 Jun 2024 23:07:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 381216B00A0; Thu, 27 Jun 2024 23:07:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F9486B00A1; Thu, 27 Jun 2024 23:07:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id F24916B009F for ; Thu, 27 Jun 2024 23:07:09 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id AF0C3A1463 for ; Fri, 28 Jun 2024 03:07:09 +0000 (UTC) X-FDA: 82278810978.12.85F7734 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf09.hostedemail.com (Postfix) with ESMTP id 6733914000E for ; Fri, 28 Jun 2024 03:07:06 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=R5F7BJaN; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544019; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NUXIPcUELjl7+6r4pYaHnzKbzhbwxFjnqXe/6x1pDeQ=; b=hcOk4T5Nad6Evpl254Btadifga0SNs9FKtA0/vabm5OA+yBIozcgWuLeBnxpNVZhE0EqIK GY9ovzXXa4aIc8lQaWj6WUTU4/mmbhpmpBHaAe0H3FDWWyTvL0K5hJ/w1azfUH9O3UkGCe boCjH+OIDdD2xk7J6DbgfSAniqetHZ0= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=R5F7BJaN; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544019; a=rsa-sha256; cv=none; b=pefKFPIodBVCPcHdRTzX6n5qBsMzSo+eHEJNWUZK8s4JwRcYOcaceKPhZxh/0SxYe4mDfV n3HFmX0FAcsXKQ9NuXfFkO68ddUJ12g3l5sHt8/owDsKHazrAX86EwkoJGVsyk4150IHnp 6+Pn/1tvdW3RZgm8Xdp6SqIQhgPsSHs= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id AD50DCE1390; Fri, 28 Jun 2024 03:07:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 04EBFC32789; Fri, 28 Jun 2024 03:07:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544024; bh=Lq3Glkac4UQFmimamfx1rDNuTCRcH+b9V+Y0ngU8Y20=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=R5F7BJaN3GA1L3ByUBwnS7JM2lJN1vZN2c9HhkzE/Knj//eX18PCVW+lJuPlcT35H Iodhqv5pltSEAMkGC+6DjTEXr5R/GE4wHanKSeG5Ve4n4uNJMu2DUsG7MPVSUVKab9 MN3HrGqv5T8xa+asV67FicGmnNgJXTGEH5M6QrH6ku1wgYja7QO+IB8lwxePqvcG/q zGq9JPJHhUehDaQp14KK4n2LvR7iWqOriqHcMFzQMWpClHFxNvzk05tMzk/VMyc1Ua SjQPZBjwy4zMx4VtSi9X1IqbtrTV2Wt0KTVwt4ecKnVIiCXM9o75u3xgKWvv2C7bUq GYAjfC8JgH6zA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 04/20] mm/zsmalloc: add and use pfn/zpdesc seeking funcs Date: Fri, 28 Jun 2024 11:11:19 +0800 Message-ID: <20240628031138.429622-5-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 6733914000E X-Stat-Signature: w1q9fyem5sgw9q6fagby5p5qhsk6jfgh X-HE-Tag: 1719544026-415372 X-HE-Meta: U2FsdGVkX19l7HJyPtL2bnf0RkVt3Q/4VdYBL+Y1rQ7IM38kVMWFPxcpVUIAhezmti6wXeWsQl4MyxkFEXxQwW2c6r9ZOuyAERzOKfeICYbVCYNGhSo3M1fvfXCZuyASJ6cjmAIndN+iYgkCZ6VU3xAOrVrw5ApO9/0kNuwxkd0CP1wcNFBCjUFNzmCUBtDCbu0azj2uOksFnhFJAeJQWl65XlpnArUPefVBsBIATgysL4pS/UfROMqSyz8+hgYYorE8KJI0jEtet5uYhDBQuWVp0+rL49ABLH3lbPkoTrXHQO0KE5ioKKwHADR2o2bLk56y+lFeSHsJ7q/uxPUVYqLZjQXaWmFnsyQtZd/I+dP4vFKbWTu6SzUCd/c9Dco7EEbIDrcATxq7+55Kt/cL+y+EzzwIAd4Qv5ZjUzMuznCpHWMv+rQsIwl/4C123ovSeAwIyY4cPjfrRiLitZl++nBZKtuJEcr4VApUdDS76IZR16o4Mqynq0rlaCaxDEhSGuPmU0vxkgP4s+AVajDGkhWFZPhFnHcqwoIoH75klNIqd5bsPZbXWa2Y7VljPFBic5ZYJzlx8J0aSy7EMvdNk7N2YeFL95yjJpSX8Neb0ZPNBIMtGkGdATUR+FnsG6adeXGViPk5dRIPTCloWqIWg1zS9j1GmNDg3AXjBuzodLbW424zqP/qFZEbgRuRmyJPJxfKHdfN6QM7oS6597CClpLJi2YCuuleN03OVc7NnQ3RMEiqM5ewyawg662I0Asm1g9RLHPfvTu4EsDY8aE/ftGex1d894xLocalA39pYi2FRToHR+yd4Sc5gJP5nSl/RFlR3sljfHQA3w1IAq3924CbA61sBy1jAjrF28qZB7+OCVDax5oYKdN9ErDjXH4iCqKXK6t3r6vXlUQkkTFi2EmjoINRRL85pnSSq/ypUYE5tgKQzcRUZFgP4SefpWhxxgEyGNqvgupqIfpoV1i QxuRauJ1 QchZq5DfiCGKXb21f2GAKU5G7bwofchFkTIeYIgAR+3rMyStX+oLIdBNKRpOo2xfEkBEmmxQpZtvZH5C31VYpOpD18twcHiWEa4/s34tMvuNbCvnAQY2laOj5bYLHDDrfR/eszB2MM6y4G8fQVPgDF/jHmxjW4NRGfYLADguo+9FR79YcvETWoe6/w3aG73H9cdYSKcyfNHEAIlm4vJTN3cjqs2lIrMa2rla9pUMJHfR5/zgiyTNbeyh58xT1eH7GgrrriO+Pioy6gUU0SQSIOiY1lYwunQ1RfJ67byDcaKeNbuiqI7Ar1aSu8SP749wX7c2JSH1VkKxwNUk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Add pfn_zpdesc conversion, convert obj_to_location() to take zpdesc and also convert its users to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 9 +++++++ mm/zsmalloc.c | 75 ++++++++++++++++++++++++++------------------------- 2 files changed, 47 insertions(+), 37 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index fd95277843d5..3c0ebdc78ff8 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -83,4 +83,13 @@ static inline void zpdesc_put(struct zpdesc *zpdesc) folio_put(zpdesc_folio(zpdesc)); } +static inline unsigned long zpdesc_pfn(struct zpdesc *zpdesc) +{ + return page_to_pfn(zpdesc_page(zpdesc)); +} + +static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) +{ + return page_zpdesc(pfn_to_page(pfn)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index cedd3dfb9124..efb1d58b3c36 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -775,15 +775,15 @@ static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) } /** - * obj_to_location - get (, ) from encoded object value + * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value - * @page: page object resides in zspage + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static void obj_to_location(unsigned long obj, struct page **page, +static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, unsigned int *obj_idx) { - *page = pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc = pfn_zpdesc(obj >> OBJ_INDEX_BITS); *obj_idx = (obj & OBJ_INDEX_MASK); } @@ -1210,13 +1210,13 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, enum zs_mapmode mm) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; struct size_class *class; struct mapping_area *area; - struct page *pages[2]; + struct zpdesc *zpdescs[2]; void *ret; /* @@ -1229,8 +1229,8 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, /* It guarantees it can get zspage from handle safely */ read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage = get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc_page(zpdesc)); /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1249,17 +1249,17 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, area->vm_mm = mm; if (off + class->size <= PAGE_SIZE) { /* this object is contained entirely within a page */ - area->vm_addr = kmap_atomic(page); + area->vm_addr = zpdesc_kmap_atomic(zpdesc); ret = area->vm_addr + off; goto out; } /* this object spans two pages */ - pages[0] = page; - pages[1] = get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] = zpdesc; + zpdescs[1] = get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); - ret = __zs_map_object(area, (struct zpdesc **)pages, off, class->size); + ret = __zs_map_object(area, zpdescs, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret += ZS_HANDLE_SIZE; @@ -1271,7 +1271,7 @@ EXPORT_SYMBOL_GPL(zs_map_object); void zs_unmap_object(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; @@ -1279,8 +1279,8 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) struct mapping_area *area; obj = handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage = get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc_page(zpdesc)); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1288,13 +1288,13 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) if (off + class->size <= PAGE_SIZE) kunmap_atomic(area->vm_addr); else { - struct page *pages[2]; + struct zpdesc *zpdescs[2]; - pages[0] = page; - pages[1] = get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] = zpdesc; + zpdescs[1] = get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); - __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); + __zs_unmap_object(area, zpdescs, off, class->size); } local_unlock(&zs_map_area.lock); @@ -1438,23 +1438,24 @@ static void obj_free(int class_size, unsigned long obj) { struct link_free *link; struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long f_offset; unsigned int f_objidx; void *vaddr; - obj_to_location(obj, &f_page, &f_objidx); + + obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); - zspage = get_zspage(f_page); + zspage = get_zspage(zpdesc_page(f_zpdesc)); - vaddr = kmap_atomic(f_page); + vaddr = zpdesc_kmap_atomic(f_zpdesc); link = (struct link_free *)(vaddr + f_offset); /* Insert this object in containing zspage's freelist */ if (likely(!ZsHugePage(zspage))) link->next = get_freeobj(zspage) << OBJ_TAG_BITS; else - f_page->index = 0; + f_zpdesc->next = NULL; set_freeobj(zspage, f_objidx); kunmap_atomic(vaddr); @@ -1499,7 +1500,7 @@ EXPORT_SYMBOL_GPL(zs_free); static void zs_object_copy(struct size_class *class, unsigned long dst, unsigned long src) { - struct page *s_page, *d_page; + struct zpdesc *s_zpdesc, *d_zpdesc; unsigned int s_objidx, d_objidx; unsigned long s_off, d_off; void *s_addr, *d_addr; @@ -1508,8 +1509,8 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, s_size = d_size = class->size; - obj_to_location(src, &s_page, &s_objidx); - obj_to_location(dst, &d_page, &d_objidx); + obj_to_location(src, &s_zpdesc, &s_objidx); + obj_to_location(dst, &d_zpdesc, &d_objidx); s_off = offset_in_page(class->size * s_objidx); d_off = offset_in_page(class->size * d_objidx); @@ -1520,8 +1521,8 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, if (d_off + class->size > PAGE_SIZE) d_size = PAGE_SIZE - d_off; - s_addr = kmap_atomic(s_page); - d_addr = kmap_atomic(d_page); + s_addr = zpdesc_kmap_atomic(s_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); while (1) { size = min(s_size, d_size); @@ -1546,17 +1547,17 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, if (s_off >= PAGE_SIZE) { kunmap_atomic(d_addr); kunmap_atomic(s_addr); - s_page = get_next_page(s_page); - s_addr = kmap_atomic(s_page); - d_addr = kmap_atomic(d_page); + s_zpdesc = get_next_zpdesc(s_zpdesc); + s_addr = zpdesc_kmap_atomic(s_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); s_size = class->size - written; s_off = 0; } if (d_off >= PAGE_SIZE) { kunmap_atomic(d_addr); - d_page = get_next_page(d_page); - d_addr = kmap_atomic(d_page); + d_zpdesc = get_next_zpdesc(d_zpdesc); + d_addr = zpdesc_kmap_atomic(d_zpdesc); d_size = class->size - written; d_off = 0; } @@ -1796,7 +1797,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct zs_pool *pool; struct size_class *class; struct zspage *zspage; - struct page *dummy; + struct zpdesc *dummy; void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; From patchwork Fri Jun 28 03:11:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715391 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3019C2BD09 for ; Fri, 28 Jun 2024 03:07:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E3A36B00A0; Thu, 27 Jun 2024 23:07:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 46B716B00A1; Thu, 27 Jun 2024 23:07:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2BD546B00A2; Thu, 27 Jun 2024 23:07:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 048DE6B00A0 for ; Thu, 27 Jun 2024 23:07:10 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 863D7141391 for ; Fri, 28 Jun 2024 03:07:10 +0000 (UTC) X-FDA: 82278811020.05.D6059EB Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf19.hostedemail.com (Postfix) with ESMTP id EBADB1A000A for ; Fri, 28 Jun 2024 03:07:08 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="fLwz/ccg"; spf=pass (imf19.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544020; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jevAal48WDlrKGLs2X9bbhtwUXQfBf6pD8AnC8XCHek=; b=6es8kPQiUsOKw/qtnSdoecONgrnmMe7pCs/gpF+vb11A3GYxGgFniZvTI78Cz8a51/A3H3 MELuHAf2QIyGh89te+FMhvF1Nbf7DJ0n1MuTRBdlwm0Wn4mW9nIFDL9FEEO/qi8zQ+3JJ6 SHogmHT7apVOrPv9yahpJZxI6s61xXM= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="fLwz/ccg"; spf=pass (imf19.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544020; a=rsa-sha256; cv=none; b=of6P1jGsSCkQT1hURqKvjoYJBt3yrvYKOYJSAnjQ7h4ktEKlgaIeyjk5ORYe8rEfn9RNqS hfxvruoFw/2liBzCGQUcUQtAvlqpfzgRcHlmwbTxvjZxf1ON/Q8C04y/UhlMaOQjfjp+9+ xRYmaZ8A52Tn155MW5rg337+5YFDlb4= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id F226061DDD; Fri, 28 Jun 2024 03:07:07 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BBB16C4AF0D; Fri, 28 Jun 2024 03:07:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544027; bh=XjQFvUcS1LHsRYGu9jnBiByPT0wYykOdlBmIkN2FtBk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fLwz/ccgEUbpiJOrFdS0JKWIACGuXo9B+lq1ll4r21yihrI4awNMbyxM+XE5ayHG3 XmK0O9G0zjrm7ibjLcKlK7WBFSzMn2Gj8217qv2X5CDFdM/u9I3EbeRkl+JrtaKfhk fQ6Wq3h+qXPYeahBSmHYSfSArtmajGSvBL7BsbIMmiX7e2BxiJFzWn5ngTydX03V+G LktDfp1xBOQ/s1nuxJRcdFMD2B//pwJQcQlM4wn4D+o3HzI8hd6yKYVAbQ7DLK6KOs dKisWok1zVs65YDd7PhvkNMpwpefhHE0rYPzWAo64qeHnYsA5F9AxbF+bN17yxu3iJ +IwgjYNBRM6JA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 05/20] mm/zsmalloc: convert obj_malloc() to use zpdesc Date: Fri, 28 Jun 2024 11:11:20 +0800 Message-ID: <20240628031138.429622-6-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: EBADB1A000A X-Stat-Signature: h3jdn7m4djnungfbxk7tsbj6g7dkafsr X-HE-Tag: 1719544028-257540 X-HE-Meta: U2FsdGVkX19qU7FHqBYImz6bBZSEMnsudlhuz3sorMrriPmKQrVWddOGebStmtYINr63sq7XCA0NRGmm5y9AptP1pL0JnAJXp9XuoqRVzdrH0qkYmb21t8R/UkpexE6LiTJtLVYE/uvHo10mDgcT+Zu62oSwreP7XeN0OgC+IDtRzcn8pKbOIe5juqxUCzNn5X1/7znoKA9922AEaQPNNu4/5RpFhaxLvtnFATYM1injkT6wrm2T11WDu3j0fWidioiP1MlRwMy2L8iezX6vloW1ysVqWUM89cg7e4MLLqhs9LzenSxJ8mQ00HjCYv5v+ujVDIn1e+WYC5vAvRbYCgcRgYwkGS2WD4ErNWgOLsTLK+KSBfw0Wl4uuG4Xlh8z2F628dsF7iPYRcdR93t+PP7E3VXKC1DUoUKaj8UBQbuPgShDgg3PrJSWJ/PqLMr+8JX42qUKQF+z+Ji19PKRXuZaOd/ilOdLjrdx+zQAnEB28aJev4Z86sf0Dw7bsifgJF5clO9uVEKL39NnyRketZAjzjuV11ZnJFQr0g8bA8qd6h1tAwnBzAIIW8BeCebdDEw3q1M4nWL1raVwoGnHGWFqGb6WBheRNevHL/E3PXMs0WF6S7wwveAmJe02BsW+YoNPDP6fumRV99vxvOgWXe/PP8T+ghY1Y7UXOizxU16GugzVGedNYeC/RYfAEuj1bGKhhsmiVWdZaqNTgPEZBdDGRJCVQYuSe4UIDhe0Ycl7RV3Ynv+3uvfFidlWNWmrsIX8+3Op7GCpWYEyYK8hCk4qv3B9UsCdvmfl5XmIYrbRC+p4LeOHT74EA1+XlSnx1NKEeVm6BNkf3ZYKr1hr8Y68jHwVo+xl0YH1x1ejm9LD5SNq4TwisqcYgwNdnTFgEPVHoRxmC8yvCSWEpN9Ds+9+ICeSGxaJ0fPCiB0p9qNMe7+tqi1HzqDFd8l0XVQWXH84QuXcmLeCFgB7pjN 7kCJY/S6 zjvUTdSi6oxGSFfM1AoF+1uExmEDVG9YVGZfBSw4tmrtZKYEC+KE5GmM1YitWwjFqJfngHoXTx+MgjR6QRjZB/CPyQGaZoZfVaRzuYCPTgMh/Dr4NROeQiXEF/Gk2o4QN+uu3f8HS4XNTkAldnI5HW73/H64MA+rTdSAW0stegF++XpH0IVsq5WAGPdHUgurVCB6TuX9RTocAS2tOKqxsPNkJcQ8mCKZLM8VUPcJIRaSXXuF1xhKNrMt3NHPv7u34tR8bh5UVQSiabaHVaKTE1hhftVSY/km+H/0Fxwj0hGsBfywBOsR8TMAFBszhU53v8O4Kq+Y8rqL3zxcdY2GAburSXhKSURSqaKYC++OcLX9sRKIkxdpSrn6uRg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first_page/get_next_page. no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index efb1d58b3c36..137b36515acf 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1324,12 +1324,12 @@ EXPORT_SYMBOL_GPL(zs_huge_class_size); static unsigned long obj_malloc(struct zs_pool *pool, struct zspage *zspage, unsigned long handle) { - int i, nr_page, offset; + int i, nr_zpdesc, offset; unsigned long obj; struct link_free *link; struct size_class *class; - struct page *m_page; + struct zpdesc *m_zpdesc; unsigned long m_offset; void *vaddr; @@ -1338,14 +1338,14 @@ static unsigned long obj_malloc(struct zs_pool *pool, obj = get_freeobj(zspage); offset = obj * class->size; - nr_page = offset >> PAGE_SHIFT; + nr_zpdesc = offset >> PAGE_SHIFT; m_offset = offset_in_page(offset); - m_page = get_first_page(zspage); + m_zpdesc = get_first_zpdesc(zspage); - for (i = 0; i < nr_page; i++) - m_page = get_next_page(m_page); + for (i = 0; i < nr_zpdesc; i++) + m_zpdesc = get_next_zpdesc(m_zpdesc); - vaddr = kmap_atomic(m_page); + vaddr = zpdesc_kmap_atomic(m_zpdesc); link = (struct link_free *)vaddr + m_offset / sizeof(*link); set_freeobj(zspage, link->next >> OBJ_TAG_BITS); if (likely(!ZsHugePage(zspage))) @@ -1358,7 +1358,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); - obj = location_to_obj(m_page, obj); + obj = location_to_obj(zpdesc_page(m_zpdesc), obj); return obj; } From patchwork Fri Jun 28 03:11:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715392 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18DC5C2BD09 for ; Fri, 28 Jun 2024 03:07:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EF4EA6B00A2; Thu, 27 Jun 2024 23:07:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E7F316B00A3; Thu, 27 Jun 2024 23:07:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5AD36B00A4; Thu, 27 Jun 2024 23:07:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A26006B00A2 for ; Thu, 27 Jun 2024 23:07:14 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5D8FB1A14C8 for ; Fri, 28 Jun 2024 03:07:14 +0000 (UTC) X-FDA: 82278811188.23.B794B83 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf23.hostedemail.com (Postfix) with ESMTP id A1DC8140009 for ; Fri, 28 Jun 2024 03:07:12 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=FSZsq3px; spf=pass (imf23.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544024; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AkpWL+9oLvJ4CI4JQdfY9bxscWHs3Ui43G+lsLhb+4w=; b=ZuItWLWmkO8OZollnWvpekVAZITmyhIbGBpQ1lAjaI2B7Ue/1bLwgPM/ZXpSgPgtTnx55k QhMdufYSZmFYoSJiD63KP51vyJ8sz1LazxrwMHMLT8XGgb4+7xGTeIRCnqnQBT0i34YiHu ii0znD4RQgbDJ30vBOl5X7d7jxX3xvM= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=FSZsq3px; spf=pass (imf23.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544024; a=rsa-sha256; cv=none; b=U714S5+YrhwjGNQcfDe13rOx9jxV2x4hvPo1RuOgU9MY2WxdADpuCm1jHI40qY1Ys+6YIo 2N6BjJWamcGRV3L5W+2t5eADOHrXNnq3SYYe4krM4SgO+MUHy4dz/+jZsvimZB2T6C7m21 fmM9gKoUSRmB0yXKtwVXMsMUiIgQX3A= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id AEA8062055; Fri, 28 Jun 2024 03:07:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 79717C2BBFC; Fri, 28 Jun 2024 03:07:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544031; bh=DEgsEkg1N6dj0FWoDYLMEtx39UiAuN8BZowdME7NzSQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FSZsq3pxtjEamZ0NO6vE08PTTiK4G3H8yUs8rLnV+WbysjiW+mZFZE0UGYeBNYpEP YB0gFO+qwGQWdfZX90SJHf9eCmRkjHRsMqgymWr9cgaXlHreST2W1QzDkqYI/sT1Ix ufqYD566GPxPVQ6LSh6bpClqC6aovilg0E20nW5vrxy2VV7iyWKMTpS658xJMnCVBU evSs3Cj4wdRZqcZsvcvaTUaOjHcXo5GZd3Xknyqg0RItte8+C0tIQnOO5QwFdQr+eP BL9EsN2UzrnTXDT4FkJJAjvPqkcs7l5c9m7jsvXXgyGFckIJes7Tw/OJ1Rm60vIol8 +bKRDIY5Blg9A== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 06/20] mm/zsmalloc: convert create_page_chain() and its users to use zpdesc Date: Fri, 28 Jun 2024 11:11:21 +0800 Message-ID: <20240628031138.429622-7-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: A1DC8140009 X-Stat-Signature: qr9myrnftn4tnng6wbzqaarkpeko37hc X-HE-Tag: 1719544032-87988 X-HE-Meta: U2FsdGVkX184yqYJ2Oh2b568UFpTSIjl9L03P07uMbdBK5Od6auczVMXJuuheWxAWv5sQZCM08zpJTho49b5hw4/qfiv2Mk81UDSxwZ55/mThyAD69ThOe7wsxULVLnUn1XWiVPwzB5xEn/5ayx6Ig4Odaq8+F6IkELSoDJ/Ljy35qQCVgurrIKnF7r+rFO1f412J0Kxe+vm8j3QqGCXSouK2sWrTBKMGRF6GK5VYgXLub5l8AH1C6yA9HwEJHKb3rM33kDoZX+MF4AapVcRhI+lzHjbFRDvUXQKB4HdUpgmq3FBshS9ewh8vpQ9G6Ix5pcE+hjDiwcQYCP7UMTVKEK1rwDrUJ1ytEmZ7qdSGE2Ry/+jxyD5FWOZH/RkdQe/aDzp0t6w9hNbDzX7p/l6o571q/+0UmOBXJMM4WQEdXIwywF6F9JXKjivMZ8T/vzo0n5kLYhJ7NmXNbqrdscB+lQPIgIUgaOnc7Uv+fgmx8HMNLmTSHw3W8JVTWpOkJqC8K6wsGizH6TFlIRHjuFW8FYaeFvUei0rVNLKwULD5RZ9PPUEPxqsqBOiG9AMl8JEgvjAnr+WhqDqlJ42OBmIZcr6sWISnn0W5qbyXru8ii4AISORnhAQy5lNT0kOsaUCRu9ZScNcgAOeH4dSAfkyPQ7igtHL7hxAI2rcCS9g9JZdKXKOy9cioE9Rj2sQg6Ccvdf+4QkZVPwU8AC6O7eoQMl4INKPKK4cw3Wt/fMGz6AJUtBkJBCQsu0iUpe83RbFnEA/V74pxVn1KIN6hO4UXHhKWCcOhD4Dr6eBkm+sVm8JiBISYXITFcOp0SBxAPTJfdbPLtH7JnSQRCW+jvPP8/LoK0ty7qNgsyL5ntlGq0n4IIf3vhgg8MzEpfBKp/KKDAwvEol9bN7B7/7oOFMSMhyu+N7PNpxp/5QbVFaZVnrEp4TmrXepnhgjTIXtCAnLMmwMI3nas/gM/OJUrMP zQalS/Vn ArWBR0J9gcDqX7xbLpAUhmACJ1cKSIaQ5I3i5Vno1fAvopnAMJR/USPL9LYo3DRmZbCsZgI+cyDet0WvlNQ3A0JM57MNMGJgCddm7XXLPXP5Sv+Kj2dDABOtlCe2INOfrGz5cl7FT+ZCwjnimulYaI+6h9zoRyO4g+Asy+/oQ5fGNLOIKc9fMrcj7o785TxxTAnLth62wNi/dFG9wUI7e9Zm8zkp48YgSFyAY0wHsJuukA0d0DkuIM6tze6Rq0rAoWiX0PyBD7EE6/vnbW/FFEGZ7Wh9SMjoNKFWYOB+TbXeTPi2xVa0IXa4roj7Wg4mGm0GqHBGDAhM7VKS3bFOdeYjtOlJQr/eVvopffP95McXyWfs7ZMe6vdlEagFkJybFN5gS X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Introduce a few helper functions for conversion to convert create_page_chain() to use zpdesc, then use zpdesc in replace_sub_page() too. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 6 +++ mm/zsmalloc.c | 115 +++++++++++++++++++++++++++++++++----------------- 2 files changed, 82 insertions(+), 39 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 3c0ebdc78ff8..f998d65c59d6 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -92,4 +92,10 @@ static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) { return page_zpdesc(pfn_to_page(pfn)); } + +static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, + const struct movable_operations *mops) +{ + __SetPageMovable(zpdesc_page(zpdesc), mops); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 137b36515acf..ee890d513f6f 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -250,6 +250,41 @@ static inline void *zpdesc_kmap_atomic(struct zpdesc *zpdesc) return kmap_atomic(zpdesc_page(zpdesc)); } +static inline void zpdesc_set_zspage(struct zpdesc *zpdesc, + struct zspage *zspage) +{ + zpdesc->zspage = zspage; +} + +static inline void zpdesc_set_first(struct zpdesc *zpdesc) +{ + SetPagePrivate(zpdesc_page(zpdesc)); +} + +static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) +{ + inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline void zpdesc_dec_zone_page_state(struct zpdesc *zpdesc) +{ + dec_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline struct zpdesc *alloc_zpdesc(gfp_t gfp) +{ + struct page *page = alloc_page(gfp); + + return page_zpdesc(page); +} + +static inline void free_zpdesc(struct zpdesc *zpdesc) +{ + struct page *page = zpdesc_page(zpdesc); + + __free_page(page); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -956,35 +991,35 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) } static void create_page_chain(struct size_class *class, struct zspage *zspage, - struct page *pages[]) + struct zpdesc *zpdescs[]) { int i; - struct page *page; - struct page *prev_page = NULL; - int nr_pages = class->pages_per_zspage; + struct zpdesc *zpdesc; + struct zpdesc *prev_zpdesc = NULL; + int nr_zpdescs = class->pages_per_zspage; /* * Allocate individual pages and link them together as: - * 1. all pages are linked together using page->index - * 2. each sub-page point to zspage using page->private + * 1. all pages are linked together using zpdesc->next + * 2. each sub-page point to zspage using zpdesc->zspage * - * we set PG_private to identify the first page (i.e. no other sub-page + * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). */ - for (i = 0; i < nr_pages; i++) { - page = pages[i]; - set_page_private(page, (unsigned long)zspage); - page->index = 0; + for (i = 0; i < nr_zpdescs; i++) { + zpdesc = zpdescs[i]; + zpdesc_set_zspage(zpdesc, zspage); + zpdesc->next = NULL; if (i == 0) { - zspage->first_zpdesc = page_zpdesc(page); - SetPagePrivate(page); + zspage->first_zpdesc = zpdesc; + zpdesc_set_first(zpdesc); if (unlikely(class->objs_per_zspage == 1 && class->pages_per_zspage == 1)) SetZsHugePage(zspage); } else { - prev_page->index = (unsigned long)page; + prev_zpdesc->next = zpdesc; } - prev_page = page; + prev_zpdesc = zpdesc; } } @@ -996,7 +1031,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, gfp_t gfp) { int i; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE]; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE]; struct zspage *zspage = cache_alloc_zspage(pool, gfp); if (!zspage) @@ -1006,25 +1041,25 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, migrate_lock_init(zspage); for (i = 0; i < class->pages_per_zspage; i++) { - struct page *page; + struct zpdesc *zpdesc; - page = alloc_page(gfp); - if (!page) { + zpdesc = alloc_zpdesc(gfp); + if (!zpdesc) { while (--i >= 0) { - dec_zone_page_state(pages[i], NR_ZSPAGES); - __ClearPageZsmalloc(pages[i]); - __free_page(pages[i]); + zpdesc_dec_zone_page_state(zpdescs[i]); + __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(page); + __SetPageZsmalloc(zpdesc_page(zpdesc)); - inc_zone_page_state(page, NR_ZSPAGES); - pages[i] = page; + zpdesc_inc_zone_page_state(zpdesc); + zpdescs[i] = zpdesc; } - create_page_chain(class, zspage, pages); + create_page_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool = pool; zspage->class = class->index; @@ -1758,26 +1793,28 @@ static void migrate_write_unlock(struct zspage *zspage) static const struct movable_operations zsmalloc_mops; static void replace_sub_page(struct size_class *class, struct zspage *zspage, - struct page *newpage, struct page *oldpage) + struct zpdesc *newzpdesc, struct zpdesc *oldzpdesc) { - struct page *page; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE] = {NULL, }; + struct zpdesc *zpdesc; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE] = {NULL, }; + unsigned int first_obj_offset; int idx = 0; - page = get_first_page(zspage); + zpdesc = get_first_zpdesc(zspage); do { - if (page == oldpage) - pages[idx] = newpage; + if (zpdesc == oldzpdesc) + zpdescs[idx] = newzpdesc; else - pages[idx] = page; + zpdescs[idx] = zpdesc; idx++; - } while ((page = get_next_page(page)) != NULL); + } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); - create_page_chain(class, zspage, pages); - set_first_obj_offset(newpage, get_first_obj_offset(oldpage)); + create_page_chain(class, zspage, zpdescs); + first_obj_offset = get_first_obj_offset(zpdesc_page(oldzpdesc)); + set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); if (unlikely(ZsHugePage(zspage))) - newpage->index = oldpage->index; - __SetPageMovable(newpage, &zsmalloc_mops); + newzpdesc->handle = oldzpdesc->handle; + __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); } static bool zs_page_isolate(struct page *page, isolate_mode_t mode) @@ -1850,7 +1887,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, } kunmap_atomic(s_addr); - replace_sub_page(class, zspage, newpage, page); + replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. From patchwork Fri Jun 28 03:11:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715393 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA89CC2BD09 for ; Fri, 28 Jun 2024 03:07:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 72C186B00A4; Thu, 27 Jun 2024 23:07:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6DA1D6B00A5; Thu, 27 Jun 2024 23:07:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 554136B00A6; Thu, 27 Jun 2024 23:07:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 336E26B00A4 for ; Thu, 27 Jun 2024 23:07:21 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id BED3F413A7 for ; Fri, 28 Jun 2024 03:07:20 +0000 (UTC) X-FDA: 82278811440.09.3E3ABC8 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf19.hostedemail.com (Postfix) with ESMTP id 85C741A000E for ; Fri, 28 Jun 2024 03:07:18 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=EsLIVXmX; spf=pass (imf19.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544015; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9YOZBj0WCGbWMxC2N8ScdBz2jFks4pm4Zy/Mo/8+yTs=; b=BuyakJoJXq1cJ2Kj/U4fFxwY3nfNrzKgMyDqGRZZmw4hUXkYetCe89//rE/OV3baZRuUYu 1R9i6PHZxL6DPSi+yWdQE9IfFY4aQm2a23UX9LxSWon9awPv8QydU388mUpFctEJ7lO7+M MGS/ss/kF+0x/AiXnNTdOF0tMU5Mq/8= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=EsLIVXmX; spf=pass (imf19.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544015; a=rsa-sha256; cv=none; b=6PEPLGgP8GCfO5UBw/I4knTBjaD4B+PZywtUlOR2aYpNdir5luXaLQSYnynCLQD/CIPzJ/ Ppxoo+PlFRCuLRxPqAlxNTgrcOKepJNUDpGj5rUMyY89+2K5mt48U1pURl8NCDDXMeqmwE fzB3fMILPQs1pmLfEqOrhSZ1jJEnAeE= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id B29D8CE1390; Fri, 28 Jun 2024 03:07:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 370D3C2BBFC; Fri, 28 Jun 2024 03:07:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544035; bh=6Pm9+ydZIsO5c9qRiniJl+L1KPC6XDYzIyrZkkU9aKI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EsLIVXmXFHNSPRxWd3m4+VHHl+VjFRdCHM8BTCvp4ASArnVdJg45eBwEpv1PTcc4+ eEJ1NHhHKtGm0C4A8uG2fnOzsLTe/goC7j/F4+vvDSYh3WgGjhkYDVegoswWQD2w4w prk/ypbbDn0J4scwSNmm3arQFeDX6m7bbKum0YUg/hBmQqvW5hthU1GNKPG49VCX5B 49mb0ZY9H9gEpbKIwx/47UF2WqV/ndYyAlhSva5OnEzxDIigPsBJWNOxaqEhf5HIGL Lxx2fFFVNA5XFHqBlSh73eJ7YpfzwDn8cAGEfxmnV26rCFrqUT3RiR/gVjKJDXYw9s cStrQQm3EJOJQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 07/20] mm/zsmalloc: convert obj_allocated() and related helpers to use zpdesc Date: Fri, 28 Jun 2024 11:11:22 +0800 Message-ID: <20240628031138.429622-8-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 85C741A000E X-Stat-Signature: nbyfb59rfnckxyq4nq31gk8ma449ib9f X-Rspam-User: X-HE-Tag: 1719544038-52347 X-HE-Meta: U2FsdGVkX19kEVypUTL7AllIrGLX2bhDliL+8qgJMps0MEN0LiEaCA+JoPe/I1DJidXIT6WERi+TnaAQRcmQvC/2aX+ugU23jUT02ZXBbYQS39WjY/GApYI4yrqhFzOfd8Dk9JOtRMyhqhuH40mJukmo9pwy25kQsMLM5JILmiiOkl7LgcHU/Pe+fKat26TqhuRYde/o/VnTIGFDDg7n6fV3cFMOzra3mQKpUlpxAwty8Bs6dQpZ4fzcisO4nwcNS+qnj9amOy+bnl9N8+kXfyk1sjImQpKLSTY++wvLkewi3eSt2luF9R9tM5tc9ZtqWH0VLFmdjafGXk2J9CKK0FmqMM36uHGujUCO9yuAWhthIR0fzHwLn3aLvIFKSlOtshxGJYGyOC97YtuhR1PE+X7sNDanHbE3YNyB54xp5q8s44zynJ1kkMVJYpraufdueyrZXcCcUfUZiPmzV918wn0NWF6RHMTziFlPBKqQtqBQc0aIFwDmyWSYSD05MVvxyqL17d3XfL5a/XLKiucifc8893RN4IlUb6lQJ5OvcINkq3ArCon+XeiED06OK5cRm2I7EkPSj5SS1grdfkWk7aA3m3qOt23/r0kFdtHi1n/yR3bpa+yj8wtiHgZRAEXP6QDzptd437L5RxeBnvZ69OkM0f2sIyEOkylrK25x+aBIqntyCNXSlP4rKgykljTmaVcV1TqL3ZK8X2eFXn48i3WNSV0NSof4vqRHWrxZT0wd228z7rLBqcWq90FKhGFm5qlpDneulnNHzHuxMOPhrQeAPWUxMHf7CDuB839kfOsf74HggaZWmbkG3crJudQbmJhefK1kjifqI475KjQXkb7tGYTc7RVag8fMlXR6UZ7Dae3FCZFo2/6fr9Gk1VcbkC0pVTfiuY9wG367Hi2LZqF93yLZN9G7G4QjEKwxi6hlRkR5pnRUVaIOqTiYSmnGBHrnNawB0POcGy1N2HO 0BApBcVl Wx2P5Ci6oUaxBPNFC/hrva3Q/X6ETHcicyZ2YSqAchsAxMjlLwojmCjHYjzm1dEVHSHfPNjBMZASCzBTURCR0B4ntzFbiubMxb26gLD2PiiZjIIBFvn2dsfDIG//BLN5h/nGqBSXuax5/B10TOzRUiQw7fHAmWNcZ4BZe5NFLorW6Ef/vprRtrLXE0YtVLYq4c5C93P2YqRWrGCoHGTmPjaleMbUjeZANS4EGufMoaEwbBajgJtREWTyC8/pH82vpYtnu7+zxzkpjDqcW2vPFJf6cdXWqhI6qf8gc2UIUdgL09UKFUQ99E/O/daDWc1+MYZ4zQizNLpTluK5eqhJD0BYsAIuH2ZYLSY5yc7RsLfA0PZhzLONebAi4PA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Convert obj_allocated(), and related helpers to take zpdesc. Also make its callers to cast (struct page *) to (struct zpdesc *) when calling them. The users will be converted gradually as there are many. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index ee890d513f6f..6efaa6279687 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -847,15 +847,15 @@ static unsigned long handle_to_obj(unsigned long handle) return *(unsigned long *)handle; } -static inline bool obj_allocated(struct page *page, void *obj, +static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage = get_zspage(page); + struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); if (unlikely(ZsHugePage(zspage))) { - VM_BUG_ON_PAGE(!is_first_page(page), page); - handle = page->index; + VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); + handle = zpdesc->handle; } else handle = *(unsigned long *)obj; @@ -1607,18 +1607,18 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, * return handle. */ static unsigned long find_alloced_obj(struct size_class *class, - struct page *page, int *obj_idx) + struct zpdesc *zpdesc, int *obj_idx) { unsigned int offset; int index = *obj_idx; unsigned long handle = 0; - void *addr = kmap_atomic(page); + void *addr = zpdesc_kmap_atomic(zpdesc); - offset = get_first_obj_offset(page); + offset = get_first_obj_offset(zpdesc_page(zpdesc)); offset += class->size * index; while (offset < PAGE_SIZE) { - if (obj_allocated(page, addr + offset, &handle)) + if (obj_allocated(zpdesc, addr + offset, &handle)) break; offset += class->size; @@ -1642,7 +1642,7 @@ static void migrate_zspage(struct zs_pool *pool, struct zspage *src_zspage, struct size_class *class = pool->size_class[src_zspage->class]; while (1) { - handle = find_alloced_obj(class, s_page, &obj_idx); + handle = find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); if (!handle) { s_page = get_next_page(s_page); if (!s_page) @@ -1876,7 +1876,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, for (addr = s_addr + offset; addr < s_addr + PAGE_SIZE; addr += class->size) { - if (obj_allocated(page, addr, &handle)) { + if (obj_allocated(page_zpdesc(page), addr, &handle)) { old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); From patchwork Fri Jun 28 03:11:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715394 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68F8CC30658 for ; Fri, 28 Jun 2024 03:07:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 001246B00A5; Thu, 27 Jun 2024 23:07:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EC9336B00A6; Thu, 27 Jun 2024 23:07:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D45946B00A7; Thu, 27 Jun 2024 23:07:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id AF99D6B00A5 for ; Thu, 27 Jun 2024 23:07:21 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7035D815BF for ; Fri, 28 Jun 2024 03:07:21 +0000 (UTC) X-FDA: 82278811482.07.C8831C6 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf08.hostedemail.com (Postfix) with ESMTP id D18C0160012 for ; Fri, 28 Jun 2024 03:07:19 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="T/NycspK"; spf=pass (imf08.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544022; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JoluYYH1w0+LRC3DRts5ANJYAAKNLdDWIwmlIS8zh+0=; b=6Z6wcSziY0wRLpRTkXRhMHWGLDgA5J5tth/074mO3SMGt7k8pSI1xUxnnuGVArlrx9F8T3 L9p18VpdZGcbK9mGULdYoqigKBZRXnzR42wAbURGn3VXBUHQ8BqHXpd2yT7UgyczOaIdZ1 WoHUDHtMhwWPWUbWy+UJPjT1MWdk0dY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544022; a=rsa-sha256; cv=none; b=FcoeESWhlJoofb8QhuCndxL/aLROFx7vVdpCpU03MjlqsJ7typHKZ7K9gfVSJHfjE+UMUn O7QRrhXkZE8nGELrjHNiRZYo2Wi5euJYJBxDH/1PE2e/sih8suDtMcqPuCt8IH7M8gkIZu rLr3HL58ZCl/9KHRKieEVCwH2qCLwv8= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="T/NycspK"; spf=pass (imf08.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 039316204C; Fri, 28 Jun 2024 03:07:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BECADC116B1; Fri, 28 Jun 2024 03:07:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544038; bh=KxgG3cMNh41F6OEFoev4W8KoSH3YuCbiC9yyKNDszWM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=T/NycspKY6Agf3MdDX1SDFtv0XHJhE1/1Ar+c+DWV0XEmuLaiYftnIc+JENyd4fOp daPtvZ95Qs3DI6xb3UCA8xlF8P5PlQ/U/k4ab1NL2hTXfOZIGYOVgEu4+CUaGkdhD7 tnIc259NZ3i5cYSavDbG2dHMBBjGtPv+RDgT2nUnnw/BAADyjfgVLMBjIpzXz5UpxI XDHb0hrUPW3d/n2EGhhU1NGpmpuktWX3xvOYAhE08BtUB6xV847wDyefrPg9uwS2fC 4rwIcyKebgAB+DWqa/OMETiqGnf3sC98/n1MmdU220AjgU75XO+WRb8rdkVVaC1/UZ ZRaOiC+UnURdg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 08/20] mm/zsmalloc: convert init_zspage() to use zpdesc Date: Fri, 28 Jun 2024 11:11:23 +0800 Message-ID: <20240628031138.429622-9-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: D18C0160012 X-Stat-Signature: xbd6ogg7ehauio6sdpcbih19ng6ef3dh X-HE-Tag: 1719544039-129355 X-HE-Meta: U2FsdGVkX18LuqSbQJQHcN90Fxg0o7Phpt5hl5iwt10sdhgiEn7Uc5Atge1MKZ3klImdUClKClBh872TFOdS0w+NnVqHQUoShATRkjI/FqLnZeZWNfeXsdPzexuUP0Acgqawm6liiNC+yoFC38DOBNBVhyzzFxMg9gTv7Gt10pZIoIQOZm4vz75OSxy6dpfEqIP/CzCvdlxYLFEwp1Azs7Mo1GDBp37TgxQC0RGOGasidvcSxlDSuM3A0qt9UhIgKYv1Ni7TY9DwSQ0btOaYmXed8c4Tjkp8H5gDLGaqUKIWqt+uoV+AHxZrW3yT8ZLlGpeFI+HCSsfJOo6J8oFsonRbqAGlu3QqGPBqPUoCu3qUPnwnM5ejfAhtK21hvWgaEsS1vvR0VceZbEXg6rvyA34J1Zh5Hxg5DHdUvDFrfWqHsFwWgFlIPKwq9cEN/PgDRyIeMnhMJL87nxHczgaa/t2RKwzcc3sFRx+EYSfrF9ikyPFZpbRfrYoZ9As5rAD06ekQsbuYpa/B4h4z0G1OIoWiEuwWsoiYkn6R9ibukbBJrRHkkx6MAUaLggWLFAZ6BgK4ylT4sv0In16U5VWFtp33bll3YQYMKSncR1qx9/8fLm+/5D3FDkCQe6/na/L9Enp5vqXYjHTgDJZLjkHyLKqYpbuYaOGXx9TsbfZZ8tk006Qu9Q+OO8oq6bQCHMd2ainbXcVZwty9AfZXOGfOhKE4x86iE79/E0dT1VdGoi9HA1tG3B6DvmhUz1XiaFPfYjKBi3jMoEuZPQDyLrQKby9FBPL5ZnNIXB0PqicSvGDwIKCYFOolMxYmH129b6tSSRcPV03agEZ1ZIz2f9NwJ30PYywWxohjR2Uk3WEJ3wUtpQ4WTsI9sEIjdG1bQaQAsNnjShaZ1MH/2pyiFdRkQUofHQQN/txqooGeScvX+G0aOAqOYZ/z09qKRct0BYX9xy24vlBDhKIVJqIw4Fs KjJMDfTR 3Ua9uB4E3lchbgMqFlFsUzL0o1rHoDA2NGwZH6FLzAOhz4k1C/MKH3H6vVkAFLxnWJbj/QjI4/8LXUrGU92lnZ8fys1tDsQG8BxR0jkKtWDlPpDepoB6kpNzHxfy7pLDL7Gf5Pmau5wWhi3zHNJzRcJzmk6Gwi9ukOjBQBXVABp2rgdzMM2YgHDS+6jZ7Fbg0NMufytmhLIvEXSYCry8syODdXsksYuGRoGc+/9ThMxGqOaVC2M6ZAojRCT/Si4SJaIId2LsOjcngLMpYkfR5nl3dLVFwYBe8W5iHrqqPhNzd0iLiRRDRBHDt8zR7YlKmN/g0UFVCYx9+tJaBWlPggMcQ50MOOhZd24r6n7nRsOXjBc7nlG4rPdBQ5Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Replace get_first/next_page func series and kmap_atomic to new helper, no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 6efaa6279687..e90e3a638068 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -950,16 +950,16 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) { unsigned int freeobj = 1; unsigned long off = 0; - struct page *page = get_first_page(zspage); + struct zpdesc *zpdesc = get_first_zpdesc(zspage); - while (page) { - struct page *next_page; + while (zpdesc) { + struct zpdesc *next_zpdesc; struct link_free *link; void *vaddr; - set_first_obj_offset(page, off); + set_first_obj_offset(zpdesc_page(zpdesc), off); - vaddr = kmap_atomic(page); + vaddr = zpdesc_kmap_atomic(zpdesc); link = (struct link_free *)vaddr + off / sizeof(*link); while ((off += class->size) < PAGE_SIZE) { @@ -972,8 +972,8 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) * page, which must point to the first object on the next * page (if present) */ - next_page = get_next_page(page); - if (next_page) { + next_zpdesc = get_next_zpdesc(zpdesc); + if (next_zpdesc) { link->next = freeobj++ << OBJ_TAG_BITS; } else { /* @@ -983,7 +983,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) link->next = -1UL << OBJ_TAG_BITS; } kunmap_atomic(vaddr); - page = next_page; + zpdesc = next_zpdesc; off %= PAGE_SIZE; } From patchwork Fri Jun 28 03:11:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715395 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17B48C30653 for ; Fri, 28 Jun 2024 03:07:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 975946B00A8; Thu, 27 Jun 2024 23:07:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8FEE56B00A9; Thu, 27 Jun 2024 23:07:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A01B6B00AA; Thu, 27 Jun 2024 23:07:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 580756B00A8 for ; Thu, 27 Jun 2024 23:07:28 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 155A8C0247 for ; Fri, 28 Jun 2024 03:07:28 +0000 (UTC) X-FDA: 82278811776.26.8B297A6 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf07.hostedemail.com (Postfix) with ESMTP id 1694440007 for ; Fri, 28 Jun 2024 03:07:25 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=asuZ1JAY; spf=pass (imf07.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544038; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dkDz35KBFrYW7wPgX7YzbuXqXCKGmsvzbaAgomoLgog=; b=3eIJt0/JKYkuw+VM5BWoP+eIddoSilX87PSjek+4BTWV/s0O0PODTe6BO0nZ372ENKKuWO qpvlfkTA88J6JeYeWQPJsSdV19ryWT51H5P8swENvOJvYVPiiF/dTOuQlVpev+ygz8qQpg jax9qkWmAjz5QSYCA8STnedmIfT9xQQ= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=asuZ1JAY; spf=pass (imf07.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544038; a=rsa-sha256; cv=none; b=njvw/dLxIFPL5AmTey3jfaXLY+U4uX0M6Hyk3ONm9t0ZDbxvxK16gjjvEQBQhZTSW7R88X Tx+Qw28qOafpChnvKT0/eOYJT3eKLHegveCG2ArIX2l7oIYkW6E4OQ67pAS03kzzuPzG7g kJOAJK5cZdB9wYQ89QppzwsUirSbjrQ= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 36627CE2D83; Fri, 28 Jun 2024 03:07:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7F2EEC4AF0C; Fri, 28 Jun 2024 03:07:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544042; bh=f1FIB9i8kXKQZtzU8kTMr1QDsZX4ItRyZ1ReJH3drr0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=asuZ1JAYGbKula5yKxocWKi3XyNBBP9kf53ah/IL0JHq81kjKvUKY3mJqyO7C04nj 55mdth8Bvias1bnoJbA9Yl82dad+PZtD0AdoIoQCbd/rEiLLmUVt7peD2R1uj4Z+kh BbN31tKf37MrNy9lcLxSbysArq5k0GH2C6A/e3qUjSoUddir51w3pGo6kBtLKfhBSB 39GgUBPrsDKVMJ+53C1BLXuD93wxHgw3Se1SSYCtGZTJWr3Tstzvbm8LDFTQIZH8M2 3ppPPAsXRhXnYvn8MycTdHq/au5R9QsL3F0ULCwRm7axJh4eFmoRLDYbAdbETSgYvu k1jLByx8bIr4Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 09/20] mm/zsmalloc: convert obj_to_page() and zs_free() to use zpdesc Date: Fri, 28 Jun 2024 11:11:24 +0800 Message-ID: <20240628031138.429622-10-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 1694440007 X-Stat-Signature: 3rdoo79b6f11zyqpuyc6u3sqb5fywb94 X-HE-Tag: 1719544045-557871 X-HE-Meta: U2FsdGVkX18AOhwXAY+7ln5kcm/te0U9ruIDGsKX7Ky2YplQXJmWLxPbWrHexkxWlMweOV5F0ZrF1ocPw+hnZ1cKDbgLeEV02b4KKEfOYGVMowky3P8zjYpD3EG9ydH+cFN9fG1oRgtzM02ZjHSBNxQGDtJOfancJqlb3Z0Jx5ZiabWb+oF49cSeC9l7D1P91rzaKSvIi90UazYGFpGjjIJ02HQJVXORg445S/sbUjcMbak05jVcfe+MafcXZCFJNiOwHkQQ5UpNFMtwLFnKXYS9bAWFr7FCg4FPIg5q9xTBebeA67yYV7N1UxUI4Pjvk/+r0G/k116ls4InOW0dD6Id1A11c1379rRu7G4kU49F9KoJW2H1qb+mirqB5gOUyCO2E8T7TwZQ39QT912tpnWIlMRfDM7a6Bb/dXLVM5Sv511N2Ek+D2llTRC519uZWSVW9lQyCn0ab0O09b7eiRV8UYbpcYJ3XF22IVRnaNCbgF9fbiusFlXnI0bkwdaONCTScPjM0iS+tKZlNKCpzZk+ItMSTjYChmBWU7/zdLv/QXxFt/sVLO8pwrmXpL6mcb32Axb62x1Mbn821OBGyRbVSZt1tX5cuIg8ttOcdKZHtJmzBK764leegM/FUx01o4TifSaEKwitfz8YvZiL65h8wasX03nkvnY3G8SBmJzzki3eFKlWgtTfi7ygJAoSVHZzSTXMQKBjbX4IXXX2fA2QsZcv9ceDhsDW0e4hNIc3QaktGdnfFJbwRtd18O/gzWAhP927KfKyjmSbGp+QZH+Od2atB6iELujJMi4r7xfl/HcDOCrTRqqc5fyM5dv6GY6AtUHuyOLDmLDd9znd2oMIUVvZPP7R9/XvgfwgNZ4jX5BTjairG48Qfz9ePC4Z1Po0hFno1IfSlJF9H9ina7gocYnrgki7xlY0eJjNb5M9ICM25VhVkaMpTkWpc9qZwYPiGEKhwgRM9FmclDK isNZ1ZxI rMTi/PV4oOzkRGr32nv+9tNFECwY05QlQB2I664hiUeHeFKODntcpfa/6OFnlTEmwDUvSfeUmOn6foz+AY8sG3nJ+wqyXBDsD+0EI71hy8b1d53wAF6MQ1aod2q7Oe8UVRydtlFI9FZh6vlG3wdSo5IOKykw8ialgImxY9cWxQc4zOKgCCw4/2rGJvKE8Aj2fyO5nKxDbrN+dxrSTBOd4VubImbpay6yCY9O/oDDzbP+l1g/x7TjNGMKN0hbjpMPoZ+x0UkvtwDfsF6K21VnEDRl5QXSiPHn01V/rlpKDpY/6+z5NbqP7wE64Fvi+NW278zhCWSKliS16DWCVW96KJghgFk97+fc3UktZcdfE9ptGPX0x6IA4WgOwSw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Rename obj_to_page() to obj_to_zpdesc() and also convert it and its user zs_free() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e90e3a638068..3ef8e13fb9e6 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -822,9 +822,9 @@ static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, *obj_idx = (obj & OBJ_INDEX_MASK); } -static void obj_to_page(unsigned long obj, struct page **page) +static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) { - *page = pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc = pfn_zpdesc(obj >> OBJ_INDEX_BITS); } /** @@ -1500,7 +1500,7 @@ static void obj_free(int class_size, unsigned long obj) void zs_free(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long obj; struct size_class *class; int fullness; @@ -1514,8 +1514,8 @@ void zs_free(struct zs_pool *pool, unsigned long handle) */ read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); - obj_to_page(obj, &f_page); - zspage = get_zspage(f_page); + obj_to_zpdesc(obj, &f_zpdesc); + zspage = get_zspage(zpdesc_page(f_zpdesc)); class = zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); From patchwork Fri Jun 28 03:11:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715396 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64C2DC2BD09 for ; Fri, 28 Jun 2024 03:07:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AFBE46B00A9; Thu, 27 Jun 2024 23:07:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A84CA6B00AA; Thu, 27 Jun 2024 23:07:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8895D6B00AB; Thu, 27 Jun 2024 23:07:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 640F56B00A9 for ; Thu, 27 Jun 2024 23:07:29 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 1D1BFC0247 for ; Fri, 28 Jun 2024 03:07:29 +0000 (UTC) X-FDA: 82278811818.28.58D9F56 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf30.hostedemail.com (Postfix) with ESMTP id 7154480002 for ; Fri, 28 Jun 2024 03:07:27 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=OYpxeUZE; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf30.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544033; a=rsa-sha256; cv=none; b=wQ9ox3EAtQw92KEKYXYSuALaWrzGbfXhvbC4ktp28MTZZmkNQt5F3QArFly9b6FfNh8jRJ 4wxNPsgyfOKWrIfI4nEak2Q6VjCgLaFgDineMzoEt+aJ0tfK/b0gCmynyIKFAA6X/23FHF 8JA2lPGBVouP4COiXiT6BibWALR3DaI= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=OYpxeUZE; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf30.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544033; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Zp7iN0uNl5MAM0ZdjSNPiBNHzo5CmutgpOndgveZu3Q=; b=vCN3alahFqD6Hr1GYah0Kq9u+EiDZYF3s21t9ErYXI83qM2L1Vl9o340jEcnX9NnnWRr8e K++quA5TCJDLFkMRFwQl993K93KRo59jJ8lIpkigTT6s0MID8Lfa1VRqrwQAK/2fWq8GrX zkctZCY5LRSwOOPUbemAyLDnfWRQ4V8= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 986906204C; Fri, 28 Jun 2024 03:07:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3DA8BC116B1; Fri, 28 Jun 2024 03:07:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544046; bh=lrAcBtSPY8wF6yEhTYORr+zHxr4lEQSsuXyA2QNdVVw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OYpxeUZEAarVD5SnSCfgAbdk57grg3isCoQNmyTfbmHWGcxRADl1KA0GOLSwNsskB 1LQ/VpAl4JEWfjQiJXUpI80DZ98V1A82SK/ZeiZy4Dik+eGYGzuIR5W4BVCiaZ3uT6 Bp4+4/OlTxkjz2cxCoEmrLAnnozkhD8LBnhHpPn+mBir2MUoS3t31oZxuqRGdiXY9+ 7YHLfuEPWLEtKBZLqm2gMIR/DXTs7xnUEYLYYZ5m6qjLL0rczZzWptVCpk7eykyEof 41Hd/z51hpHJWSSAZXHTHQuNpIjqUI+MH33JNtZgqR8PQmPy0us8dObV2536bZ6yNP 03ZYClyRuJqKg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 10/20] mm/zsmalloc: add zpdesc_is_isolated/zpdesc_zone helper for zs_page_migrate Date: Fri, 28 Jun 2024 11:11:25 +0800 Message-ID: <20240628031138.429622-11-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 7154480002 X-Stat-Signature: eyzbfauen7jwjawdx6g9619ptqkzophc X-Rspam-User: X-HE-Tag: 1719544047-388195 X-HE-Meta: U2FsdGVkX18y4dy1vh/jtwpu3zoIxDrZ5wh7SLaWoq/DjXwmRZg1pordI+67GG7LivbhzomYfUOV+wiqFNmy5bgQOw59OCZHJ24/K/xxr3yeEKlsVzESCnu5IiCIg4gsgGgGvSWAuBaoxg4jBK3JM9PIIcsSMifY+6pGGqRbtGYB1MVHhwQ5TUVKYSIN9u+/nEfgfo3sstic6ZFjYXJX6uY65W266DM+M8L15TwqWPWsKzqRmrxJyo7sOFr91Sq4yHFDis5/yW/wOLY35R8GeyiUJBVYKyF9oZvOHlK/InfiidnkVg/i2cwdAfFtENDHL5jM56YOliBmESGz+J9QRJ6aUl3OEvFio77TyCXMppcTmUcqibdpVaKEMknpqhJU8O8vHAcapsRYg7xmsb/Le9BQiXRE1RAp6S+hgS1mJYAGqVwJVtQbLWZriey3vuD0xidPFX73MxTe2UTfHF9QGkq6TZj6g08zXczrnqfjc5nCxrcgJP2ZDO806mJHhsilN+d48XKLvP0w2cdPbJOscoXqG0hcm7j2KPLvnqsDix4SiWben1A05cp81vmODmt69k/+fCL/gD5woZRgRAOf2aifakeNOBJw+5EP7SUIcCqILzLsUind3hU/N2l+rBksDgYd278dIW+wWpyJiURIWj2FfqPvPta1Jlx4tQ9ooT81q2Fo7+Idti/iOf6asGzJjf6wncFGaDfo+Tc+UDj7BgjkP62/48VgoHyKgH2C4d/4kY2lwUkSjHiVYlQXxSCnbO0o6Q+SY5hCqG434+/JE+uEkMeRIvHFRTnRoX3bGPHexJ4bZIKFtpHrKBwtzrZo+s6bBd26AtRsFPGjmcZQx/dqleryC0Mmi1sYWMVQiMujE2LbwCF8O8rxHFw1K8q/GURBFJ9oywfxL4gZlqUwSkWl0kE+J4UVka9U1yE2I+rzIMeDh3+m4F/TUYTG0xxCQ08pSAVXwkA3NgS+oW5 iLjlwcbk j0k7zLCF9bzTA+WLJW6viBSh19dY/JXz2ur8dJIjPK+MLV1jYitYkCV0GU3UoJhKLYzUj0aCnvojw4KpdPwstnFxww279KeLaX88J3HPB0IbGUZW3XA2OWR4QzLKuL57AZsDrm8gkvOEU+4SNyBwvmwcUbvaSqdEUUWxjkzLX/DdXurk9wVo4pwo32FVE6hb22D9eFuFAzPHPBvY51CzYVPZsvngoLiPxa/1vgr+hi0nE6hTMEe/ZBU0fcwVJ7YwWeeiPNQwetEGnk4paT6us6w3Hx/5mHEmspgF7JJDw4rRS5YEytMKqAy4PINzps2eX0k56cV48d49FSu4T+lti7UiabZ0jcgaZtyZixyBxoSn/2b4Nw3qo0JVy6g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To convert page to zpdesc in zs_page_migrate(), we added zpdesc_is_isolated/zpdesc_zone helpers. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 11 +++++++++++ mm/zsmalloc.c | 30 ++++++++++++++++-------------- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index f998d65c59d6..2e35b8220e05 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -98,4 +98,15 @@ static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, { __SetPageMovable(zpdesc_page(zpdesc), mops); } + +static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) +{ + return PageIsolated(zpdesc_page(zpdesc)); +} + +static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) +{ + return page_zone(zpdesc_page(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 3ef8e13fb9e6..251a6e63e58a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1835,19 +1835,21 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct size_class *class; struct zspage *zspage; struct zpdesc *dummy; + struct zpdesc *newzpdesc = page_zpdesc(newpage); + struct zpdesc *zpdesc = page_zpdesc(page); void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; unsigned long old_obj, new_obj; unsigned int obj_idx; - VM_BUG_ON_PAGE(!PageIsolated(page), page); + VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(newpage); + __SetPageZsmalloc(zpdesc_page(newzpdesc)); /* The page is locked, so this pointer must remain valid */ - zspage = get_zspage(page); + zspage = get_zspage(zpdesc_page(zpdesc)); pool = zspage->pool; /* @@ -1864,30 +1866,30 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); - offset = get_first_obj_offset(page); - s_addr = kmap_atomic(page); + offset = get_first_obj_offset(zpdesc_page(zpdesc)); + s_addr = zpdesc_kmap_atomic(zpdesc); /* * Here, any user cannot access all objects in the zspage so let's move. */ - d_addr = kmap_atomic(newpage); + d_addr = zpdesc_kmap_atomic(newzpdesc); copy_page(d_addr, s_addr); kunmap_atomic(d_addr); for (addr = s_addr + offset; addr < s_addr + PAGE_SIZE; addr += class->size) { - if (obj_allocated(page_zpdesc(page), addr, &handle)) { + if (obj_allocated(zpdesc, addr, &handle)) { old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj = (unsigned long)location_to_obj(newpage, + new_obj = (unsigned long)location_to_obj(zpdesc_page(newzpdesc), obj_idx); record_obj(handle, new_obj); } } kunmap_atomic(s_addr); - replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); + replace_sub_page(class, zspage, newzpdesc, zpdesc); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. @@ -1896,14 +1898,14 @@ static int zs_page_migrate(struct page *newpage, struct page *page, spin_unlock(&class->lock); migrate_write_unlock(zspage); - get_page(newpage); - if (page_zone(newpage) != page_zone(page)) { - dec_zone_page_state(page, NR_ZSPAGES); - inc_zone_page_state(newpage, NR_ZSPAGES); + zpdesc_get(newzpdesc); + if (zpdesc_zone(newzpdesc) != zpdesc_zone(zpdesc)) { + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_inc_zone_page_state(newzpdesc); } reset_page(page); - put_page(page); + zpdesc_put(zpdesc); return MIGRATEPAGE_SUCCESS; } From patchwork Fri Jun 28 03:11:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715397 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30977C30653 for ; Fri, 28 Jun 2024 03:07:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B72206B00AD; Thu, 27 Jun 2024 23:07:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B20E06B00AE; Thu, 27 Jun 2024 23:07:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 99A2E6B00AF; Thu, 27 Jun 2024 23:07:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 77E876B00AD for ; Thu, 27 Jun 2024 23:07:35 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 369321A14B3 for ; Fri, 28 Jun 2024 03:07:35 +0000 (UTC) X-FDA: 82278812070.03.E88354D Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf09.hostedemail.com (Postfix) with ESMTP id 37485140006 for ; Fri, 28 Jun 2024 03:07:32 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=G93JlV8Y; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544040; a=rsa-sha256; cv=none; b=vSrZyc9Icx2UaJqDZ9YjOu7BVC0tXOts88Bum9aS50GJLcRdwIIplQiSHubeZNlerBBFf0 hXS8ptb/+2Iyq7HMwFb08BuxDu2Miv0WsyTgxyF5zrnBcg+o8+2x0YCRRQ5hsqp15xR2c4 TZfFlUrKO1QvAcQhPeQkFii2+UM5wpE= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=G93JlV8Y; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544040; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ByZbEuL2MDpf+Zxk5cYlFhi4HqAeoW8UCkKwELZKWzo=; b=0Hmzt13sbylDDauASU0w1aquwss4s1I7amaCzXC39jb5qV1B3I9h20uBfonyyMa8eu8WQH Qxoy0BJ3kH4vBh/8FmfrOPybFg8ZSLC6nQZy9DvC6x3pENF8KpRlVfvySOghHu8sPDFkqB JweWc9IR+ov4dxUzlCtIJsRbLsSdH6I= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 774BECE2D9A; Fri, 28 Jun 2024 03:07:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EF836C116B1; Fri, 28 Jun 2024 03:07:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544049; bh=WVCo5FocoDi8mJGyEARAUtW59QMXLmGoSt30iD6IwAg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=G93JlV8YYzxWgG9GYE5JjU2ygzVYPZRfSW0u+Lv+XKD+uGl5mDGQzi1sW2Ses9l1a jAqs4GDKakVW2T7olETXuSfQGUxe3N79I+c8/cGoDeT2dP/Qw0Y2SuS5rGQLiufXU5 cOLPgU28kk/KZq11ozpMKKNWpsjd4HHmGqk0MlEem4feI7RiQOLPstDBogmGArrQPa LcgEy1h/AuWnFZj+aEkAbS3r6jb/tOYSB09HoP/8yXn74PQtbDa7t5ldfdFxCMsP7Q a2HDHZ12nEeMpUneZSsUIak2KUFzhCLVi5SF4Or2usMTBhpCCoUqLLHZLhaZQkpljK xnmC1xGKfZrYA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 11/20] mm/zsmalloc: rename reset_page to reset_zpdesc and use zpdesc in it Date: Fri, 28 Jun 2024 11:11:26 +0800 Message-ID: <20240628031138.429622-12-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 37485140006 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 4zcw6xdjp1h1jq43z6ewoajkbwjz3ior X-HE-Tag: 1719544052-611839 X-HE-Meta: U2FsdGVkX1+bv1eyF6eiw9QMJjyPt5o8phY8+MuWaRn3QjGshIxpsYwhHVMMSww9NNjGWmZtu45Ap/TsGmvXfUeldOSsqgehNEPxZOSq1F6YBCfISHJG5Ek0wgiwUHA7xwBqpLwmsra22re+8wvl/Bt4W+VuOG0LD7q11tUmdrROCwjm5gOZP5VxVVKuJbe8TL7xw09i6cehn12q8OchshWhgnIWlBQ3l1ow3gTsUyzjZgnRV3lS2w33zkWkW3XMOR8lWVkpkErgkmejezpgMH7Q9xx3w32rcaA4WEvikF+p0UwS5mOlpXjAEtfGHK261uOimRMj9elqirPZ5oFmJilSX5uXEN8QKKFrX7HAc9VxpT0B2/zs2AyZ56Fj8esvN2UVvMErd4K7NhjNoXfXI1xqdJ4mNebu7Yq3zCz/hyAQak5lIElugF+8TdaXxXLMZkEdzdgWXECOrkWW7PaPDkoLXUQ5kyWDkjfQrXCtwkqVop4iWmLwk5d+T+q+clCzQpg31pnw3x5Mumhya9M7tx9Fc2H2a+Vn9TcxBhqRpB+i3344nuuL7Hh4zFXWlHG0RMuWkedDBntQCHJAw+cl72FK3/6Oi6tCVohpni5XRQ52k7mpElmgUctfIRbGFU3wMXPMDoGFdaqU2JQ6XMZfCezpxvodZEbYhgbJr1bTAb6ZWeQrEnIJzZIgRCJfH0ffWY/wTzMGKo2GMhnEDj6hu2AAVqcf4AOopDr4s7qFnpfvoUpJ/DzS1mrfIOZvClZeXFGx+lT7PI9QfF7kb4abGm6jK5ACaIc+tImhJZFiL0TTLhTyDTtGF8iPKVd/pVQk/XRmdz0CahrfxVS3sOysUWt3aSuM3djhrpZnFcBcx2+ShfccbtxJRxy1T5KM/yQ1j5XJyNynBiqk22T6rdgY3bDq97h+j44hZbMqmCbrzfW400711BabFuJuAXCM7N1n4nSnxODutf58Nbt300A BsdJlVZt 4/Fwp1zmn9LhMN9B1DqMu1SwrZAZbc2MI2vRVNuN5fTTzTQFklfrdvHXgiGOgvcqecjqn9Xe0SonYsRdScqZjLG2XCu9tWN7Ew40/GIWsvuw3pqgd88CkXTNbFGOwPzxlX+I8oMhvoGHAeOd4nYe2c9ioK5lmawRHr9hXZdG5GiHQ62qZxpfjdAwWuifR0J+hfK/tg8y0Puzb1DtOQQEpWXC8zib26wKXtMXOV7GEduH4T/wQFSbEFbRvSfBSB9qZimMF7nAyZnudOqo37OEjQp2EUpdFZHpOBmfjibVBt14qg8mO2Q93V8cSIq+dJhH2LmTpReKBYyjNkPxbjFDaXor+zj7kNWFoZu0i X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi zpdesc.zspage matches with page.private, zpdesc.next matches with page.index. They will be reset in reset_page() wich is called prior to free base pages of a zspage. Use zpdesc to replace page struct and rename it to reset_zpdesc(), few page helper still left since they are used too widely. Signed-off-by: Alex Shi --- mm/zsmalloc.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 251a6e63e58a..216f8e6950ef 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -867,12 +867,14 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, return true; } -static void reset_page(struct page *page) +static void reset_zpdesc(struct zpdesc *zpdesc) { + struct page *page = zpdesc_page(zpdesc); + __ClearPageMovable(page); ClearPagePrivate(page); - set_page_private(page, 0); - page->index = 0; + zpdesc->zspage = NULL; + zpdesc->next = NULL; reset_first_obj_offset(page); __ClearPageZsmalloc(page); } @@ -912,7 +914,7 @@ static void __free_zspage(struct zs_pool *pool, struct size_class *class, do { VM_BUG_ON_PAGE(!PageLocked(page), page); next = get_next_page(page); - reset_page(page); + reset_zpdesc(page_zpdesc(page)); unlock_page(page); dec_zone_page_state(page, NR_ZSPAGES); put_page(page); @@ -1904,7 +1906,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, zpdesc_inc_zone_page_state(newzpdesc); } - reset_page(page); + reset_zpdesc(zpdesc); zpdesc_put(zpdesc); return MIGRATEPAGE_SUCCESS; From patchwork Fri Jun 28 03:11:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715398 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8171FC2BD09 for ; Fri, 28 Jun 2024 03:07:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D968B6B00AE; Thu, 27 Jun 2024 23:07:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CA7F46B00AF; Thu, 27 Jun 2024 23:07:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A5F476B00B0; Thu, 27 Jun 2024 23:07:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 837656B00AE for ; Thu, 27 Jun 2024 23:07:36 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 47433A1445 for ; Fri, 28 Jun 2024 03:07:36 +0000 (UTC) X-FDA: 82278812112.07.742C2B1 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf25.hostedemail.com (Postfix) with ESMTP id 8905CA0013 for ; Fri, 28 Jun 2024 03:07:34 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="mLYMc/7u"; spf=pass (imf25.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544037; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bAhIIaqab/APipgO1TGwSy9N4CpIWlezM1hq6ZRi7hI=; b=CIsMKdrdoGVEMnVR9Pc7/IGlWTewnZjB8N2hT7pBIHKjZJ13eTNCLQygzKAIta743UtbZj 1OfA7LeY7Hwm1oZMdbx/djqkasKLPRH7fdcNEV08xNrL50YUUGcpv5bFWp16gYkD3qdJLH aZipexa6o9qX5P/v5RVsJdToiaXjxRM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544037; a=rsa-sha256; cv=none; b=ipSSfLxmhBWMtPfv3Q1NvTSYHm4V7GSsD985WQ+oWGwbfPP54dpBD80mJCY0gm8rr2RO9X IKEkBVZtTBNFfuRLkjerKlKWO9hO6iuViu/ftY8bdm+9pzL2Hq1lykbUT/hEV8ltArBIhl Yluqjv6TfBhK2V9lS03R2bQV+zOPqms= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="mLYMc/7u"; spf=pass (imf25.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id B1F376204C; Fri, 28 Jun 2024 03:07:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 812C8C32789; Fri, 28 Jun 2024 03:07:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544053; bh=rDBA9PQhfrX2sld8/x0O9UG/mVNkTt8p9w0g00Zl/Is=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mLYMc/7u901OlvwzejVebGRJEyPAEygoLhV89brKlhhYjxZ0icIH+7gy5VLMEhstY nC7sgqj0XdoqBL5QlBfnBN+qP+BIpEJN0PjpCOSOS2WuR74IrOaEn/iLirkfQxTzS+ UkAx8p2uK7Z8em0W7vDigSNNjGWkjfZxdeqM90qox2/0MhzJ8EBVvPhWtBO936eT5i AnutNkwsiJvqnDczkg2qwn5VxLl1uIwZDool0UvbpMRsJO7a5R6tn9KT0sHs1oQWWC 7L62XReP/pejWZpdCCQx8KXAehDgaiohDLewX5nNd2thVBKgKwWv6LbGWHl/ZVXaNP WDThjKrP9FPBg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 12/20] mm/zsmalloc: convert __free_zspage() to use zdsesc Date: Fri, 28 Jun 2024 11:11:27 +0800 Message-ID: <20240628031138.429622-13-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8905CA0013 X-Stat-Signature: akp96eh5mpxr31pq8sd1fxdbjfhuf6pw X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1719544054-533878 X-HE-Meta: U2FsdGVkX19Fd901C3U0I9sMZJGup8NYS3YIAgtQMjCBlg3AC51IIsBf5TaWsO5ZFy2FwD5MAYqtb8zc42V0bQQ2ll2khag006OocdSgg5SxydZzibtIlxOdHKW/O+aslnPVFN3Tka7SG+HlnNEC3EPoVP/f0UB1OOThgdCE3/aVhFFvQ71JwWb06mQk8goRovX93eqZSriAT89pEaWxGpBb1fWyTQbsHcxBqJswEqqQCTp41QV9DfSeCqi4mKasSLT5WkEZZIBeCooSSoXUaEIQ/JZan6g5vLrA82gU1L1XeDh7u77Os77+0Jk/p+LZUo6JwRTRUQIzBrm+CXiFISwEydQ1tE15uV6I4BDs96fltdh7EWoLpRbipqafrEb5CNEmgkPr3IYv8e8knlVFb+dgNRNLPIQaVXfi0qbKEs9G3DN8QzyY6ZAEUVPq+siY7Y8GDAx5ogCQOwRUbZuWYTVBwNh/KxvzKMs7cqg8v+lIBVbARzbA/Z2RTP+QKCzoTGdwAMTWq2+DTI6z+u0Cg3WMeP/LLYPbSrlo2PRYbhwZ5qevOViWORn9ZzkwzS9cYIogSbu4D/+VMZ5qoBcIG9q5DspQvHxdG1TxbLAQqxNKfqqrKBvvJEcj/WT1EZ2phinyycVaf3YGxn+9F/XVLGTV91+uzTc5dQ5lV464X+tG9Nb2KtpTYCQO1EpfhXfPhkxCyTRTCzgv6N67CjfpzUxQlv3Xw1eGufYXZOXN+JLXlwkhuOpJwTMk4XhCKrxh8fDXpZtiu4Fd1huoqvTl+AABePk7hFzi/fPG4gFnpKa6xrxrL6jl3vciAleb+GmB+U4XCNEyrohDuTHD7pzgR29Gp384NHrDAjXWXAAvoZzHjexIntLKXriyjynL5ITGqQioER4l5jXyoH0dAJKpyvZTY2qaNDMapOGebS9gtERQrqTFSYSwUFfV2iIOEiATCnM7MnkUktB633kPaVZ w6vEmT1l ZaUpHIRU713tGqxTUny8dVS+6OffKA6q+dncCJHPp4V2ggAkRdqR26miSZCVltbykf6iMeNubBDNlMywq+wCbMm5OHZ3MTEfKHuUZFAfEYrE+tOFGElKR+HukyeeyH1i13DnR9EfaZIIsl93uhj9wtfh8s0KxNrb6AvRIcKXxZCOsr2ABfW5DNkXCgEYdzKgchWSsUxHXsEkGCXVGC3XUInH68OqTiPlMar2RPev8h4TaEWVISQ98XzHasleXoBxbcfE1fMTF3508l4qdWya1SLPUTuDEcJYrhVA6b8PIV/Fvb6q1Nd5gxZiJc6REZ+BGhxqo9Guod0l75+4dTF/OUgjq0jAQYmQGrdsaW0cgg57HB+g+5yYjzNKrVA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Introduce zpdesc_is_locked() and convert __free_zspage() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 4 ++++ mm/zsmalloc.c | 20 ++++++++++---------- 2 files changed, 14 insertions(+), 10 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 2e35b8220e05..186ff9711ffb 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -109,4 +109,8 @@ static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) return page_zone(zpdesc_page(zpdesc)); } +static inline bool zpdesc_is_locked(struct zpdesc *zpdesc) +{ + return PageLocked(zpdesc_page(zpdesc)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 216f8e6950ef..bf5a1b63bb17 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -903,23 +903,23 @@ static int trylock_zspage(struct zspage *zspage) static void __free_zspage(struct zs_pool *pool, struct size_class *class, struct zspage *zspage) { - struct page *page, *next; + struct zpdesc *zpdesc, *next; assert_spin_locked(&class->lock); VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness != ZS_INUSE_RATIO_0); - next = page = get_first_page(zspage); + next = zpdesc = get_first_zpdesc(zspage); do { - VM_BUG_ON_PAGE(!PageLocked(page), page); - next = get_next_page(page); - reset_zpdesc(page_zpdesc(page)); - unlock_page(page); - dec_zone_page_state(page, NR_ZSPAGES); - put_page(page); - page = next; - } while (page != NULL); + VM_BUG_ON_PAGE(!zpdesc_is_locked(zpdesc), zpdesc_page(zpdesc)); + next = get_next_zpdesc(zpdesc); + reset_zpdesc(zpdesc); + zpdesc_unlock(zpdesc); + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_put(zpdesc); + zpdesc = next; + } while (zpdesc != NULL); cache_free_zspage(pool, zspage); From patchwork Fri Jun 28 03:11:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715399 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7AB7C2BD09 for ; Fri, 28 Jun 2024 03:07:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4979D6B00AF; Thu, 27 Jun 2024 23:07:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 444356B00B0; Thu, 27 Jun 2024 23:07:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2989E6B00B1; Thu, 27 Jun 2024 23:07:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 048656B00AF for ; Thu, 27 Jun 2024 23:07:42 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B9E0B161438 for ; Fri, 28 Jun 2024 03:07:42 +0000 (UTC) X-FDA: 82278812364.01.9C40993 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf27.hostedemail.com (Postfix) with ESMTP id A41EE40017 for ; Fri, 28 Jun 2024 03:07:40 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=oIUTcWXK; spf=pass (imf27.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544051; a=rsa-sha256; cv=none; b=bmZGkIc20tcsO2STFCDK+VblejKF+66iLYJnCtmnQIBO3bGoaFMiXacSxIR/lVvm6NEmlt WjvoOZFxIGGOegKn3Qwy8eOeWNgZlr/WgFpBvOhJjScwBKUuFzV+S19fWgt/eVm0KF574v UBCyzSVDo8o0zmd1iEgDThWYwdYeCVw= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=oIUTcWXK; spf=pass (imf27.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544051; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CDWNHuXI1LJKO4XxIIxJ4PlMSqJF0GDetzQJMcSAuJg=; b=fA7+A+J8J1zSD1KZqBnPcWrwAVvseKFVSsyO1GkEvWmxd+CvcX6m80rmo6JXiXgZt9bD+9 Oe7xMGLEtlaSMZzIVVwGEQv/Xb/spHm/nvBiDCtN5AsyRXTh2QoCeKq3e/Pm2kFDyI6Y4K RQ9IEqzNS/sfwkqjxWC3OLGm1X0v+8A= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id C0516CE2F67; Fri, 28 Jun 2024 03:07:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 14C8CC116B1; Fri, 28 Jun 2024 03:07:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544057; bh=gedDp0RHU7Zb19ZIAumudrHJ1RWmViS9C8nGeod0zds=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oIUTcWXK+xs8W0Bn+nZd+E2zqjNEJ8xxmE/ynQKaenuBx0s++2m7yO67tPFf51YlB QNtpHUsRZ3/KM9uiJkqmmWApsGNcD5Z4brLMBJm7rD9XT6E+m8POORQcq0+eM6IZhX A3xTV3/hWWTqz+QxjbnhnqWH3t0NQdJefRAGWm4AaAoiasyGi7TJXpct7XkGKoyPEf bo8vRCiWJxJtEFuXHFms5scP7tUJ2i+4wyOJBLnYWfUmoIVDKACDbY8HbwIfv5bR8q OPBReFRDgjZSfMO/MF1XAy2s/p8kAdKbeWbZLIC+70s8CeBaHItKYNJoWCiVttKUU3 6qDrwLqLmgX3w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 13/20] mm/zsmalloc: convert location_to_obj() to take zpdesc Date: Fri, 28 Jun 2024 11:11:28 +0800 Message-ID: <20240628031138.429622-14-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: homo8e9bqtqf3eix6qnj9o4cyui9paep X-Rspamd-Queue-Id: A41EE40017 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1719544060-921539 X-HE-Meta: U2FsdGVkX19rQ8q2++KTFo3HkbfTdLkZcARGpNrgkKPCQkk96sbjkYzx8H/kZfGZA8V84hbxKb0r9wnzdHrBEmMGyoyThKo+fYOzqKYPtdNc9NKl2oIcidGSU+u0CGnhX/zDAMUT56iJ5SG95mukz7XLFaY6gd17/svvjZdtl0v8oBoYZIhg6pRHqCSK8zvyWAVI/YOgUvwHKNKkNfZ5CDvBXFi07539b8/uSUliRu/ab9WwUt9YIVIOe4q6R0jZq3zPzAIHn5K11FNbOyurv+4PM0NNrdfHfapKhXUuSLlBXA7Gwb0kOp7qG26yWQICeb6ECPvVw/38uAZdTmH4rZtYAZticylIR5hgleDNshQmHT21mJWx3hv1LcVp0UQQ1eRTGvt7/Qv6QrogvmKIjFYq/4oAo1e82VRGSffwFHASjdQDa2e0oqazWK+C5isSKq+g28wZ9uAVrtfNgmvaV2FBWqLhnS0YGGlMgXbH9bjtOwCO0JoADSOr8ReqaSIR95NU8ixr8A15RatCrhI5EfBvXW/MqsB22V/+wgw/nMSRN8Xgm/eeKs6r9+iL4j3FRox2sp0rj5qghm+qudTasoLonvU8VKkv9h2daPsSCzIIjU50cb6Aixyl5sSsu7aRjm3ViBXZpjlwqFV+mkfBHmqr2WRIQcr7k0Rh/NK+8o8fWj+9R/xYbkI6g3qiO8uuaK+Gmye0MetlTt9jEHeJ7vjruhm0MpKMUdPuvbCGTa/dXTX1YLyFaThyTe0tZnXFWe3/WI7jPMXlZXCHH6AtJAsNAgyGKyrRrY7d6Kw+GtY/n440QMIyrIWeBJUYx+lEhf53T116ZGCydW0qXX1OW21TN7VJzDFm5XgrZx4Y3nanaAZtKCapVfqrruyusDRZvAEIKRn0QxHJVNTE27hsjVWMEYzLHU9uXsZu8V0yLSizuHQ7wqfPDXyrCwJ21llieSo+2/x1nmLxqsfentg IAoRVQsH g2H6vWm/vZg7VtPxTRcOwDBYNQZJal4El8uPPE5Bwei3jLpjqBSBWVtVyJ8R+ktUMgCv+OL/cxZBLLIYjIP4EFvfS8YK2uDJGfQRlE2vwKwQheOEybzs5C5kHUYO/z+OIOo91rWMk45WBmlZd5B3ck4485ZkmfqVmk6qDl5W9vPaViE6odPCltMJChvUEhjVMc1n52N3w4gNJf+Y68SUQFGez6+TDEBtQMqxgIf4Je9tbHAbln7m8v5z3oJxl58dRWgl4Aa0dyvBS0gm0uMvX4Wyy2t9tlWbE6oZ7QweaQ6PYvyxJLoJjUweEKIkrQdASfUpYDbUKs2u9gZ/tapKaScP92zAnqkki1P+UJG2beRojc45NPI+zor35jw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> As all users of location_to_obj() now use zpdesc, convert location_to_obj() to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index bf5a1b63bb17..e8af01768ae0 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -828,15 +828,15 @@ static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) } /** - * location_to_obj - get obj value encoded from (, ) - * @page: page object resides in zspage + * location_to_obj - get obj value encoded from (, ) + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static unsigned long location_to_obj(struct page *page, unsigned int obj_idx) +static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int obj_idx) { unsigned long obj; - obj = page_to_pfn(page) << OBJ_INDEX_BITS; + obj = zpdesc_pfn(zpdesc) << OBJ_INDEX_BITS; obj |= obj_idx & OBJ_INDEX_MASK; return obj; @@ -1395,7 +1395,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); - obj = location_to_obj(zpdesc_page(m_zpdesc), obj); + obj = location_to_obj(m_zpdesc, obj); return obj; } @@ -1884,8 +1884,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj = (unsigned long)location_to_obj(zpdesc_page(newzpdesc), - obj_idx); + new_obj = (unsigned long)location_to_obj(newzpdesc, obj_idx); record_obj(handle, new_obj); } } From patchwork Fri Jun 28 03:11:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715400 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFDF7C30658 for ; Fri, 28 Jun 2024 03:07:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 509466B00B0; Thu, 27 Jun 2024 23:07:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4BB036B00B1; Thu, 27 Jun 2024 23:07:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 33E086B00B2; Thu, 27 Jun 2024 23:07:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id F1EF06B00B0 for ; Thu, 27 Jun 2024 23:07:43 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 778F6815C9 for ; Fri, 28 Jun 2024 03:07:43 +0000 (UTC) X-FDA: 82278812406.20.840FA9A Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf12.hostedemail.com (Postfix) with ESMTP id CF9D340008 for ; Fri, 28 Jun 2024 03:07:41 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=aECRD5kR; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf12.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544042; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+bkD7jUtGiZOZPGmOXqtYdB1Yh2uJS4UWLM8jt2hvrk=; b=7ODnG7Efw5p/WP6WnjWsZt17TSTZtLqgyUnPXwdI7/Yq4RpAX+ywqkOfGmlAYYW20ZS6Ak cIBp92Emm0dpBvGbWxB+vcZmNV+nQdshHrDDSFq4cbOq7uvyI16oCpGWAOqCA0p6lyJo+V dNNzI1YPMTxMbrYUroKuINbRjtCWMXs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544042; a=rsa-sha256; cv=none; b=ZrzyOYLY0CfmXfVvPMl7l2EX9TE9nFLznjpKMpclfrgQBzjbJfYoGKPrK8PiTcgdxctAWA pJVk+F7/gCWtoYzxRbTwqfnIkekrqiclpQMSoNWugnkOFJF+1TCjMtHBW+CP7MEU/y4+OX DzP9E7RVWQW0C0aLFq5kPMOZl/5+nNU= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=aECRD5kR; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf12.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 078FA6205C; Fri, 28 Jun 2024 03:07:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C6B02C32789; Fri, 28 Jun 2024 03:07:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544060; bh=6AxrbjsULXI2U/XKKk6mycuO4iJeWrzhsSGBcy6kdOQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aECRD5kR9xoAvSpndBIFlgKo2JqqxBip4pTFrYHYczi9pt2fEs0YDBL3FL9ihGLPN dglkNp8v2wVgW4fRUFamWff3pQLICuo6u4rcQytZJhWVKePCQKo2Z1pc4Keqr24Min kCxKz85MfgScoKYa8Jmkhm5LUaoIo6ll2NeyVgC8MU1CIjZATofW4L/z07K/VsOoNp Hg0xPuVEK8REdGAmwy7pa3YxAZ/RfCoVSfYm+87q30iT1p6LByqLPnnSQGNB3DHnmn L5Ph4XaeiMvUxy/hGgtGom2xh/BcTHyMdCpa9KAncBtu+ct1oNvBGLFPyrramw0rrJ Gi41UA6Yh6veA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 14/20] mm/zsmalloc: convert migrate_zspage() to use zpdesc Date: Fri, 28 Jun 2024 11:11:29 +0800 Message-ID: <20240628031138.429622-15-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: CF9D340008 X-Stat-Signature: zmb6mhy4bx5qwzbxf9cnbatitn9kr8xk X-Rspam-User: X-HE-Tag: 1719544061-459113 X-HE-Meta: U2FsdGVkX18UFOHyNDNTQiNu1accEZyBS0QZVruM0fEoof9X/TQR9O4VfsJrP8oqmqdMageFKrs2BY6UT/2ZHwKv24KEYCIUZgYgZxQi0AWe1dtRcrb9vfK4vI+YYLoDbDOxCzjCtf3GXniN6S6JStQvYSi+XeUo7QXI+uEMaQZHBqjZjZBfQQWaEhx5DuqzFMViFU07xcVIDQUZ+YB57fzpV5DvFg3toiA06jJSk7WvicGmdNZxo8ZwSB1N4PUdQsDBbO7kQyIGG0VvvqLFnsouWS6rJRvZhKsIRSWnTGKN5cAeDF4xO6/wdzXutgk5AIyUyVcgTeIVWdyiWRtY1SleiChktfXambQvpx0+ENwXpCUvpAv3ybGxlkVob9QveQQy9nOByLQhVq81ATpO7HFzwBomaMG8wwTxiIboo3qRbNvFHSAcT7HWV6KeC36VLhO8UT0EEbzmTU20f/8nZ/JrjIQaF3+uR8afPVL0ppgCUvk9l/faprAFPVTP3eeZIy0Vzk/3442NEAjlV6F/mLBx6LBAdNZsOHPEpYF5Xb3hMSn73okRVGsxrMKT86sXwurOXF4244iC1ZxVDm5hfXvV6XxaiucTUzZg6YjYNn/p9TFkYmwbHc2o34OcsS18M6g2XSDuNS/sB1/z49iAwqZgJHjTvJCuHnRzikkRRzsDWH3dhNEvlCT0BkfPeCzhTD3t6RShvND2/w12CV8qFJy58XZZNw69xaA7c2x/Tns60xAoHbwVqxr+GUladE/gs5v5xNPeki9Lbdz2sBOvIgNLL88480jtsDHsTd7HjTaphXAHMLGwdStH33nw89BbUD5QUd5YMuYovyE+zOy/Th84fFK8ds7ypOtjxdR+VxP78sAbqnnKLW1sCe8eFmUk+UuRkUqyRODuknqmNfuFVw5uCNnAx3uTtD4gYS/KXMf5fRJArcJrAjFLWO2S5IZ0rCwHSQ0bfr5cxc8d/t3 pclgjV2n ERolmhnYFBwl7HPhkLjC5hBdw5MHUxk1XQp4lMydYK10iZUvwRg84qFAe6ip8oyAX7FxHPLZtjNpjDTbt/YzbhTVe8DSfoJVy6q8yV0gvwcU3kPQSjPAHUKkjAWELPT1Ts5IYGOwTkrjM7IM03ySGxCbxwYYEVJAwU/cxFjyf9S1WL9PLW2PdvzQQsIGb91HCGY/OgrTUVvNTVGKPQiqlOSyj6HjHnD1T7TqTVeENYMAuacLqgnRLYCDZb2Fc2DDpZow9Y3SwYJoZ91HjLhGJjwh6IxlWj7nCzfEiksgm9BE04bmLBb+2yA4orwxnt8HzDRamVTxctw9Xj99ay80m5EhNXQN1fXvlD7nDZ/1IE9BGv8Gf8WCv8U9JN9AJwZrHUWCH X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first/next_page. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e8af01768ae0..a10af49d8d08 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1640,14 +1640,14 @@ static void migrate_zspage(struct zs_pool *pool, struct zspage *src_zspage, unsigned long used_obj, free_obj; unsigned long handle; int obj_idx = 0; - struct page *s_page = get_first_page(src_zspage); + struct zpdesc *s_zpdesc = get_first_zpdesc(src_zspage); struct size_class *class = pool->size_class[src_zspage->class]; while (1) { - handle = find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); + handle = find_alloced_obj(class, s_zpdesc, &obj_idx); if (!handle) { - s_page = get_next_page(s_page); - if (!s_page) + s_zpdesc = get_next_zpdesc(s_zpdesc); + if (!s_zpdesc) break; obj_idx = 0; continue; From patchwork Fri Jun 28 03:11:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715401 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67110C2BD09 for ; Fri, 28 Jun 2024 03:07:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E35CD6B00B3; Thu, 27 Jun 2024 23:07:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DE1786B00B4; Thu, 27 Jun 2024 23:07:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5A896B00B5; Thu, 27 Jun 2024 23:07:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A06F46B00B3 for ; Thu, 27 Jun 2024 23:07:47 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 5C9F2801F7 for ; Fri, 28 Jun 2024 03:07:47 +0000 (UTC) X-FDA: 82278812574.21.518835B Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf30.hostedemail.com (Postfix) with ESMTP id B2F7C80002 for ; Fri, 28 Jun 2024 03:07:45 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="ZZ/oDvEH"; spf=pass (imf30.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544042; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=B+/Hb8/S5Lk7yyk8AjSko509zSVfs0oko6ujW15zpCQ=; b=CqKIBPkyOaBC238g88UPf6azsJNljg7ytL30DZM9yT5Yu03XU/YfCZ2h8B0fZPI+H4WNuW YW5s3HloYKj1rZbprfK7ocS24zjuIR6PlthjFwMik1KAfH9twIH1f1mRix2T2xHZphm2AU fqm++HZ1a29KAO6mcbB12tB+xqwaHtU= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="ZZ/oDvEH"; spf=pass (imf30.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544042; a=rsa-sha256; cv=none; b=VXe/wFGsNJWmST9yN/3nHoFczrJe+KEDPsL4FYhYl8JrXWxoeeaIKK5mz1BIh3rF7/DovA 3YRDcz4wqUmCzQP2W/j8PgScFIMgZR3BPN3re1AGu37hgoPSsjaEy2F+7g2YkzGXIfCsrK MO6rJEQVvjQHvVHRVF3LgBbRhDFVfE4= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id DC35061DDD; Fri, 28 Jun 2024 03:07:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 84569C116B1; Fri, 28 Jun 2024 03:07:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544064; bh=vusdBHzfqHkaDK8baoOjr47f4AAjSRZFJhrtyFn5Ac8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZZ/oDvEHhBu8s6dToln3pesB6GaFXokhxzB7lNQwP7xSG6j/NBhslRWc1e3jMBzEO kiPYgyN2CWYo+tKF0U246DL7nSEXR4GnlsXz+SrRPmAc2lbptCrK5YiJF2xKiZdbnZ hv+O/C6NuSsMex3Y3mffzQo0IqaUOvYQc3E9iDlogODu7Y6/LHWRVBJwFSQnHwjGE7 GPzFRFyB0QXF9PAMKQW1jWAmIuF5gguivX0DnSVuME3aIdikODNiyyUvaqE/uv4A/9 PxxgTIaZaU8gQeEhxtl35LiMSPjI6rNW8ra32YGAVam6NcgKExAEY3Xxq9dui01BGi 68jFL4HIjdE6g== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 15/20] mm/zsmalloc: convert get_zspage() to take zpdesc Date: Fri, 28 Jun 2024 11:11:30 +0800 Message-ID: <20240628031138.429622-16-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B2F7C80002 X-Stat-Signature: n8yz93rytiufabz5gx89h333jryc5f43 X-Rspam-User: X-HE-Tag: 1719544065-273848 X-HE-Meta: U2FsdGVkX1818d8QZF0BT5Gw//79a9L0Dln3e3srcspALorTSVdstI0OtjZMKX4whr8Sy4YaUPpibnf7BvHYveKxODCt+tOGdx7souz4K2GMLQch7dbNsZgIz3epJpbMwJicepT2TVXznLoJCusfDpvbLJ9VZIJaptiUI+It8o116waUwAnf9gg+waQgaeqEw7iG9NaAms0SR2SokbJJ0fqRdQw0NrW0ayhyHaz9tv/3dGJ6zCaTB7I8CqmWE5JIGapy5/yMYuuaGUNy0dK+/muJFO5HsJvAC3w7bpO4gfmc33nHVvFS2/Uo94prrJtmcoN4s3z47DItfNSeq0ikiHiY6lNR2zt4kKBqr0XVPG3kkNJ1Jbn4oJtA8TMCvaCyq+ZHgEmoiw0djG7q2hLJXwqdtkQH9Ubhm1j1thvpygTi7Hma14+p9hmnNczrw+jWIBSr4UeBfcPKnLOcknCq8Xlb314bsjRw9llut9s5V5MMd+AxnIfjJPmLHx7F6xOw8ZznTbRTfugogdl6AK58Uy/B8T3hZGSScW5QXCZHxyB1NKel40Hd2JmlwHQv26yLYT16hOAChiy2pPH8AXU5cUxdNkbUxY4HA9ZwFoz8i+2G+DOq+xM7jDe2GxDh72Z91b1OnAoMZMYoxCEMnpCWAwezoqCOzTw2Witx7eDCSWNP/92DldIl3BzZlAt+NJbhnAHOuqGFE2zYlFH2ATT8aEN52+iEy+iiU86gDNw0Bb5eRwZPvUs76toqiETc0FPU5PrLhEAOfHWqtx9karbqKmuhHZdaBGF4JPCNOIto1PDW71C6fp8i7kdhdWUXFaYm/sMUq7FAOXtKuutHMzTi2mh5sRXE34zSGcs+86bFAxG4P3WnGM49a8fLbvZoyFQvuhJgMCxShrzphccf29jUnOAedTkErdwFcSpo2mSvg4LYG+egIDLSMReGNe104uqugRGeSmPOPMYvIkPpxzH 2eJDQ7KW Cn4eep3McUjt3U/wQO1KrbdjEEnWtnCPvZ29onxUR7w8kcuxiDX4q/rCXnm4BagGzUtRumJWac86S5sqbb6TALOHtwUc8Rd8EdnmHErwbZoKQDXprgxsf2PuQFThazUvUdadn+/v2CjO/f+9pGopS5g7es0wXOawV0B7gV5T1dlsP1L8pLJXNJ0LzaZI60clVU5ioImeMwKpFh8sSDtpNzd6Drpgx5EeMTMsD3wRf0C4hs7Qz0SGFN9GxHm4/ntjoc8V7194hPlRbdVfyx6PSyy2j/ZFS8nEDstVMmcBLZ8797TiHRoIzDAt01xiTPZlrENdTagUqeR0uFSJqd9lN2RMKcMEt5Wy4xg2+REdA6li2hSP87hhTiq0qbQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Now that all users except get_next_page() (which will be removed in later patch) use zpdesc, convert get_zspage() to take zpdesc instead of page. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index a10af49d8d08..c136124adb10 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -781,9 +781,9 @@ static int fix_fullness_group(struct size_class *class, struct zspage *zspage) return newfg; } -static struct zspage *get_zspage(struct page *page) +static struct zspage *get_zspage(struct zpdesc *zpdesc) { - struct zspage *zspage = (struct zspage *)page_private(page); + struct zspage *zspage = zpdesc->zspage; BUG_ON(zspage->magic != ZSPAGE_MAGIC); return zspage; @@ -791,7 +791,7 @@ static struct zspage *get_zspage(struct page *page) static struct page *get_next_page(struct page *page) { - struct zspage *zspage = get_zspage(page); + struct zspage *zspage = get_zspage(page_zpdesc(page)); if (unlikely(ZsHugePage(zspage))) return NULL; @@ -801,7 +801,7 @@ static struct page *get_next_page(struct page *page) static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { - struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage = get_zspage(zpdesc); if (unlikely(ZsHugePage(zspage))) return NULL; @@ -851,7 +851,7 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage = get_zspage(zpdesc); if (unlikely(ZsHugePage(zspage))) { VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); @@ -1267,7 +1267,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1317,7 +1317,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1483,7 +1483,7 @@ static void obj_free(int class_size, unsigned long obj) obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); - zspage = get_zspage(zpdesc_page(f_zpdesc)); + zspage = get_zspage(f_zpdesc); vaddr = zpdesc_kmap_atomic(f_zpdesc); link = (struct link_free *)(vaddr + f_offset); @@ -1517,7 +1517,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle) read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); obj_to_zpdesc(obj, &f_zpdesc); - zspage = get_zspage(zpdesc_page(f_zpdesc)); + zspage = get_zspage(f_zpdesc); class = zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); @@ -1851,7 +1851,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, __SetPageZsmalloc(zpdesc_page(newzpdesc)); /* The page is locked, so this pointer must remain valid */ - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); pool = zspage->pool; /* From patchwork Fri Jun 28 03:11:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715402 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80EF6C30653 for ; Fri, 28 Jun 2024 03:07:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 09C5C6B00B7; Thu, 27 Jun 2024 23:07:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 024566B00B8; Thu, 27 Jun 2024 23:07:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DB7A96B00B9; Thu, 27 Jun 2024 23:07:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id BE3B46B00B7 for ; Thu, 27 Jun 2024 23:07:53 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 6D9D9815D4 for ; Fri, 28 Jun 2024 03:07:53 +0000 (UTC) X-FDA: 82278812826.07.BD7E604 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf02.hostedemail.com (Postfix) with ESMTP id 6770380011 for ; Fri, 28 Jun 2024 03:07:50 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=cnRaB13D; spf=pass (imf02.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544053; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GkL7q3elqWvB35+rnNqWKh98ckKaqopv23PrO3Uu+68=; b=h/s56s8Sm0IefSpC6s97Xw2aMio/S08Y+1Qm5VbXDFy0o/E1X8FFX855LBA2g0GQ3hbIHN WyfONFRZVdQg0+0GKQ5d4FHFLQNhy9idqJUOHLq83Lh2QW33u9kqqKVYYopVxz/yLVMTTL B2OT7Wl/EUV7mxB8H0qsFebRiP+tl2o= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544053; a=rsa-sha256; cv=none; b=4qX1eQoUN+qh5kYPcpoYDDyVqXo1sf9bNCBmgaOnMAjB1I+3Nq+Pthptp/HR5QJ479ioiV c+qQUTLXLHJvpnXWLt//4vrkrIWeA62Bvr4d7N8/JR3wcrm5QBPp7mFmG7X5ycw6ReRu1o Xbxka+7HCXCg0/Fr/+TEctJn9kjwrVU= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=cnRaB13D; spf=pass (imf02.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 720B0CE38E7; Fri, 28 Jun 2024 03:07:48 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 41863C116B1; Fri, 28 Jun 2024 03:07:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544068; bh=kldqwAh9cZgKWc/qnQGuGYAkpd0utSr37gCIdaPuGYU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cnRaB13DEdjhJTai8RSwypjifYR0A/D24GZqzbB/taIZOGJfx5ukrATQw9p8OyWzM iAAxQfNiplB4VzN0rtbVW16IMBdZ/RsWZzSYGoCNXE7P0FcRAh0awod5grcab4giG/ H9STRRKZnmZoXd6hwu7YPDdhTGj85zNkM/x+g2gmKq3nr/+1Yqa5SxXdBusHA0/gHA gavi3MwMdlDst2QYjmWdcNLkXxzrNsfj+AkN1SOSbDG0fQAH2wr8szA84Zub72pyP6 UeXApiZ2X+rmvh2mr8QAuT5tuyHFj2yDnC3+V4z8vxOPqUv+lI8tAvJlryj3mSk/gL rH841R6zzOW+w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 16/20] mm/zsmalloc: convert SetZsPageMovable and remove unused funcs Date: Fri, 28 Jun 2024 11:11:31 +0800 Message-ID: <20240628031138.429622-17-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Stat-Signature: aobchqkbztp37zfuk34wibe5qt6h3se1 X-Rspamd-Queue-Id: 6770380011 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1719544070-452116 X-HE-Meta: U2FsdGVkX19UGqbm/uFfKA96PmWwZYppeLXfS2qVFchSZHfGXeNai3YxbYIhG+p8UpvWEv100ncVVqHbHvXJXIqiH5LwoXvgw6BvItdS7NDUvykiWiOJ9vz5feZTRDzfXZ60pMTAGIPuruQDGAvTxtM8tf4I806k6hDVb+iEEDoOdrmwbJn2Hhc3fuGhoxwIB+q/SfIUUMCewZUwos61/DHfdB7DOKIigAUQ1DEqHUQ6gsGZuaAkMcrwF9ZMs3YsD+IVc2pJCoyeAvHB4qxhOCmRdAHdefICvkgcmmZXYVLlyGPOViQESBCeNSmTA7iUy1Gv4cL3T1yaF1PjRiQfgZmKZXMmgZCLrfF+rejZdG6Zk4BR67Dx4ktT5lTtBB/gzYTD6s6Z+/1ilfT7bvGnJK80ml5rjpZ5c0NkWg/TktufxomZH2E+V20WckyxZNQuXmG982elxyFgCRPd3Grb3GUjlNU7X7sdqUCR917nKW0ES11A0rHYFzpLiH7WdZ9ZsESMtAhPhgnJwJe3TmHqYAP8lkF9Kg4aL7P++dBKYqCQ/f/3wEweMLUvZmKbEpeJhUJWn9mB/7lpboOSsWQaoEE9CKXDU27tekSTGMDvuSJzZp4lblHis4wpxajuBmlBZDi8bbIGHW6NE4xZhAHuMXu6wP+S6iIFHY9SeMWAYRJHxikxy4Q8ELRDQg5+rqRWJaM5OqVxb83RI+PwYllAiH9sSvuSKvi8inZ3Kn90Q8Iymy9avBL8K0ss70obMOPGpfJwUpVy2fxKXaoBU6TgcOms1vN3nYCVhGU11xnKAtytSnMVzH0jTNE2PWuVdyueuUzjAt6UFIU8QKoijpO5JptZ+ONRdSHYq8cNTdJlxBf96izHeZ1lbRP9RMTXt1+BBYkS0Cb0eMTS3kvXdwarDKWEEHS4b/TGaxopyFZ5th59BoQg3jSjNuugcxUHhGcy8twrnq86GtNWIrYgDiF uqUWgWQu ec/64Z+5NMwAJNP72pJauuD6Xrmj1ijehDMXojquUZcUJP8rRLh54o7x8HQirkSsicYzw+oSudzvjWWppFn8LoQiTM7L0Lpt3lXf/G6FTx9pe538tO1dMjfDkDUfgXovPJWUbkLorOSZDcLbXvVq8kiTkvVbu8US2Zl097723ROpMtuypNjz51qtZSzsL/E/WKwOX/zHxE6qas17D2KPCHF/ldNo1YX48PxnlKAjFPJppB4kGC4u0w0oZa1OzgJBSZQ5r+D3uhlyUHYM9IzqqLWJvThej1zMiDnboPWXIHES1/58c88UnfRxMjIfESnGX/kGtAXVLpt9EZ7E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Convert SetZsPageMovable() to use zpdesc, and then remove unused funcs:get_next_page/get_first_page/is_first_page. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 33 +++++---------------------------- 1 file changed, 5 insertions(+), 28 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index c136124adb10..951292dbcae4 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -470,11 +470,6 @@ static DEFINE_PER_CPU(struct mapping_area, zs_map_area) = { .lock = INIT_LOCAL_LOCK(lock), }; -static __maybe_unused int is_first_page(struct page *page) -{ - return PagePrivate(page); -} - static int is_first_zpdesc(struct zpdesc *zpdesc) { return PagePrivate(zpdesc_page(zpdesc)); @@ -491,14 +486,6 @@ static inline void mod_zspage_inuse(struct zspage *zspage, int val) zspage->inuse += val; } -static inline struct page *get_first_page(struct zspage *zspage) -{ - struct page *first_page = zpdesc_page(zspage->first_zpdesc); - - VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); - return first_page; -} - static struct zpdesc *get_first_zpdesc(struct zspage *zspage) { struct zpdesc *first_zpdesc = zspage->first_zpdesc; @@ -789,16 +776,6 @@ static struct zspage *get_zspage(struct zpdesc *zpdesc) return zspage; } -static struct page *get_next_page(struct page *page) -{ - struct zspage *zspage = get_zspage(page_zpdesc(page)); - - if (unlikely(ZsHugePage(zspage))) - return NULL; - - return (struct page *)page->index; -} - static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { struct zspage *zspage = get_zspage(zpdesc); @@ -1974,13 +1951,13 @@ static void init_deferred_free(struct zs_pool *pool) static void SetZsPageMovable(struct zs_pool *pool, struct zspage *zspage) { - struct page *page = get_first_page(zspage); + struct zpdesc *zpdesc = get_first_zpdesc(zspage); do { - WARN_ON(!trylock_page(page)); - __SetPageMovable(page, &zsmalloc_mops); - unlock_page(page); - } while ((page = get_next_page(page)) != NULL); + WARN_ON(!zpdesc_trylock(zpdesc)); + __zpdesc_set_movable(zpdesc, &zsmalloc_mops); + zpdesc_unlock(zpdesc); + } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); } #else static inline void zs_flush_migration(struct zs_pool *pool) { } From patchwork Fri Jun 28 03:11:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715403 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 545A2C2BD09 for ; Fri, 28 Jun 2024 03:07:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E56A6B00B8; Thu, 27 Jun 2024 23:07:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2973E6B00B9; Thu, 27 Jun 2024 23:07:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0C3056B00BA; Thu, 27 Jun 2024 23:07:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id DDAE76B00B8 for ; Thu, 27 Jun 2024 23:07:54 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A10A41A149E for ; Fri, 28 Jun 2024 03:07:54 +0000 (UTC) X-FDA: 82278812868.09.837E66D Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf18.hostedemail.com (Postfix) with ESMTP id F2D001C000D for ; Fri, 28 Jun 2024 03:07:52 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Kn8GoGgZ; spf=pass (imf18.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544064; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cLzn0lEwP9g6CtaSp7tP7hd1tIoDPUEGDmoCjPN3erE=; b=6FN2YJQ+kqrgbYmrHrI7pzBd1DZPVR71NJqAajTKGDhlVYQZXCcsxa1uhbg9p2BBd1hUa8 6H7PZLi9XRq6HUUEehBXiKrAVFeqwrncGFzkTtVQfpieWxLy9X3IkLeBc5VJsXZJKmt/rr oCiGzP74IUiI1LtAT6YkXVg1Uibkb/8= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Kn8GoGgZ; spf=pass (imf18.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544064; a=rsa-sha256; cv=none; b=eeZAXOIJBMOd/v9K10tW6DQYVfQVahio7gEu78x2iBA9W05xMWP00BQPTRuviyuKUJOlz/ cRoeZvF7Ci0kpKGarbBCbq+PHieAukW/AmRmxBMOcyoKvuVthF6e3YTBovBW6/cxFg+bbx mshjNVMD3Cx9ys6tedWrItCnTNzX7V8= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 2AB9D61DDD; Fri, 28 Jun 2024 03:07:52 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C7603C116B1; Fri, 28 Jun 2024 03:07:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544071; bh=BULUfQtc3EjNlGEEdYDtph/07BaO7bkwBtlyH1/5iBg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Kn8GoGgZJ1gvzJR2ncaJOTr4cft0ptFYqqqvbtAwTpCfjfvz4TvWV5uAi2Eez6HFe fKUWmp/Ts+D6L4Djx91kxmxRNiKIpJnkx8+z1/ZcZdL8U1zcxymua+7Gi4mAGVdSPM JORUVVR6FgEOCUCxesPq43knn7yJph28Reeb3A8CaTzJ0LJYlKmw1JaZJKOtO85Pce R/zlvqlS7vVfrxk14GzwY82GipQfBVN9n7LaPjttYrKJmpNLAq4aXSGvIaD6Gh5t+u QRlJ7PvfPSXp5TlA6TInbxbPPsrHJyyGJD7s2T5vpgnRSaqu8yy/uWEXSp7AbPLic2 7bibTnUJh3iLA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 17/20] mm/zsmalloc: convert get/set_first_obj_offset() to take zpdesc Date: Fri, 28 Jun 2024 11:11:32 +0800 Message-ID: <20240628031138.429622-18-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: F2D001C000D X-Stat-Signature: 17zuembbwoifj5ewjomhzdb14a1tdy6z X-HE-Tag: 1719544072-351474 X-HE-Meta: U2FsdGVkX1+1Jgy9kDRw74a+TVIAI9PFc3P5wDIn85Pzojz9C0q7Hil4NhTm/+vIuW5Av+l/UMW2vJ/YVxBJFuFWBRaVhlLCzWZ1ifs5WXAGDN06GGhPiTyHDwCmyKyi4gSpwHMKGpn0vlXI11aVKr7URtRMWkVXdjXiL9agfgSAz1Ckkdx4vlfv6dbMuduoRjdxrgUwy+vlAGM840rKn+imhGKiddpfaYaYPg0+EDpI32TbT7a0epQ8oQzfqbqL5mfErzIzos5bN0NMXcnKN9YR1hrJtIO+jpONUsJeDwOfv078JR94VjavbZ4KHU6R0WwJwQHoOkB79J0GpnOpPdf/r2z2hOpuU4uYEvZ/XpwG2akU0fkbAkb5u4eCP63upu7uaGFYANgijAA4lQKDrh6AgQYSrhBJWzVIXh5jvht5lkuqits6jOd5AnJH2YT7TdXHFqEftsTIDu47eXNoSaZMWxC7T7Ok/FhNzuVfBEy5rtfzU/IDdbYvVhcmlc53j0zlAl3UVuRMse+VOlUNLDZSm8YsK8RxnENKeVMvIH1eweTTI80v5GBrxN6wckUXV4I2Cy4h7X6wXVk2bzx5pHx/JKLvl3AN2o6yv5iGKP2VgAkruKpn/FiLLq4V5ujst3/dpJg5MqVrddWF2aBVJoPOP1R2nqsXka6+0bBIBXxYIcS/KaoBn5zr4YV3j4AlYmTd+qiP0S0pLd1/v3qzXMfYgkYqvfIHT5mW34825mq4AmofuRiQ0+9Y3LicmlxPxIUFhlL42n5wdVolv4Trwkdl8N1QhiXv//Yic44dz8mfHZJSiPVPf1nZVArb/7n4SLGe5JbNeeNBxTlmURfYqab7rs7aNvrDY05WIFmspobwGyX5znvtdvmc8SAfeEcNnNIBl6U9DqiuBlkZAPuT5SNYlbjm9WNWBegkVipqG+jO/huLhUZx0A0lZwj8ozsxa/1ndWi4S9LbKWTk1U2 DAe+mNAD vg+9CIEax2Cyh3yKho/5TIbWggdfRd380NzDXZ+I3VvX9Y4iBmNhpkuUcUJQvrkXm0qUEiN6jym1ZFvnC/elJGroCFqVY9NWeMNyGMGsPP931Cn1MNFmOQSqS4KnPQPizOIJ6islkOgk36xauNgqUBBhXhXVJQoNIFnP9ox2LdSVHHGyOgyPv4SVvkYQ8RJfnPCAKBF3iAR1oB2wyvdms6zWdiggZKl4pUCf6lYZwxVzubti7tElvqho5lTp7Jn2ZYv6bAlzkmQGpFz8dcwkSUNBsqgwUbS487MCe1kmiOr8WW94x6bZ9reYj6oVEjZkJV0qk4KkrVpYRhUH8ywQ+RGg52dXlrddhCp4asNALwKlmmn8+eToKfaag1sPJKrlQCFt5 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Now that all users of get/set_first_obj_offset() are converted to use zpdesc, convert them to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 3 +++ mm/zsmalloc.c | 32 ++++++++++++++++---------------- 2 files changed, 19 insertions(+), 16 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 186ff9711ffb..17941774c46e 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -14,6 +14,7 @@ * @next: Next zpdesc in a zspage in zsmalloc zpool * @handle: For huge zspage in zsmalloc zpool * @zspage: Pointer to zspage in zsmalloc + * @first_obj_offset: First object offset in zsmalloc zpool * * This struct overlays struct page for now. Do not modify without a good * understanding of the issues. @@ -29,6 +30,7 @@ struct zpdesc { unsigned long handle; }; struct zspage *zspage; + unsigned int first_obj_offset; }; #define ZPDESC_MATCH(pg, zp) \ static_assert(offsetof(struct page, pg) == offsetof(struct zpdesc, zp)) @@ -38,6 +40,7 @@ ZPDESC_MATCH(lru, lru); ZPDESC_MATCH(index, next); ZPDESC_MATCH(index, handle); ZPDESC_MATCH(private, zspage); +ZPDESC_MATCH(page_type, first_obj_offset); #undef ZPDESC_MATCH static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 951292dbcae4..3ce49f0372cc 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -496,26 +496,26 @@ static struct zpdesc *get_first_zpdesc(struct zspage *zspage) #define FIRST_OBJ_PAGE_TYPE_MASK 0xffff -static inline void reset_first_obj_offset(struct page *page) +static inline void reset_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - page->page_type |= FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + zpdesc->first_obj_offset |= FIRST_OBJ_PAGE_TYPE_MASK; } -static inline unsigned int get_first_obj_offset(struct page *page) +static inline unsigned int get_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - return page->page_type & FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + return zpdesc->first_obj_offset & FIRST_OBJ_PAGE_TYPE_MASK; } -static inline void set_first_obj_offset(struct page *page, unsigned int offset) +static inline void set_first_obj_offset(struct zpdesc *zpdesc, unsigned int offset) { /* With 16 bit available, we can support offsets into 64 KiB pages. */ BUILD_BUG_ON(PAGE_SIZE > SZ_64K); - VM_WARN_ON_ONCE(!PageZsmalloc(page)); + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); VM_WARN_ON_ONCE(offset & ~FIRST_OBJ_PAGE_TYPE_MASK); - page->page_type &= ~FIRST_OBJ_PAGE_TYPE_MASK; - page->page_type |= offset & FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset &= ~FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset |= offset & FIRST_OBJ_PAGE_TYPE_MASK; } static inline unsigned int get_freeobj(struct zspage *zspage) @@ -852,7 +852,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) ClearPagePrivate(page); zpdesc->zspage = NULL; zpdesc->next = NULL; - reset_first_obj_offset(page); + reset_first_obj_offset(zpdesc); __ClearPageZsmalloc(page); } @@ -936,7 +936,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) struct link_free *link; void *vaddr; - set_first_obj_offset(zpdesc_page(zpdesc), off); + set_first_obj_offset(zpdesc, off); vaddr = zpdesc_kmap_atomic(zpdesc); link = (struct link_free *)vaddr + off / sizeof(*link); @@ -1593,7 +1593,7 @@ static unsigned long find_alloced_obj(struct size_class *class, unsigned long handle = 0; void *addr = zpdesc_kmap_atomic(zpdesc); - offset = get_first_obj_offset(zpdesc_page(zpdesc)); + offset = get_first_obj_offset(zpdesc); offset += class->size * index; while (offset < PAGE_SIZE) { @@ -1789,8 +1789,8 @@ static void replace_sub_page(struct size_class *class, struct zspage *zspage, } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); create_page_chain(class, zspage, zpdescs); - first_obj_offset = get_first_obj_offset(zpdesc_page(oldzpdesc)); - set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); + first_obj_offset = get_first_obj_offset(oldzpdesc); + set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) newzpdesc->handle = oldzpdesc->handle; __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); @@ -1845,7 +1845,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); - offset = get_first_obj_offset(zpdesc_page(zpdesc)); + offset = get_first_obj_offset(zpdesc); s_addr = zpdesc_kmap_atomic(zpdesc); /* From patchwork Fri Jun 28 03:11:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715404 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BBB3C2BD09 for ; Fri, 28 Jun 2024 03:07:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A11506B00B9; Thu, 27 Jun 2024 23:07:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9C4A76B00BA; Thu, 27 Jun 2024 23:07:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8166E6B00BB; Thu, 27 Jun 2024 23:07:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 5F63C6B00B9 for ; Thu, 27 Jun 2024 23:07:58 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1E52C1C2920 for ; Fri, 28 Jun 2024 03:07:58 +0000 (UTC) X-FDA: 82278813036.25.381938C Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf09.hostedemail.com (Postfix) with ESMTP id 7F83E140009 for ; Fri, 28 Jun 2024 03:07:56 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=DvOnLEDk; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544059; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6tqRDJG/cEYMTiNmBkhl9lqAdqcMqlFpJcsAKZZNF9w=; b=5a0NN/q4Lc9lpXxu5K2y28/mJNKmyd5CYqEKgjTrCT8dUfu/WLl+r7/MQQEy0R/nJ4pzeG ozzPt5dop7Ceip398xVYAOUjdLkgYUBMSJVIOVjWD8feht1Il/oSEN5UZBIar+KINLydsb ZwaG3x9ygJQAAKlYwcWg599QGylptHQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544059; a=rsa-sha256; cv=none; b=vQFEA22PFpe5We3qytK50ewHZaPeppGPJkBzfUcG2SV0glK5pIdSjxhA9ZmE0dGpvGofA4 WD7nb72gbF964MFuYD7wXPDecqJarigHlSdpMPZ62yuJuThhnRxXMqsL8qW3X6Kp3N3FIf YzYv9zxEHDisxsOApO7bHelEX0c/ToE= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=DvOnLEDk; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id B05BD6204C; Fri, 28 Jun 2024 03:07:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8258FC32781; Fri, 28 Jun 2024 03:07:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544075; bh=fr0OSLBdVEu4PiREH2sgZbNiunlztN2rB5Cg12p8Bfw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DvOnLEDkbTMaJdXPPoE1HQN7ICZRZxu/wx2iuZ1D7NCvHtiBu/yFUSmpp5eoK+R+Z TvQqHcw2vcNpwhurl3xIfOeKvMoBgVhkiFsLwk0VU4++awCje1qxplJZfPZn0cbjYD 5x1ZXJrtgAeHdcFkb8/i8YpR1zJv1PKsrAXbqKRKqZPO0plgI5cGurmLzHbirBSPEc dhk0stZxfzY2xA42dD8R6xyMrrBjAVjxbF2X/2AT1IGkp3TraKKTbrLLOCjiOnZ3+K 08leqFILaTWgWWtRcj3nawG+h99Vs1nRjoc37yqyeL2bdDuvr6V5RNmcnnWddzGbGs yfGXWvqCkDi6g== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 18/20] mm/zsmalloc: introduce __zpdesc_clear_movable Date: Fri, 28 Jun 2024 11:11:33 +0800 Message-ID: <20240628031138.429622-19-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7F83E140009 X-Stat-Signature: jau7jai95wpg5ssofk8gfonxn4ei51sk X-HE-Tag: 1719544076-222401 X-HE-Meta: U2FsdGVkX183htGhgAmn/Z7YXNndeV8a5g3ZRmL2KrhNLXmQZOhxsNDhfJb6bGNIKtaYpoGTP9+qGZP2LwtJF1Pp/weFXe8MoZAubWnH9MJZwoamL+rAMTozeBxaYWD8MVLmMl1P+VnR51okMFzLV596X5O94O1YbSDh2PTdMWRoiNyc1vuZWEaMuTPg1o//OSVMvgeimNfeP0fxMO4C3N7FX8BKgmUxrHDQMGuTq70mFE4B0O0MhYHO6B8jOh7zbN6RqJMhWzW19L485AdYO/RJZvbYVkQrZ3KSyvx78ysxHhg3pRN+DHsk/ih/7ZaBcfSaiuhVlltkAheGTAkeJ4pIGcGZgNA4ZYUfm4o2uUyXw80hh8mqJp8APhFgF9vWm7N6tFoqYnXQW5FlAdVax1/M335nY7YWaXEDuzHZ0aDkwK0iRalNO2628M4IENcs/KYkhIqQuBOAZ+DPiSIlSXG3VtC7r2lONnB0hHokiu8xE/8FoKk6gyjORwBnA5Zwc5fEUy8KEQSE/jh/OLOeFaQMcXxTvV3OCfSdoZobOxiZpFdUkWZF2wDPVONteo1hIb+B+N2JSRPc4cP+y8Ahwn3C5bLikvx+lI0N/64FweTE9diqklqVlT6Op+2LzMStZFrQfFrqz/Dw9+jICyznSPC/VwYa5loOqmCyBG+D+JeGeB362g6Q/gm43cT7oGJEIP98+pFWBoWO+Cxs091ql3i49jwhSIblMPV+MsXK/CAmdBF7jIffkKiIXGIsitpDoMd9CdV/DZyjehcRzm1jhPt09tpWZ12OLPqNWcvgj1NXBgvYRNcDtUtCp3N+THo+Kqu79h1rHeHyWTFVnS+hA9vohGkA8jDx8aCIadoveIwEuBmdft11lcjDLmgqxSzgpVcmyc/LRR24Ldau5fWLoCUbIPOrvwPpT9dWmpQwJqR5CCKGwjOuRar15sWuQEsypATQhD/pL2t0TW++LOL fp4RRsFl VCnyjQeV3JWQsRgDdRHX+9OgaBvsGChixTeR09j4TcmRRxVo7OiLmcOhhaLJCpn61nFAKkp9wWcxWd10MyJbTTvYVgFh3lBJWmmPaJvlrb5sAyMxXKqfIDkux1NgL2d/jIhKQCCfpN1/m/uhJIEi6IDS1pkhnkA8onpvRbCcZsjO+kS63AXfzhpyXqGEqceUGN/xxte78K1hOhokmEBDbapGeBdgNy6XovTL4iKVxPp6iXHDMe1VgRfLErghi4Az+hbZjQvdW7boDBnU2gWUMw9OzEFf3qgaZ4oFvqAJ8uh8pTMYEAVFGQ4TN77QH/PZUtCddRS5ckD2V6C+lQH/jAEEQJus0zfp0AaGh X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Add a helper __zpdesc_clear_movable() for __ClearPageMovable(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 17941774c46e..c6a44fe04f97 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -102,6 +102,11 @@ static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, __SetPageMovable(zpdesc_page(zpdesc), mops); } +static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) +{ + __ClearPageMovable(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 3ce49f0372cc..abecc8a5c566 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -848,7 +848,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) { struct page *page = zpdesc_page(zpdesc); - __ClearPageMovable(page); + __zpdesc_clear_movable(zpdesc); ClearPagePrivate(page); zpdesc->zspage = NULL; zpdesc->next = NULL; From patchwork Fri Jun 28 03:11:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715405 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24DEFC2BD09 for ; Fri, 28 Jun 2024 03:08:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AAE336B00BA; Thu, 27 Jun 2024 23:08:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A5CA86B00BB; Thu, 27 Jun 2024 23:08:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D8CA6B00BC; Thu, 27 Jun 2024 23:08:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 6D5246B00BA for ; Thu, 27 Jun 2024 23:08:04 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 2B1E4C12A8 for ; Fri, 28 Jun 2024 03:08:04 +0000 (UTC) X-FDA: 82278813288.24.DC56074 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf02.hostedemail.com (Postfix) with ESMTP id 29BD080019 for ; Fri, 28 Jun 2024 03:08:01 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Mk0Y3czk; spf=pass (imf02.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544065; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Iv3iNXi/eIe9volJtnVPjvF8mgGNp8ouqEHlmLtpLEA=; b=vt4xmP7Qrg0JoZ232KFBLDzFtsez3Tf68AKwHv0knLs5JWt8FlLwMX3Lxn2gvSfoi913P9 h7XFcFS0B663L/9gfkrIMojBW9f2x+CdGu5BSLUR/1BSxC85ftbQyUyTTIDr2V6gm+mOWM EMyPPgtCvvP184gfXhPx4duP9mWkxUw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544065; a=rsa-sha256; cv=none; b=d2YEVgfI1RUbvKwOwq9SvsBKONzU4F6ToqIwi2+kzzelcCvOIlqc+5yPHB/bK1ZCDZc5uW Ed3Hhc7cndA8vEe6CjjlmEVHPx0vlBzvdArXz8J/1GPfqsywQG1fagwWn6+53Axe1Zkj/L zfGqRZZOgKRz9gub81IrtjdRXhmR6Cc= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Mk0Y3czk; spf=pass (imf02.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 8F0F6CE33BE; Fri, 28 Jun 2024 03:07:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 13BFEC2BBFC; Fri, 28 Jun 2024 03:07:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544078; bh=1b2JRlKwTFXQCKXls2arw2ZYXd32r3yXKlFqVVcujQo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Mk0Y3czkLU95tgGbkMh1dc7Ujh5UVx4rFxHwXuxVsgEsk/u+S4AReBO1g56nW+Nhh 3YmhlRIT03b0vDYM21OStct7U9LXxp1hcClg6Ir+2o7e3MxGkjqKbb5TODMs8B/69m s+9yoffNOB1dnVjRcLRz7Za2COjRzhajsk6g2M9mBVyu1yDZ0XyVjnPwzMhEO2SUBx oWnWAdZdNEGIBe+7XR2MbwfFfO024TyLiN67b3jo42V6s/nL20YHE/H5HsXiKGt9jr 6ax09zDtLSK7qDG6FRD4gBONZiiKEkcRz4bX6SQ4ufwhNd7vi0wDAs2fplpKZqB0GJ 479dAcAhfe1uA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 19/20] mm/zsmalloc: introduce __zpdesc_clear_zsmalloc Date: Fri, 28 Jun 2024 11:11:34 +0800 Message-ID: <20240628031138.429622-20-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 29BD080019 X-Stat-Signature: hma74iju8t45bidockyas9k6hnta6p13 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1719544081-763015 X-HE-Meta: U2FsdGVkX1/9864Gg0D53ucyuLi1VxGtOu6p/JPlgqGxfoZbA+e8Gq1IKRv4Z8/a89/wcvMU3W1GtWZhpdc0CiNPug9wfIGhnRDa8TNpgcJ4R7W/u8XAG6pRC455NilsZjKoMUzSs9ImPEWiUZJcynO5o6kss5n8BBzLwVG7ucvlT26lQazUxPo5xMlOICl7n/VD5du1fwFOCFFH952MXTet2ZntaKXKTlsJdj8t8pRt8ZdoZAuuTDeqmvA27nBETf9ENbNpV3EbjtC5yoluaP2LYKCRqJb+eEln7okI5p623uKghGHRVoN8Q0FT1prO7YnY0hybRTDEuxtpf/TnWLVcVjfV/t7zZUwIp/s6A0Xl3wgdXtFivGga4Jrm7/ElWXUo/EpvyLCeW0KqW02ZD14qcYO4ogY+Jdmf+oyHm1W4pn1gJ1enqmyaJzJZDSyXkVn0/360ZCdxg2GFg+MtPc35XTKeHNX9lyEWMeHwpFHgUwVec3VcjjaSDR0DSkBKhYzGg83gM4iLWvtHHTAwMfGNeXrUUIh7Rz2zTkYyZME+mI3b2lfr1l4CzgXfCcomEt5YoXtZmKPfhejirDtMyW6wfLIGP+toOdugKnX28XImwPdkt5r7NJ7QJ2Q52632NOw6ToQJl63PYwoSnBTZ3NNjNYVC9bHbXHR6X49GE5E2OtLqQFQNfatz74YUaK8pSsDDTFVoVdiIb0Wh2AQES5PouBAGRx6c5zXK+GfOsqHFZ4VcNLhb4RlbU72e+jQYgeoZmuxOsSah5GYnnjH/qWAjlE8jn2IM3GLVJ7+8OhRuSNb1ervdb9lEti0gvTkEuDR9oGuPB/0mWOSRLrkw+z+uaCrpncwPf3dwXFxObx+bE/MWOr4rzY0zWzvgpxU9Xh3UhH0tkoTpOdgCZB77GYWpUJpbrHdjIVgf8Hblt9q8jhx8vavh2uWyj5eapgD7QwcRA5vFs7RQ92Y9TVR FwmZ2/J1 FXoQCp21tkNeVsMYcb77dWpHf0SlhanvKrzsYAQQd+AMcPRU+xFXqQI3Bp4ShalsPB85Omwk+sSaKan7ght1ysi9NCwyHjnd4asxubYKOifaIxzUI7GlMW05w9s0Vn2jc1e2yuVBungi4gD69qvmM9eRdoU6B9/Jf5pVG1kdVKbuGazI02H3JHlpYCLX90DztmkhI6FTKokwDRxQ4AwFJFs4rKmjOZ0MxoWA5kef7oiJ8dN89mYXGTazq3rhY+nCNlTsgoCqoS1eT/zTFFEvfeh/h/41M7sybXgNkteCtDkL2jqBlpL7hudykpAEXFTchZ7k/L4t3IYXOBJG6r9mvjN9PCy6O4AqNflfy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Add a helper __zpdesc_clear_zsmalloc() for __ClearPageZsmalloc(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index c6a44fe04f97..e346b39aba59 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -107,6 +107,11 @@ static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) __ClearPageMovable(zpdesc_page(zpdesc)); } +static inline void __zpdesc_clear_zsmalloc(struct zpdesc *zpdesc) +{ + __ClearPageZsmalloc(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index abecc8a5c566..eaf8dad04f2c 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -853,7 +853,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) zpdesc->zspage = NULL; zpdesc->next = NULL; reset_first_obj_offset(zpdesc); - __ClearPageZsmalloc(page); + __zpdesc_clear_zsmalloc(zpdesc); } static int trylock_zspage(struct zspage *zspage) @@ -1026,7 +1026,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, if (!zpdesc) { while (--i >= 0) { zpdesc_dec_zone_page_state(zpdescs[i]); - __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + __zpdesc_clear_zsmalloc(zpdescs[i]); free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); From patchwork Fri Jun 28 03:11:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13715406 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F22D7C30653 for ; Fri, 28 Jun 2024 03:08:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 027E86B00BB; Thu, 27 Jun 2024 23:08:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ECAB86B00BC; Thu, 27 Jun 2024 23:08:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D1F6C6B00BD; Thu, 27 Jun 2024 23:08:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A78C16B00BB for ; Thu, 27 Jun 2024 23:08:05 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 5C395141388 for ; Fri, 28 Jun 2024 03:08:05 +0000 (UTC) X-FDA: 82278813330.06.972D847 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf07.hostedemail.com (Postfix) with ESMTP id BAB7D40010 for ; Fri, 28 Jun 2024 03:08:03 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=cbU3x2Dd; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf07.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719544069; a=rsa-sha256; cv=none; b=FvqlGlb/zB5Hlwy8V0+mltq6DTy9XPWioQxdnjGD1ssB+GihN9UzbQSuc/G3BkS06jcdaQ HZd4F2SxfaPZwdPz0yInEEgSppPMAFq53MGTV/UEtShupogthU+kinrJWTRxRseZN96Rn2 tgS+2rQhEpBAL3xyDFDkYr06SX/M6ak= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=cbU3x2Dd; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf07.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719544069; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kWcP3+EG93bRygnAaI9FVjOhd9/5VSHv8cgJ50j0qu0=; b=RGlvnxhrp47nIxtfWSklUlM58Ipw0hmf6IQmatd5rLYdzCaKi1j9hU55W10jhLXmV2PkWg pfK4GZ3NHCvwwN633nvZGmHTBJY31+2rqFWK6s+Xl6xaui8NdLC8h1bCevc5xHQjyJTPFd gzxXqfLnmZ3ltrOYlJiwT9pgVVpcCGo= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id F1EC061DDD; Fri, 28 Jun 2024 03:08:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 97FBBC32789; Fri, 28 Jun 2024 03:07:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544082; bh=4MKTXWqqoUrRELMNSTMj+qMl+m60BitHu1nDeXKuUgs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cbU3x2Ddh8UT1d2qoVfthnhMM+GJKywAez5C89nw56pBJLHMwtkvKj/JzIwEwMFRk wzmpdRUrNHYRvjA6DyL9hLgF2gtdCELLqOT0Ac2uw6PTQ1z+HCS0CPop6sQ5JrxKaL H6DNIP2g8MFFZxd1i8wYTSKZYOUXhQ+x0WRhjXVCeosCcHVhUCN33Xlf7v6uT8OUDw i/46Pz0IAIjX67naNUMaB2vjL8T6s7Fh8s8wMLWP9XALYQ8N1EidkmLu2LRwvgLF0J yE2ElJWpvcfyCD92w/NKVFID3nhC2abtUH7BW1bSfu8OtUlsZUF724zhamgTRYry3Z 7pie/+HAHWdWg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 20/20] mm/zsmalloc: introduce __zpdesc_set_zsmalloc() Date: Fri, 28 Jun 2024 11:11:35 +0800 Message-ID: <20240628031138.429622-21-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: BAB7D40010 X-Stat-Signature: 6m7hsbijnk8jahpipj4759kdrfwbsjij X-Rspam-User: X-HE-Tag: 1719544083-245715 X-HE-Meta: U2FsdGVkX1/GHbO/zI5mzxMBy1VPCIGL3xJxjFIkRl72uDCZ3M3uJW5AG1AL+6rCsp7Z/FqliNk+ScMlgtF9qyCOfJxWwU0zkV1oAYsl0RDSbSx7oEdbpLGg9bycsezmTakpcRelMeJyb0siXXNsLyksSEQ7MyrOoC3DV/CIDZXLZ+J7Qsb3KGRMGZfdBa71oBcl2cNx7Yq7eAzF/9efM3VOWX/t7c86K2vR6innSYPZNxv8/BVoJhBCajatZdI13o0foc/68XuMe4ECmcjJf1StJNHbmyTYSuPzr6KIbFLDhqTU6sxW4BAWGzuSOZWkUYRxv8S1pIWZsyUeZW1jSukpZDz34d9pvljCc2ZA3uE7yJq6gr3OeyIAtIcLFKobemvh5c3vRBjFAT1oZivoEOXWQCwx29PPySE44yIQnIpZpcWQg4XqHIbEm/WfkA44NcdimUNiVAbRvguIa7uFb6gDAwt4BVnOoCU3rU/3Dprlcz7lPVuVFo3K0d6HT05so+B40/WvmSxL+jCCvl+jB4rw6wH05BjFAPDmwX5qEPR1gmKw2zE05zlWEAfbEKYnJD1/Qm1CkvfRyPcYMjxx3zqjCTfT2CyIq5Jl2eSTbQSyIxP7ERKNx/P7Ns6nKgEv259t06K5Z5spbgo7p2DxyxhgYcsXgY3GMa986OkaayCYJjJkE2YQpkT9HlOXRpl9ICAjQRj09mJv04+0OtF/0MryzNwOXc134yp0Fb2E6xfw1Ru9rr2EI8iQki9ilfjt0leRw7KVsEU14GJPT7dYbJoBka/U/gM83SrF5O8q3Iq4x077J+O2NuSDTr4lUNVJ/9572+vVimUeSkqaIbNQnOjKnb65F39+sIgBZH9e6bSgB+HFQaEPAwwXCmFtz43M9m+ffPtJ+RFl5tOlrX1npdfIU42/7NVGwEbvkqGv1MMtGIjAD6R/HHyHt4U+YsuEdfiWKgR7tOiXJ7qYtMx Xscc7BkZ TZF4TWu0EvsqF03Ug7KpgHkixlQltpWvkoUxTAD6JNJPjvGCK43SYNCFxzzMwTSbRX176z8HUipLiHPOwzhFdUL3gGvT1pNQIUN++kh2CTKb5QdhrI5jzPoj/mF5h6FEsrw09SvXSaCBF8oy+d8p9G5tdAbDNGs6nFcoZea7uQ9DSm1NsL/PNTbThvWG2ZOwKPZ+S3zxmCG3Ua6+nHFyRQ/G3NysV0WfG0M4UUFKUusckbjErEQKyFGmR+FRfwIXIwDdFgYMpRJLsS3catiP2BXfhxuhQjedG1H0t4HdtG2bBg2oWwqa3skxx0Z/5sbcpxq9ERXpg6mq2zyF600f8fYEyXJF5QVG8uUiA X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Add a helper __zpdesc_set_zsmalloc() for __SetPageZsmalloc(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index e346b39aba59..ef36fc3de75d 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -107,6 +107,11 @@ static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) __ClearPageMovable(zpdesc_page(zpdesc)); } +static inline void __zpdesc_set_zsmalloc(struct zpdesc *zpdesc) +{ + __SetPageZsmalloc(zpdesc_page(zpdesc)); +} + static inline void __zpdesc_clear_zsmalloc(struct zpdesc *zpdesc) { __ClearPageZsmalloc(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index eaf8dad04f2c..e2a506f53957 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1032,7 +1032,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(zpdesc_page(zpdesc)); + __zpdesc_set_zsmalloc(zpdesc); zpdesc_inc_zone_page_state(zpdesc); zpdescs[i] = zpdesc; @@ -1825,7 +1825,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(zpdesc_page(newzpdesc)); + __zpdesc_set_zsmalloc(newzpdesc); /* The page is locked, so this pointer must remain valid */ zspage = get_zspage(zpdesc);