From patchwork Thu Dec 5 17:49:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895903 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C17EE7716E for ; Thu, 5 Dec 2024 17:50:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5FC226B0180; Thu, 5 Dec 2024 12:50:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 131536B0189; Thu, 5 Dec 2024 12:50:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C06CD6B0196; Thu, 5 Dec 2024 12:50:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id CC5BA6B0184 for ; Thu, 5 Dec 2024 12:50:09 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 8286D40593 for ; Thu, 5 Dec 2024 17:50:06 +0000 (UTC) X-FDA: 82861643508.30.DFF47A8 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf19.hostedemail.com (Postfix) with ESMTP id 0E4471A001B for ; Thu, 5 Dec 2024 17:49:46 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="i7qSmtS/"; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420997; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ESBy/PC7w3Ubzt0too5hvxQOmgaQszQL5ZR2Oy7o5YE=; b=RGA6Ia66nwiUA+3LXVU46DRbrfh4tDZxbmpH/1ZmvOkxidbY/By+xe2N2DyFpDYIGHsa2w HVnBKUKjZlKkdU2VmcvK7shXC4rHSiT8zMD1D5jWWTzhy+57claUSF+gkxnIvR5bm+1jpw lXml7Y5sZItCSKmr8FZXkvzYKosprvU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420997; a=rsa-sha256; cv=none; b=yxTivoLi3u/X+raSSpJkOtZ7093KJxV4tSSWONhcKh1ztgB7/1/tz7CUbEkD6LYm/8C0ud H3lA/3mNJiH/hLxLvXag5xPtr32uSgBIFZakJvP1fO3kQXX7pExYT6lnAiUuHR+OFgGeYB 0LxiK5timvV5VJhJ+4XDpjLsYeizh4I= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="i7qSmtS/"; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=ESBy/PC7w3Ubzt0too5hvxQOmgaQszQL5ZR2Oy7o5YE=; b=i7qSmtS/Gp3ZPGdzl9iC1H1xZW EcfexljHS28KMCWTq2dLwHoVk1VBCV+OKAcRR6hdW3gg2mgA3omeCWyEchOjgal933PShzvFzFr/D wCRUikXyp+pTYTW182AWIgiXT9a08FfkP6YfFjPTC/fONPxrdbrYDmcFSdaaLWIXIve8LjCFeF2Zm g3+Z9N+y7Toz3eeyOKa4UMRML8RLPk0FGhFP1Y6WvaW/yEDwVI+QBh7W1R5NPZzK9HTOCLsfR9rnH 7vs474SBegWYZAQzkfNdXSVmIYhGGAWXAQIHjPfE0vfiU5wfpr+cOk/1cXhZlL8EMjQJev4BnzBrg NmbH+h8g==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzG-0000000DN6s-1Z1C; Thu, 05 Dec 2024 17:50:02 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Alex Shi , linux-mm@kvack.org, Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [PATCH v8 01/21] mm/zsmalloc: add zpdesc memory descriptor for zswap.zpool Date: Thu, 5 Dec 2024 17:49:38 +0000 Message-ID: <20241205175000.3187069-2-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 0E4471A001B X-Stat-Signature: pr5z31z5hg3y7p711jc8y1uoheb8c4pi X-Rspam-User: X-HE-Tag: 1733420986-255665 X-HE-Meta: U2FsdGVkX19rlDn/QmUlQpmOmouO0sEBuf+NCX93amiX5mzz+nw4wvLanpqNSKzKE9V3xg8AO6cJFqrMMCK0qMlTSEx82JkFxdLZNj2+heNoYmWiZNJ3wZHZIN0FWfDcKM+PZUGRgwC6u/aWzSCMzrvAVglYbHb9TPEa0HQKvfMhZIcY9BGdnvfwcHDWtWHdPx4UVgf/0aIr+uhubeceOJG7vARPclAQlpjOeeAAPL3aNGKuxED9deVuNpLlxYNb2e2h+e6VHcTkwRrNEiXGcJYLsksPjB21zO1THhcWEJkKJ0h2gAvGBIddhIOYo+969SYIi6UILPiy0HggbOi16WZhC97BzvNKb+jnWLeKVD1bUTD3bn62ivkzRvgCCaSERCeMW1//tjeSWwjdIjHBiuX5Shnsga3aIH4+dZ5X3+PvjnCFg33VG5qXRcylLrxxvp+430DDSalVvWRs1jPcuzz0cTuSW4N/33P+P9MEGEnKqO5erKqYvTNZxZr1CNYhYxSM3Q0dcszvZ3CEWab+h2piDmuzShdR4Jl2kbSW9Nar84uPzmAFjDgAxzvB9yHKnFK/oQVx2pzS0qBXLNSQGAE3qDCIgswaJGT0wVJHbnjSzULk/R3rcJKlQEWOzWY2MYtz/4rZ0zCyJJ1PWRV3FIFS4HqgLLwqV+liHfKYePGWuTftFwofWCMp2LU31XDKTJqfQqJwDJle9NUg21Nwr7EEJ0k3kixG3zHQA3pY+zst/Qs0NUkbePACdNt7FamWt9ebLxO0mSF/wcqvWOKQZEgIQcHDtXfehIsEmyB21UZChhyo6y3gsXDiMvIUdDfkCqlYcs51Kt1kWhMWW2a0wqPN4+q4mFex6hWXGjjsOt2HM8LH9MilFNIai6bOqIzuatD2oP/A4wOPMYQ+a+EWoGSiWB0zLna8DYm3YGFta6oFp5jBO88CCxBUJmwA3sWA7F1Pk9v5QlwrED0dcu3 IUMcE1DK n2iwV+hJvaxzsfv4KJI+0u4DuXVAqfmEQ4Jh9VjK+YrQtdwv6I28S0YPmEnt2w738iVhUJXdgyB9tXNyhIW9grBpGTV+OqpjqKtM6+9M9jUkP3BR76eK4TQfh+dP8vRI1Ykq16rsqlnH3+qa5CI/V8vP5JxzXJPHaKXi1L9e/IqORPA9l37ArMxSfv9PR+d9PffVs88SoTPane/DNKei1+5CIZ23ZoymgtP+98yqs9JcgoYigZaWwLXe1Pr2S6HrOWihJu1Hy5Lfn4AVyZclHIgh6CX77X4BQDy2eHmtX7nbUBsI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi The 1st patch introduces new memory descriptor zpdesc and rename zspage.first_page to zspage.first_zpdesc, no functional change. We removed PG_owner_priv_1 since it was moved to zspage after commit a41ec880aa7b ("zsmalloc: move huge compressed obj from page to zspage"). Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++ mm/zsmalloc.c | 9 ++++--- 2 files changed, 74 insertions(+), 4 deletions(-) create mode 100644 mm/zpdesc.h diff --git a/mm/zpdesc.h b/mm/zpdesc.h new file mode 100644 index 000000000000..9ad232774469 --- /dev/null +++ b/mm/zpdesc.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* zpdesc.h: zswap.zpool memory descriptor + * + * Written by Alex Shi + * Hyeonggon Yoo <42.hyeyoo@gmail.com> + */ +#ifndef __MM_ZPDESC_H__ +#define __MM_ZPDESC_H__ + +/* + * struct zpdesc - Memory descriptor for zpool memory + * @flags: Page flags, mostly unused. + * @lru: Indirectly used by page migration + * @movable_ops: Used by page migration + * @next: Next zpdesc in a zspage in zsmalloc zpool + * @handle: For huge zspage in zsmalloc zpool + * @zspage: Points to the zspage this zpdesc is a part of + * @first_obj_offset: First object offset in zsmalloc zpool + * @_refcount: Indirectly used by page migration + * @memcg_data: Memory Control Group data. + * + * This struct overlays struct page for now. Do not modify without a good + * understanding of the issues. In particular, do not expand into + * the overlap with memcg_data. + * + * Page flags used: + * * PG_private identifies the first component page + * * PG_locked is used by page migration code + */ +struct zpdesc { + unsigned long flags; + struct list_head lru; + unsigned long movable_ops; + union { + struct zpdesc *next; + unsigned long handle; + }; + struct zspage *zspage; + unsigned int first_obj_offset; + atomic_t _refcount; +}; +#define ZPDESC_MATCH(pg, zp) \ + static_assert(offsetof(struct page, pg) == offsetof(struct zpdesc, zp)) + +ZPDESC_MATCH(flags, flags); +ZPDESC_MATCH(lru, lru); +ZPDESC_MATCH(mapping, movable_ops); +ZPDESC_MATCH(index, next); +ZPDESC_MATCH(index, handle); +ZPDESC_MATCH(private, zspage); +ZPDESC_MATCH(page_type, first_obj_offset); +ZPDESC_MATCH(_refcount, _refcount); +#undef ZPDESC_MATCH +static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); + +#define zpdesc_page(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct page *)(zp), \ + struct zpdesc *: (struct page *)(zp))) + +/* Using folio conversion to skip compound_head checking */ +#define zpdesc_folio(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct folio *)(zp), \ + struct zpdesc *: (struct folio *)(zp))) + +#define page_zpdesc(p) (_Generic((p), \ + const struct page *: (const struct zpdesc *)(p), \ + struct page *: (struct zpdesc *)(p))) + +#endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 64b66a4d3e6e..469fda76ed8a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -67,6 +67,7 @@ #include #include #include +#include "zpdesc.h" #define ZSPAGE_MAGIC 0x58 @@ -254,7 +255,7 @@ struct zspage { }; unsigned int inuse; unsigned int freeobj; - struct page *first_page; + struct zpdesc *first_zpdesc; struct list_head list; /* fullness list */ struct zs_pool *pool; rwlock_t lock; @@ -459,7 +460,7 @@ static inline void mod_zspage_inuse(struct zspage *zspage, int val) static inline struct page *get_first_page(struct zspage *zspage) { - struct page *first_page = zspage->first_page; + struct page *first_page = zpdesc_page(zspage->first_zpdesc); VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); return first_page; @@ -952,7 +953,7 @@ static void create_page_chain(struct size_class *class, struct zspage *zspage, set_page_private(page, (unsigned long)zspage); page->index = 0; if (i == 0) { - zspage->first_page = page; + zspage->first_zpdesc = page_zpdesc(page); SetPagePrivate(page); if (unlikely(class->objs_per_zspage == 1 && class->pages_per_zspage == 1)) @@ -1318,7 +1319,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, link->handle = handle | OBJ_ALLOCATED_TAG; else /* record handle to page->index */ - zspage->first_page->index = handle | OBJ_ALLOCATED_TAG; + zspage->first_zpdesc->handle = handle | OBJ_ALLOCATED_TAG; kunmap_local(vaddr); mod_zspage_inuse(zspage, 1); From patchwork Thu Dec 5 17:49:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895905 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77A93E7716E for ; Thu, 5 Dec 2024 17:50:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE7306B0193; Thu, 5 Dec 2024 12:50:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7EEF06B0194; Thu, 5 Dec 2024 12:50:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0EF4D6B0197; Thu, 5 Dec 2024 12:50:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 00A046B018B for ; Thu, 5 Dec 2024 12:50:09 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 82B8E161644 for ; Thu, 5 Dec 2024 17:50:06 +0000 (UTC) X-FDA: 82861643550.22.5266931 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf20.hostedemail.com (Postfix) with ESMTP id 6CC931C0006 for ; Thu, 5 Dec 2024 17:49:47 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=WXmQ6CMF; spf=none (imf20.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420997; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Sm9ppCk60yVPpbeM79vwVZ4QIq0HArnQvFPHx/TPa7k=; b=n+48E7v7JK8v5FH/8aY5wa2NMlFeQ3DzIIicsP9GS5cRbMFOf4srjQDcZeawb5MTG5CXKS K+6rvejy1qams7wotOzRwP687CmrEQK/4f1+OUaNUv0dmwNKvimiyz78ClDA9RY8hrnQPf 9tIaGBNDAbwOTE4QrJjE/bmg8oAGr5A= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420997; a=rsa-sha256; cv=none; b=woOuCMTwy2E6Gj4mYXvXU+JfHuZd78vJdzZ2bKb3+uF4I0rXCH+ktrDfwIxYOxiIterwY3 xFds13ZLnib9wKQCrE/sqCdPyDHxTV4xNde2tktoBmlFvpccUT6qSg4xvf/F9WlMx8yK+k uqF1/a8Qz6WfFOsibcjqBQbLN2MyMG4= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=WXmQ6CMF; spf=none (imf20.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Sm9ppCk60yVPpbeM79vwVZ4QIq0HArnQvFPHx/TPa7k=; b=WXmQ6CMFmYwHp8ep7piXYN6wJe o3w1IJMCGuEMapPAnBj25RPerEPWNqj42Le90GnQY7alQIycjl0VRW8AIOwiMf7Pqd4A14yMuqeKY 3dj/xt+rWIhmvKPEak39UjpiyPOssmt8/J1ZplIys0tI0/y69111O9i2I5W/XavU/+mnWjp+jR0fH QSGK/voxgKzZBg/SN8JYIoEn5dgUzqpikOixQa3aTVM41Kym58SfnbjRC3nma29z/dnIrs5Go47QR c8U1BQVAZ9P5Lw52uKuFmWoZHorpeZxQxHXuRWi9FP3ztPYGYts848MWDhmEblMoydcN9etO+QRRs FMmOD2MA==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzG-0000000DN6u-2AhD; Thu, 05 Dec 2024 17:50:02 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Alex Shi , linux-mm@kvack.org, Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [PATCH v8 02/21] mm/zsmalloc: use zpdesc in trylock_zspage()/lock_zspage() Date: Thu, 5 Dec 2024 17:49:39 +0000 Message-ID: <20241205175000.3187069-3-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 6CC931C0006 X-Stat-Signature: rap44f8aaczgz9tn71feb9xqjaewoqp9 X-Rspam-User: X-HE-Tag: 1733420987-21935 X-HE-Meta: U2FsdGVkX18dT/gJGBrWlOVhk9HpSB5Dx/2Y2I8I4tpUx4oLN0FDdAz5ZHCLhAntGIc9LAMgDrpJlXwUV/WeZmEHEkPDHguvtYUiLN4j7JBP7zyTzWSGX/LacgKuZhTXQE0Rovz048HFnhkW1La6/fGEdELkeNe/rEcmNglY914wkId3JLiF6u+yyBhqdlSeyQUAO4iPv5WrScdda8f7kGTCKECLDkvuUwa5maGmxdbuqrGlI7H6momMfx4/HfgYOhPaWZngOvRc4Nsj++hh8i5AXcdPfzLvtTgWc+ERZxZbJfbBiHIhbpXKq0mLcDDXGFbqM2Cu39mPtS6w1vWwnGOPBnCBllErjXkZH0A9aodHh2SOYYljy4byZ/XycVG4AMyqoE1GSdplimaquS9NGi8W7e1p3jsgxPbiRoOPvvxjJAYOrb3XZaN+z42nI28OO0sSqVIjny9a1XyG38a9N35+4hEMD6tKgVlG7hU/BXYNO/4YZT3TkFXjvWZetQg16MTVnk8fwCPd9PdTx0xtLmuMA8egClxquqYR8pvwHworCA4zUofYC+rlG7Xxz8UXReyq7b52h379/Xt5Q6vDVXNkwjXWBzAkjeQVJbgBrIAQlW1q0pzDrKQBadjXccxSsQ+wR2Z4Ns0fYsJqFTWRucx/qnKiA2wOWirEd3+LAeZoO+mXEdAkVgTjdobFDYFg05kMXZCLRc/vgfcLK0Jwi6Ctx7p1bjRJCABy6Awi3DpnI408r5M/1OaE64o62GyLU9ipJLB1/FShV86nWiCrVOTwJp69x+6az+Kuhf3VcqLlKZh5uFgpmcpHPs2VmwWRkHZCfdGl75Y07WVdwPySslkgyGQxQAxJwRaVYSsW0SEy53gOemKaBpmZ+S+g4tIDR4NyVXw61cFBXgURRE5rSAP0SXQv7Zwt9WcfrPi/mwSqMywg1rzTrVR/sTnDXHiW5s1P0IL6qz9skEl2Wxd YXymkynB pXDo9VXFPM0CE/ocXTqE3wetEnJii/nCog3nLO2npB5Uoi8YieUK+7/qXToYBJTYapqJWJX/GgjFUJDTqt2Xz3BYrtMhjbk1Uj5Gdb24u2QZZGwXCzCl9OcW0rqOZZoJ71C8+Ao6Pc0zICQhVMMK9rDhdvBoZf8a94KRLR70YttTzxOPzurmNy3RkgzAYdjSt7JMRwQpNrUwInep29HKCzEgGL1wxcl1c5rq0vNbNaEIiuImWvuk/YYYAIsTTwnuP1pNt1Iuwxc+4TOXZu6cV+yzyk9cP4qH/hLBNE7G/+sEjayzX3AnQPaGHW23gPvEIFq76xP//p6cuC8hu1owyGAZB4A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi To use zpdesc in trylock_zspage()/lock_zspage() funcs, we add couple of helpers: zpdesc_lock()/zpdesc_unlock()/zpdesc_trylock()/zpdesc_wait_locked() and zpdesc_get()/zpdesc_put() for this purpose. Here we use the folio series func in guts for 2 reasons, one zswap.zpool only get single page, and use folio could save some compound_head checking; two, folio_put could bypass devmap checking that we don't need. BTW, thanks Intel LKP found a build warning on the patch. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 30 ++++++++++++++++++++++++ mm/zsmalloc.c | 64 ++++++++++++++++++++++++++++++++++----------------- 2 files changed, 73 insertions(+), 21 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 9ad232774469..4c7feee5ef1a 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -66,4 +66,34 @@ static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); const struct page *: (const struct zpdesc *)(p), \ struct page *: (struct zpdesc *)(p))) +static inline void zpdesc_lock(struct zpdesc *zpdesc) +{ + folio_lock(zpdesc_folio(zpdesc)); +} + +static inline bool zpdesc_trylock(struct zpdesc *zpdesc) +{ + return folio_trylock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_unlock(struct zpdesc *zpdesc) +{ + folio_unlock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_wait_locked(struct zpdesc *zpdesc) +{ + folio_wait_locked(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_get(struct zpdesc *zpdesc) +{ + folio_get(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_put(struct zpdesc *zpdesc) +{ + folio_put(zpdesc_folio(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 469fda76ed8a..1d1dd4578ae3 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -446,13 +446,17 @@ static __maybe_unused int is_first_page(struct page *page) return PagePrivate(page); } +static inline bool is_first_zpdesc(struct zpdesc *zpdesc) +{ + return PagePrivate(zpdesc_page(zpdesc)); +} + /* Protected by class->lock */ static inline int get_zspage_inuse(struct zspage *zspage) { return zspage->inuse; } - static inline void mod_zspage_inuse(struct zspage *zspage, int val) { zspage->inuse += val; @@ -466,6 +470,14 @@ static inline struct page *get_first_page(struct zspage *zspage) return first_page; } +static struct zpdesc *get_first_zpdesc(struct zspage *zspage) +{ + struct zpdesc *first_zpdesc = zspage->first_zpdesc; + + VM_BUG_ON_PAGE(!is_first_zpdesc(first_zpdesc), zpdesc_page(first_zpdesc)); + return first_zpdesc; +} + #define FIRST_OBJ_PAGE_TYPE_MASK 0xffffff static inline unsigned int get_first_obj_offset(struct page *page) @@ -752,6 +764,16 @@ static struct page *get_next_page(struct page *page) return (struct page *)page->index; } +static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) +{ + struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + + if (unlikely(ZsHugePage(zspage))) + return NULL; + + return zpdesc->next; +} + /** * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value @@ -821,11 +843,11 @@ static void reset_page(struct page *page) static int trylock_zspage(struct zspage *zspage) { - struct page *cursor, *fail; + struct zpdesc *cursor, *fail; - for (cursor = get_first_page(zspage); cursor != NULL; cursor = - get_next_page(cursor)) { - if (!trylock_page(cursor)) { + for (cursor = get_first_zpdesc(zspage); cursor != NULL; cursor = + get_next_zpdesc(cursor)) { + if (!zpdesc_trylock(cursor)) { fail = cursor; goto unlock; } @@ -833,9 +855,9 @@ static int trylock_zspage(struct zspage *zspage) return 1; unlock: - for (cursor = get_first_page(zspage); cursor != fail; cursor = - get_next_page(cursor)) - unlock_page(cursor); + for (cursor = get_first_zpdesc(zspage); cursor != fail; cursor = + get_next_zpdesc(cursor)) + zpdesc_unlock(cursor); return 0; } @@ -1654,7 +1676,7 @@ static int putback_zspage(struct size_class *class, struct zspage *zspage) */ static void lock_zspage(struct zspage *zspage) { - struct page *curr_page, *page; + struct zpdesc *curr_zpdesc, *zpdesc; /* * Pages we haven't locked yet can be migrated off the list while we're @@ -1666,24 +1688,24 @@ static void lock_zspage(struct zspage *zspage) */ while (1) { migrate_read_lock(zspage); - page = get_first_page(zspage); - if (trylock_page(page)) + zpdesc = get_first_zpdesc(zspage); + if (zpdesc_trylock(zpdesc)) break; - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); } - curr_page = page; - while ((page = get_next_page(curr_page))) { - if (trylock_page(page)) { - curr_page = page; + curr_zpdesc = zpdesc; + while ((zpdesc = get_next_zpdesc(curr_zpdesc))) { + if (zpdesc_trylock(zpdesc)) { + curr_zpdesc = zpdesc; } else { - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); migrate_read_lock(zspage); } } From patchwork Thu Dec 5 17:49:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895897 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BD57E7716E for ; Thu, 5 Dec 2024 17:50:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B5D656B0175; Thu, 5 Dec 2024 12:50:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 57EF66B0117; Thu, 5 Dec 2024 12:50:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC8BD6B0180; Thu, 5 Dec 2024 12:50:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0771A6B017D for ; Thu, 5 Dec 2024 12:50:08 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A48FE14187B for ; Thu, 5 Dec 2024 17:50:06 +0000 (UTC) X-FDA: 82861643508.23.FB55128 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf22.hostedemail.com (Postfix) with ESMTP id 206F0C0019 for ; Thu, 5 Dec 2024 17:49:46 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=qvbgNOYr; spf=none (imf22.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420994; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/rElCw5FqAKtEBYfTYEarV2wo4tbB8FYMNSzg+Qr0Lw=; b=lJZhZeO5jC2PDJDnjWvfmr0HWKz3l1ZCAMS7GGvFgvHPgzKYUQ6gegONi4D5CTb3XwRunk 9KjLApsGD+CA/JFv+id5ujv3eMu4Kz5zn+StYGCT1CEJGxgclkUpGhjm6fAxm8V2UeyOvI ZtTNAdinLlnUMfIsodB7VMhqJcq/OPM= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=qvbgNOYr; spf=none (imf22.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420994; a=rsa-sha256; cv=none; b=oEicfLhes//8nmYqwmVSolOFt5y0FF+PWfLZLgD1MhSB6ylEUxl1+5tGxjik+J/DFYuOK6 joyI7SP0w0zZ1H/N+NsZNpoE7Ryg+A/WxV4owZlPNMAzbUP1cLmoBry/ppJ68vMWg2YFs6 cN8WaJXngpdKeBoM4rOWRpVoYsPNADQ= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=/rElCw5FqAKtEBYfTYEarV2wo4tbB8FYMNSzg+Qr0Lw=; b=qvbgNOYrWEpPlJppZS0ylA9jU1 C+2Z2dX4C+aqkzqZEaFDo4rQbYpugtweWLFyJnRNhsoaukJQztMnwda7gKgb1uoljIYHK/j0I5GuL qpaovY/+zqeu1bHiejgD2qXAns+rAtXGioE579FTCPTP1Vm6qRmuJ/JhR4RBJM00x5bYxr5QbUX8D MXur2xPmWcvYnBw3NmMlS/m6oXr1nVlZ9wmrzFqpc/sy8RqmLZP0rTzQX63JPzI6JZ7k99B6no+8D lnz9GafgOWdWW2Mk0JEsi83cXPc10/5/uKG6xiJHqm0FY6qKEUpOpR0lxhEad0isMbGqU30BPlpNL YxEcdKkw==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzG-0000000DN6w-2bak; Thu, 05 Dec 2024 17:50:02 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Alex Shi Subject: [PATCH v8 03/21] mm/zsmalloc: convert __zs_map_object/__zs_unmap_object to use zpdesc Date: Thu, 5 Dec 2024 17:49:40 +0000 Message-ID: <20241205175000.3187069-4-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Stat-Signature: fiojpcyi5yr7pdsi44orjhzwns975sfi X-Rspamd-Queue-Id: 206F0C0019 X-Rspam-User: X-HE-Tag: 1733420986-621435 X-HE-Meta: U2FsdGVkX1/dLPdurBaiQ5fX9yMjls2ea3JYPSRV64jb1BvFx54fVFSV1x4tnacOJRUgU8zbZEq/EeakzNG6OHDMfK2v+wV4ZYCw4ibagM4mJ4C8BFoDB09aIEYGO3hr9UGi+Karb6t69ArwWe4YgYKMGofmJcgTcPu+WCaZcn0Lk7jAGI80I1eqo6uGZyZr9cGnxpS6tOZH7YO5WWbIwCjJf/dYSwAt0HZrcJvHxd6sdXleDICjxCms55k77YR0q7sQxPtWB27Dxca9M/MHCR9KvQvVR6zF/VDjDRPs+xpymu5bGyAna/O8AtNGGMsKKbijZgp6b7hystM7u+ZvwFzsfqoxLh7sYjZX/9wWeqDNGvQ8Wz811Az11zn5jrDeeMiJn6s7fKgkvYfB9/UHs4d+v+1XGTIW00H6Y3IsvkgduERUqnRhVcGYx7E39cuQjsggsyGeADWtUXo1KKa3J8m3C5lKwaJWxOx5Q87Ef0yROW45r0LSLJ/rZCuiYMFH1HvxB/eUGTHM6AoxUseBWnrBTmAPmXl0OF7m7Mi74HqB9g4+xK3aUBBiJTpXyiSKhEFkajYQIDK1vxYgDkdzD3Bhs96fdETzUNxGD2SOHguCZkml239LMW++p2G/yRIeXlZwGuGiWEr5kNUXn4tKgXEfWRxXjcq1AU6U0BPy/smYCkmahgoxSMaJ752tS/tQGWSc8i4KQsyhQRLbPoL36RnlqopMa3evXJzEM9ttGg83dzvMby+QRdowudxfd6AV2VhtWCe9ZG26LkesC3zTa7z6+R5A/zg4UnlelJBwv8UR5OpGgACDrvTQsZtszaHBjLt3Ow9rMpimcenZXiQqpouomzvUK6eTLZHpm3dgUAD6sl5AEdhloSsaY1vcyLrNyoTCTuOkoIxCeQOdefgMjRluhjEeiNp/WzyIR+6PpjyW6/YuRRSEyVnoJVHNbDmkryel1MUkrotNUlQjsUB nt/5Fl11 2Gv/faM62tTtxQp7orX6Z1jGXfx3tze94yZt3WOWsCcdlxw9guVv/adabp2RadbaA5tQKsLCq0tte24V3YHhS911ncADxq3W/ykmnwYZav+t36tzbhCropkdtUzDhaDzk9okYxwL6sD5pfB03gHODsgSilLvd6j7JTk2iSnuCRudgT7XksM/fMPC2Nytn9cLuoSTdqUpLdgdVsrwg/bV72jS6YivzucotPiSYtq1us7SrGff9nxlVBNgioi9XLappI3e6CMFPZJh8r9wgDtD0GwaEwqCFRBQ/lf7OGwy3livjkWCHl5a/pi2ysUyIHlWC+Y3+K/Eeehafcb57LEHDkM8zsQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> These two functions take a pointer to an array of struct page. Make __zs_{map,unmap}_object() take pointer to an array of zpdesc instead of page. Add silly type casting when calling them. Casting will be removed later. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 1d1dd4578ae3..5e6467c10757 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1067,7 +1067,7 @@ static inline void __zs_cpu_down(struct mapping_area *area) } static void *__zs_map_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { size_t sizes[2]; char *buf = area->vm_buf; @@ -1083,14 +1083,14 @@ static void *__zs_map_object(struct mapping_area *area, sizes[1] = size - sizes[0]; /* copy object to per-cpu buffer */ - memcpy_from_page(buf, pages[0], off, sizes[0]); - memcpy_from_page(buf + sizes[0], pages[1], 0, sizes[1]); + memcpy_from_page(buf, zpdesc_page(zpdescs[0]), off, sizes[0]); + memcpy_from_page(buf + sizes[0], zpdesc_page(zpdescs[1]), 0, sizes[1]); out: return area->vm_buf; } static void __zs_unmap_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { size_t sizes[2]; char *buf; @@ -1108,8 +1108,8 @@ static void __zs_unmap_object(struct mapping_area *area, sizes[1] = size - sizes[0]; /* copy per-cpu buffer to object */ - memcpy_to_page(pages[0], off, buf, sizes[0]); - memcpy_to_page(pages[1], 0, buf + sizes[0], sizes[1]); + memcpy_to_page(zpdesc_page(zpdescs[0]), off, buf, sizes[0]); + memcpy_to_page(zpdesc_page(zpdescs[1]), 0, buf + sizes[0], sizes[1]); out: /* enable page faults to match kunmap_local() return conditions */ @@ -1248,7 +1248,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, pages[1] = get_next_page(page); BUG_ON(!pages[1]); - ret = __zs_map_object(area, pages, off, class->size); + ret = __zs_map_object(area, (struct zpdesc **)pages, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret += ZS_HANDLE_SIZE; @@ -1283,7 +1283,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) pages[1] = get_next_page(page); BUG_ON(!pages[1]); - __zs_unmap_object(area, pages, off, class->size); + __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); } local_unlock(&zs_map_area.lock); From patchwork Thu Dec 5 17:49:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895896 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCBECE77179 for ; Thu, 5 Dec 2024 17:50:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 687B66B016B; Thu, 5 Dec 2024 12:50:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 047136B00BB; Thu, 5 Dec 2024 12:50:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B244C6B0185; Thu, 5 Dec 2024 12:50:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id ED37E6B017C for ; Thu, 5 Dec 2024 12:50:07 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 87D161C8698 for ; Thu, 5 Dec 2024 17:50:06 +0000 (UTC) X-FDA: 82861643256.19.D90FF47 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf23.hostedemail.com (Postfix) with ESMTP id F1B45140009 for ; Thu, 5 Dec 2024 17:49:53 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ck+h9ETu; spf=none (imf23.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420997; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6IwBfvloigEVpr0LprokmasSm5zXiqgw0uLRSU20BAA=; b=5XEYZgZzz5AqV7+UObjJIAyYlV6EIYSuBRTxQOWsl6RLNBas2trHvXznqJuf9dSbOfHwMO KNZ9/dmqhGPNsJ3Qua3eoyROuXbdkUJO1h572HkCzEfc/0F3Wm1CJnMXFQinTU9BZevF1L 4q+mD8zZaHkZFReImuK2F/eagJLsbz0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420997; a=rsa-sha256; cv=none; b=XHbA2ist2DWsUJuzXUnN/qBCqtFah9V2ACJ6HWgVmSs1TgYVuToH4RFaaf7/xlKjBzjqjD JpKQDT6yxAvX02YFlvz3H+LaG78DhEgHH+jfGY0dLvL0cfu93Zd/jDvbzeeBE1ZJPHIqPA HJKpapCR4hUSNOw5fG4GYwvNJjX3wQY= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ck+h9ETu; spf=none (imf23.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=6IwBfvloigEVpr0LprokmasSm5zXiqgw0uLRSU20BAA=; b=ck+h9ETuPIRtaQqvp9uF81QHYr pbzfbYVdsdPylfS5KFSt03D/q5ApCAe6qykLqQH3tbm9DH8xe8jNGjNaY8IyL8GIwG0zkXhpMAphP 9BrBCneLK5efP8+fLEMsiNKlmmMF0gRFGq3YX4ALfJaiIsbGP5twx3DW/DXikEmvAG9jUGwWl5xbG /3MCjheT8iyO70zDrxX33yfv8KOJUjfqAViDWR3k9uiQivk65TpltXC9E6O0WpTAT7LtkHjmdAseH GVOA6nsIr3VWVytP0yijmjSKCasUaf5Ps5cpPbI0kk4YDmzEfExp4mkZgWHpWKtW/XrYd3B82ucyP wCosQAaQ==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzG-0000000DN7P-30UE; Thu, 05 Dec 2024 17:50:02 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Alex Shi Subject: [PATCH v8 04/21] mm/zsmalloc: add and use pfn/zpdesc seeking funcs Date: Thu, 5 Dec 2024 17:49:41 +0000 Message-ID: <20241205175000.3187069-5-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: F1B45140009 X-Stat-Signature: hye8xcf9dcpri3yowptxackzp9phbtky X-Rspam-User: X-HE-Tag: 1733420993-522173 X-HE-Meta: U2FsdGVkX1+6id2h9gq7/hGDO/i3zpnxVhVVEFfoRfmPmUuUnx/899SFvEcwPeD0UmIbs6CGclhD6DapvePB1hec/ogY69dJqaZm1kZ9+4uy1rVVcycsp9c13k6Gf7rZhTQm84AgjAKtDJbkTDf8dfOlIwjuOac399Ss+yUXljLBG3VUs4aoQvJwLkYJG2BZNdZejaIbvmCbqizZc42k6KbCH5ShM68l0sInxTtsgx+b9euiF9XBqmE6/cAZk30VqIb03FFZtk5SWkR2xQx9OO/zWYOCf+NnQfjEFTtk9+u5aSA9OOMUYbYqyKBT6t9UBd9m6lc9yQ8JPn+wye7mwpXFhPAvv/N/ZB6mM9Fp6oVGtZR059ziFDa9b2r+FH/kQvRC1aGuhp7wg9bDkAEPb9GluKnkS2o0ZH/sv7hbC/CWzFMrbrvJzbdRdFb7fZnC+7ETdmRwjIccYHYXmk1AdbvAKYYYh2DB+WB7G1w7Yc8Wr+BlBpK78PBlDSV0P4HSgUQfBs003N8s0lgdldukytEUHAbrN7RvjFIybTN40UhM3Mo00rxuGRxzfEygEqcVpuesL1VwZbMRHQp8NzWxmWlqytudorOz7T0iFV+v7vMq6U481aJn7QAcvB6MXclTe37A7/ldol7KRK9Ns9RYJeAR+WwpV+aC5Nh7cNLbgk9PrJLtzLuNhdOCaILd+bUKyAMnn9eMcyfBoIHryu596d4PhB4d4DUYgYjIAcSYCfVhbZwjkERCblLdhjwytEAN1+lJEVmWQ2Ja3E0e8y6WN2f2l5yVRjNwM9hCTapBZZbtjSdH5H87y5FWt3WRDxL8EipkC4JLf2VR1wl6WXAR6oESR9t1aYqHWzS7pS6yzmN4N4ECG9ZaeUSsukQG3WjVU74tA7zLG2I9RzuI1LffvOzXmya3qO92ciPJfCtjSqYSx506TDSs7igzGeZ9lkfOTIwo9wVl9WvqlO4BveI W8ZzQ4gX 3an6+G/KDnIOxXryECBs2ZEFWBdA+FJ7OOshaN+bDMynEWOakpzHObqSsDCqAhjYQbG2D0XTk9QgCgwKfSDVgYdZxW939gUQYD5M85Uzwq1k3AzBtunNbHP1nt2vsc7xDKGOX/lmAIPH0mRQx8qU9RmT9Qf/DyHH+vzhsNyXeDja2l6HJG+XUMyQKTeSd1+JEs+LhTFFYjBAMrK7O0ugtQXVFGqX0Cxpm5UFGP935WJsztgGfYhDMsLMmpksO9fhl7r2Hkhw237cGtNusDmyoCf1/jCA3HmlbAlUC9jBY0etqybg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Add pfn_zpdesc(), pfn_zpdesc() and kmap_local_zpdesc(). Convert obj_to_location() to take zpdesc and also convert its users to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 14 ++++++++++ mm/zsmalloc.c | 75 ++++++++++++++++++++++++++------------------------- 2 files changed, 52 insertions(+), 37 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 4c7feee5ef1a..937de815a4ac 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -96,4 +96,18 @@ static inline void zpdesc_put(struct zpdesc *zpdesc) folio_put(zpdesc_folio(zpdesc)); } +static inline void *kmap_local_zpdesc(struct zpdesc *zpdesc) +{ + return kmap_local_page(zpdesc_page(zpdesc)); +} + +static inline unsigned long zpdesc_pfn(struct zpdesc *zpdesc) +{ + return page_to_pfn(zpdesc_page(zpdesc)); +} + +static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) +{ + return page_zpdesc(pfn_to_page(pfn)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5e6467c10757..03b77690c616 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -775,15 +775,15 @@ static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) } /** - * obj_to_location - get (, ) from encoded object value + * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value - * @page: page object resides in zspage + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static void obj_to_location(unsigned long obj, struct page **page, +static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, unsigned int *obj_idx) { - *page = pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc = pfn_zpdesc(obj >> OBJ_INDEX_BITS); *obj_idx = (obj & OBJ_INDEX_MASK); } @@ -1199,13 +1199,13 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, enum zs_mapmode mm) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; struct size_class *class; struct mapping_area *area; - struct page *pages[2]; + struct zpdesc *zpdescs[2]; void *ret; /* @@ -1218,8 +1218,8 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, /* It guarantees it can get zspage from handle safely */ read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage = get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc_page(zpdesc)); /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1238,17 +1238,17 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, area->vm_mm = mm; if (off + class->size <= PAGE_SIZE) { /* this object is contained entirely within a page */ - area->vm_addr = kmap_local_page(page); + area->vm_addr = kmap_local_zpdesc(zpdesc); ret = area->vm_addr + off; goto out; } /* this object spans two pages */ - pages[0] = page; - pages[1] = get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] = zpdesc; + zpdescs[1] = get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); - ret = __zs_map_object(area, (struct zpdesc **)pages, off, class->size); + ret = __zs_map_object(area, zpdescs, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret += ZS_HANDLE_SIZE; @@ -1260,7 +1260,7 @@ EXPORT_SYMBOL_GPL(zs_map_object); void zs_unmap_object(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; @@ -1268,8 +1268,8 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) struct mapping_area *area; obj = handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage = get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage = get_zspage(zpdesc_page(zpdesc)); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1277,13 +1277,13 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) if (off + class->size <= PAGE_SIZE) kunmap_local(area->vm_addr); else { - struct page *pages[2]; + struct zpdesc *zpdescs[2]; - pages[0] = page; - pages[1] = get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] = zpdesc; + zpdescs[1] = get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); - __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); + __zs_unmap_object(area, zpdescs, off, class->size); } local_unlock(&zs_map_area.lock); @@ -1425,23 +1425,24 @@ static void obj_free(int class_size, unsigned long obj) { struct link_free *link; struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long f_offset; unsigned int f_objidx; void *vaddr; - obj_to_location(obj, &f_page, &f_objidx); + + obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); - zspage = get_zspage(f_page); + zspage = get_zspage(zpdesc_page(f_zpdesc)); - vaddr = kmap_local_page(f_page); + vaddr = kmap_local_zpdesc(f_zpdesc); link = (struct link_free *)(vaddr + f_offset); /* Insert this object in containing zspage's freelist */ if (likely(!ZsHugePage(zspage))) link->next = get_freeobj(zspage) << OBJ_TAG_BITS; else - f_page->index = 0; + f_zpdesc->next = NULL; set_freeobj(zspage, f_objidx); kunmap_local(vaddr); @@ -1486,7 +1487,7 @@ EXPORT_SYMBOL_GPL(zs_free); static void zs_object_copy(struct size_class *class, unsigned long dst, unsigned long src) { - struct page *s_page, *d_page; + struct zpdesc *s_zpdesc, *d_zpdesc; unsigned int s_objidx, d_objidx; unsigned long s_off, d_off; void *s_addr, *d_addr; @@ -1495,8 +1496,8 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, s_size = d_size = class->size; - obj_to_location(src, &s_page, &s_objidx); - obj_to_location(dst, &d_page, &d_objidx); + obj_to_location(src, &s_zpdesc, &s_objidx); + obj_to_location(dst, &d_zpdesc, &d_objidx); s_off = offset_in_page(class->size * s_objidx); d_off = offset_in_page(class->size * d_objidx); @@ -1507,8 +1508,8 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, if (d_off + class->size > PAGE_SIZE) d_size = PAGE_SIZE - d_off; - s_addr = kmap_local_page(s_page); - d_addr = kmap_local_page(d_page); + s_addr = kmap_local_zpdesc(s_zpdesc); + d_addr = kmap_local_zpdesc(d_zpdesc); while (1) { size = min(s_size, d_size); @@ -1533,17 +1534,17 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, if (s_off >= PAGE_SIZE) { kunmap_local(d_addr); kunmap_local(s_addr); - s_page = get_next_page(s_page); - s_addr = kmap_local_page(s_page); - d_addr = kmap_local_page(d_page); + s_zpdesc = get_next_zpdesc(s_zpdesc); + s_addr = kmap_local_zpdesc(s_zpdesc); + d_addr = kmap_local_zpdesc(d_zpdesc); s_size = class->size - written; s_off = 0; } if (d_off >= PAGE_SIZE) { kunmap_local(d_addr); - d_page = get_next_page(d_page); - d_addr = kmap_local_page(d_page); + d_zpdesc = get_next_zpdesc(d_zpdesc); + d_addr = kmap_local_zpdesc(d_zpdesc); d_size = class->size - written; d_off = 0; } @@ -1782,7 +1783,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct zs_pool *pool; struct size_class *class; struct zspage *zspage; - struct page *dummy; + struct zpdesc *dummy; void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; From patchwork Thu Dec 5 17:49:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895904 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BFEAE77170 for ; Thu, 5 Dec 2024 17:50:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A4896B0189; Thu, 5 Dec 2024 12:50:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 54E986B0193; Thu, 5 Dec 2024 12:50:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D8CBF6B0194; Thu, 5 Dec 2024 12:50:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id F3DE46B0189 for ; Thu, 5 Dec 2024 12:50:09 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 6FEF2C1667 for ; Thu, 5 Dec 2024 17:50:06 +0000 (UTC) X-FDA: 82861643256.08.75C27BC Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf05.hostedemail.com (Postfix) with ESMTP id 07667100003 for ; Thu, 5 Dec 2024 17:49:33 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=joDubJBJ; spf=none (imf05.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420994; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=teUYba9V0lgZDe9TAPyaRDbtKArjvl8uhoFbMpANONw=; b=VLlho7j7Kgi+fLf91dcr7AibtumHae+oDC3n8NgOP55MjjW5yALiwCYUFiVOujgQ0wySwf KPMndjPZLgxv4/YDBfiqSybHm4+PsaILJSCkkuebb1poEOBNBGsyi5siVLgByuv9XXi8fL /IThg/LMu8cf/4PsG8Qcp7mLwCE5DaU= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=joDubJBJ; spf=none (imf05.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420994; a=rsa-sha256; cv=none; b=EAsissv7tzN6uUEpC2uTMq7JDdVpexOgwV3z3chKSYsTduuJkGD7G2+O4WZxISFApmWgv1 oJ5NTKfZPuuraiqlbLZMt5ygFM0mooAdRfvix/s/oIFokAfc9wmNdlCYhN7GPjtCv5fw7N AL7E+K7omzpkZHkfLkcTK2EHIino9aU= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=teUYba9V0lgZDe9TAPyaRDbtKArjvl8uhoFbMpANONw=; b=joDubJBJeIY6hlBT7FcmGBeOk2 dhU+19HVTqKNhTN483sUjxSWNnpO6ZWrUk0eBmLrHCORKQudoMiIrU0fNIIESAZW95DVsuoiL9p+o +p9hSAbHw/p4KQ9KfMTw8THku6YfOUKAVhOwUUFLQT7vhC2f5SgYoCLd2/dfM6mCtjzOnN0FuUIX2 hVAfimph2m05vzIFJmPwauYScOHQkyNwd/PDgxlVneSj3vLxa0jrAVC0gpLKiEqOtFNA2Op08eSPm FHOxrDDrVFgCnEoWuNCzVJzqEVG8F6JwW6sX7xP1LNLhamlOl5Rocomg5eoPa1Pb5E9Ft61b3uoSy sGHrI2VA==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzG-0000000DN7e-3ZUR; Thu, 05 Dec 2024 17:50:02 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Alex Shi Subject: [PATCH v8 05/21] mm/zsmalloc: convert obj_malloc() to use zpdesc Date: Thu, 5 Dec 2024 17:49:42 +0000 Message-ID: <20241205175000.3187069-6-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Stat-Signature: 4fwxd6bm44xw3poyy77kj4sbnj1kcny9 X-Rspamd-Queue-Id: 07667100003 X-Rspam-User: X-HE-Tag: 1733420973-855889 X-HE-Meta: U2FsdGVkX1/SUE0eE70IgF1NJhA8yT02/5BlP5pcY3UexXjTmkLeBCBdlsLnLmjquXm7Vml78GtrF/b4ag29XZHxudBlQTzFl8Y4oW7acrye/rbPhNMhx0uhmOg7BCAnQS8U8H24eV8DFQ8vo1zekh4kn7ElySuyzrNJvvOMID8Xm90CD+jQAUq4T2a4YbW/xhhm4VxqTiCr+Euh2DBkk3i62hyJEEFuBuouPLT7SehnDdPszCT8Eg8xTotxr04HTpXB5uk2Unqd6CKM6T/osSEi9YApKjQ1qjRrP/jBtwr1ljGUoKzBZdZsk9N3eNQFIrKFeZNC76GtjMSnOF5FuTm6bCce5RyR+UB7GpZoo3bhJOZatGS7MYxbHlGwKHyMs/+mXD0eaN+lVd+xBYQpJrxgqu5eQko99yFKDtjekuww4CqkdvOoA0asg+LkLeDbCvblfrwP7rWP9cCQ9Zd3/IQitI/rh+S5UvlhSpL8nACyOMqch2rM8Rcritb0RZreh1FrrAW+L0615UpX+Xs+/MoNrLZV/TO0XRKIl3mCsnZ+WoHma4vkJ6vfGR5ge9rZU9skgA2s8IiMRpTzesSORiWaYsj0Hqt/Kprj0nGfM3abM7B7cHIP7pxkhglH/i2/rM8Sg/Xeerw6vjh+HQu9d1H0gzAYswomgesdYFhUm9mO0Pz+2nuPPFpd+UtJxBJxoQy0xdjoY7j07f/QiPWIepvBYAhVr/3AKMl9hk+fYPxPEG+0grLCFXHfHvUhY1c/E9kUIAnSFMW6LOMUPl7bs2d0Xg0r+oN3z7Et5nourzPuLLb2KhkGqf4mbp3FMB7bUX9aMygBI66ISzYuw6R1fS+Wrd2JSPvrb1kZ0biq98kbVtez4v12HPKpLTYTfa8nQsGyGPTpUV4WX7nNYzfo5RnaIim4kP8+XR1udmwHaiqMH47BFS1yFrDniEE1g/OGbUYR8akSRX3P6lhwLS6 JLgzRVUr ygAtJQ2c63AlSglvFIIMNmwmgqnxoWewlTTvu6Cphf0GcaJR8GmXxo/Sf2FO0SwD4FhZQJ95Ub9uy8jpx4OJVhrXof8ixJkVqSGQcNV5dpI0ra8wqSx7oJxLzMtAkZx8fPz+S+cQq0vsjjB1ihwUZsSqRpGiDUz98QFIaWsE8zea/KXYOFBaLg2Q/ZdPxmL+ULQiyBJd0tNhQU5ZdvuMlCwCAAzbTsjo4QcD7AndmeD57QkgEOyKuvIWLbqi5zNPpl4OorgVMvYaSefcSVGj1EjxH0Lvz8ZkXUVcRQhBSgQHzBybRdJ9u5nOxmY1jgkdVBfpfu40+7SsEGMt1mfWftTpGhw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first_page/get_next_page. no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 03b77690c616..af8a6cd6b814 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1313,12 +1313,12 @@ EXPORT_SYMBOL_GPL(zs_huge_class_size); static unsigned long obj_malloc(struct zs_pool *pool, struct zspage *zspage, unsigned long handle) { - int i, nr_page, offset; + int i, nr_zpdesc, offset; unsigned long obj; struct link_free *link; struct size_class *class; - struct page *m_page; + struct zpdesc *m_zpdesc; unsigned long m_offset; void *vaddr; @@ -1326,14 +1326,14 @@ static unsigned long obj_malloc(struct zs_pool *pool, obj = get_freeobj(zspage); offset = obj * class->size; - nr_page = offset >> PAGE_SHIFT; + nr_zpdesc = offset >> PAGE_SHIFT; m_offset = offset_in_page(offset); - m_page = get_first_page(zspage); + m_zpdesc = get_first_zpdesc(zspage); - for (i = 0; i < nr_page; i++) - m_page = get_next_page(m_page); + for (i = 0; i < nr_zpdesc; i++) + m_zpdesc = get_next_zpdesc(m_zpdesc); - vaddr = kmap_local_page(m_page); + vaddr = kmap_local_zpdesc(m_zpdesc); link = (struct link_free *)vaddr + m_offset / sizeof(*link); set_freeobj(zspage, link->next >> OBJ_TAG_BITS); if (likely(!ZsHugePage(zspage))) @@ -1346,7 +1346,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_local(vaddr); mod_zspage_inuse(zspage, 1); - obj = location_to_obj(m_page, obj); + obj = location_to_obj(zpdesc_page(m_zpdesc), obj); record_obj(handle, obj); return obj; From patchwork Thu Dec 5 17:49:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895906 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25F6BE77170 for ; Thu, 5 Dec 2024 17:50:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D59C6B019A; Thu, 5 Dec 2024 12:50:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8840D6B019D; Thu, 5 Dec 2024 12:50:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 660E76B019E; Thu, 5 Dec 2024 12:50:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 45CC86B019A for ; Thu, 5 Dec 2024 12:50:24 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 76C70121788 for ; Thu, 5 Dec 2024 17:50:06 +0000 (UTC) X-FDA: 82861643298.27.904C69A Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf10.hostedemail.com (Postfix) with ESMTP id 6022FC0005 for ; Thu, 5 Dec 2024 17:49:57 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=OzDD7MDy; dmarc=none; spf=none (imf10.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420990; a=rsa-sha256; cv=none; b=OUtLkzl95k2TOPb/6P/taRkqwJYs+Rdmy/isR8QwYPwG8LvJIzhnjQtwkzMKtsNN7JTXyb L/CaykmuczoZYoWPG/BW2x1bhR46573qXbxDBpYgpETF+9mt01nebpVKKXcMlzBiIJKvn5 XLO6BnIryCHumd3ZGhqm841qyJ5sZdU= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=OzDD7MDy; dmarc=none; spf=none (imf10.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420990; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0Tc+hr3l1ay6sXsQgfONXbNVWWG0dupWjQT5pbW/O6E=; b=ObwrWuRz8Cki/gWityzLbg1oUBk+IgOiPL13uZxa/iupTxcC+4YO+/HItgMopbKGAM/2GT oFqml+2pkKDuqAo86LHNGFHA60OdDfBs8akm2sJhqS/3lhk+NcYy0vECFlKy41hC6/fnn/ kQKxKTOm/lzxlIJyDNMI8Vb+u+bM95g= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=0Tc+hr3l1ay6sXsQgfONXbNVWWG0dupWjQT5pbW/O6E=; b=OzDD7MDy3Y3dZb6gbb1H5YwWPF wwqRi/Jp70sdF2Y0qH8CyAdLumV3L1k5sAb5jJqxMIRoa6DtxPntdy7P/PrcbTwI9OwVm++rmskaK +Pq4ehteCDDkZEOAUki3ClocYO3iAjCiuUpxu+rlls+nW59a9QV4iMLSGhQclvR7N5AEJU6tpQxrE +qjD8tUB3O1M613y8xsJc5XXWzwInTmZLiTsKI3gVDPYTHLGwCnB5IAY5/an9sBDZemrd0g/YGUft BQpQkALzCakS16M5WRMwGHMVVIgm4l/9LuXngQWh8rys383R7YwaDRdfMUtxXj0giYmoKysaG84Ag NIhDbTSA==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzG-0000000DN7o-3v98; Thu, 05 Dec 2024 17:50:02 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Alex Shi , linux-mm@kvack.org, Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [PATCH v8 06/21] mm/zsmalloc: convert create_page_chain() and its users to use zpdesc Date: Thu, 5 Dec 2024 17:49:43 +0000 Message-ID: <20241205175000.3187069-7-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Stat-Signature: 1qos4u4hamkjcjbcxbnx7o3fsn35agaz X-Rspam-User: X-Rspamd-Queue-Id: 6022FC0005 X-Rspamd-Server: rspam08 X-HE-Tag: 1733420997-280339 X-HE-Meta: U2FsdGVkX18WLD92FE147cMZRoQvwxjr4P8HiN4txD+OyvTa+Wt++zVsa3Pef3DHjzPtOskBTCD658omPtkwvA8o/EVsbL38PJykH+FhlPmR9gwzNz8PtyrxPrGA1zv9i+7txyfXyylHxv9Uwd+vzuuGq1QlHUzmOrmsxbIx4FUh6393VEqG/M2+XvAOsYqGxdfmVN/f0lepdQKj/7ksVUdEAqNSk0aOlhucWuQEXFxTGMQmUmdBFoNlpLFTohwruP2/GqwmKR+oK/KDXsXlvWlXF0cFm+WBiaKjP0B1QKNFDA3srojswWB++ojbuOjmvAwGc2EBL3Xe0U18LlIlJ9zFZaTAJ6SzeEtmeXSNDMRgr1xI3LqVnRTcm4TMHux21KiCeh61n+vY24/TQtUCbKPZcc2keNshlYlHNRJPkn6vO6qTUfftcFVkdIwBKJG0KdO2s5lWXrO82V6rxVie40jILCPpkRJK4qfsT4l5biFJWglc9lluwc+RtGe2gqZJ7jeUXRtyKMarMa7ZeaUgyBHc3wUvQY2mC4Mi9NLzj9v8UMvnjJgNHB+Kc35jV1Swb+ORIuOkinnohfL+AK8faayzWLLjQkREuFBliDFhtXSU0qMZVtP5Fq6NGFNg9CW4HFxWAOdQmQ/NDCRvCITdC4LhC4hGHNrNxJeXkeoxNBLVodl+11e4Zl1qTrstBeny29TLRY1r/cUGR+q1PLenEdvYG2tgNtUwErLrIM/vByTJLye/Kk1mPV1zZ1yXTiCRHU51lWNo5vpMENawB8TdgOFSbztJ6DIqQQ23WEa8ka5pKmG0g5xS1yt5WV+Od5yhDZGOMSHTo6CsFBQ3PrV1wGcYQiUqkHyPzOr1Two86ySu3zJndjuqIXtMP1JATtb9zhKKQ2vfyhGMbNuoR+VRSttNyNjuw1podWo9NoMlw1ivcDKFav00ePJHVrQV4dhQnFKUTRYFhDOIzfUiY6W sLsAj2Rs pEs0n1/0SCX60mL95tUlqiixVO9j9m9an8KAfm2faV13QIyitznY4t5Bnuq2a18QJAQJdKM7TIjgwgdE8DUVUN+HjsKd/RSGvOVZTzYdWJPj/7xoCnnI3e+uq538BBCp1jdB+MIR5h/Nvt2YkzEQx5svsvXsbLjFu/CFHYb291zMS7qZTnjeCCy0iqveyTd9gTQjXXjN3bHOPGCVG5isv4EFBwML+OqDyiEDh1rZkY5mB82lQnK6HG3ADRBkLkHUa/X0wgIdKpVipoAGODLHwSs7mef36Awu2aWPakbiPwkVSk1rSMiO4RI+YiCO+WL5W8rN3lzV9U9aHgbKLnGnCi7q7rQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Introduce a few helper functions for conversion to convert create_page_chain() to use zpdesc, then use zpdesc in replace_sub_page() too. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- mm/zpdesc.h | 6 +++ mm/zsmalloc.c | 109 ++++++++++++++++++++++++++++++++------------------ 2 files changed, 76 insertions(+), 39 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 937de815a4ac..0387f5771dc6 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -110,4 +110,10 @@ static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) { return page_zpdesc(pfn_to_page(pfn)); } + +static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, + const struct movable_operations *mops) +{ + __SetPageMovable(zpdesc_page(zpdesc), mops); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index af8a6cd6b814..6beb7cce4c31 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -246,6 +246,35 @@ struct zs_pool { atomic_t compaction_in_progress; }; +static inline void zpdesc_set_first(struct zpdesc *zpdesc) +{ + SetPagePrivate(zpdesc_page(zpdesc)); +} + +static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) +{ + inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline void zpdesc_dec_zone_page_state(struct zpdesc *zpdesc) +{ + dec_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline struct zpdesc *alloc_zpdesc(gfp_t gfp) +{ + struct page *page = alloc_page(gfp); + + return page_zpdesc(page); +} + +static inline void free_zpdesc(struct zpdesc *zpdesc) +{ + struct page *page = zpdesc_page(zpdesc); + + __free_page(page); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -955,35 +984,35 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) } static void create_page_chain(struct size_class *class, struct zspage *zspage, - struct page *pages[]) + struct zpdesc *zpdescs[]) { int i; - struct page *page; - struct page *prev_page = NULL; - int nr_pages = class->pages_per_zspage; + struct zpdesc *zpdesc; + struct zpdesc *prev_zpdesc = NULL; + int nr_zpdescs = class->pages_per_zspage; /* * Allocate individual pages and link them together as: - * 1. all pages are linked together using page->index - * 2. each sub-page point to zspage using page->private + * 1. all pages are linked together using zpdesc->next + * 2. each sub-page point to zspage using zpdesc->zspage * - * we set PG_private to identify the first page (i.e. no other sub-page + * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). */ - for (i = 0; i < nr_pages; i++) { - page = pages[i]; - set_page_private(page, (unsigned long)zspage); - page->index = 0; + for (i = 0; i < nr_zpdescs; i++) { + zpdesc = zpdescs[i]; + zpdesc->zspage = zspage; + zpdesc->next = NULL; if (i == 0) { - zspage->first_zpdesc = page_zpdesc(page); - SetPagePrivate(page); + zspage->first_zpdesc = zpdesc; + zpdesc_set_first(zpdesc); if (unlikely(class->objs_per_zspage == 1 && class->pages_per_zspage == 1)) SetZsHugePage(zspage); } else { - prev_page->index = (unsigned long)page; + prev_zpdesc->next = zpdesc; } - prev_page = page; + prev_zpdesc = zpdesc; } } @@ -995,7 +1024,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, gfp_t gfp) { int i; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE]; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE]; struct zspage *zspage = cache_alloc_zspage(pool, gfp); if (!zspage) @@ -1005,25 +1034,25 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, migrate_lock_init(zspage); for (i = 0; i < class->pages_per_zspage; i++) { - struct page *page; + struct zpdesc *zpdesc; - page = alloc_page(gfp); - if (!page) { + zpdesc = alloc_zpdesc(gfp); + if (!zpdesc) { while (--i >= 0) { - dec_zone_page_state(pages[i], NR_ZSPAGES); - __ClearPageZsmalloc(pages[i]); - __free_page(pages[i]); + zpdesc_dec_zone_page_state(zpdescs[i]); + __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(page); + __SetPageZsmalloc(zpdesc_page(zpdesc)); - inc_zone_page_state(page, NR_ZSPAGES); - pages[i] = page; + zpdesc_inc_zone_page_state(zpdesc); + zpdescs[i] = zpdesc; } - create_page_chain(class, zspage, pages); + create_page_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool = pool; zspage->class = class->index; @@ -1744,26 +1773,28 @@ static void migrate_write_unlock(struct zspage *zspage) static const struct movable_operations zsmalloc_mops; static void replace_sub_page(struct size_class *class, struct zspage *zspage, - struct page *newpage, struct page *oldpage) + struct zpdesc *newzpdesc, struct zpdesc *oldzpdesc) { - struct page *page; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE] = {NULL, }; + struct zpdesc *zpdesc; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE] = {NULL, }; + unsigned int first_obj_offset; int idx = 0; - page = get_first_page(zspage); + zpdesc = get_first_zpdesc(zspage); do { - if (page == oldpage) - pages[idx] = newpage; + if (zpdesc == oldzpdesc) + zpdescs[idx] = newzpdesc; else - pages[idx] = page; + zpdescs[idx] = zpdesc; idx++; - } while ((page = get_next_page(page)) != NULL); + } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); - create_page_chain(class, zspage, pages); - set_first_obj_offset(newpage, get_first_obj_offset(oldpage)); + create_page_chain(class, zspage, zpdescs); + first_obj_offset = get_first_obj_offset(zpdesc_page(oldzpdesc)); + set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); if (unlikely(ZsHugePage(zspage))) - newpage->index = oldpage->index; - __SetPageMovable(newpage, &zsmalloc_mops); + newzpdesc->handle = oldzpdesc->handle; + __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); } static bool zs_page_isolate(struct page *page, isolate_mode_t mode) @@ -1836,7 +1867,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, } kunmap_local(s_addr); - replace_sub_page(class, zspage, newpage, page); + replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. From patchwork Thu Dec 5 17:49:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895902 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9201E77179 for ; Thu, 5 Dec 2024 17:50:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 273146B0184; Thu, 5 Dec 2024 12:50:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CD08E6B0180; Thu, 5 Dec 2024 12:50:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8AEBA6B018E; Thu, 5 Dec 2024 12:50:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id CE0B36B0185 for ; Thu, 5 Dec 2024 12:50:09 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 7CFE8141879 for ; Thu, 5 Dec 2024 17:50:06 +0000 (UTC) X-FDA: 82861643676.05.D9331D4 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf17.hostedemail.com (Postfix) with ESMTP id 69A6E40014 for ; Thu, 5 Dec 2024 17:49:52 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=efDVzj+1; spf=none (imf17.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420990; a=rsa-sha256; cv=none; b=AQYZcghT43rgUTB3UQzlrG2RTVxYVogMC53yb1xXFcSTWZR5CqeKAoPwJ/v5l4Cup/T5cd Yw7ZNYKBUDJUbXZ2apXc9o9aErXiUKHvf9jq0bs2tYxIAOXQFaWPupUH9eH/mkfZ+svQaQ 0IRHHe/1fEx5YsMfcR2Jqh6+FfpMPg8= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=efDVzj+1; spf=none (imf17.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420990; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HVYO14EAznZJJrat10eB6Gq5Pa+UXwqYVmPbaoMVuzM=; b=VJR3c2ttABZ2mhIAcMe+45r+qbnlqhSPsY10TSZwu1PT1GCez/yLiuNbAwVxktXPDugCjy pgebxjka6vIhB5rFmwmxMiCVXnv7KJCZ7wlai2W8zo9jJRK85dDB71RoLDp5umUflLEn+6 c3Wq29Rv97trOjR5YLWWMV+fBKQ+mCI= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=HVYO14EAznZJJrat10eB6Gq5Pa+UXwqYVmPbaoMVuzM=; b=efDVzj+1SkMmZUl80u9mk3pQ/8 gD5TTdksVUS2W+Y91mVZkq3jpAzUDbGpSoFUF4tsLqdCfESIVKNnf14yzK1tFu6kD5iZynFs/QarP E8KBYMlvubVxEOK8K9IuU/tnuDnVq+6ZqhS/7LxqOENtrjy4WOOQnIJG8EjoCwxY77ekGcfrrTY55 9tFDUgqiH8210A8UL1GVA3X3/jlAgf5YKtzti99/SgYvt/0ZWF6Iw/miAKkkolixnseLnnY8yCyCA DhNYlq5p5oVr7kGH0/6aU3eo4xzgynYVsBwBD9SCkNaRZvRsVrFf9WBAOe2oy+LBaR2KdCc/sTTFm I9nit+Pg==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzH-0000000DN7v-0NF7; Thu, 05 Dec 2024 17:50:03 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Alex Shi Subject: [PATCH v8 07/21] mm/zsmalloc: convert obj_allocated() and related helpers to use zpdesc Date: Thu, 5 Dec 2024 17:49:44 +0000 Message-ID: <20241205175000.3187069-8-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 69A6E40014 X-Stat-Signature: w6sacp8s158gtgyfanf1mrew6o1issf7 X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1733420992-569908 X-HE-Meta: U2FsdGVkX18XLp8TiSvHxya5jjJ4LLJWhSHxB8pwX9oEN+9SBNNuOIdacfE1wyzulYNXmH98GZSmOUq6rOWUfZ+I8+OfiJ3WbBdvSeN8YGXF/FAAKza7rsg1omKEoCubc6mMHGPxg8w2O8+DCyowGajUezFj42ql4Ge+41X63+uZVIvE0XGimXlodFxfeu//IHx+XaQoTDT/z6YHi4M6w5yCbkrwYnMYcR/UV40by+d64SuNV1iK44nU7BCjXSPi/H+uMsYjnfGRxX9av16ZqWpuyM3U4TpJiG2Zbnnsmh66JsL9a+1ATXKSdWbmZzif0B11lXKwUNrvOOpZ8JQdfx3jWbYkl5N7A7v5kBdJyb6cIN+9vuyJRxWmF52egEo6YiprBm4KNy0mQSzgRrlx7rFuPvLv/EgM1rw4b3yeBEaOvSOdTGzSztQW2vs0XuNy33rwma3/2ccFifC6nlfpi6iWcc4sjF/E6mOTw2+n+z7vqPhMRdUZIvy7LDh7aYDdfnfqiRs96Ib/4vmOWbAedKQOl5OSMqH5XxyKR7L9C/xlbQvoG3II2qwTAnoMPUk8oJVLGm0GgtjKHNfGKHDkgdBSdnYhX225q6gElqhB1QMN42ery9CUXjYVyOxk3z17Z1dMyKzE27rPnK4BDOKbmNwFJY4KAlH7EJ7lmRfInMemfG8khgVaygXjV/rf5ZA+cOwLeQqly7Pmd4I4uouN05CSOUyLruykFMVueepKeRjfmWqahFKT5pcd0U2riQCjVB6qwSF0IUx1PUou+s5m6G2ehiDxsDKrttOBxnnnI8Bb7osuZGQdx23z6KhyUHJH2mnv1BZZ1PSc7n0K9saWq8UVeXWFzK4SNs5lDjelUY2p5F1jF4oL99bNYrQrMKUTUesI/7/saypjCeb/4TPsRTVbNKfuEEXm0OiXLOc2lmv3FJXAQADNkBfOGhrT/BgUbBxuyMk4nlq/GLafimv NHm1/tT5 mOQlO9BIP/b2XpXmP4CMOPmZYAUjLyQ1o6buIDrr/rWYbf9uI5FF3O0K1oH1Ut61p6i40G8HEX/Xwto5UuBTCVNpTSGWIibtD55LgkLFe1sB12S57sV93EHXbf/waarqASaUkXg0GL00HAFeCjS/9FwFkmZ9YZWh99K0P1D6qyXRfr7ysng8Kcw89ZP45SckxQ7Em+0fpH9b6dwdx2lZQzY4Qr/I4XD9lMvXhIfeLWHrz5AYRyxR2oMYnkvlFu73yzopn9k6YAxt3urEdPl2Bbj5Y9+yTQGmgnDEL6ZWBa2Sh5RPyfzZ02eVKA2XHYrMfPmubhKLAc74ca0KpFM9rxm6TSA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Convert obj_allocated(), and related helpers to take zpdesc. Also make its callers to cast (struct page *) to (struct zpdesc *) when calling them. The users will be converted gradually as there are many. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 6beb7cce4c31..894a3131ea2f 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -841,15 +841,15 @@ static unsigned long handle_to_obj(unsigned long handle) return *(unsigned long *)handle; } -static inline bool obj_allocated(struct page *page, void *obj, +static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage = get_zspage(page); + struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); if (unlikely(ZsHugePage(zspage))) { - VM_BUG_ON_PAGE(!is_first_page(page), page); - handle = page->index; + VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); + handle = zpdesc->handle; } else handle = *(unsigned long *)obj; @@ -1588,18 +1588,18 @@ static void zs_object_copy(struct size_class *class, unsigned long dst, * return handle. */ static unsigned long find_alloced_obj(struct size_class *class, - struct page *page, int *obj_idx) + struct zpdesc *zpdesc, int *obj_idx) { unsigned int offset; int index = *obj_idx; unsigned long handle = 0; - void *addr = kmap_local_page(page); + void *addr = kmap_local_zpdesc(zpdesc); - offset = get_first_obj_offset(page); + offset = get_first_obj_offset(zpdesc_page(zpdesc)); offset += class->size * index; while (offset < PAGE_SIZE) { - if (obj_allocated(page, addr + offset, &handle)) + if (obj_allocated(zpdesc, addr + offset, &handle)) break; offset += class->size; @@ -1623,7 +1623,7 @@ static void migrate_zspage(struct zs_pool *pool, struct zspage *src_zspage, struct size_class *class = pool->size_class[src_zspage->class]; while (1) { - handle = find_alloced_obj(class, s_page, &obj_idx); + handle = find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); if (!handle) { s_page = get_next_page(s_page); if (!s_page) @@ -1856,7 +1856,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, for (addr = s_addr + offset; addr < s_addr + PAGE_SIZE; addr += class->size) { - if (obj_allocated(page, addr, &handle)) { + if (obj_allocated(page_zpdesc(page), addr, &handle)) { old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); From patchwork Thu Dec 5 17:49:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895890 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A308E77170 for ; Thu, 5 Dec 2024 17:50:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B60256B0170; Thu, 5 Dec 2024 12:50:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A5E266B0175; Thu, 5 Dec 2024 12:50:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 688FD6B0171; Thu, 5 Dec 2024 12:50:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 3B6606B0170 for ; Thu, 5 Dec 2024 12:50:07 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 780611C8696 for ; Thu, 5 Dec 2024 17:50:06 +0000 (UTC) X-FDA: 82861643844.29.76877FC Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf08.hostedemail.com (Postfix) with ESMTP id B75D716000A for ; Thu, 5 Dec 2024 17:49:53 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="fg2Wkhv/"; dmarc=none; spf=none (imf08.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420997; a=rsa-sha256; cv=none; b=R4TvYRVV6r/6pxVRBw8ajBLdYJYy+NZqBTQBTN9F8T/3NzvayScaNOrU+Cq//iVAbqPJ8y NoO17ul5rMc76s3+cX2O2hyvHbBgaxpAp9Ykm8TQUjjsX+BfAYN7h9s41gLii5R7fyzcS5 EFKyMhz+INv9O8+TtxYfYI3SBprofFg= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="fg2Wkhv/"; dmarc=none; spf=none (imf08.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420997; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3sVBsQPSpwyLfYEIPOdm6XD2XQTQyrW9fL+uk5Se/l4=; b=PA71dJaUjOyDpA+6KCSL+ZuiUXNft/WSHkJ26O7i2A+zNovVy4u6LQJMp9MdpxzIusGt2U weREMhP/69kCLNZ1Gc1iCVK2KFVypi3AJiIFKgNmKZge7ugheGGkSxbA7Ponm0SS06ktDW HoRYleBCRRwnJA3JMOV2OsEez6/etwk= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=3sVBsQPSpwyLfYEIPOdm6XD2XQTQyrW9fL+uk5Se/l4=; b=fg2Wkhv/pzgh0fJAbaTYhwkNNj Nd6w6cFbyzrl78/3O1MyoemEug9jy6AfzmUaJMJB4WDfNFPMJVxMrOxt8ag4WnSiqHbyqWxEfk7Fg cbQ1nuQ9A82HjYtWhQGRNX9GpdcUFi0ZIxAjSXFYvjFsLbM1LCw+Ge/T0QiZaqW98WRT964bQmzxC PG0MyCtBGdIECklG4OzTbTtm5C1bbTeWEchJJrIlLCOPi0nUtRbcnnKz/8/weLyri05CMptbe4Pzw /3pCAd6McskgTsV+w1l8uuQXFLXcwZcF7Qx+WAupdhZkqNmY/KDIFKiKA6415i/6u/Id3MSTkG7E1 j4UV1AoA==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzH-0000000DN81-0f5l; Thu, 05 Dec 2024 17:50:03 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Alex Shi Subject: [PATCH v8 08/21] mm/zsmalloc: convert init_zspage() to use zpdesc Date: Thu, 5 Dec 2024 17:49:45 +0000 Message-ID: <20241205175000.3187069-9-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: B75D716000A X-Stat-Signature: ao3gpchzifkyupeej5h7xowp46kyjw8a X-HE-Tag: 1733420993-311665 X-HE-Meta: U2FsdGVkX19KfPojqIsHyv/XtCjzQdvz6bEe4mZ09cjHzriYmTMIso9Gjwgnr0UH8SrVPMl0FEXGrnGvN//6m8g/zGDCRuA4yF9Se5lD0w9U8kofLhjjPpYfQKaxLBU+C2PlygF8Ik39LTKWeKuzWQ/NC/zKlopczjH1hsghLhT//1kTKZAUUe7ttOQ7M6aJH1eF+K9ubV7AOsuJ4RkarTuYA6B2+rsRCPlxSzyvjHLqDW73xjieT16r2/DAlEetL3tWwwueiy8x/8/qGvlcOkGLLlEk2dTBpeYUZimEdAf+BA1R4pem2MrwKOmGdXN+9YbROZcO1X8uRvvE6r2uJ/NPnyt0M7EA1l9AXZr3/OnPubMqnCO8yL2d2LEIep5+lMJcfMZaxaImDXlFJFDr+S/5YPaIyJ2aKiYyM2VmE8RakN7O+FDdwWWUq2RgsJqKJDVgPkCKGthbHahl1vCqGB9t9Ko4i3wlsXHkIFyvqmeTkeTzjx9LAd7sYkrjZiJgILeg7ZBJ9PQLSR5KWltw80wklz/MY96Fe+IJz1LYsxzAB4EHwRfrQb+mZxo2jtQclIxKIgunId3MmxNX+tY3yE4eEE3C3x7tPFcaYBTiuJGr3RQO/kwjl5FL1ajsZS9jQWAcBoJdDjvsRo0OtL8yP69/W2CtVYbrSALMf0avHtaQtsxfXvLgthzZfukk6556bnAJOkbb9qiCANj0M2C+eNU4RBMGp1uRd8uIRy2YaEQVrAD0V3VF5lDNlHMY0PRoOA6v6zHdwRlD3w6txghXgN5eF/pljFXtOLGdFMbZubviNKOH1GqUDBiL3e/k9x0cC0ekrRzeB9vKSnLP30ZyMjzolr8dVksnVyXbg0tTcypsc/Y+IWMTI6QSsxNDR3Tt3eaQvfyvtRSc9oPMJ/xqfuYNnmv7YY2YuDqPhjaLhsC4wqAsz0VbL67XPrG8vNjzaEaR90cf8pih0KT2WMr 5RoJUyAa dQF58ZK71XnIJd5QKvoNAy/cJYyhFyFIasvk8y8Pa//iqfXuEsyQddVDWtiI0n7CGoKM4GLL2ILRtP/FEAqMMf3jH2CoKHkxT9dWkhWlrsZHeMbTlDdD9ZhYkBto+EGQQP6cwB4Rd2JuruQ/dS36l0yZziRiGkfS7k4y0D+3MI4seoRxf4GEyQcxzW/zwJWXEp91/lgCiR2Pgn8kzWjBDUlB3Hd/1XW8F/G99yyDX691hrouvp3nhMAVpE+28xOmlwSq4QOATdezsLOsgyQIlW+oVVqpcVrnNd58VLMWckFu/3BpIuv0USVLLzgulGuyPNcjlaLU84+G1ta90oZEIROdbtQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Replace get_first/next_page func series and kmap_atomic to new helper, no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 894a3131ea2f..faaebc1514ed 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -943,16 +943,16 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) { unsigned int freeobj = 1; unsigned long off = 0; - struct page *page = get_first_page(zspage); + struct zpdesc *zpdesc = get_first_zpdesc(zspage); - while (page) { - struct page *next_page; + while (zpdesc) { + struct zpdesc *next_zpdesc; struct link_free *link; void *vaddr; - set_first_obj_offset(page, off); + set_first_obj_offset(zpdesc_page(zpdesc), off); - vaddr = kmap_local_page(page); + vaddr = kmap_local_zpdesc(zpdesc); link = (struct link_free *)vaddr + off / sizeof(*link); while ((off += class->size) < PAGE_SIZE) { @@ -965,8 +965,8 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) * page, which must point to the first object on the next * page (if present) */ - next_page = get_next_page(page); - if (next_page) { + next_zpdesc = get_next_zpdesc(zpdesc); + if (next_zpdesc) { link->next = freeobj++ << OBJ_TAG_BITS; } else { /* @@ -976,7 +976,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) link->next = -1UL << OBJ_TAG_BITS; } kunmap_local(vaddr); - page = next_page; + zpdesc = next_zpdesc; off %= PAGE_SIZE; } From patchwork Thu Dec 5 17:49:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895901 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9711E77179 for ; Thu, 5 Dec 2024 17:50:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE8006B0179; Thu, 5 Dec 2024 12:50:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6FCE46B0193; Thu, 5 Dec 2024 12:50:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A4E96B018E; Thu, 5 Dec 2024 12:50:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B605C6B017E for ; Thu, 5 Dec 2024 12:50:09 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6B43D1C863C for ; Thu, 5 Dec 2024 17:50:06 +0000 (UTC) X-FDA: 82861643046.09.E99C2A8 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf21.hostedemail.com (Postfix) with ESMTP id 8DE591C000F for ; Thu, 5 Dec 2024 17:49:34 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=X77s57vN; dmarc=none; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420997; a=rsa-sha256; cv=none; b=QhHVO3eCaEoKcVPwh5xhPMA3RyD4a5vblKLENTnVJ9FYwOaVS1NdbcxuYQFj5HZdbJuR/0 JGEUEtSGIiur8O2QAbOTOYERLS2u9cmMMyymL2NWz0Ai1BI+fFPtEfCi+0ODAysOKhyRVX UnDckRCBTr8FE20AM/HvjfWlgF77QjU= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=X77s57vN; dmarc=none; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420997; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ytR2DMOpT9l70a9J16/2c3nlbTXk/DpV9lwOjz46sSI=; b=x4vjSZNxAsdFFdCQ8iVHMtlzts1YI5ddRi23dZO0lHxUNzQy8LbR0xDSmjaZC9oYFDe8WS cnG6iMoc5FUDlvoZfZku8jIOR5ab0iHyY4fK2MKLQ8/i0HI46Tr7bhu2GVbn/+ZMZi+S3e /nb7IiAxIjVAYupiCcn/DuYlEfuQEV4= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=ytR2DMOpT9l70a9J16/2c3nlbTXk/DpV9lwOjz46sSI=; b=X77s57vNgrj4SiZG2HwYY+NOKa NLz70l2rDKSTtEfE+ogYZDUuXze/7HHCADFjN6RAL17cdl0a62zUbqBVgGZioFND1gvn3nnY6MkvO 12IUhQYJ7IctdG/fByrwJIqNM8WT/+usTU568xwCT5o5752BiQikYM6x4jrZHHO+vMdMwRvZmyv1I kFOZ77c4E+7Kk7+YhFPhAgHMkQ8lr9HqRG6PoVie/BTdZskxpD669oDVxkzXxulRJF2n4kYibPwwi gjL/DBPXkXl8lA84uiiVGiagtqI0DveDOFiNeutqxX4RuQKCrBd1lt5l86wqI4O+P+Nj+mbpE3Fw8 0BtFJa+Q==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzH-0000000DN89-18Db; Thu, 05 Dec 2024 17:50:03 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Alex Shi Subject: [PATCH v8 09/21] mm/zsmalloc: convert obj_to_page() and zs_free() to use zpdesc Date: Thu, 5 Dec 2024 17:49:46 +0000 Message-ID: <20241205175000.3187069-10-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 8DE591C000F X-Stat-Signature: 3g1ogzwmy48tm9jgs8xysonhj79pxsge X-HE-Tag: 1733420974-996678 X-HE-Meta: U2FsdGVkX18C+mx/h/ZwqvY4dmwLRMQ99g64pUHHvuY6Qh167c6zB9ftkIUv8xhkJuty3s9dSPNMD0sugqeNY7Rd7/gqGuTPFq4a37U0YJOe5Drn0nCsugf7CYpJbxmRwDVCBsdS87/t3l0NIIh0OGHszdRiScJnQTf3eh+gnjDoCqMdS42yWugsLaYuSgkiqAmWKMt0RQ3FOqWTu+nblSJj4zVvd2kiPCJ8rUlYn93cCwhNuTKmJ+wWBqBRHyXB7erQ2W6EeNF9j04DSO9VDxoV2hh2TvAb0eVx7mbBKWgkQyEBwYKxkANBFUv+UR2AkNpR10XF0+VpMTB4rDgKqIg2PEM6oK2bA3zkyKFr/+eSWdpoeNZOmshzs3XPEcplU6dIy/zSHc6sIfcYRSdcYIHCRhKNRTq4t/ISBLoyCsj17D185Stnbk+GMFHkhMzK3GFQM9PlUXqkMSfUseuF2S2T1DL7/8wdSfg20fLwjB8Vd4FcA5As5khROdym9gzOfmDw5enkrmugFB2hcQRzH5DLtzXRBKIugf2sr7fu4Ux+ViSBE63u4R25gg6bLah2qbFFg+WcVaj96TaUCfjMOXnGKJcPNfNiSxYBStCSQCxG5/B7TqwTi1PKyZrrwnlw4PLcjCNFrQxc0dI1nqgf96DxAQkIaaa7RLboJLOIgTgUdg/MzcjmfYGNezPEPgMgZ/zBHCdEwmIlQBAe/IkE6YvJUt7obNlynREdLXvqYhkEesMsyZFqiABXw6a+RFojYbLvFPAg+Vr3DJi2rVk6GEcirqJvqPfztxzCYcnC+qw7YmvrqVT69Kq//esmKtw3dlvXoCI01gZmeRaPUsaDNAxN6dBTjtUE9ElIq4Ouq3OwfoL3TPtfJP1bCmiq+Pr3gMpqEklB+4CwuyPZuC+zxG3DD2YAzOIrRZnFbdwr2y45BwPCeIlq+mGz3yhASNS++XwqjnPb74TCSz55kQN CYgfbX3V 74Z63tRQnsX6JcSxmAGRUal8ayNhZofOwd/eO+HD1F4Za8/hNZtvd+dkZU6yd7BcjLRaLbprgR86M4bof2KJiqalcMjE75QBA3Q3QwDpXOi+9dOLzl2GyLP/IaJJ9nVSNwIi26ltKnBIrtVg3KhF7iAgGjCKovcoBZw0B2v3TKrEktyAYxGdr2NGNdv+PPF+oEr78sU+A6dpaumKcV7MFnQn8cHQh13ECZoRiU5fwLPp+26ycHJNIYIyzAixxKEwhltkdpTbQwr4xh9Ee9QDSscNNI05n1UyVt1vhAX4pwcgg9FtvuvpCBtk0XEYTC+duigSLKnAB0FiwVRG656ddSM+Tjg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Rename obj_to_page() to obj_to_zpdesc() and also convert it and its user zs_free() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index faaebc1514ed..6145f7169c67 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -816,9 +816,9 @@ static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, *obj_idx = (obj & OBJ_INDEX_MASK); } -static void obj_to_page(unsigned long obj, struct page **page) +static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) { - *page = pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc = pfn_zpdesc(obj >> OBJ_INDEX_BITS); } /** @@ -1481,7 +1481,7 @@ static void obj_free(int class_size, unsigned long obj) void zs_free(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long obj; struct size_class *class; int fullness; @@ -1495,8 +1495,8 @@ void zs_free(struct zs_pool *pool, unsigned long handle) */ read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); - obj_to_page(obj, &f_page); - zspage = get_zspage(f_page); + obj_to_zpdesc(obj, &f_zpdesc); + zspage = get_zspage(zpdesc_page(f_zpdesc)); class = zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); From patchwork Thu Dec 5 17:49:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895889 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80B97E7716E for ; Thu, 5 Dec 2024 17:50:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7E9636B016D; Thu, 5 Dec 2024 12:50:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 798A16B0170; Thu, 5 Dec 2024 12:50:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5EAF26B0172; Thu, 5 Dec 2024 12:50:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3266C6B016D for ; Thu, 5 Dec 2024 12:50:07 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 717C11A160D for ; Thu, 5 Dec 2024 17:50:06 +0000 (UTC) X-FDA: 82861643844.02.260D919 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf15.hostedemail.com (Postfix) with ESMTP id 60DB0A001B for ; Thu, 5 Dec 2024 17:49:48 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=qbZSmH5D; spf=none (imf15.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420994; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fcJ6cZyqlI8kYhrx6tlQPh4Zb+MB5UuATNGQ59JcAWs=; b=Cr3as9FMxeYfpy2TrycSgDmm7SVyfBo0nvfuIbr2vD4zgDfh4eLs2Nx0beRED3lCxuahUf PbGLYCdwDASkgUJnBOYxXda6QTQqPrhf8jClsfC19/fFiQtP9tkE24+xDGN/VHUmElnffD jZi1IiWyQ0qHH14yf5672hs838a5PHA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420994; a=rsa-sha256; cv=none; b=LL2mnRYgf7ImgSYwWOGIaD9CbvwsoMe75o3Gg2Qf39SlbhxA+vw2eVR27oyqco1IUJNayE bkaiZEUqPm0GfI3wXDxgU+LmWJyTdXXAIfLwAauEosUgaB3r/u0joE4Tq+l/qW8uDly1Ak YFs0Iwx68SIBSccvmZ/9XgaDdiDZWSM= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=qbZSmH5D; spf=none (imf15.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=fcJ6cZyqlI8kYhrx6tlQPh4Zb+MB5UuATNGQ59JcAWs=; b=qbZSmH5DLc9tWQawTtY0cKavfi 7NgXmaaALQ381FoK4izKQ10noORt2Iynb+R541C8P27oG6l4bQTKSrImFy3SrsioOCxVg2xBZcYtr OBDfzTYX7SQpkDzWjIuZQM8sm56tuFKJRVNcXYj4CYYxFvIq6JiDTvn+NgdhKhv+1jd4H2Anty3pp 94DdtpaS0kuIvGDyq5GfKdFrUQidmNhPQOLiN7RXk/c5Wg7QPmu9265aQMfA6zIN4ma/4MKindWZd bJf8fhcB0Pz3Yhn6graqGoOMxTWz9i+Syo0/VXNGJ6cja07N9K/72GFmJtdeiXqhihsg5UAzH4nx8 cT2tqLhA==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzH-0000000DN8F-1XC4; Thu, 05 Dec 2024 17:50:03 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Alex Shi Subject: [PATCH v8 10/21] mm/zsmalloc: add zpdesc_is_isolated()/zpdesc_zone() helper for zs_page_migrate() Date: Thu, 5 Dec 2024 17:49:47 +0000 Message-ID: <20241205175000.3187069-11-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 60DB0A001B X-Stat-Signature: hfa8mabjcgn88m6sqwbmux1s93adya4q X-Rspam-User: X-HE-Tag: 1733420988-393788 X-HE-Meta: U2FsdGVkX1/M02szS1YMvknEeD2iLGfeQM0IACuWgdRmLCELlvRL8JaHCUi58xgAou3M3lOzx8VgXOp665BhKCtoh2/NvzBfsmjMfV6zsmLWiGhAZV+Tqd5EbY6ClZefrxJQT0rDiABRDa6UybJPZ1QKfF0l9Qsu2wrhH+rrkoVebRWeT7lz/HZc+cdDBaXJsLc5f7GRq2nMlh/LDdER++LCd2y1UgBoHxE1KaeBlGiYZCFIKXfMNgr4FBNuHjIMgs+I4JjTTgeHlzaPC05nnKxqypy1xw4+vCRI1PDoxvXMDtfMCl4fCeUrWbWgDUQtPJCuYgUqcSQuORv7nHw/j+h3exyX8KKoV75bz4c01Vj6TBYs1KULiGCNzTpVC6I9npEcv4eFA4l5FsMeUWVfm5zd0epVTtEeRov20zgK8trSctA0D+ZBHmNErucJ3mZMjEDvRqfWDk8C4Zck9+lLtwEeHpEsl4fFDORMvR8oRrS8FmUaIu42VBDG8oPTATF66LroWfmQspOBatANIF8+AwwvjtA19vdhO/g9J/MhrLG9A74aMYqDK5/cJpO41zmlXrEa/cAMnUQ2MEcMbwTtWhSB7qQcc5n51+N1jsgtY6c9ESCWTjKs8plYOPgufcArWQv7pyOREtCOrcm8wQwq4AHG1bAJr2LShXWTV9y4Bzs26t0zPL882qqcdu4Po3U0Abdpm1ksfG7F6AV0O1bVVszeNf/iJu+9H1WsriRp9r4/72VnBSmr5zpDwem/x+sCHAyJq4dQnpdCW2uALSJsf+gUxwy1LhQ9f4o0d8hg/MVJuJN1tDs/jKXHY7CzzqGlZksUsv2H9B08Ne5r1v9HtnIoStWTxmErzC2DEDt9054i8HVURQHTr3QPbX4c9KzHMmJ3h/ppsZocFNkEu2ej/miFjt/xCJirv8jodqCbNSGsOwXBtSTYVzQCyOJIc50YjyUM8CdIZYWzXBr5THW HEZS8T4h FEzhKxwF7ILZYZ+mWSk2Ode2Fadonab5E0/KtuwCkXu79vWOcdLgpF3uFTSiiLZbMO611HBa2g1BHfZkT1XoGU7WueAJCQfGgBVFIdVczePtFQsiwayr4HqZW5F9Srr3yhdk6hMzx+nJa+BASR7RfEBCBrhY//dy5kLCLGhx3hjVMSUo6OjLKAQycwp0SRBQgI5UceVikJ/BCDKMTMB/FMNiU4u14ZNWzEL/jSZVy0Xw40f2E/TVtoiQnjc0v3CobxlqyJRzV6ONBbhgXxLG4mvo7Anve+rr+Hgp6yl4ZVWzw0uveqz29Ap9x3GiEUhWkqMbH1PjBXRX4Twknz7t8j/1pzw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To convert page to zpdesc in zs_page_migrate(), we added zpdesc_is_isolated()/zpdesc_zone() helpers. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 11 +++++++++++ mm/zsmalloc.c | 30 ++++++++++++++++-------------- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 0387f5771dc6..7d59bc2587bf 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -116,4 +116,15 @@ static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, { __SetPageMovable(zpdesc_page(zpdesc), mops); } + +static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) +{ + return PageIsolated(zpdesc_page(zpdesc)); +} + +static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) +{ + return page_zone(zpdesc_page(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 6145f7169c67..d8f2f8e5c877 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1815,19 +1815,21 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct size_class *class; struct zspage *zspage; struct zpdesc *dummy; + struct zpdesc *newzpdesc = page_zpdesc(newpage); + struct zpdesc *zpdesc = page_zpdesc(page); void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; unsigned long old_obj, new_obj; unsigned int obj_idx; - VM_BUG_ON_PAGE(!PageIsolated(page), page); + VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(newpage); + __SetPageZsmalloc(zpdesc_page(newzpdesc)); /* The page is locked, so this pointer must remain valid */ - zspage = get_zspage(page); + zspage = get_zspage(zpdesc_page(zpdesc)); pool = zspage->pool; /* @@ -1844,30 +1846,30 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); - offset = get_first_obj_offset(page); - s_addr = kmap_local_page(page); + offset = get_first_obj_offset(zpdesc_page(zpdesc)); + s_addr = kmap_local_zpdesc(zpdesc); /* * Here, any user cannot access all objects in the zspage so let's move. */ - d_addr = kmap_local_page(newpage); + d_addr = kmap_local_zpdesc(newzpdesc); copy_page(d_addr, s_addr); kunmap_local(d_addr); for (addr = s_addr + offset; addr < s_addr + PAGE_SIZE; addr += class->size) { - if (obj_allocated(page_zpdesc(page), addr, &handle)) { + if (obj_allocated(zpdesc, addr, &handle)) { old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj = (unsigned long)location_to_obj(newpage, + new_obj = (unsigned long)location_to_obj(zpdesc_page(newzpdesc), obj_idx); record_obj(handle, new_obj); } } kunmap_local(s_addr); - replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); + replace_sub_page(class, zspage, newzpdesc, zpdesc); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. @@ -1876,14 +1878,14 @@ static int zs_page_migrate(struct page *newpage, struct page *page, spin_unlock(&class->lock); migrate_write_unlock(zspage); - get_page(newpage); - if (page_zone(newpage) != page_zone(page)) { - dec_zone_page_state(page, NR_ZSPAGES); - inc_zone_page_state(newpage, NR_ZSPAGES); + zpdesc_get(newzpdesc); + if (zpdesc_zone(newzpdesc) != zpdesc_zone(zpdesc)) { + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_inc_zone_page_state(newzpdesc); } reset_page(page); - put_page(page); + zpdesc_put(zpdesc); return MIGRATEPAGE_SUCCESS; } From patchwork Thu Dec 5 17:49:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895894 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF2CDE7716E for ; Thu, 5 Dec 2024 17:50:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D8B0E6B017D; Thu, 5 Dec 2024 12:50:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C08956B017C; Thu, 5 Dec 2024 12:50:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5C3B76B0184; Thu, 5 Dec 2024 12:50:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id EBEDD6B017B for ; Thu, 5 Dec 2024 12:50:07 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 85FD4C166E for ; Thu, 5 Dec 2024 17:50:06 +0000 (UTC) X-FDA: 82861643382.25.2FA7FCC Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf16.hostedemail.com (Postfix) with ESMTP id 8DB1B180005 for ; Thu, 5 Dec 2024 17:49:48 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=cHXKYiCa; dmarc=none; spf=none (imf16.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420997; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4xHbmQCRdJwLFxwWLnDqxLyGzIaNRK/5CMiN435paJY=; b=7Q/KxaN4o8xmmTSP9IW18b82KGrM/zfKyvO5wfGvFwKr34tct4eRgpnvdiu0DdzknnaySB iWMWj1DYM28uBwmSckZVLSsSklA3AuJx704pxkkmqPNeuUptZpD11+kpS4f6c8+FKzR7j9 mtEB6KjSBulhM7x0MMq/w/ZRubwK790= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420997; a=rsa-sha256; cv=none; b=7mUo0KMk+muwwPautDjjxt278RiE4+id7b9Qu96EPnwix3KBeVclw+ROiS6H/eNpsIOlD9 eYoVzkRFKr8s3QO6GFX8sucdpcOLfC2P8ALkKv2BMHu+f6THoOFQuaPvwWp//yHUVjF7JR ygQbVsSxd4B8S1EQ9Ivh3EdxyimjPVo= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=cHXKYiCa; dmarc=none; spf=none (imf16.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=4xHbmQCRdJwLFxwWLnDqxLyGzIaNRK/5CMiN435paJY=; b=cHXKYiCa1o7H9pqjRBd1A5G3aD umYbo2kLMcPB+kF48Gi/kUl/07Ha5LiSvwwCyGziYORRgfRSa5K7OjmDQYy/pngQOo/WKA6fq6bD/ B6OX2Q6fZqXGj5RripaSHJag8y/AMo7hpHAeg7y1XUG+bBZmmlu4bTHX+ttoNN/XIepM9pZOmgpyW n+uOxBIptB85V8kAvq7XyI88YGMaByPuXayauzh+GSz5GkXIpxh5d7z6vD6wC28CgMubM7Wda/k2Y 1wtd4ytNgDrsti24CPaKEZ2scjl2uvRuwNar1CQBldIxsjyA/Q0kZCkxz4e8ZJlp5fUEFRi6YpwBR 043MPE3A==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzH-0000000DN8Q-2Cws; Thu, 05 Dec 2024 17:50:03 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Alex Shi , linux-mm@kvack.org Subject: [PATCH v8 11/21] mm/zsmalloc: rename reset_page to reset_zpdesc and use zpdesc in it Date: Thu, 5 Dec 2024 17:49:48 +0000 Message-ID: <20241205175000.3187069-12-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Stat-Signature: b7fjkyt9793mxqt3jbcfb73ndqguqsk6 X-Rspamd-Queue-Id: 8DB1B180005 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1733420988-916231 X-HE-Meta: U2FsdGVkX19p74PwWsvTRwT5gHr6mPr9CsStIhCaDzu0nb5Gox2eukRMvkMZqO6lKhan9oxm1KR7FZthCfBlvEWzL1I5nuSPiQ5C08KyZ05kRXk4fZmSIJzqr1fxutls1UsOznBt+vB5+NHtT0/4jvxulLgBIPhBAaQbaTDiK2Y5lDwx8yeHRUAms+wAyP5+OjGjypckhguS0AI2JDOhrwGzl+iPr1DS3Cc6RVb1I/JOZlLXCV4Q1UbY3AU4iaBiRhHtT/RIJAGwLLOwyQjbGRowheu+nV/poDr0C13DUSvQArZI2vuoccCkc/Qk4b+6u6pD0jL3J60dMV4ZjlBsxsgKYClFtNGdful+qqKkosPwicsRNUQ2vmz5fIf/l8X7Vmr9DJ+qbdEURGrijkYzySMxQnyZD4hjMuIK+RsISPocvexc9Yiu6hIecKtZ39e6RZFn9E9XkLN8TLjrKZJG9JiDIsD/SyOon5dQgfMr3jlx3573kyDyB8UDibq1ezzvgzp0nmpb/fYQdNadytaC2NilOaGDShb7/3VJ31W0qIsp0TRFY4pRD3sfajP8XObLXSKMoJFroJSqjJiltp5kh0A4G3AmimBAVgqfPPIb9EwjmZmDjsLT9lta1/SwKAIS7eE7g3Jcr6m9+gj/YUSfnM0DooIc2ju/wnipsHeyGKIcmMKXlZDftRgGxXlU9sl32euLuFT6FadEzo/NURej+Q1jZbliQjzk7EYtnW3IhHhBhneo/HDfwjP/xSRf4lZWEYzj8Hqgf/qP9gYcWJIs269xMG7uHfva6WewTvm4+oLL8NYeqqnm0GFk1BzY4AM/GEP6tQJhZJg1GfTfp++MiUYdqBfhmkhtjb8UOfkZwbScOUn+z5R47y+fmjNleI980JN6OsaqeA+JZ0yKmKI1VdhELzP7/tLy3qhE1pSdS19ZxBwfxYbi+NmIzCTnUt8KKv0FVlPVufkmBAqmvdP GzGviukV bxRDNGWB9TGtQUzTw2Qx9l5SfHw5V/H7rgl/J6vfcame/oF3XcWi18LczMz1lLAbWtfGhung1qc/pv5R1EZIkxmZ1uqkNUX59/UY0fzv1N7E/GiwulQYY1LfE8Bj2n1kfrteoDlEfh8TEC0i8BRA3wsT7clyF5DJYhQC+SkV+PGZ8h20ZiAYVUOwH4xwdurWBpx60MfnGNXEE8tcNg5V39Ic8l9ULqMw7x1WRdkfWUbx3GaUZmqp0771k4bsZU8KCVjvd X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi zpdesc.zspage matches with page.private, zpdesc.next matches with page.index. They will be reset in reset_page() which is called prior to free base pages of a zspage. Use zpdesc to replace page struct and rename it to reset_zpdesc(), few page helper still left since they are used too widely. Signed-off-by: Alex Shi Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- mm/zsmalloc.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index d8f2f8e5c877..87b2b100782f 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -861,12 +861,14 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, return true; } -static void reset_page(struct page *page) +static void reset_zpdesc(struct zpdesc *zpdesc) { + struct page *page = zpdesc_page(zpdesc); + __ClearPageMovable(page); ClearPagePrivate(page); - set_page_private(page, 0); - page->index = 0; + zpdesc->zspage = NULL; + zpdesc->next = NULL; __ClearPageZsmalloc(page); } @@ -905,7 +907,7 @@ static void __free_zspage(struct zs_pool *pool, struct size_class *class, do { VM_BUG_ON_PAGE(!PageLocked(page), page); next = get_next_page(page); - reset_page(page); + reset_zpdesc(page_zpdesc(page)); unlock_page(page); dec_zone_page_state(page, NR_ZSPAGES); put_page(page); @@ -1884,7 +1886,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, zpdesc_inc_zone_page_state(newzpdesc); } - reset_page(page); + reset_zpdesc(zpdesc); zpdesc_put(zpdesc); return MIGRATEPAGE_SUCCESS; From patchwork Thu Dec 5 17:49:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895895 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBE9CE77170 for ; Thu, 5 Dec 2024 17:50:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1A5DD6B00A1; Thu, 5 Dec 2024 12:50:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C6D3C6B0179; Thu, 5 Dec 2024 12:50:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 920326B017E; Thu, 5 Dec 2024 12:50:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id DFED16B0179 for ; Thu, 5 Dec 2024 12:50:07 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 944FA161660 for ; Thu, 5 Dec 2024 17:50:06 +0000 (UTC) X-FDA: 82861643634.15.D374D44 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf25.hostedemail.com (Postfix) with ESMTP id 6D220A0004 for ; Thu, 5 Dec 2024 17:49:53 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="df/MmR1R"; spf=none (imf25.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420997; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=crjKswtSjiQfBwQcXTW2Ux6px6WFKzD5tBr9/cgLJDg=; b=jCXHlCAd+utTUvd2a4Mt0VxlG9kn49ot9p76jkof/sF/dk+oq3lCqUOOPeg1vPw0Tcw0QO 4Qsq5magR96kywNPCQ9DoGSI7qbEEuE57EE6MA7p55tfhF7mRb4L+UrPvZXBPbY7Cp/23k 5YuYW8Y+O8vHsnF0aFdiJQ+3ak695XA= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="df/MmR1R"; spf=none (imf25.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420997; a=rsa-sha256; cv=none; b=Zdzs8dJa+gnyeCvjctV/UPSLVGBB6yUkg1OmiP7V4U9eUS2kb3adnIaGpNm/cXnbgmMEOL hMQs27zZpLwhnBlNyXBK0/58eZyiVtzXbBfDVDlEO1fTDgKz2msp4mYMQR+9NcWq464SWT AXhjlV9nQGmdgFWXcGO/92v8MAs3it4= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=crjKswtSjiQfBwQcXTW2Ux6px6WFKzD5tBr9/cgLJDg=; b=df/MmR1RZK10mtnfYnzbPIbb5U /gFttjvrNlMCxREJOLkPV5DJbNNgyRgS9RAVBmfq6ZMd469NaqUbWC16HiyinSZltF6SnwOzEn5cI z8WKCUgTBjYGRyhM9V5njXdd9JCB/0todTVB2A8YXvYCeqyXrGjPhTNGlMoc6EmQUH/z8cdq0oazP F5mpXuwsx9Pm3L+OUOUqj02TDzi/bvycnei9fHJSv23ZQXTAPy3IRY82QAkpDE//eOqsYLUHsgHmP d3MZRQwWYpyK3KHT1s7TtxAssLdLz1P0XSpOWwAVPXqDqjQNgaTwgFOaGBcYL8d/bUCRhXYciJFlu qad6ARlA==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzH-0000000DN8Y-2s4R; Thu, 05 Dec 2024 17:50:03 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Alex Shi Subject: [PATCH v8 12/21] mm/zsmalloc: convert __free_zspage() to use zpdesc Date: Thu, 5 Dec 2024 17:49:49 +0000 Message-ID: <20241205175000.3187069-13-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6D220A0004 X-Stat-Signature: 3bychrbkzetqrjaye1p1kj45qxejc77f X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1733420993-767155 X-HE-Meta: U2FsdGVkX18Jo/33wjFzQf+jOex/Z9a3WKy4/EBID2CGeBbbv+9uTRtF5l9LZA+RLZU3Kj/+O6hOiBS6JA5Oi08kIrWiCrZPn5kmz/C6wk7pqcWMkPDQC+vjSLdgzmxwEbjyPlD2qfJXh4PXqxRveSbM+bGugqzioCpGJMW+pvNW9WckT82eMxUMi3lOuX5AFpvFQvWcqjb2edg2BML9vbDtP1lao1PVJ38k89Xoihi6pi22Aiwxn9ge+X1JonDxUAl7tst7YEjhNjs6LM7yUM7GxC7L0nffTv2JIFoVAL8Dm2lpnPJR8v8cHcEZyymZLhMVvFnM2dxIglPezo3PWRMHO2lGIW69SsfUYCJbPSCRB+IrIvhTSAhoN7/48DN42prZXZBk4crbgma0jTCcxqBVwLLCf41VY81x3+YAQp0piMwX4viqPdItu7yS8KV9RBr59JyKqz7pYJOueBrHFiiTmJBV4xUSk8SuIw65lyrPAE80Nd0I/Vdn59dVUv2djb+Fm+2a2DYe8l5eKCJBKLcLp8BfscqaKyD6+r/VdV4S7zSDLlPc262NlDdTlOn9ZnM3kcz2AEu7yzEfsUWN8dlhIICrQIdpARk903krbQU7lCTbnedj0DDgcMEMbVhiIdyv6PiqS1OF4qVdSwCkAYMGFewbZFfbQGJCUrZx412Ki2go0+uMQgtROFGxwzB/xwLKYwrsv9WpqoSZzu4jTNIONt54p592vcchfJtTiGlX119GCssm4LRA6oTBaz/KPlwKdWM2Rq62XEqH1JINpZLYYqBKgSye3OhBWTmN7Kperm04ZwxlHVlZWDEKzS2tnfcXjtdiu9dOF+A8XUlsz99hhq4FfS4Bahb3Hb6aOz012oyoh4KSjwj8xQAvhc1bZc4ycUlvPetrSImUGQncNK5GeBrPLMBScP8CMPgLaNKF5NCNDFt+94+dIlevCGN6Rkejq+/SFRe2hNDHQaz RZeC0qVZ BBpjGpC39ud7nTgfhIG5wm80p6xy4RUCuh4zG8PB3+Dv4cHy8V5ELuo9IFeV+//jAedpo99XjibWdMXfSac4k3xHEFWwmvDyaj94BU6PphcruDidl+c5uq69c/HK6G2WYwE8kDGcSA4tQShYOK+0Anz1/a4My4NlGKNHJ2Ob9bA3kb3LfozG0bS/oN0Xi2brPpilZvjBG35CU+vKgpmJGotmEfh+Bb4g5/tt+RbYAm65PGANtpmuiUzhy+yxGBnaqCAJd6/vfjmlYVohsAd4YW4I1Okvd1gYGz5zujg48nQ9xiCiVm7gKMhJvf4WXGXs7xoVUMadNPbAhEkgGY/Z7fDyfeA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Introduce zpdesc_is_locked() and convert __free_zspage() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 4 ++++ mm/zsmalloc.c | 20 ++++++++++---------- 2 files changed, 14 insertions(+), 10 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 7d59bc2587bf..89fe78a7f5ca 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -127,4 +127,8 @@ static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) return page_zone(zpdesc_page(zpdesc)); } +static inline bool zpdesc_is_locked(struct zpdesc *zpdesc) +{ + return folio_test_locked(zpdesc_folio(zpdesc)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 87b2b100782f..2932351f4698 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -896,23 +896,23 @@ static int trylock_zspage(struct zspage *zspage) static void __free_zspage(struct zs_pool *pool, struct size_class *class, struct zspage *zspage) { - struct page *page, *next; + struct zpdesc *zpdesc, *next; assert_spin_locked(&class->lock); VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness != ZS_INUSE_RATIO_0); - next = page = get_first_page(zspage); + next = zpdesc = get_first_zpdesc(zspage); do { - VM_BUG_ON_PAGE(!PageLocked(page), page); - next = get_next_page(page); - reset_zpdesc(page_zpdesc(page)); - unlock_page(page); - dec_zone_page_state(page, NR_ZSPAGES); - put_page(page); - page = next; - } while (page != NULL); + VM_BUG_ON_PAGE(!zpdesc_is_locked(zpdesc), zpdesc_page(zpdesc)); + next = get_next_zpdesc(zpdesc); + reset_zpdesc(zpdesc); + zpdesc_unlock(zpdesc); + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_put(zpdesc); + zpdesc = next; + } while (zpdesc != NULL); cache_free_zspage(pool, zspage); From patchwork Thu Dec 5 17:49:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895898 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B23F1E77170 for ; Thu, 5 Dec 2024 17:50:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E813C6B0188; Thu, 5 Dec 2024 12:50:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 797116B017C; Thu, 5 Dec 2024 12:50:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F05866B0188; Thu, 5 Dec 2024 12:50:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 061236B0175 for ; Thu, 5 Dec 2024 12:50:07 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 9C626AF51A for ; Thu, 5 Dec 2024 17:50:06 +0000 (UTC) X-FDA: 82861643382.06.A29A132 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf18.hostedemail.com (Postfix) with ESMTP id CC25A1C0008 for ; Thu, 5 Dec 2024 17:49:57 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=gocPOtoL; dmarc=none; spf=none (imf18.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420990; a=rsa-sha256; cv=none; b=nijN0IB4RLTKxRwiD0GGgEnm98xTVeJrnK33An4nDV+M1zwN98gJo9a8jqjfirnzF4QZ6C bT7PvtnoA285EXhj1Xb7BT4z94FjG/aL3wU428CufdYLPpEYroNZnehc50Seo2oQ9gf2LS kti/D6A2ijg4G4MSCNJglckAqXNCX3k= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=gocPOtoL; dmarc=none; spf=none (imf18.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420990; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=v1D7rbUNhCE1bX9EOMFHnijZuqUzg7mt/pt8cCRyQ4o=; b=0GoGV1TH2HQ/8Zv26iB5yIXnH3DpiEUQ7RYscTp5zn/vb0rojPmZZy5eOssBLFFM8OscM0 KLh8f3Se+fLhceXiT/4v/p1UZkv9Q6UrqXehNEAeOd8UPBXOebILyiebEVfzjsy6TVkYiM FvIULtZ2uDSmNANDj8AqmHyIF9f6sl4= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=v1D7rbUNhCE1bX9EOMFHnijZuqUzg7mt/pt8cCRyQ4o=; b=gocPOtoLWxs4c+F/5+VlLX4O+r 2K7KcS3yTFsdYJBk8bKpzRBbtVy6kBs9VZskE5aCia6i/G25BpJrmdFFZ2uuiywgknyHBiGJuuXsq yNYCfsmNAtrU+olZJy3JrztPt4iRvbofyi6Ij80WeARfUu7TeIYeO29r+fN7HDfAX/4U2y7MYkuLY 6YbnfG5Mb+Tw0eZFQAZgBZGt1XlgwfYE+kaLLYuNSGgU6G5Hlun5lHiArinoEzLQOkDPSThsl/+G7 4wOSLlchMd3xUf2TABTks0WvSUiy8wNrvGcJyycBZCAatSNznbHRwSxWzQYMxZ1rfavYDTA0ZK34D ZQGZHr5g==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzH-0000000DN8e-3PgI; Thu, 05 Dec 2024 17:50:03 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Alex Shi Subject: [PATCH v8 13/21] mm/zsmalloc: convert location_to_obj() to take zpdesc Date: Thu, 5 Dec 2024 17:49:50 +0000 Message-ID: <20241205175000.3187069-14-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: CC25A1C0008 X-Stat-Signature: k9qzhx5w1op6u8xymjozx5qcoyorc7dt X-Rspam-User: X-HE-Tag: 1733420997-523985 X-HE-Meta: U2FsdGVkX18mBwIWbXh+eEYhMidhZ+8sptq89jtTzQDwV1G73fCkA8u/dAbuqFs20WyAYWn5zCK+efX74ygSLkD2FORUoKDSDMSf6RH4jjnOi/7bSq1c1YnsJX3LK9Qu92ThqC79IuNP7OGPFfATaxMFcec9S4zbm7+MNBJBB92AeNoLyzP5EP9JjVLIdODNJDAu7Nnk7QbeT2yE3LKwX/u53xLYVaJeDkto1daQeNhlHmbh0K1vkNsHctOs44jJSVP7ANkA1mCRAOre41XxBm7JIOlFKSg9qtt+Ch9cm+BlcMVHtkdpIt0GQpexfxUNANeJquGyrAJpVcHlhhPNKjN1Ubq4jXV3IWE1HKlCR2sdTegm4k9ArrEOgDfI37R7Far2kMImEy2+B2hHkfdxjAPCQI06sZHiwLYjuubAPJBsyrgqiLQvkxMU2JlvsDE5le9/tJuqa7mmZrZGCcKweatOdwWUhVEQsg+XiPZdqxWPcgGN3F55ne+4DsF9449nfh6YXSIyTHuWGCvyUaEWXSjFWFIWMYpb52toGoOcUDsdE/gfYhXnMZ9LcBjtv36mWEcAUrE6xt2zTAvdgtWf9opUhEqEGnJmof5zw6BV6/M+SPTXr3DscCYxXS0EJVoKk3jdw0sOvB7G1SRwKxy4urE13Gz3eRr8zrWs7asfdqh7WQFT7MJIYCSr90BRElsbOPY14Uhr6bfGQGZ/6mN06JHvy+UfloSiAPrKgQsuMdtqe72KE8dRRhJvjCkMwhG3ZHz7aXsy1P6DLKu744gjpD2xHuJbt0eEsUeEubnmoXPs85dpgfO8JM+MiL/NRzou9WyqA/k0W1vztN0prkHcW2c/ZucKxefJi7r0oKxfbBuFlYSLpUruLCA7m6hIpv/T529MS9ct5qcnX53n5zUKS6KLk2JNa5gB0Gjw4Z+PRYNMgy8jDOg+iHqnBO4fjSggtc6nfAtW0kwyQU9yMOK h1aFdiq3 xPJRfmXsA7naSaBvx4VUESOeDZKHprivDly0BdtKQsqGWbE8Wl8XT910irOftF6Ih/Dza+7lIHNFR7FTBckvabaOzlnSu2ffvZSIpMNou+Nbv3PqR083a+W/6G56txInwQt7YTXKe7VlCIu7zUC0RmBbgovvXzyb3Sce0pd0r6H/PFMDwoOYtPN315Zy9ggBktSgU9EH+w/RjM+qhU2Spiz4ortZahYWbwAFcDEFHQFM9wzxX3DpBPWHu6BjlDpSRnlTBfaOuJxkESPfG2H3tGpQqcOjBvRUCDvGz99zBgsZrNHz/+leUv0OYC54SIl8lfFAYAwSUGYOh6VcS1esPYXwEKw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> As all users of location_to_obj() now use zpdesc, convert location_to_obj() to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 2932351f4698..2960258c9cda 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -822,15 +822,15 @@ static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) } /** - * location_to_obj - get obj value encoded from (, ) - * @page: page object resides in zspage + * location_to_obj - get obj value encoded from (, ) + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static unsigned long location_to_obj(struct page *page, unsigned int obj_idx) +static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int obj_idx) { unsigned long obj; - obj = page_to_pfn(page) << OBJ_INDEX_BITS; + obj = zpdesc_pfn(zpdesc) << OBJ_INDEX_BITS; obj |= obj_idx & OBJ_INDEX_MASK; return obj; @@ -1377,7 +1377,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_local(vaddr); mod_zspage_inuse(zspage, 1); - obj = location_to_obj(zpdesc_page(m_zpdesc), obj); + obj = location_to_obj(m_zpdesc, obj); record_obj(handle, obj); return obj; @@ -1864,8 +1864,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj = (unsigned long)location_to_obj(zpdesc_page(newzpdesc), - obj_idx); + new_obj = (unsigned long)location_to_obj(newzpdesc, obj_idx); record_obj(handle, new_obj); } } From patchwork Thu Dec 5 17:49:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895899 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D467AE77179 for ; Thu, 5 Dec 2024 17:50:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 638DA6B017E; Thu, 5 Dec 2024 12:50:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8D1376B0180; Thu, 5 Dec 2024 12:50:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 132956B0175; Thu, 5 Dec 2024 12:50:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 00CB16B0173 for ; Thu, 5 Dec 2024 12:50:07 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id AC377A15AC for ; Thu, 5 Dec 2024 17:50:06 +0000 (UTC) X-FDA: 82861643508.30.C59B97C Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf07.hostedemail.com (Postfix) with ESMTP id CEC804000C for ; Thu, 5 Dec 2024 17:49:48 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=lrzOXJaO; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420994; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=f9iCiFRqV3HwmoRr2YS5Im3FVGBNrODuI7pmeIvlk8g=; b=D5UlXoS2LJEN1gDhAn0OLZqPrxv8RcJMlTfTwuTv6cKvnOmyUxYV6ttmC6NYBzm6XVtZpU No5ZoCW/XTOIPTLQEouufiO/ZDUYTtAs8MEged3Ni8ZbhYLac3UykrRJrUc8W8Oyc7Ddu8 SQEyFPD9XbsmBkjof+DJ1LU9rv77Z5k= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420994; a=rsa-sha256; cv=none; b=TuAOVSWPTNJBIq4B8MvTY5Ji51W4/vcmcK66K+SPmmRzL2OEQ7xKDdQbHmBZhLE/bBonVs kXgIbwonpQM10wuumLCCmnUbQfnx1UdltVo10C1tmKXp5jT8g3nbBuXUgDYV8tF8bJeRBf VzJ65r7ujeH1Nn0isBoheMD8qZyBojg= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=lrzOXJaO; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=f9iCiFRqV3HwmoRr2YS5Im3FVGBNrODuI7pmeIvlk8g=; b=lrzOXJaOEbmyRI2EPx98aASDUo rBwBePTubdOlgckSeyIlvKOj1rUkJJvBQ9N82hMfC2tj2gJRdH47d+E4otZv9DbbBMETkXL2dhKy1 ElZeLTMDSCQb+Kbm1EeaRZiIi4hqqmJuFCD3HwqsXAToWHyRvWguZ+SUlbjOPNuE/4XH2z60wTgjk YBeg8XVh1bkWOGGg/dKB0lqKxR+NuZm8zqhr1zpbDO7UV0LBBy/tXRV+ZcZNUfAn/HbYyPqTHtGc9 iJBOXSHInQ2hHjHXGufPNAW7L5Etw7NHSggYtvKJeS+XXeurCYGAliY9zTuZY3IJqEJ/RyWeoxZI9 OrmY+2bQ==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzH-0000000DN8m-3wci; Thu, 05 Dec 2024 17:50:03 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Alex Shi Subject: [PATCH v8 14/21] mm/zsmalloc: convert migrate_zspage() to use zpdesc Date: Thu, 5 Dec 2024 17:49:51 +0000 Message-ID: <20241205175000.3187069-15-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: CEC804000C X-Stat-Signature: mbounycjgm5hjhff516yx6yokswgcz1x X-Rspam-User: X-HE-Tag: 1733420988-717085 X-HE-Meta: U2FsdGVkX1/0hTQHJDOjbXHJ1d/KoB1kVI0o/RUXIm50yIIPfYsrXBigB/ya9c5a8KGs285+wMcVaOyRe2zJrPMTdDNQl3KeW19iWDKrh1/trbLE3jGXZTfIHKohn4MnYWbBwBO9qq9rFsUyUZIwG+5Ex3Oc7D7cVqznyVHhN4/a8ruifSFSGhYt76PIPfqxcVWbGf0aiYKIjZDJFnQEQ1ynzf0tTmCIPeZPSAC23DFc2wV0frOOqapoX22kXlNeEF84Ngk0ThPsbx2tQSTQLZI8uf8RA458aPqVPOpB66ppVJkWs76fghnfPGqxLJ4p2mp1l9W5DZsYusYpryvv6Sf9X7qsXC1zj8jN/MAxbNCosk/hkUV8jRNn5/X7OKPBuF7idYPlBRRN0LPnMGH5FfcwYXCqP78KFyuH7uGuMoJy/Uhr+X/VKOTR+nuUbine2hJH5jiAYqQDDIJY6oRY00y0rTR+fC57SHc/eSekmj3PAOfe/sGbcn33nDrmgd2XUSoFqotiRvoNzgsNj0NWHUbVnBiQOK1sVDOvlZ1NnnFUAcPAAm7AnB8F+Y3/LRehTX9/kjKvDT1nLTg8tGOx+G208A7sslD+tsA8Y0Zbl94u+3oX8h+hKNp4MVKyus6RtlIU1BnnORacD9DfnR514ocMWL52kPC2v0HUS69fMGjC5Q76zkXzwq+sKekz9oyneOTuEjZkUrvSWT9DMNgGfjE6IvcE6z9IhYT4TmpznN3PG7Qi+qdnMf1pTka0yJ3jWIkI5FKDwVbamkLlJNDLoDARzQmQYSLcd2/ccXUL2ofgMgJL1rDKnfJ2cn7UMnEbOdWeBHQBNf9mQ8JG4cmqUgaKv9WYj5MQ6R6QiVs/paaauRp1QcceXZ/1HTueN3XGuLmdxNmRz6Dk/RGgMJ1EhZWIJc+IwtFcd1yAoeLT5q3cNcu8ccxJwsgev4l3onTH1G7A5VmFwbJkRG50i1W Yn8hzNhS NeOS8akMBb7VXtw0BLhELnxmB/b9KTCRSf7duZWlKdsigKBbHB2NXZn9p3JocJq6WgywoHZBb7b5A2DJc17/WI7uW0YEVT2zlxw/rsCX1xgiXndwF6GyqTstLYQusFPOhoKwZ+pG4t8OmeCkerXjt0IYMOYSOHtAb0IL82DJ17ivhiIm3w/8Vjq1kDzw8iTjLx/o7bpuByidv8wm8ZcLD+wRJd+vH61fro+eRT7GfNPtrl15/1YkRIDd7UQ+1jqQUNmqYkPoz0tnnYdsld05OJSQrOJfTdELu5aVCR7u+KVoHOa+Gng5CYl5IYRW2nv96nSF7HQJmRRNsFkuskUcM2E5T4Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first/next_page. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 2960258c9cda..6b05d0a47ed7 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1621,14 +1621,14 @@ static void migrate_zspage(struct zs_pool *pool, struct zspage *src_zspage, unsigned long used_obj, free_obj; unsigned long handle; int obj_idx = 0; - struct page *s_page = get_first_page(src_zspage); + struct zpdesc *s_zpdesc = get_first_zpdesc(src_zspage); struct size_class *class = pool->size_class[src_zspage->class]; while (1) { - handle = find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); + handle = find_alloced_obj(class, s_zpdesc, &obj_idx); if (!handle) { - s_page = get_next_page(s_page); - if (!s_page) + s_zpdesc = get_next_zpdesc(s_zpdesc); + if (!s_zpdesc) break; obj_idx = 0; continue; From patchwork Thu Dec 5 17:49:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895933 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A1D7E7716E for ; Thu, 5 Dec 2024 18:29:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BB4B86B0096; Thu, 5 Dec 2024 13:29:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B63C66B018D; Thu, 5 Dec 2024 13:29:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A523F6B00C1; Thu, 5 Dec 2024 13:29:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 8866E6B018D for ; Thu, 5 Dec 2024 13:29:14 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 2F7D31216D3 for ; Thu, 5 Dec 2024 18:29:14 +0000 (UTC) X-FDA: 82861742166.12.FF1355F Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf29.hostedemail.com (Postfix) with ESMTP id 904DA120013 for ; Thu, 5 Dec 2024 18:28:48 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="cr/QlDlt"; dmarc=none; spf=none (imf29.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733423337; a=rsa-sha256; cv=none; b=cjY26ygJU0eH8R3yWvKChaNEu1n81JElhxRSBo+hqRU2ApCZqbM8isjdOcDB12NULXc8VQ KDuMlnwAybrJ46RndhdDhX8rJq4M4ZhCgXZo56HDWWPiZudUTSMG9n3sRWUKEJBu0uv7rI ab4zCGDaHrD60SPo8yNchPnAzz/d3ug= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="cr/QlDlt"; dmarc=none; spf=none (imf29.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733423337; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vjWgaQhdgIZ5U15RH/nX3b3vESf852BAXj6Bv+8+rfA=; b=gABR2uZMLloyFKxRx/ZwuVvsYeXmcfA6EoBPUWf2HzxazR27PcpO+YAnloWdUB6rKHo86S dnwvzVCgaSy4Oep4wv+XSiDYPqYBoYSiIR9cscPBt7WLOxJNEE6H2Vx6VrrmRFuRjlkSMg i3KYipKxgBEy3coI7Gnawz3DHsPDq/0= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=vjWgaQhdgIZ5U15RH/nX3b3vESf852BAXj6Bv+8+rfA=; b=cr/QlDltOCHAX3DUuQFK+WrNTe iex8wr8HcWU7zyno4zBHkVLHvi+w/IcwvQCrds1+8OX7r5k+n+r8n4tYtL4T/yc/01GknxmVNuBtY C/IZQHX7Q639QFnlg+JfEqJfyqq3ZOZN8n3KMrtGTgf1hgXM2pX6BU5XMp7q0butHkKRiSNZMNeGA iIKRt7fRPjoctniZdv/rFiNnedcs425OXSFUzocKfKcigf6FIpfCC/3NzVOCe7nIny8kqwXQQMxWx z22+Rtsfbqy32B49TQ49Ds/b87dOwV7HcNUpLRpnwZcSIcLaNFMKgD0kL3B2vShwbPBYCH9W+xj7X rI00S6JA==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzI-0000000DN8s-0DQH; Thu, 05 Dec 2024 17:50:04 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Alex Shi Subject: [PATCH v8 15/21] mm/zsmalloc: convert get_zspage() to take zpdesc Date: Thu, 5 Dec 2024 17:49:52 +0000 Message-ID: <20241205175000.3187069-16-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Stat-Signature: bg9fcwjh5hnbmc7g3y5x6gqa797cz3x4 X-Rspam-User: X-Rspamd-Queue-Id: 904DA120013 X-Rspamd-Server: rspam08 X-HE-Tag: 1733423328-868654 X-HE-Meta: U2FsdGVkX18v0A/WSNej7zEEhaTI9iUtmeAiYxrqfsWoOFT+f9f3VXJbzRV30ai3pF3eLDjjQFVEE/LHlgm9rOCdclzusz4HJFfzjV/eNvuLCewuqZkLFpJeNB6bibyHC5VMtHxhSh+5otK6CADF/pBPSPDXYsIm0dGmVapW7FwWHGNSF7vZJqqvCWxi995D5Y8aY1fL9n/nPFokSVBY83hpIig7U1Xm7+dvPUb+RPa23/6qovIyaYP1RVXRQZokhWdGXKcy2SBGQO7Gp2zKhiNDu3sEWZ4OPkwryJrs4Lakocc6DB27o4FN8ToPhBssS20Y80f0iV/OoY8xvv3IBZmoz4+hPEd1sMHuu9tZRKirMqusjnVd4zWw8APWuZ8lFD1ifLmzY3lFkjyDWjgFhk2OIC6KpJGeIwT4tq64wLZnuOVDlf4CAH+X7OSMQsE9iOkXHYKtzaBa2O/mr0J/K/HUfeMO3O3HnNG3LXbgnhyzRkD5DY0qD+oiLqFU6sTHI1BFd6PxflJ4yqscp192o6o0r43lGeNusecpm5b3eTlo6LKAY6eTZ8Su6bX3N5QtDr4k43pzkE5QCbCa00H9jo0ze2Usd9rtutWlsIKwTAX4AbJepQotXwgOCcsEeYicHPGLZogF7io0wwA/F5e0Is7XRf8CT2S3sY2qyesnReiQ1jTMeAFhMSU5Hfv5cnG6OoVG658fJqNzsS0M5Iv/5zk0FY5WqBJtGgVKk8kQ8pc9LpbwXrDK8kNTGrVCdyZhGJPQF2d+z9Y7kTm3OZeXFNCbZ59vVd1M07oVgaAcapALBOl8+0GBG/taP6lfcjxxvPYm6VIizaq6HERtCgfRZSyWBIDVTe8jk8ZGR2me5FcbQjGGdDmZl1GRxTfHTosL1oONZnsrPahI/nkqnqR69BGijMeD1mqhDyu5bfRNVgnLQuMfkt1BpnSUnbo2Sp1LcT+Im8LiSwwRkE+cbVB 6VCQAuQg UNkK4B5qga0F3uuW0AN3vnAMflxo4e95jaKKm1QG8fE9O7pwi5kXQ/d0Uuae1f1sR7cX7sDdHFmkMukVBkxfkaPj/Jsg+mSvy/xhTPCTcsOx0dC4/3Q7RBhLBGIkWx3OuR8ISL/g3hWThEs1lGYpsvcAtF53IyVAKGnnZdLUGc9O2NmlHLu6a2RNWF+KuPWSXFPUMVr3YGQg4peI3jzcQzCAHqGPAnF1rrQUXIUU4aubTZ4oIi7Nrd8ONx1EWnMSczEgsMkRXd6tNn09QKXFWqwdfCMgJAEwClzB/g8+/qwAUisQadUOSh13q+BSXcQCL4vHW2i5tjZWnCOk11C1oT13cQA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Now that all users except get_next_page() (which will be removed in later patch) use zpdesc, convert get_zspage() to take zpdesc instead of page. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 6b05d0a47ed7..bf1db07c58cf 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -775,9 +775,9 @@ static int fix_fullness_group(struct size_class *class, struct zspage *zspage) return newfg; } -static struct zspage *get_zspage(struct page *page) +static struct zspage *get_zspage(struct zpdesc *zpdesc) { - struct zspage *zspage = (struct zspage *)page_private(page); + struct zspage *zspage = zpdesc->zspage; BUG_ON(zspage->magic != ZSPAGE_MAGIC); return zspage; @@ -785,7 +785,7 @@ static struct zspage *get_zspage(struct page *page) static struct page *get_next_page(struct page *page) { - struct zspage *zspage = get_zspage(page); + struct zspage *zspage = get_zspage(page_zpdesc(page)); if (unlikely(ZsHugePage(zspage))) return NULL; @@ -795,7 +795,7 @@ static struct page *get_next_page(struct page *page) static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { - struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage = get_zspage(zpdesc); if (unlikely(ZsHugePage(zspage))) return NULL; @@ -845,7 +845,7 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage = get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage = get_zspage(zpdesc); if (unlikely(ZsHugePage(zspage))) { VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); @@ -1250,7 +1250,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1300,7 +1300,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) obj = handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); class = zspage_class(pool, zspage); off = offset_in_page(class->size * obj_idx); @@ -1464,7 +1464,7 @@ static void obj_free(int class_size, unsigned long obj) obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); - zspage = get_zspage(zpdesc_page(f_zpdesc)); + zspage = get_zspage(f_zpdesc); vaddr = kmap_local_zpdesc(f_zpdesc); link = (struct link_free *)(vaddr + f_offset); @@ -1498,7 +1498,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle) read_lock(&pool->migrate_lock); obj = handle_to_obj(handle); obj_to_zpdesc(obj, &f_zpdesc); - zspage = get_zspage(zpdesc_page(f_zpdesc)); + zspage = get_zspage(f_zpdesc); class = zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); @@ -1831,7 +1831,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, __SetPageZsmalloc(zpdesc_page(newzpdesc)); /* The page is locked, so this pointer must remain valid */ - zspage = get_zspage(zpdesc_page(zpdesc)); + zspage = get_zspage(zpdesc); pool = zspage->pool; /* From patchwork Thu Dec 5 17:49:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895939 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF8D6E77170 for ; Thu, 5 Dec 2024 18:49:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1D7CE6B0159; Thu, 5 Dec 2024 13:49:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 186986B015A; Thu, 5 Dec 2024 13:49:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 04E936B0174; Thu, 5 Dec 2024 13:49:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id DB42B6B0159 for ; Thu, 5 Dec 2024 13:49:13 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 45097426B4 for ; Thu, 5 Dec 2024 18:49:13 +0000 (UTC) X-FDA: 82861792650.11.29996D7 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf27.hostedemail.com (Postfix) with ESMTP id 4679240005 for ; Thu, 5 Dec 2024 18:48:52 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=juIPpL6F; spf=none (imf27.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733424535; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+0KpkIbIdWpRoLKWbvLyo2kOfKZAUmAJeeYsmD3sRK0=; b=kw91/QL7xKKBg5akLjrNErVbGfl3vMKab8QmFbp+non5paeY8cnfHhBYgLOt4G9vdLFE1o xlfiPloCDf6nxBKrDuygoEi79TDmpLN/Wdo16RH406tGyULZLlftpl5QdJ7ld4tYVlqNVC pXFwQQEOyUfadGjFO9SK1Y3VA/GWeg8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733424535; a=rsa-sha256; cv=none; b=0lIUWWCKD/eHOJCFiU+Cm+IADHZk5dL98xywRnBy80FoqTm0znziiz98pe3qw/xgYXi8b4 5XjrIHNiePeCQHm1ViM/I4wbNUaecuDJAFti/3Ddc4XtSiRKT/5Zbwmkg8Yt6mCWa2Nky9 LJ9TrgJ+9YmA5TaI/KWatKnNsWghu44= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=juIPpL6F; spf=none (imf27.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=+0KpkIbIdWpRoLKWbvLyo2kOfKZAUmAJeeYsmD3sRK0=; b=juIPpL6FcQ2xiieMijXu7M+ceA jgfy68Re2EzSIkjZRm362NZ79vqg/6NGp7HhOuJ2xzVMgxj3GE3cF7QVY/Pi7537msf2Es/aJVPIX +RLW8iSQgvBzf+Dkvy7UrLB83QzloseQXh5hSw4vlDJyzeY+AskUYupf1Sn+TvaaMUqMm75gkCq4X Q22lcwpyyc50u4Ga5IEavMpFupujSB/fCKJ1DNY5O8BTol18qyEWuGlNHTdHPVQmgshu+0Kcc/e+n 9z7AMH67u7Cua+7kfwbbuDQO8CPfctThshPm9ZVMZXMDFjIIrvNj7fwYqIGzVkSgKKDgD3Y6/ImjF 18lBuadQ==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzI-0000000DN8z-0cBP; Thu, 05 Dec 2024 17:50:04 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Alex Shi , linux-mm@kvack.org, Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [PATCH v8 16/21] mm/zsmalloc: convert SetZsPageMovable and remove unused funcs Date: Thu, 5 Dec 2024 17:49:53 +0000 Message-ID: <20241205175000.3187069-17-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4679240005 X-Rspam-User: X-Rspamd-Server: rspam07 X-Stat-Signature: 9nzhgsjrrsyzm3eahzcfkd8j6jbw4hti X-HE-Tag: 1733424532-843413 X-HE-Meta: U2FsdGVkX191alRBwWIa/RLyqsT9o4OsgDpYuRZiQvb9uQ219G4sVrQivS2GblSYNx1D7Cz5NE8r8szzAt+k66owFSK9rxwCcef/P/8DcAu8wV/asn1DQsW922BdZ09wePBTlmULJPu0eBrfeo1A/pLsL/UKs321F5iVbju56/K1KWdc10OJs3GJTc7QY7h+rAhLrkwUQrB+/IkQd0kU3128b4Taq5+URk0cioNfc/p8vMDWv+yN+T0lyUAEKcUL07OPChd7TSxi+Lb45bZyNvzyl/YCGDI+HAKljBAwg/PxZHJVBR/htmjVfBBd/8QfwytGGgk2YSWnMQRcar9smf2Vrj3RyPz8d2l0MMTVTbQTZjN+F6GpJrQ6SKB5Qtpu6pqN0FjZQsUS1suhEpG+u5VttjYwO0vq8nRxR40IuzD1+1D4xLho4nvMu4nC4d8IspIOSPfplhD3WGbonCsSbbskiTjNKTGSLn5YgeL8Qx8K12dj5eGdpgGPyrfOs6emlkwc75+yB43EJ+LLzqH3mivlgAdJt9ppmoxndHSauYWlYWoicGIY30KhLqOVw0RkMxTkgSmSMCAtbsF+LpVfB83KT2W04AIgFMDm2YuBsZ+4U0cf1gFm1tASjq9q3l30sj/ZijzDYHANSK0YzNAbqddEje5bNT2NyUGNwSK2BK3PHhj9XeIPkqC5and1mRSwWo8EEOPhUZPIrtqEptZVQVPH/sd7FrSKPLr3TcXQS0gSj4a4YjYPs0Vv/ePTgD7bbvcTz1GvapLvHwZ4UoCsWvoYaaev4ZwGHovi6ZnsyP26n9wtB6xD8uWZhzfqJesESL8L9q231SlVZ8oNTrpZ8RyQwohjP5RjZP0upHC2VEHdyha+mzYIFSpnh8szQKOIYQjHnIqzMz+FwQTrJpniLYu8NIKnsE5IwiScnY9C7gZKHQMFfVfHS3s0eHaYQxORuH3bpWwEYEADzTKkbN/ sm0ONyXY ytP/epMqVlJjS2/KWVmvSmuiUF7SZAGeKsNV4JR6Ijev5mnmcHRtVxpLa9HvaAyjTCPKJk+xPfcGQxkDVx98U5RUQegs+v4PL3tY+INvfr1okzy8ULNeONDGF0/PBzLjxRYHreFMsZLgWzMDqzbOkq1dptk3ZgnxBZMxoKFw8QsIhGJ+tF9bq0+MXsPCTIGjPYJ22rj/vavlSZsMh8WRulBU0Co/8EC686gLrb+6JKrXdFME3vQzmggeGQKcPBEBXjpeW/WXZxcj+X3gqG7noezUBjNaguMPBYW9MhQz1QHcy6LM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Convert SetZsPageMovable() to use zpdesc, and then remove unused funcs: get_next_page()/get_first_page()/is_first_page(). Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 33 +++++---------------------------- 1 file changed, 5 insertions(+), 28 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index bf1db07c58cf..16e3d6705563 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -470,11 +470,6 @@ static DEFINE_PER_CPU(struct mapping_area, zs_map_area) = { .lock = INIT_LOCAL_LOCK(lock), }; -static __maybe_unused int is_first_page(struct page *page) -{ - return PagePrivate(page); -} - static inline bool is_first_zpdesc(struct zpdesc *zpdesc) { return PagePrivate(zpdesc_page(zpdesc)); @@ -491,14 +486,6 @@ static inline void mod_zspage_inuse(struct zspage *zspage, int val) zspage->inuse += val; } -static inline struct page *get_first_page(struct zspage *zspage) -{ - struct page *first_page = zpdesc_page(zspage->first_zpdesc); - - VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); - return first_page; -} - static struct zpdesc *get_first_zpdesc(struct zspage *zspage) { struct zpdesc *first_zpdesc = zspage->first_zpdesc; @@ -783,16 +770,6 @@ static struct zspage *get_zspage(struct zpdesc *zpdesc) return zspage; } -static struct page *get_next_page(struct page *page) -{ - struct zspage *zspage = get_zspage(page_zpdesc(page)); - - if (unlikely(ZsHugePage(zspage))) - return NULL; - - return (struct page *)page->index; -} - static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { struct zspage *zspage = get_zspage(zpdesc); @@ -1955,13 +1932,13 @@ static void init_deferred_free(struct zs_pool *pool) static void SetZsPageMovable(struct zs_pool *pool, struct zspage *zspage) { - struct page *page = get_first_page(zspage); + struct zpdesc *zpdesc = get_first_zpdesc(zspage); do { - WARN_ON(!trylock_page(page)); - __SetPageMovable(page, &zsmalloc_mops); - unlock_page(page); - } while ((page = get_next_page(page)) != NULL); + WARN_ON(!zpdesc_trylock(zpdesc)); + __zpdesc_set_movable(zpdesc, &zsmalloc_mops); + zpdesc_unlock(zpdesc); + } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); } #else static inline void zs_flush_migration(struct zs_pool *pool) { } From patchwork Thu Dec 5 17:49:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895930 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50D70E7716E for ; Thu, 5 Dec 2024 18:09:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7D05B6B013A; Thu, 5 Dec 2024 13:09:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 780386B014A; Thu, 5 Dec 2024 13:09:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 647D86B014B; Thu, 5 Dec 2024 13:09:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 43B006B013A for ; Thu, 5 Dec 2024 13:09:13 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D99C61416D2 for ; Thu, 5 Dec 2024 18:09:12 +0000 (UTC) X-FDA: 82861691640.30.A355041 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf18.hostedemail.com (Postfix) with ESMTP id C68791C0007 for ; Thu, 5 Dec 2024 18:09:03 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=iXP5YhWZ; spf=none (imf18.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733422136; a=rsa-sha256; cv=none; b=ZJkg0GICOTAyM99q8MGpuhvcGrkHDPrPecDa2nD7kOcZv8jAxb4tHXJo59vTyAcjWhE6o3 STLygAq8qutqBFplKVKKVp0i1OzNhHso+zA7ie8P+BLIfTKqtR0ilAYNq085J4sUuq3Dkx nMk995wFG7UPkugv6jAB+/v/GiYclHo= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=iXP5YhWZ; spf=none (imf18.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733422136; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7a3VGducrMVXQ9/iNeGwmenmfpj1ag31Du6oIOBWz8g=; b=eJEx2m1ndDeqhIFxx7Pg0vikqG2i6QxWyu3Sip2qxQwsRNVCorG0wVQvMIJJEXuNxNpYsl diXKEh5G9UnuQLZxfkWNDa7acy7F/ghl9pzyxDc7oZFd55bHOrexjsCDbBhBDlKLrgdOo7 m2aWA3JyraP/mpKXIk2G5yE0R//a8WU= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=7a3VGducrMVXQ9/iNeGwmenmfpj1ag31Du6oIOBWz8g=; b=iXP5YhWZzbxSYWZ7PtRqxOtzVl oT86dz9cIxtF9tjbyO0EDZxeGZXqZ8Drjj9Ws3bDD4ylhQJ6cEc3fpqK2cZXi3hMUfqIod0vOiVq0 5VTVxRn3Aq84wonM2UuA068DW5B6kjUn/211I2SdU/dzED4EeU89PE3BautwLLMrfYz5hQfaBGhBa 1o+5L9d+OhMfqtQlPNszdB7vMP5LEKfHyziWmk7ft5MTO7Mqx28LjArPN19bJIYg7pZvu8uWW+Tgr R4UV1UVtST7SPdUwEFXqNsSPovN7cjS3kK0qSRyZPQJtz9kyyeSGiEhrIav3JXLc7u1NsVg5sI02r Pae9TZaw==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzI-0000000DN95-147l; Thu, 05 Dec 2024 17:50:04 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Alex Shi , linux-mm@kvack.org, Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [PATCH v8 17/21] mm/zsmalloc: convert get/set_first_obj_offset() to take zpdesc Date: Thu, 5 Dec 2024 17:49:54 +0000 Message-ID: <20241205175000.3187069-18-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: C68791C0007 X-Stat-Signature: 88jskarg3xqscmdk8qmkum9ra1iaci33 X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1733422143-779078 X-HE-Meta: U2FsdGVkX19/yogg30lzyYJLWLpBGR4RWIaGArNMbGTmIJpNq99n2S80OV0wIUPCaq1ONt0y9W6oSefCZodIJGy9f7fH50HNzFx4//0AsuFfUoxWGeBEs6cq1sgIaNjpJmD/hLRe7KRRcsJ5cjytzGW5VTh8yNIVRa57PBwNSDvq85qw8/Ov31UwCmY9z2YAPJTMXFYXxXLM9nsl3qCakGaEy1cpX7e7vJBexDMhaHVIYmXdBkRihT/o2/B0QkJ72CwWBngEjix6bLmflZc2jJ7PjTXckzN6ajOagMir18Gq3pFRp78CKfwFTcU8bJ2z0q+syPvMjhegHpFih2TSrky5LxlofOaqrngF3CBd+bcefswCAZTLRpXyDYtMuBxkRd3VesQYJXFkWel23LQv62a7HmR9p0yIC9Ix9JxtVR6kqDwq1hXOJMM5TkawkbPffK0sIlPxkayCPKJIBBbWpNJxN0knHM/lCzkyP2QxBglcvo+/9ekQq302of4BrExHlyh4jlgIWHzVOFURhInGgEzBjvVoDTtMUsF5epx2QcJCJkyE9oBEpL2p1hkEtouzvetGQIvZgBzNz2rMGF0g70Ez4mRiNDB1TepYqYBuGiqHVlNY+uPb0sNNh7Ln9aUrxZZbmKGWR4ZlRWRAAqMYzCwhu/9p+LnhDeCpildfZX5n9Wkcyav8RCg2cTGMde8FzN/wR8N5ilvIeR0RTxfEuXO+ImKlD02DxlOznH7wuDzHUc5WTueV0PGW2ovpTDDsczY5Wrh3TpMiv6LdBztmSDHV1IlifP4+ksQw31qkaH/9wWB7BOENGARd3mllTziTV6kp0MBm21HQ8/6FBASK4xlb9tC+EDWyRYI6I7bivOio6713eTUQNeJfsgmrGIEVTIknNAHuRpXVQh6pyaub/+jEuZnpRuYh3BceAwUC+iZvGqna3ZK16BU2qD/u+AXU7DVpd5gg12QvxeYkMH6 GT3ZmhLv +IN+ymxZSXfq23xFpF2anYpBGSD3oQWEboOkmD93hWF/7xvanoC/jdNL6PstlpEy4YAs2k1Fo1Mfe1Z9h6FZeNPB6u6sbB9a1qfXdVaaRoTSKoFrLUuFA/2H2Kk38+TvD1udwcZFq3aSFq1uXMApVjlOHVsjB1ot62o53hpWleku8NyVTE6TqH7dW1JXUWQXlHdaL47X/jDpPtiUIsOY2lYpapMiL8H8fJWhwYZbtKglFjg1Ia6FYd4/0mC1+s49F9WW9HWBLOlLsYxh2/eO7mBwQLb0i6dbbuKc67DyDDOSY+WttGvziU29Ny02YguTVE8L3HBH0zZC4hcd8A+UEm3DOWQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Now that all users of get/set_first_obj_offset() are converted to use zpdesc, convert them to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 16e3d6705563..a15bdcc25bb3 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -496,20 +496,20 @@ static struct zpdesc *get_first_zpdesc(struct zspage *zspage) #define FIRST_OBJ_PAGE_TYPE_MASK 0xffffff -static inline unsigned int get_first_obj_offset(struct page *page) +static inline unsigned int get_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - return page->page_type & FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + return zpdesc->first_obj_offset & FIRST_OBJ_PAGE_TYPE_MASK; } -static inline void set_first_obj_offset(struct page *page, unsigned int offset) +static inline void set_first_obj_offset(struct zpdesc *zpdesc, unsigned int offset) { - /* With 24 bits available, we can support offsets into 16 MiB pages. */ - BUILD_BUG_ON(PAGE_SIZE > SZ_16M); - VM_WARN_ON_ONCE(!PageZsmalloc(page)); + /* With 16 bit available, we can support offsets into 64 KiB pages. */ + BUILD_BUG_ON(PAGE_SIZE > SZ_64K); + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); VM_WARN_ON_ONCE(offset & ~FIRST_OBJ_PAGE_TYPE_MASK); - page->page_type &= ~FIRST_OBJ_PAGE_TYPE_MASK; - page->page_type |= offset & FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset &= ~FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset |= offset & FIRST_OBJ_PAGE_TYPE_MASK; } static inline unsigned int get_freeobj(struct zspage *zspage) @@ -929,7 +929,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) struct link_free *link; void *vaddr; - set_first_obj_offset(zpdesc_page(zpdesc), off); + set_first_obj_offset(zpdesc, off); vaddr = kmap_local_zpdesc(zpdesc); link = (struct link_free *)vaddr + off / sizeof(*link); @@ -1574,7 +1574,7 @@ static unsigned long find_alloced_obj(struct size_class *class, unsigned long handle = 0; void *addr = kmap_local_zpdesc(zpdesc); - offset = get_first_obj_offset(zpdesc_page(zpdesc)); + offset = get_first_obj_offset(zpdesc); offset += class->size * index; while (offset < PAGE_SIZE) { @@ -1769,8 +1769,8 @@ static void replace_sub_page(struct size_class *class, struct zspage *zspage, } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); create_page_chain(class, zspage, zpdescs); - first_obj_offset = get_first_obj_offset(zpdesc_page(oldzpdesc)); - set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); + first_obj_offset = get_first_obj_offset(oldzpdesc); + set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) newzpdesc->handle = oldzpdesc->handle; __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); @@ -1825,7 +1825,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); - offset = get_first_obj_offset(zpdesc_page(zpdesc)); + offset = get_first_obj_offset(zpdesc); s_addr = kmap_local_zpdesc(zpdesc); /* From patchwork Thu Dec 5 17:49:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895891 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47788E7716E for ; Thu, 5 Dec 2024 17:50:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 295026B017A; Thu, 5 Dec 2024 12:50:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 268B96B0180; Thu, 5 Dec 2024 12:50:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E08B76B017A; Thu, 5 Dec 2024 12:50:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9C7706B0173 for ; Thu, 5 Dec 2024 12:50:07 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 2C3971610EE for ; Thu, 5 Dec 2024 17:50:07 +0000 (UTC) X-FDA: 82861643424.13.F786C8D Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf27.hostedemail.com (Postfix) with ESMTP id 7040140015 for ; Thu, 5 Dec 2024 17:49:46 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=GwW7d49N; spf=none (imf27.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420995; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jRSmlyBZyfO/cN0lHKIx0H5Tn/myqOCAR0TLubJ1a9Q=; b=RoDM3/0pJWt3hUfdyi0fB37iIfC3RV1dYR+NW6VtuULCLzZP+zRJf1upk6rgGP/y0qtW5u Br6TdzV6tcEWm5S2B2zRSrU7/Bb8gnP6rf8RMN/+caghzi391jfUD6QaiqW3d2G75jgIM8 bv4BzKBCdn2Tl+C/nuXF0NCoqZBN11Q= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420995; a=rsa-sha256; cv=none; b=Fh9Oj+FdmMEJjzT5Ox0r4f8tKCQ5PwOirLA98Gg0gyFSLUMskVPxQafNgFLoCkhkLF7p3s i8XrcaFhPeDhAIAa9U1NA1iHjzlhAiL7kg2dPlYs3McM4N25CuHGEsf0m54O9jhgcoGtso 3EDiqdNh1jrW6zq5xlmIUIJAgvqMGDc= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=GwW7d49N; spf=none (imf27.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=jRSmlyBZyfO/cN0lHKIx0H5Tn/myqOCAR0TLubJ1a9Q=; b=GwW7d49Nk/niKHN5xontrmItfE X6n13nJ0cusvgCdwX37GztGvg9PxSepFTg/9KMtAgJLw+cKCPEDGo4oAqboQbb+vNye39zsvIbjvE XE7nB4vw+Z1Vo1GLToKhUsPyvhOYL3OIsGDNsVnj284UFLBPc4tdaMaMszjp2xQfTARzADo9qu4dE 2l7n4JEwMYUKNQdzqXNToWNAImY+C7mHrE2M0w89K8whugCPNBHhPn9akhA5BlPtVUGcXYIcDZ1IO hBupPDrmE4TXe5oM3MMuk3CR/KfPLAsz6WchYPHHrKyvcSUPGbyHNmwQ4Qz7bAmG8TP+brXJlyQK+ DgjUNOnQ==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzI-0000000DN9B-1VYx; Thu, 05 Dec 2024 17:50:04 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Alex Shi , linux-mm@kvack.org Subject: [PATCH v8 18/21] mm/zsmalloc: introduce __zpdesc_clear_movable Date: Thu, 5 Dec 2024 17:49:55 +0000 Message-ID: <20241205175000.3187069-19-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 7040140015 X-Stat-Signature: 8qhh3uxj9ha4q5yh8h3dhq77qfm5ztm1 X-Rspam-User: X-HE-Tag: 1733420986-715910 X-HE-Meta: U2FsdGVkX19PNir5ORnZA87RCN1ZJJw1V3L0QM3KBP05DzhL43OFUyQVWafCr1AgOjH6OQhLZSkoauW1acPGO3dp0x66tZ4Xt4QXbtdyV2WbxIJ57UhSzNVlPEWGPTPBf9LkGOadmDIkMnKiqG1Mlz1rkGNaL7yxTyZSDtPh7iT8lzhMycnX6stYFNY2eVoHN7AvyqpV/n1RUpgqkLNM4TOG46ReCR2wopps3JISQ5HBo39J37y879j/3ZOjvJdQXysTd3u/Lb9W+cd+ta+jWcvFW9jeIm+ru1nkrLJ33KjqL3c4+r50jcWhK/vmFXVUO4Upf4RsR/Zn60KKhVk218WeL7m8HDsZB80y/IRMj+BHaUOsbjpmw04GzEe7BuDFVeND6tP3aObvODMJCbUKv0rlZWe+rI90wTWmkVBf9F7rB54uwRBRBN11xmzMTZ8KImqihb7bJ2IvzZ/oDyYy+A3CWobexaz4yYyubWPLTO05A3j1vTbGpOLQhuTsJASHjLh2YyrUcj285+f1lDoZrHb1rNYTbgRepDro262cS+an7IKWsjRZZadfBBh3ik6G0nvnIPSVqlU48Yem4GFhJPUDq5ddHx7Ee31+Jzg2JYbv8oQSEbJbGRnYW2Io/5txH3COMEfSSkxISFdFtWNHMQM4NdxDUWQvpMiOJwWLP4+NHd3pExGm8SRZlaKS88cVONXUgILExmdsNUIj68QcNc2Hj7+T6nOHqrbpRD9nYId3YbuNmG1RlK9O6bY8JJKYPJxQtdGsk70/T9ko5ccTqnxP72elczNdpuhYm1l8ahjhsZyC8vNif+f+gBzIFoYERDF6JHbL1UF0rZUbyz1LnSTzoo5aGMTu/Kms5SzCZIo+pu364YRJYd4LoDj95hNUKcBP6N1b1q9pZzNOaAtYC/zXPZi8tdHLRGpVbdB5W/XUBjhMB7rgIYcKDPiRF8UZr+edO31Ai5UOEVcwe/h 86d5Np3K GViNpFdgw1sXpyDoSsWLVGezK6MikuC0U0nw38xI6RHajvv0RVP/rgyylFIA49ilSUfVVyVVsiXr+gQlE9WVWx6MLeyKp6BWzQ9AtdVCotyQBZbCpPhjFS5iPWk6VJZhAGu4yBJ8xK8fJ3WoJwBs8GDsvFVN1bZHqFINRkl7EbO1OhiF3ALJ3Yz4swi7FFwJ1ZmD45Q/3waNSjRR48o9P01M9g31XRalEZKwY46gPjIseAVdUmZLXyzXdkrdSMYToxmmr X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Add a helper __zpdesc_clear_movable() for __ClearPageMovable(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 89fe78a7f5ca..d468003c7ebd 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -117,6 +117,11 @@ static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, __SetPageMovable(zpdesc_page(zpdesc), mops); } +static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) +{ + __ClearPageMovable(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index a15bdcc25bb3..bcd5f1b9f382 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -842,7 +842,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) { struct page *page = zpdesc_page(zpdesc); - __ClearPageMovable(page); + __zpdesc_clear_movable(zpdesc); ClearPagePrivate(page); zpdesc->zspage = NULL; zpdesc->next = NULL; From patchwork Thu Dec 5 17:49:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895892 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B122E77170 for ; Thu, 5 Dec 2024 17:50:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 618B76B017B; Thu, 5 Dec 2024 12:50:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 48E956B0185; Thu, 5 Dec 2024 12:50:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1CEC06B0183; Thu, 5 Dec 2024 12:50:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A01576B0174 for ; Thu, 5 Dec 2024 12:50:07 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 568318163A for ; Thu, 5 Dec 2024 17:50:07 +0000 (UTC) X-FDA: 82861643550.30.6FFC33C Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf01.hostedemail.com (Postfix) with ESMTP id 8E9BE40011 for ; Thu, 5 Dec 2024 17:49:53 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="m/6qltxX"; spf=none (imf01.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420988; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=M0a99fmCVrOPq+PPvnzMWS5DYYkjS0DjuUk6shW5dpQ=; b=fs/A94eNm9GL2ihjXas61k7f/B57AF9j3Fhaj540jiltK1KNSw5evmtHLUvgNKmi6epwxH 4KuVhKIHWU2IkPBac0NQVNLTRnStrGkiWlW+iAP2JUM8JeFcCpcnRR6X3muVnW2ZhMOJSs FHiYxYLbaU/L2zFoOJp2U4DWr5UtNNc= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="m/6qltxX"; spf=none (imf01.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420988; a=rsa-sha256; cv=none; b=UQ6EH/hzdbLZTNJI3wpdemk073mJJTfHHzOe3EwPTWQp7jeqmYxrp8NNQNLEN7JNs3+gE7 VKVA7aRkESSDGl8/GeWcPIgvrkqzdIXbfIrxA79vv9h3n23ZROfgUSaWkKA0sdwQTN+0Wj uhhZzg1Ym4w9zMGIzDY8WYAaEQ4Nk28= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=M0a99fmCVrOPq+PPvnzMWS5DYYkjS0DjuUk6shW5dpQ=; b=m/6qltxXXAIe/Xxd1xNLYmLfif DN2syc/DlwdS5QCpdmTfmMwV3QcbyXP3RqxqRDJF+b13nrvDf+n7mXzlpTJZ83TREkfpF08qza8JW L9NFwlaQ/H0BRsKAIjwUqJAE+qPVOSKvrgQ8aPbL4LGZ9MIuTn+elaV3NFQoBqHTrT8XbEkVqis2H MlYR+fXx05iF2fRjJS4c5oABFG5keNS4nKGvc6VRKQZBFnE9Z71kuVMusi3J4qCivTMoAabP4KOha gb+phkT+m7LTZaV41Zzdhn2XT0DUcUX32zeQ5N8CSnAkwaMAtk9lVnV5YD0Y5rHZpqfRSTU/YRYs/ wiTvuMQA==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzI-0000000DN9J-1uM8; Thu, 05 Dec 2024 17:50:04 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Alex Shi , linux-mm@kvack.org Subject: [PATCH v8 19/21] mm/zsmalloc: introduce __zpdesc_clear/set_zsmalloc() Date: Thu, 5 Dec 2024 17:49:56 +0000 Message-ID: <20241205175000.3187069-20-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 8E9BE40011 X-Rspam-User: X-Stat-Signature: jm8e5uxs1g5s31j5o6de4ihi4rknpaxh X-HE-Tag: 1733420993-605937 X-HE-Meta: U2FsdGVkX18jeycoXYLmQSIwAYzCSV2XjvC9GgHcih9J55GwbPYzIJ8lJBNLjMd8ubij5YCCMdJZ2viMtCgp4cL4oJHOMaUYLwjcaYBdfUbJ4L1Zq+Qc23iRHrWTEJBWyDtupNOEbjT/I2UrSyh7DDGKyqSSToDeGMeIy8l6FwzVqR0HcfU7YpNdkeA4GHR1NhGfiHtOBCAmgK+2/y3ePnYQGCAOzl1LuNWI4lmBFikN2lGTVeI6pZCAlZQCes2AlqxxebNcbcEMJ8xon4yv4Q96y6HOk3qj0jwrP8VdGslkZC9OF/ujFQTkpSw+UUXcTHqv9smwhs7wtC1q8XE/Rz9mbKyxNU//par/0imCVU8uR+bVm3yERDm8rRaWQQbxkNhoNA/CJUz0YCwtaM84oCOO7/Nmw/BS0vHLzn/2wQHWJlOfl68txdxUS07z6+b+lTQYbmeSJ57plaJwsJTrdVESTfuUi7z36zz8Oyek8URG3/yX967fZZLA1g5eS7bOIWlVfe3x5Cc5tKhjt+FFGcjRD5uSvv50HQg2B/MUMvfglrcrsscUV01Qf0WIrXS1y80G8JGcC+OXCuasSDrm5/Xu65C/SlxQR1KPoNtvdVb8e6O1on0048oTcjzHVC6jv2rrbKG+0bVPpRHLe93n7SYFW7kxkLIiXwo3PnJW6+yBtjesQPz+UVMgH7x1/jenY+jQTOtjEkATujRy8qB2Cw0sSGoTUbTBjYHK5e1XNbc3encB8f8Ult/JxwWNZK3tTqpOZ4b5SgfQG9lCsubQj9qznysHsVUvN9HG9U+HvslkDUSqfUnEbuFfIP+2zHK9lz9GQ6FCbCXsZl024RHwP6hLCp6ltcP2KjEf/HB1n+XHQZGgDMC/BH9wAzYmrjgMn2L1AYadDJYPH/uaLaR3xtzFTQxtb+AktuQOXMMpMuVdRhbCjAB3U8RsZLsINi3+cOti2NFV4nwyvGztrw3 Ft0gysh2 o0wqL1y00ZE/+uWcTI4/RTGOWAaWA4/AEMz49edt4YZjP33pT8/HmeOvHTeULbOp2RwsaLKteHYib7hht2/wa8RFarh35BhZtJb0GCaHNRqyLvs55FbnFSnz8uBhM5TVdz8frDdrzH4XCii9CBinL31DKdpmxIsc2baWSHW2KJonbGEhJWGA86FqcdFVaPqeCWdpjfCXegWUdG+he1lE7lNM80ZNI4l15g2pHf/dfAidP+ckykfbbr7DR23UuysZNBYCY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Add helper __zpdesc_clear_zsmalloc() for __ClearPageZsmalloc(), __zpdesc_set_zsmalloc() for __SetPageZsmalloc(), and use them in callers. Signed-off-by: Alex Shi --- mm/zpdesc.h | 10 ++++++++++ mm/zsmalloc.c | 8 ++++---- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index d468003c7ebd..a09a63be929e 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -122,6 +122,16 @@ static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) __ClearPageMovable(zpdesc_page(zpdesc)); } +static inline void __zpdesc_set_zsmalloc(struct zpdesc *zpdesc) +{ + __SetPageZsmalloc(zpdesc_page(zpdesc)); +} + +static inline void __zpdesc_clear_zsmalloc(struct zpdesc *zpdesc) +{ + __ClearPageZsmalloc(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index bcd5f1b9f382..f9a9c2400e41 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -846,7 +846,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) ClearPagePrivate(page); zpdesc->zspage = NULL; zpdesc->next = NULL; - __ClearPageZsmalloc(page); + __zpdesc_clear_zsmalloc(zpdesc); } static int trylock_zspage(struct zspage *zspage) @@ -1019,13 +1019,13 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, if (!zpdesc) { while (--i >= 0) { zpdesc_dec_zone_page_state(zpdescs[i]); - __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + __zpdesc_clear_zsmalloc(zpdescs[i]); free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(zpdesc_page(zpdesc)); + __zpdesc_set_zsmalloc(zpdesc); zpdesc_inc_zone_page_state(zpdesc); zpdescs[i] = zpdesc; @@ -1805,7 +1805,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(zpdesc_page(newzpdesc)); + __zpdesc_set_zsmalloc(newzpdesc); /* The page is locked, so this pointer must remain valid */ zspage = get_zspage(zpdesc); From patchwork Thu Dec 5 17:49:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895893 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66023E77179 for ; Thu, 5 Dec 2024 17:50:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9956A6B0177; Thu, 5 Dec 2024 12:50:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 816CE6B0186; Thu, 5 Dec 2024 12:50:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 52D4C6B0177; Thu, 5 Dec 2024 12:50:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id ACF0C6B0177 for ; Thu, 5 Dec 2024 12:50:07 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6363F1C8696 for ; Thu, 5 Dec 2024 17:50:07 +0000 (UTC) X-FDA: 82861643508.21.B6ADA7A Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf03.hostedemail.com (Postfix) with ESMTP id 998C720015 for ; Thu, 5 Dec 2024 17:49:58 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=IBxU5un+; spf=none (imf03.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420995; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OZrTVTlx3nBqz9b5XE0au5WzgdiWje429f4/0vfxIOQ=; b=dV5/X/kPT2tnRjFXxsnte0M9bv6+NZpAy68rWtxv/Rwn/bEJ6LCZfFCA+YVJH8RY1CuRNB /S2yWUP9sSWNUceJsS/LQZDCHlSRbLV5o4p8vNhHybjHr2TMEs98qpreKiCHgl/0DUOdb2 1rmWwqqW4CxbqFitsa5CnafXgQdZS5U= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420995; a=rsa-sha256; cv=none; b=M6ku2eob0oppP6LWMrYtkmuGU1jNHkcw/SvYdKZen59UQAm/wQeweBjqTssw82pg8/Pptr vC8l960+9a3xi/LOsnde0mWJZUqTwAdYxaimlRURD6JIs4rWz+nFYEvTf8AXffhfQT5Y9e o04XhTEOniVRcZNLwmSvVM0+t2kvaQ4= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=IBxU5un+; spf=none (imf03.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=OZrTVTlx3nBqz9b5XE0au5WzgdiWje429f4/0vfxIOQ=; b=IBxU5un+UAvIX7arDihWj5LZ5R m4RqM4n2T0Sb+WZx7akVK1koE5KVkg447TDvTSihHKpPWazEFnCzG8/vU7UQXztHcvljO3/R4eJRu Fzt/oUENOPWDrGuWT6BXmCvpdwOkiE/iuBoQUXXxOKmLOUhQuApSNSG7z5WsKO/zCIFw9f8Nukma9 nZztN8KUVtbIcz1ILsjGRBBprqMNLmc2119sStJGHyBTU6Xdx4HIv43L5YWE4HoK2BvWw0QNcHjeA pbQRTJioY2DEDmnOak2iPnMWYz+rhX3yfDTNr48QBfUPfuPKjHFyvUXfld+vduiyFIAIAv/+Zc0Wj BWkAzVUw==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzI-0000000DN9R-2VVI; Thu, 05 Dec 2024 17:50:04 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Alex Shi , linux-mm@kvack.org Subject: [PATCH v8 20/21] mm/zsmalloc: introduce zpdesc_clear_first() helper Date: Thu, 5 Dec 2024 17:49:57 +0000 Message-ID: <20241205175000.3187069-21-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 998C720015 X-Stat-Signature: doakf91jgize1r53tfykz8tanm1zqbu8 X-Rspam-User: X-HE-Tag: 1733420998-237549 X-HE-Meta: U2FsdGVkX18HUaFlQUIANSEvt3NFUsRAHCg/fgUDpMx+4rHsvfm5gBW5VREaLs9IGc6H7HRSH7Oxy2r0vu1BCvw+Pr93QUDvBVbYJftMPsuwpQ8ZMSrst94E9+7z/grt2jaxVDruDBMDQolcph37J8a5LqxM/tZq0repn/3a2C53KeTaPpTXevXXi0e0i8Boou3DdRdzy0ZgzyPKP+vTyq99pIkq8I7XD/ahIIvwaH5u2Vm9WKowSZeoxzEbqn15EZEF9HIOsCehpR9NffZBmT4MUGI0itxixniaIdUfntGf4SvZotblkRRHB9m4r6Xu8/tEtVM7pGqXmveqEbZSnjxumx+UZonZwTFIlaYOTM7RUqX70bh2uIZ3eV/vaYDbvb3eGMfbpKjQ9hJHZqQLRJr7duuD1fVFkRexFwYN6YIwlWVElxqWCkJpPKqCCy40rfmxKa7MVAOD7ZHXmnqMZCrEUjkJJAxwp7UfimwOkUWKSfCkpLcvnZlPcU/jEXTiz9S5ztMGs14GNlkZHnjXmwjN5EKAJD93iLu7jx6JRiD8uHjuwbvE6sKhRogYdkjy4TKoDCXgLUzlsGjO9ldXdudiTV3W7C/T6D99ldN7MVKA6UfTAUNuBCBeRXpv8GxKUUV+hSHzP2NbahBK4ZOJPlL3rXlxmktxQSLIBBeSLLpfZ6FtriduimItgDJ4ykv+yUQzRFGTdDXW8DHdi9rMHolH+VrPhU9STf8ERz6Gm8Ah8yTRDZkgtxXUgMkQRKklOWh7orT8jt+HUDAR7Rg/SK3QkFMQgWvZG8g+DE+6B4m0q7PHq1fPbHNPc91pXy+SsvJMm93kzP0Z943emExNUqywPF7Bfa6pfu+5YTEdUJgXwUFJb35rNFOCZigg5hbW8ZeTABBGkcxSeOoQqtMrOdBeJuZ95COkgmk60zdUTuN6yKUJKsZcd/s8suKE2LXBbaYJfKbnazNLujAEc1V glUwhQ+C /cA0pDeIU1fQRf+B6aXzTm0aiK0D5o7fkfsoChXEEgbF+JvcEH85uRT7i8hLb/1hL+i+77dDFagb1t+XC+4jbMsHtfWm2kE9FbVs+kU8iJm0wAfS0wIINGUBdX2GiTAOHOzfg5Fp1G4Nck54N55t/2q75ksA+e36ycZKGFx4n2TT5wSF5Yd5moOOkKWqiamzSI8b8RxdMo4a1T+B91MuPwJGbWRrlPJSaXJdZIrKlsYcPfeFhoe5g8H0qDDTru9R9zSfd X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Like the zpdesc_set_first(), introduce zpdesc_clear_first() helper for ClearPagePrivate(), then clean up a 'struct page' usage in reset_zpdesc(). Signed-off-by: Alex Shi --- mm/zsmalloc.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index f9a9c2400e41..c0e7c055847a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -251,6 +251,11 @@ static inline void zpdesc_set_first(struct zpdesc *zpdesc) SetPagePrivate(zpdesc_page(zpdesc)); } +static inline void zpdesc_clear_first(struct zpdesc *zpdesc) +{ + ClearPagePrivate(zpdesc_page(zpdesc)); +} + static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) { inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); @@ -840,10 +845,8 @@ static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, static void reset_zpdesc(struct zpdesc *zpdesc) { - struct page *page = zpdesc_page(zpdesc); - __zpdesc_clear_movable(zpdesc); - ClearPagePrivate(page); + zpdesc_clear_first(zpdesc); zpdesc->zspage = NULL; zpdesc->next = NULL; __zpdesc_clear_zsmalloc(zpdesc); From patchwork Thu Dec 5 17:49:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13895900 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20AC2E77170 for ; Thu, 5 Dec 2024 17:50:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ABE316B0185; Thu, 5 Dec 2024 12:50:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A346F6B0179; Thu, 5 Dec 2024 12:50:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3D2C16B017E; Thu, 5 Dec 2024 12:50:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 277A56B0174 for ; Thu, 5 Dec 2024 12:50:08 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id CCB911C869B for ; Thu, 5 Dec 2024 17:50:07 +0000 (UTC) X-FDA: 82861643340.08.498203B Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf02.hostedemail.com (Postfix) with ESMTP id C5F0280016 for ; Thu, 5 Dec 2024 17:49:35 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=pVlyfgSt; spf=none (imf02.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420989; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dAjGyiQGNKWMBww5yB85pPs2W1dcE6q7ezO2w5wKNtU=; b=fVxBaL26E7YvzB0RAEn8n+FDK30cY8/MicI2G2/hVlOCAR6NkWIi3laHdxTHnGo38EjVQz GDasUso2rnVq5ViSJVt6MnIiwb5GOor5uPCig9QRqfJx25n2P/TEDr5DTzbrrEoW7xBIT1 5VqcIpfCqw0tLO4q5E9q7ypRzftu9vw= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=pVlyfgSt; spf=none (imf02.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420989; a=rsa-sha256; cv=none; b=jYHMcHBKbdV8+ZUB2R8FuBXXj/uOsLv96ehfjP8Tx/fhOti5BHKF7iZEE5I/4YH6XR1CAI 8MMHZCbtwEWVCZul4LV6hrrOtSUZWmHwam5pKrHp1hPmnrJ2qSFlq/KbrtlHvS6AzYRjx1 Lw1XmA0t8n74i7UHPe6wMi0AVmUH69A= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=dAjGyiQGNKWMBww5yB85pPs2W1dcE6q7ezO2w5wKNtU=; b=pVlyfgStC16x9IFkzfilpykGJy wE6o9OTJt3gg8k4mVEM1tXLr8cl/bfUPqgDt/hpbyssRvD+2H8nJ0ql8N6Lp4YkERKiVweqVaHe6W LpGmafpjAMN+5j/eXhc4ByQP6EC5stttMI9S7oz4zjcBZWGahyP9oCBnFf/+mZplwM1xBjytQhNYQ VSblhFoz3/elHXNgp3bQmEmejDbtIri+ydirZ1vw4po032cqLoHwuCBG3rO2A96NDbfPAFm+pb7ao bjuHdjmUktU03K4db+l+wVpKktMLixc38KdjLoTs8EUbGhJMKQfxiOXP3bjO+9KR8Qst/mGwqysbn +XiB1qfA==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzI-0000000DN9Y-34jM; Thu, 05 Dec 2024 17:50:04 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Alex Shi , linux-mm@kvack.org Subject: [PATCH v8 21/21] mm/zsmalloc: update comments for page->zpdesc changes Date: Thu, 5 Dec 2024 17:49:58 +0000 Message-ID: <20241205175000.3187069-22-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: C5F0280016 X-Rspam-User: X-Stat-Signature: 8b5ryerh9xwb7hgsiu3me5giw3prk96w X-HE-Tag: 1733420975-613907 X-HE-Meta: U2FsdGVkX18d5zceNpMi7CUOL0v4RReVmfd3Dprsaf8IRiEe46gvJlelIqaBVusel8DB/jNzIpvrZFrOkSpqZ/NLTBld+x43R7Z0AOY7yA1xBUWr2qpnTb/cQzRSPzf6ag0R3Tx4FkMACCd2jXYPZpqd3TrgzEX5y6ESaoOZdHLOMGqcI/lzj4LFGc9VbLJ06DZQr9E9gjs0ObZbPDUk3FTTz5YkR/JK38cgpwLOm3aEAQRhC2FaiXAODO5nUACbrV2wjiAIRs8MmoQ7M9jVWWJ9pBr0/PLDIfe3xkjaTU5GXsJKejNZ4AvoCrXJXka9bKL/GgdNugsPHJtvYcaVN1p5lZ1aT2l8zm/07JzEed01DP1erk3Jqseokienx3ACg4ZnrEAaUzcv3aRbwu/LTbQeg0IdwIGFC2RFJhq4+a0HpEL/Jj4aKwjc+JBtC8+/9UnyFci+2Vj8O4qG0ClEiypvUgjkuXrPBBE7ThFIWXzINfzwOA94BKa0beuecHpyjdYJ0pkLJYkmudjWOIn6qBRcxvHqB8haU6LuaDOW4yS91VOgo1tMC8r+QIoY5GcFpDlVllB63TfBs6OqvEXsIidHwpkhG/LQ6CDWsWYFuMF1eDfRSM2qvneGYLXR7L4IGHaijIEkcg6ae3krzWZP19/5Mw1nb/LkrOirYzblL0ktzCWeuLkwce1K8psqBEjVH12jPaJeCeyrneZ4RK/aqIOIThB+OEUIb2U59ieYd9zNCTZeVwNArf68Q/Zvfw3cTTWdeQwH3lH/HwOTKG+CfrWSfsn3GSWySwzBqPwZ9OBtZPtUD5IGy23k2A4ljGqRvvFNsNMmbOb3SefGlzwRwu8aEyexpZQkRujTkyh47yujNBeHxstA6Vultql0dFrpWNykxibOfazWLZBBqXOn9QJnlgiSwsz4jlRWqNbTRKTQqkdOk63kvM4WmDnVIN+2ELRRD8FlUsjUmqN3M8R nooJ1BIW Qe/+eiOm4GIclSHKJC0XGu6jZ6K2L25CIE49Z2IPPEBwVk78d5OVWCoVe9VqAolVSl3FPzr2HPOZPbwzcp12RUDJkETPB1hjixsjAyzJPIhHZx492nar9IMbqFag2YgwzuiHyW6vs0ZxIeZetdbGEDp30nsYXh78yCodLqnuaBcotZftMFyJKa/PsnVkArKK4qOF5cmDBWsgChuc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi After the page to zpdesc conversion, there still left few comments or function named with page not zpdesc, let's update the comments and rename function create_page_chain() as create_zpdesc_chain(). Signed-off-by: Alex Shi --- mm/zsmalloc.c | 61 ++++++++++++++++++++++++++------------------------- 1 file changed, 31 insertions(+), 30 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index c0e7c055847a..1f5ff0fdeb42 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -15,20 +15,19 @@ /* * Following is how we use various fields and flags of underlying - * struct page(s) to form a zspage. + * struct zpdesc(page) to form a zspage. * - * Usage of struct page fields: - * page->private: points to zspage - * page->index: links together all component pages of a zspage + * Usage of struct zpdesc fields: + * zpdesc->zspage: points to zspage + * zpdesc->next: links together all component zpdescs of a zspage * For the huge page, this is always 0, so we use this field * to store handle. - * page->page_type: PGTY_zsmalloc, lower 24 bits locate the first object - * offset in a subpage of a zspage - * - * Usage of struct page flags: - * PG_private: identifies the first component page - * PG_owner_priv_1: identifies the huge component page + * zpdesc->first_obj_offset: PGTY_zsmalloc, lower 24 bits locate the first + * object offset in a subpage of a zspage * + * Usage of struct zpdesc(page) flags: + * PG_private: identifies the first component zpdesc + * PG_lock: lock all component zpdescs for a zspage free, serialize with */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt @@ -194,7 +193,10 @@ struct size_class { */ int size; int objs_per_zspage; - /* Number of PAGE_SIZE sized pages to combine to form a 'zspage' */ + /* + * Number of PAGE_SIZE sized zpdescs/pages to combine to + * form a 'zspage' + */ int pages_per_zspage; unsigned int index; @@ -908,7 +910,7 @@ static void free_zspage(struct zs_pool *pool, struct size_class *class, /* * Since zs_free couldn't be sleepable, this function cannot call - * lock_page. The page locks trylock_zspage got will be released + * lock_page. The zpdesc locks trylock_zspage got will be released * by __free_zspage. */ if (!trylock_zspage(zspage)) { @@ -965,7 +967,7 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) set_freeobj(zspage, 0); } -static void create_page_chain(struct size_class *class, struct zspage *zspage, +static void create_zpdesc_chain(struct size_class *class, struct zspage *zspage, struct zpdesc *zpdescs[]) { int i; @@ -974,9 +976,9 @@ static void create_page_chain(struct size_class *class, struct zspage *zspage, int nr_zpdescs = class->pages_per_zspage; /* - * Allocate individual pages and link them together as: - * 1. all pages are linked together using zpdesc->next - * 2. each sub-page point to zspage using zpdesc->zspage + * Allocate individual zpdescs and link them together as: + * 1. all zpdescs are linked together using zpdesc->next + * 2. each sub-zpdesc point to zspage using zpdesc->zspage * * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). @@ -1034,7 +1036,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, zpdescs[i] = zpdesc; } - create_page_chain(class, zspage, zpdescs); + create_zpdesc_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool = pool; zspage->class = class->index; @@ -1351,7 +1353,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, /* record handle in the header of allocated chunk */ link->handle = handle | OBJ_ALLOCATED_TAG; else - /* record handle to page->index */ + /* record handle to zpdesc->handle */ zspage->first_zpdesc->handle = handle | OBJ_ALLOCATED_TAG; kunmap_local(vaddr); @@ -1441,7 +1443,6 @@ static void obj_free(int class_size, unsigned long obj) unsigned int f_objidx; void *vaddr; - obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset = offset_in_page(class_size * f_objidx); zspage = get_zspage(f_zpdesc); @@ -1684,19 +1685,19 @@ static int putback_zspage(struct size_class *class, struct zspage *zspage) #ifdef CONFIG_COMPACTION /* * To prevent zspage destroy during migration, zspage freeing should - * hold locks of all pages in the zspage. + * hold locks of all component zpdesc in the zspage. */ static void lock_zspage(struct zspage *zspage) { struct zpdesc *curr_zpdesc, *zpdesc; /* - * Pages we haven't locked yet can be migrated off the list while we're + * Zpdesc we haven't locked yet can be migrated off the list while we're * trying to lock them, so we need to be careful and only attempt to - * lock each page under migrate_read_lock(). Otherwise, the page we lock - * may no longer belong to the zspage. This means that we may wait for - * the wrong page to unlock, so we must take a reference to the page - * prior to waiting for it to unlock outside migrate_read_lock(). + * lock each zpdesc under migrate_read_lock(). Otherwise, the zpdesc we + * lock may no longer belong to the zspage. This means that we may wait + * for the wrong zpdesc to unlock, so we must take a reference to the + * zpdesc prior to waiting for it to unlock outside migrate_read_lock(). */ while (1) { migrate_read_lock(zspage); @@ -1771,7 +1772,7 @@ static void replace_sub_page(struct size_class *class, struct zspage *zspage, idx++; } while ((zpdesc = get_next_zpdesc(zpdesc)) != NULL); - create_page_chain(class, zspage, zpdescs); + create_zpdesc_chain(class, zspage, zpdescs); first_obj_offset = get_first_obj_offset(oldzpdesc); set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) @@ -1782,8 +1783,8 @@ static void replace_sub_page(struct size_class *class, struct zspage *zspage, static bool zs_page_isolate(struct page *page, isolate_mode_t mode) { /* - * Page is locked so zspage couldn't be destroyed. For detail, look at - * lock_zspage in free_zspage. + * Page/zpdesc is locked so zspage couldn't be destroyed. For detail, + * look at lock_zspage in free_zspage. */ VM_BUG_ON_PAGE(PageIsolated(page), page); @@ -1810,7 +1811,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* We're committed, tell the world that this is a Zsmalloc page. */ __zpdesc_set_zsmalloc(newzpdesc); - /* The page is locked, so this pointer must remain valid */ + /* The zpdesc/page is locked, so this pointer must remain valid */ zspage = get_zspage(zpdesc); pool = zspage->pool; @@ -1883,7 +1884,7 @@ static const struct movable_operations zsmalloc_mops = { }; /* - * Caller should hold page_lock of all pages in the zspage + * Caller should hold zpdesc locks of all in the zspage * In here, we cannot use zspage meta data. */ static void async_free_zspage(struct work_struct *work)