From patchwork Mon Feb 20 13:22:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hyeonggon Yoo <42.hyeyoo@gmail.com> X-Patchwork-Id: 13146442 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5EF2C636CC for ; Mon, 20 Feb 2023 13:23:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 565BA6B0087; Mon, 20 Feb 2023 08:23:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 517726B0088; Mon, 20 Feb 2023 08:23:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3B72E6B0089; Mon, 20 Feb 2023 08:23:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 2D3816B0087 for ; Mon, 20 Feb 2023 08:23:14 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 075A21603F8 for ; Mon, 20 Feb 2023 13:23:14 +0000 (UTC) X-FDA: 80487736308.09.F50F6DD Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf23.hostedemail.com (Postfix) with ESMTP id 37540140015 for ; Mon, 20 Feb 2023 13:23:12 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="FTYnX/Wr"; spf=pass (imf23.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676899392; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EsO1dtiPmIqk89+L0PC/QY+hoYO9HQ5kr0OEemlC8x0=; b=jf+joEqmqbzsGlWVFbv2d+W5j/elIRYGFhLxu2OBdpNFyyfd2evWnZrxCOADgpz0AQdtVs BgMpeAqYZm0r7ZklImeHo2ND7IDTzNMQU0WdWJqUZFMfUYXdCr3xSaTytRFfWi8HUWT/N5 KfG79Wgh/N9wZDGE3Otbl2IVEUeg2KA= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="FTYnX/Wr"; spf=pass (imf23.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676899392; a=rsa-sha256; cv=none; b=btQqwl4T44Hup4GAhqFYEvcoMrsgCLdYmtjwR9sSenkTWdoIx2Tqu7JuFHRP41FPu/wpow uMq5Ya34htT7NtGAsoDgpCdkt6jh5inG/rkw5Eu6jWGcZ47yWn+ZSlVKoPl3tTSdNL0cXw tx0PKEfuPr8iK/h0haArhogOcWQjjX0= Received: by mail-pl1-f173.google.com with SMTP id bh1so1341363plb.11 for ; Mon, 20 Feb 2023 05:23:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EsO1dtiPmIqk89+L0PC/QY+hoYO9HQ5kr0OEemlC8x0=; b=FTYnX/WrBP2C7Al3tx/fyF9LK3NZE927AGA1l/PFkr6pDPPMdOy0IGL0z9c5nJTu8p DvW+1j4qxtpDhjTzPIVYcBO5fKG5jnLhAFIpbLlPlo9TLlD+yF2XU6hqx/kgXeu5Thb8 fOMEOkv8TDftL/MN29fPMOGnx3hYJpX06isEcNCqBApslt2fz2XY0U7Y+cQqGLb7U7gh 0C+tSU5BvTr+kpGIDd6/YBMFVr1JTjh0gzD/VIV+hb0iBcav/xCr+fIwtSpIcfxbDDZ2 wBdST9GUGDtz7bGSrvGY2ET+4xqTFuSZ3SVT7CkXogv/RahCt1Ch4Blmz+YPvq2FOvjZ zBTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EsO1dtiPmIqk89+L0PC/QY+hoYO9HQ5kr0OEemlC8x0=; b=Fa46aqkIOt52wrZj/5Ceh1WWN1VK3+HLusFcF8BQ0m9AsCXyx+KzY/6wnWyl9JNWHO PSHykj0xD1YWtGduUaHucSAbpeXtTdjdAVNtofHugpT8XAYswkFVQWR+39gxemXcWLTK g/JqyA5C8P4OUAtIia08pqaIvdPOphAJRR1yXHkYa0Sb2kNFAUHtRjmy78aZSOmPVD+m QmoZRR8w4Crz8XnMuzivHx+RkN0NHvLfSRjpQI2pZ+lt3fl7seX4t/bIRRoqN9tAOd9V 0mt+zkvaNEomyh6qPUUXFu02QcSLJJ3JBrYEJVswCsSMjJ/VLAWiFukY0wbp4E/zduIh injQ== X-Gm-Message-State: AO0yUKUHmAQrWppatX+XxKb5wYNKNgfz/sYnjHQXSZSRz9VyReqB88Aa cQ25RoZjvwLDy9vygDBbeaKuXX3JKOyCWA== X-Google-Smtp-Source: AK7set+1cSNyKxU0IijJuq2QUw9fX3DUb+UXhA4usFhzh4AKREjeecXa38GbX4nwRe426gaAdT6ONQ== X-Received: by 2002:a17:902:db03:b0:19a:96a0:4af5 with SMTP id m3-20020a170902db0300b0019a96a04af5mr1412304plx.57.1676899391660; Mon, 20 Feb 2023 05:23:11 -0800 (PST) Received: from test-ha.kr-central-1.c.kakaoi.io ([210.109.63.229]) by smtp.gmail.com with ESMTPSA id x2-20020a170902fe8200b0019c13d032d8sm6125436plm.253.2023.02.20.05.23.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Feb 2023 05:23:10 -0800 (PST) From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To: Minchan Kim , Sergey Senozhatsky , Matthew Wilcox Cc: Andrew Morton , linux-mm@kvack.org, Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [RFC PATCH 14/25] mm/zsmalloc: convert zs_page_{isolate,migrate,putback} to use zsdesc Date: Mon, 20 Feb 2023 13:22:07 +0000 Message-Id: <20230220132218.546369-15-42.hyeyoo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230220132218.546369-1-42.hyeyoo@gmail.com> References: <20230220132218.546369-1-42.hyeyoo@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 37540140015 X-Rspam-User: X-Stat-Signature: kt5rze7bge6r1nb55zjjnoacegnoy4zx X-HE-Tag: 1676899392-817301 X-HE-Meta: U2FsdGVkX186bc7pokZeZ/4RQEZI6DpBhS5VZvtn2c/YCweGGkyIbYYslVA19u34/Wg5gx/nF6GRXNyNI+xWHFNSQAiIm9+Wf8Flw84/ccB3exvQRQZf5mBvICcPvhh6Wf4mQnGjTm0r2OFslnrnkYf+eQ2yItsyvJH0Xsm93NI82Pu1KUuvIzKMNZbu06AzOu5N2czDDsAeG6OFef4e5o+LyYBBO/yzpQsyxaOIsvFQlNVh3I8GJzzqcfkpnoBZuO29pHEwcatoJYt5Q6IrMveHlen3XUCpu5aCap/ukP1nWbQ6ADWUdiby0FT5LuFc8ZDMNVnNfey8kb33qLvulZ+qp8whrs/c3qMsl033iy12yxkrCSWjLtmCLRzjXBoP6dfrAi3GH2EQqPxeZ7SxkQqfzdWUkHvXbmkWMQB/y7vM9rIQPDylIBBxmpQXNe6WhEH5RYdY9k5OUqWyvMo29AlkogV6Hrbdfqp5tHvKD0nY55f8bEAjHBS8DPsmF0MO0Qyat4w17vWRI9Ae+y8d2onowHmFofgLS4mt5RTGPIyG9quMPolDXObqPl7lIU9rxcAWloFUu26VGNCsyOEzUGUO9CGwvFkfFqU6lZYzzu7fVokCotD/ebuLVsXYxcQuHKW/Wb1GobRjKBeB5JxWGeGwWh7pw4/YFaOL+QJVjYIimYPc27Bhi+YbbcKg0+rlC1ClRWp0EllxQeysiurwNGLxJQyheAUAi4cHWiC6AWx5gomvq7+qWjjmtfTmCTfeLqm+sZp7O5pvJ/WvhSn3GkaIXTMMAVvt1+V82jxsRwZ9tfQmhKN2CuV/XtKA4lTvTBpfG7VgfRyNp7WzrLsx+l00EixqxmxbXXdLU9vDmXb6y7F4vKBTmQDnUqpqB4x6xaOS2q+IS/eQfnXb7HyfeXm3cm7UbEDmgejgdCLVvUapS6xswJ8qDn0zf1bEaI1ZTWonNhWwjoi+f+yuZJL tnuHq9NB t6sRT9arH2PZb4Od66+xTsgVEEm7B740nim+1fDCw4qt94qrrYo4lxojskewxFiz55afY6sPdyEB99kELjY+ez3KPXtvwnPQyUMfaft5UtnKiUeXsD1OItjFaputwGKwMj/fWIn+o4uiy66zBtiAcybU4/npswLr5VtdgFJLleYZNVxNHWqVPv9ly7YTbFU2Sb67WF43eQU9xV0NO4c11rRYv9Cqi0Y8M3G+/3qG8GRsGi6Ce6EppNp4uF5YB6Ou/CkfBsuTkbFbgv8tQ2ueiUuGQJwGrUQlRmAFfcRNgasp5M1L4Et9Fzav1rgfysfhiUk6Ujmhq5siRE6Aiv5CWXNurzimmxp/62vRhPVoHucLV7JLufZkuR2jrN35bJ5Ea/lwag/vmxJ250vVawa2JZ/TSNjRM97ceh0mQLVKvJlQD3N3V6SozuA/OYwN9W7mkJB/L33BR99kQBRRKixEXrMw3x6EiJ2ODLRW1vdmirrvqiuinl5zFwtbtujCg4ai6Wx6H6QkDxPHK+lXDZaHq0juVQUfQrTsd5gAB/+c2KgU8VTaX8niDJY3KTNZP0FoyLpM4UQp5PT2clV+F+DPgj6UKQNVSytchswOX5J43/3BNMS8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert the functions for movable operations of zsmalloc to use zsdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- mm/zsmalloc.c | 40 ++++++++++++++++++++++------------------ 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5a3948cbe06f..ced7f144b884 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -2181,14 +2181,15 @@ static void replace_sub_zsdesc(struct size_class *class, struct zspage *zspage, static bool zs_page_isolate(struct page *page, isolate_mode_t mode) { struct zspage *zspage; + struct zsdesc *zsdesc = page_zsdesc(page); /* * Page is locked so zspage couldn't be destroyed. For detail, look at * lock_zspage in free_zspage. */ - VM_BUG_ON_PAGE(PageIsolated(page), page); + VM_BUG_ON_PAGE(zsdesc_is_isolated(zsdesc), zsdesc_page(zsdesc)); - zspage = get_zspage(page); + zspage = get_zspage(zsdesc_page(zsdesc)); migrate_write_lock(zspage); inc_zspage_isolation(zspage); migrate_write_unlock(zspage); @@ -2203,6 +2204,8 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct size_class *class; struct zspage *zspage; struct zsdesc *dummy; + struct zsdesc *new_zsdesc = page_zsdesc(newpage); + struct zsdesc *zsdesc = page_zsdesc(page); void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; @@ -2217,10 +2220,10 @@ static int zs_page_migrate(struct page *newpage, struct page *page, if (mode == MIGRATE_SYNC_NO_COPY) return -EINVAL; - VM_BUG_ON_PAGE(!PageIsolated(page), page); + VM_BUG_ON_PAGE(!zsdesc_is_isolated(zsdesc), zsdesc_page(zsdesc)); /* The page is locked, so this pointer must remain valid */ - zspage = get_zspage(page); + zspage = get_zspage(zsdesc_page(zsdesc)); pool = zspage->pool; /* @@ -2233,30 +2236,30 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); - offset = get_first_obj_offset(page); - s_addr = kmap_atomic(page); + offset = get_first_obj_offset(zsdesc_page(zsdesc)); + s_addr = zsdesc_kmap_atomic(zsdesc); /* * Here, any user cannot access all objects in the zspage so let's move. */ - d_addr = kmap_atomic(newpage); + d_addr = zsdesc_kmap_atomic(new_zsdesc); memcpy(d_addr, s_addr, PAGE_SIZE); kunmap_atomic(d_addr); for (addr = s_addr + offset; addr < s_addr + PAGE_SIZE; addr += class->size) { - if (obj_allocated(page_zsdesc(page), addr, &handle)) { + if (obj_allocated(zsdesc, addr, &handle)) { old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj = (unsigned long)location_to_obj(newpage, + new_obj = (unsigned long)location_to_obj(zsdesc_page(new_zsdesc), obj_idx); record_obj(handle, new_obj); } } kunmap_atomic(s_addr); - replace_sub_zsdesc(class, zspage, page_zsdesc(newpage), page_zsdesc(page)); + replace_sub_zsdesc(class, zspage, new_zsdesc, zsdesc); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release the pool's lock. @@ -2265,14 +2268,14 @@ static int zs_page_migrate(struct page *newpage, struct page *page, dec_zspage_isolation(zspage); migrate_write_unlock(zspage); - get_page(newpage); - if (page_zone(newpage) != page_zone(page)) { - dec_zone_page_state(page, NR_ZSPAGES); - inc_zone_page_state(newpage, NR_ZSPAGES); + zsdesc_get(new_zsdesc); + if (zsdesc_zone(new_zsdesc) != zsdesc_zone(zsdesc)) { + zsdesc_dec_zone_page_state(zsdesc); + zsdesc_inc_zone_page_state(new_zsdesc); } - reset_zsdesc(page_zsdesc(page)); - put_page(page); + reset_zsdesc(zsdesc); + zsdesc_put(zsdesc); return MIGRATEPAGE_SUCCESS; } @@ -2280,10 +2283,11 @@ static int zs_page_migrate(struct page *newpage, struct page *page, static void zs_page_putback(struct page *page) { struct zspage *zspage; + struct zsdesc *zsdesc = page_zsdesc(page); - VM_BUG_ON_PAGE(!PageIsolated(page), page); + VM_BUG_ON_PAGE(!zsdesc_is_isolated(zsdesc), zsdesc_page(zsdesc)); - zspage = get_zspage(page); + zspage = get_zspage(zsdesc_page(zsdesc)); migrate_write_lock(zspage); dec_zspage_isolation(zspage); migrate_write_unlock(zspage);