From patchwork Mon Feb 20 13:22:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hyeonggon Yoo <42.hyeyoo@gmail.com> X-Patchwork-Id: 13146437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63748C05027 for ; Mon, 20 Feb 2023 13:23:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BCE756B0081; Mon, 20 Feb 2023 08:23:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B7F326B0082; Mon, 20 Feb 2023 08:23:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A203A6B0083; Mon, 20 Feb 2023 08:23:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 90D566B0081 for ; Mon, 20 Feb 2023 08:23:01 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 63E581406F7 for ; Mon, 20 Feb 2023 13:23:01 +0000 (UTC) X-FDA: 80487735762.29.193F394 Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) by imf22.hostedemail.com (Postfix) with ESMTP id 68984C001D for ; Mon, 20 Feb 2023 13:22:59 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="AKns/q0d"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.216.49 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676899379; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LmknqlF2BVMkgwnYUzvUrAsPXq6CGDnWGEGBtOvFYx0=; b=DMG34tYlFUL6sMh22ydlFzGSzJ7yutezu7jgu3QGS3niZ1/6nHqCmZ1haqsTAwCZMCNqXM A3BMLj2+lvcXm6JisU6EX3dnwROiE50dUXt255NpKAwpqDvo9PM7Q9YxXC9hD0cvL9bJja uPd45YOhqNuU7aVC4EjeU0VFSMfZvJ0= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="AKns/q0d"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.216.49 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676899379; a=rsa-sha256; cv=none; b=Lw21ZRfQTE4pe31DBqF8FFFYq8K8Zxobm31VbYprUGwv01TmeGyWB9c44qLCKZO/3rqIut jbSG2ROGEuVRbabz0RhCIVGwyxVVUCLwPQuyzsKrMD+5+lkGabtDbL6e9jxC1mzlTpTISp Xgmya1J0skY4eyxizGhjJ9j0AMKMRXs= Received: by mail-pj1-f49.google.com with SMTP id co23-20020a17090afe9700b002341fadc370so1374665pjb.1 for ; Mon, 20 Feb 2023 05:22:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LmknqlF2BVMkgwnYUzvUrAsPXq6CGDnWGEGBtOvFYx0=; b=AKns/q0dWCNsi+Q0WJs0KmVkRxy9UgpXDmnrZCvdQ+bF7fvheAPeJybGESt98RBSBw q3Xz4iq1g2TsO3ikScytlU6/I+EjfAn6t0gBP2oQQwMoFFi7qo3BQGG2GcOs0CvzBBtc jyeABx8Uup0uNj7DS/XOddomldDfopbDokqkwJksR7DQAgc5E3RTd/Ua8NXAboWY6Ltq p54DgQDK5kalaSai1qg9PcVSlpHHdCcmxOgDKWdgMeFDnhCgTfoQQEVi68vKoPwKf2wg BAeZT8i9Bk9QF8BePqW/GxvXdByXhsyC773MqiPe+F83GllkySPR6a3jRgqavOP36D+V ZY9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LmknqlF2BVMkgwnYUzvUrAsPXq6CGDnWGEGBtOvFYx0=; b=EPhPdbMJCsFiEBfhC5pT9DJ6gLJfPcP5YygN/V4ZDxdzP+ZTzY5cbdtfdv4Nsm7k5A YOHBVyNSKUocdRLD1Xx5zXdMwx/70cZAe0vQin6w82tdLfS6yxU60pPhqBJY47E/omY1 ZqYcA0/EQL/rrUTCB6WMOm47lquWdeC3aUZlhr3OL7sj5UYw6WAUDbH8sUQPn6KRbyR6 XZvdtXzt8Etsr0KAvA3D5UX/s4arlF5NHj+OrFE/6qzKNdzXOy27CMLgZflX6Hg2/XO+ C7qS4TvrSIbN9dZmxMEtaIF1J53oKqGpAGPPoW0wCKup264sdezi9Ge7BWYwupqUz0qh PHVg== X-Gm-Message-State: AO0yUKWrhaXI1lTzV5zbnbOb7zkMFPLgxdD/U8XZo8Hj+/G6t8CXuBg9 PWmofgYqv2PnzAhG4t8frzEAr5mFar9bFg== X-Google-Smtp-Source: AK7set829BmS51GF4J4REjokHwIstpy2gNmJ7yItnok5kIcoVF/xsQMadDBkyMgsu+0b9h5j7fbVAw== X-Received: by 2002:a17:902:c40d:b0:198:e13e:e73e with SMTP id k13-20020a170902c40d00b00198e13ee73emr3264330plk.53.1676899378260; Mon, 20 Feb 2023 05:22:58 -0800 (PST) Received: from test-ha.kr-central-1.c.kakaoi.io ([210.109.63.229]) by smtp.gmail.com with ESMTPSA id x2-20020a170902fe8200b0019c13d032d8sm6125436plm.253.2023.02.20.05.22.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Feb 2023 05:22:57 -0800 (PST) From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To: Minchan Kim , Sergey Senozhatsky , Matthew Wilcox Cc: Andrew Morton , linux-mm@kvack.org, Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [RFC PATCH 09/25] mm/zsmalloc: convert create_page_chain() and its users to use zsdesc Date: Mon, 20 Feb 2023 13:22:02 +0000 Message-Id: <20230220132218.546369-10-42.hyeyoo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230220132218.546369-1-42.hyeyoo@gmail.com> References: <20230220132218.546369-1-42.hyeyoo@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 68984C001D X-Stat-Signature: dx15hu5sn5xh6t7rbky9uo5qhbygnyfe X-HE-Tag: 1676899379-533135 X-HE-Meta: U2FsdGVkX1+LCgRtFAOAw/zpqJow1pFSVE76aGhxo+sqxzsW/SC9Aoy4813Scbn/wvDuc82OHma7W0e52U2kn1pCrbX6jTiGAMMKk8U5Pj1Svez+IzR/kUaZYAiYGmtdDLNSVyKJV+UCnw+TVg53mhyKJPo7cEnCk7cIaH+pqGKnqjyaSL4epWsWvGFzJk5uXjc+HyGgpYgJzvA545aoyGo2/faGUlrItV8kGJ+QVS6MWrK0ntOWUWWmacL5sEtG3DHP4sAhn8WPqe6mP/dMAiZVdTEaZnWhPOIgb+tDWDKC8gWiEHMTAHXsoU57CqCJvP3DBJo8bkx5pHT3XK+2pDoybHQG0jUUOI1KKfeZRWsGV1/Q07UT7A33YgFZXebzMjKcNeMOI52oXa2eb6qvlb+oJ2CLSIc5DX03VyFZBNocVKjpT47EFbN0d0iaP0oQCdJozKRRH5oyRFypT7TzmqO7nuvALpmkoz2ICl+wcdwvozN2qDSjsPiyuzPMl66g2Mgc22CifxlI60/U7mlJSxnccyC0aKxGcI+X3jh3e42eQJrkGSDI9IcDIHgzAiJjMjs6q9KUiT3aWcoV2ej7eHAyRlsJFqYHqS3POytV6zZq04xfdp8OrXUz6KMl51fiME2fi9pYR4epU/IvY4xroeHVp3ZVrnl6UdaBK0Q7kpt3VZOAfFylvqy/DKD/Hx5KB0LmKRkz0tsdlJkXrIKGLLTs/3g1L60dhmXAes8tOp93OC1JDlXzS6Tye9gPLAC81TVumOS15cpM6X8z+sftFyzOIF6ex9XmBUX7M6run5z48jXucRi/jAmHRRqQ83PS+EAqQH+LXzaScZ5BNqA1nHkix/mKN6SzTOPVnHfPzzHxOL7uDMlXuuhnQIWtPrq2Cdkf8whU9yXiWi/SawPQRy4UDNSbCX3Ih4XuY4FDReBcIWdVaujHGkgdpzBhxP7zrVOU7hVotjuZlVdP42Q wJZqLD9w XgS9alSsRlo7esh1O0lFRWnLte3GfEgxngLLNQvzHpPa+XBWdjfPEybqBacinyfNnvfDUW6nsY4CSZpVmTQV1zsRFKQOyb29me5CWYF6mpJgEXLv+BcdzQ5Typ+V4XIU6tjyPB4PGnwdZj4otpFwtpbWR5824TlVqkFhzGUksQ17NYeigusZ5m4fjKPe7ZFFKYsRJH4dhAatCWbthX08196y5bWA3OgZnN4IL9+BN1uPpoPjTidbK5JG9Q79UaZFrwV8GArP+CS2aZPHSgllTO5Vwdb7u0Owv6hspJW66Weq+0L5TZ0dbNQMYsgF+Tqj3nnLRA2vNib3vkAOTqewFVgCinujA1YAbp54K0s0Q8lS10QVFEZP3w1u0/t5O6b3cDqrjyoyq5XDtylGsU35x5J/i9FLKxWqFYD+C0fliiYuPzsqbFih0BqsmWyWLUftHdyT+neNRRF+YIEJAdDDDjdTQUJPx5SBuOhYBQ7y66wys6AFphtx0Q7bwmIKXSOu7qFSeVWb7hHzocCyQLmdkjSnwf/NsvFF7QaXj8Osx84JEqk85WOH4PxVPTSsILV1h5Y53l6RZcpRdex/enaqXHBRxaQ4uUJs5FrkdQCtmUxMWle+gWUHltpYF9YFECDHHPZFBjY7xg2QKs8Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert create_page_chain() to use zsdesc and rename it to create_zsdesc_chain(), update comments accordingly. Also, convert its callers to use zsdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- mm/zsmalloc.c | 80 +++++++++++++++++++++++++-------------------------- 1 file changed, 40 insertions(+), 40 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 4386a24a246c..c65bdce987e9 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1251,36 +1251,36 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) set_freeobj(zspage, 0); } -static void create_page_chain(struct size_class *class, struct zspage *zspage, - struct page *pages[]) +static void create_zsdesc_chain(struct size_class *class, struct zspage *zspage, + struct zsdesc *zsdescs[]) { int i; - struct page *page; - struct page *prev_page = NULL; - int nr_pages = class->pages_per_zspage; + struct zsdesc *zsdesc; + struct zsdesc *prev_zsdesc = NULL; + int nr_zsdescs = class->pages_per_zspage; /* - * Allocate individual pages and link them together as: - * 1. all pages are linked together using page->index - * 2. each sub-page point to zspage using page->private + * Allocate individual zsdescs and link them together as: + * 1. all zsdescs are linked together using zsdesc->next + * 2. each sub-zsdesc point to zspage using zsdesc->zspage * - * we set PG_private to identify the first page (i.e. no other sub-page + * we set PG_private to identify the first zsdesc (i.e. no other sub-zsdesc * has this flag set). */ - for (i = 0; i < nr_pages; i++) { - page = pages[i]; - set_page_private(page, (unsigned long)zspage); - page->index = 0; + for (i = 0; i < nr_zsdescs; i++) { + zsdesc = zsdescs[i]; + zsdesc_set_zspage(zsdesc, zspage); + zsdesc->next = NULL; if (i == 0) { - zspage->first_zsdesc = page_zsdesc(page); - SetPagePrivate(page); + zspage->first_zsdesc = zsdesc; + zsdesc_set_first(zsdesc); if (unlikely(class->objs_per_zspage == 1 && class->pages_per_zspage == 1)) SetZsHugePage(zspage); } else { - prev_page->index = (unsigned long)page; + prev_zsdesc->next = zsdesc; } - prev_page = page; + prev_zsdesc = zsdesc; } } @@ -1292,7 +1292,7 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, gfp_t gfp) { int i; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE]; + struct zsdesc *zsdescs[ZS_MAX_PAGES_PER_ZSPAGE]; struct zspage *zspage = cache_alloc_zspage(pool, gfp); if (!zspage) @@ -1302,23 +1302,21 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, migrate_lock_init(zspage); for (i = 0; i < class->pages_per_zspage; i++) { - struct page *page; + struct zsdesc *zsdesc; - page = alloc_page(gfp); - if (!page) { + zsdesc = alloc_zsdesc(gfp); + if (!zsdesc) { while (--i >= 0) { - dec_zone_page_state(pages[i], NR_ZSPAGES); - __free_page(pages[i]); + free_zsdesc(zsdescs[i]); } cache_free_zspage(pool, zspage); return NULL; } - inc_zone_page_state(page, NR_ZSPAGES); - pages[i] = page; + zsdescs[i] = zsdesc; } - create_page_chain(class, zspage, pages); + create_zsdesc_chain(class, zspage, zsdescs); init_zspage(class, zspage); zspage->pool = pool; @@ -2153,27 +2151,29 @@ static void dec_zspage_isolation(struct zspage *zspage) zspage->isolated--; } -static void replace_sub_page(struct size_class *class, struct zspage *zspage, - struct page *newpage, struct page *oldpage) +static void replace_sub_zsdesc(struct size_class *class, struct zspage *zspage, + struct zsdesc *new_zsdesc, struct zsdesc *old_zsdesc) { - struct page *page; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE] = {NULL, }; + struct zsdesc *zsdesc; + struct zsdesc *zsdescs[ZS_MAX_PAGES_PER_ZSPAGE] = {NULL, }; + unsigned int first_obj_offset; int idx = 0; - page = get_first_page(zspage); + zsdesc = get_first_zsdesc(zspage); do { - if (page == oldpage) - pages[idx] = newpage; + if (zsdesc == old_zsdesc) + zsdescs[idx] = new_zsdesc; else - pages[idx] = page; + zsdescs[idx] = zsdesc; idx++; - } while ((page = get_next_page(page)) != NULL); + } while ((zsdesc = get_next_zsdesc(zsdesc)) != NULL); - create_page_chain(class, zspage, pages); - set_first_obj_offset(newpage, get_first_obj_offset(oldpage)); + create_zsdesc_chain(class, zspage, zsdescs); + first_obj_offset = get_first_obj_offset(zsdesc_page(old_zsdesc)); + set_first_obj_offset(zsdesc_page(new_zsdesc), first_obj_offset); if (unlikely(ZsHugePage(zspage))) - newpage->index = oldpage->index; - __SetPageMovable(newpage, &zsmalloc_mops); + new_zsdesc->handle = old_zsdesc->handle; + zsdesc_set_movable(new_zsdesc); } static bool zs_page_isolate(struct page *page, isolate_mode_t mode) @@ -2254,7 +2254,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, } kunmap_atomic(s_addr); - replace_sub_page(class, zspage, newpage, page); + replace_sub_zsdesc(class, zspage, page_zsdesc(newpage), page_zsdesc(page)); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release the pool's lock.