From patchwork Thu Feb 23 03:04:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13149802 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2CEAC64EC7 for ; Thu, 23 Feb 2023 03:05:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 31B356B007D; Wed, 22 Feb 2023 22:05:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2CC4D6B0080; Wed, 22 Feb 2023 22:05:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 16CAC6B0082; Wed, 22 Feb 2023 22:05:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 0ABD86B007D for ; Wed, 22 Feb 2023 22:05:07 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id CE9AE81027 for ; Thu, 23 Feb 2023 03:05:06 +0000 (UTC) X-FDA: 80497065012.06.8732B75 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf27.hostedemail.com (Postfix) with ESMTP id 06CB34000D for ; Thu, 23 Feb 2023 03:05:04 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="PVE/woFY"; spf=pass (imf27.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.179 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677121505; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=I+a7RvRhurz1Ai+VZTI1SaXOAjw7CakUvWehQGGNcJ0=; b=z2fjYiKgsxHBWHbqL6RAY8VpfFLA/GfQjEHChIbYLRSgsYzTKUyVmGf90VHmQku+b594Jp Iw3GKy+nWiw+241G9x9/UuO+4IFC7HmqNK0K/SjRfsUOSAdzDby5lpqq3nKJpcPa/YLHSg mkaXbeuYSFtz9tTDWczThRL7u0lCl9Q= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="PVE/woFY"; spf=pass (imf27.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.179 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677121505; a=rsa-sha256; cv=none; b=Qw2OJFNGdxbpSPtZ6jPtQjRnXcrP3TBVz8r7IfIPu74iZ1vGi7iLL7E5jSHRT0t7WRuyZX R0fSzqLMLa3V169a5Ssew8+P2ODiWTJVek5Xye0b/sON/v2BlaBqbTMB1BsUfQB8cWHnEB q0vQdjxuBJm6IQHYvG1vbZ5rHEMfeqY= Received: by mail-pl1-f179.google.com with SMTP id h14so11397687plf.10 for ; Wed, 22 Feb 2023 19:05:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=I+a7RvRhurz1Ai+VZTI1SaXOAjw7CakUvWehQGGNcJ0=; b=PVE/woFYX/y2Nv/sSWN/ThQ7WpqOXClosC23oGEVbNLw+KQEHwBO6RQseLQLRiTTO8 rQ3VjL8xKc7Wgb/Jl357h+iW+SZvTOhE+otycFTz0/T665RZa055Y/XJ5GzLILgml5tC 2n9hS+yqHu+gRYPQ1oJQZHmz4yUqpgjh5Y0UU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=I+a7RvRhurz1Ai+VZTI1SaXOAjw7CakUvWehQGGNcJ0=; b=K/NwU8jUDYrIwGyvrm2JIK4+8YFmcze4oxL1HLZPsx1WpNmg/yhFNGVAagCieFarP2 vcQZ+/dkBZzVVDOJBG2dcnOJ+aDTsz18ijj3mf9RAUUYdp5WKb7nXCqDCaFYc2nQP5Uy 44LaKxluCQnJaCHEbvUjUd2MTYSPaJeD0DRE+FbQn9V4Skua7lbUAgsKlPRIc7EYJtBI AuY6Vg7wDweifIDl6VY/6UiQJetx9ED5xgt/JWe14LYvBAEt+KQCN0m+hABTp8BKOxPj olAXXbLXaKVmT1pHpJ92fPRf4qQst+cteLm5cHa+hxyZeV+CpjFeg9a0SkNYvhfgXUQ9 6rAg== X-Gm-Message-State: AO0yUKW0fMoO6qR6nD5RsaUz6z+AgS4aZ3QZewgm1tsXc3T30zk6Yi7m mnwsUGck1APvZH8sCJR4LTfidw== X-Google-Smtp-Source: AK7set9iWFHlpCYsAD2np6Tp3JHdZO5a9W4ONaB5Ya2yJKg+38hrP2KF/ufMSur/h6V0NBGOjezMpQ== X-Received: by 2002:a17:902:ecd0:b0:19b:2332:18cb with SMTP id a16-20020a170902ecd000b0019b233218cbmr12943938plh.1.1677121503834; Wed, 22 Feb 2023 19:05:03 -0800 (PST) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:6de2:9e85:b508:57b8]) by smtp.gmail.com with ESMTPSA id jl21-20020a170903135500b0019926c77577sm608520plb.90.2023.02.22.19.05.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Feb 2023 19:05:03 -0800 (PST) From: Sergey Senozhatsky To: Minchan Kim , Andrew Morton Cc: Yosry Ahmed , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv2 2/6] zsmalloc: remove stat and fullness enums Date: Thu, 23 Feb 2023 12:04:47 +0900 Message-Id: <20230223030451.543162-3-senozhatsky@chromium.org> X-Mailer: git-send-email 2.39.2.637.g21b0678d19-goog In-Reply-To: <20230223030451.543162-1-senozhatsky@chromium.org> References: <20230223030451.543162-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 06CB34000D X-Stat-Signature: toydxr1xqyx6wk18o8uqigxrcfztia9z X-Rspam-User: X-HE-Tag: 1677121504-486287 X-HE-Meta: U2FsdGVkX18fKUQIDkMlRRwJzjGCv04vaGxDKkHSM0kHXx70WRhg91fSIl/u4Y8oNz3QcmzdEDfjaSbgb4KrepKIpwg1RyOVMV5YeDI9y75VoP/9rQI4Al/36xtUhtp3LGB0TL2dH5uhC9gu09EBkoWVJxOEsJFf3isT3a2ynzlXQ26nz9f+5k1qmV/qM1JY2lGsmkONxHBBLhKLo1CDh5tPxMmhakWhoSUxOteYGsTWCPQqnISknFtzTvny1gBQGTU+tEX6HqvyjTIjDeiqFVJGTvQx2VwJUqGxkbqfp1DVLSAHfKzEsSF8fsn9IeCR55uDh9sG5w683/Cop63bs7cDDp4dKwVsMdiJig9/X5gplfbHVrZ7xzdVez95SPVISpTlQWcYzl9j2aYHUvE4bcg1jPpKYBjjSwi7eWVc2stf8mAG1R+xvMk/pMOK02secBVyAZ5BbkZvwB9nwmtX/b8bcTXlVm59cg3Zc1aVKojOiSNWanwURliml69JToZHIX6Y3f3KECeQnqzdL6gqVF6ZnScBpMxvaBAvxl9YjtsHcAUM5XnqmVZ0LqcJEqW5TtIuco2NCQ/CTQNbW1yPQxUU9WIKztr6zF6gx/QEm/K7F/yWUpw0ZXCtvL/8u+H/aCzv0KL/ltekLf0BeBiRHZem8fl3WlV842sOamWdIIVjX+4XVSvSRXUEJNhvNJGZkjjCQ1CA7ar5FbfquvKIiGno22fsE0y4ZpunDUoNGc5jryozdnQ8W5oau+xa3Cauy+TyioY5tid3ZMuE+ZnrzzEAzatOnZ4zUaNcvRlTlWmwMmH+C18K0/c0z9hu+AS25JbMFjsonxD8aGWMb6w+cnQf+IJSxGoKFxF9bV1mbiZOJauaY0gZRgnLekZ3tomtlriI+C2wj9AJjK6zi4bw3JhE+7i7NIEwOuOApSRwYD2YcO0GB0vzz8mR8rxYMjB/W+LDtip4/ZVvpB6cjSh fPMRjIEW rnp/2SZEvPUTqW6tjbt6TeonRBM2fL4ilzNIefHJcOfsXMou2vOUucPMQ91WLZp2fJEcwR3IcPB32Bn694Jbk9LmHncR3X3M1ZF4kYO+TzOXMg+pzuO9KYAg031NsRQjzy733wXWMl5NlSrO3cjZ3h9HSFg/TnQdQuACEsfn47v4NG95JyXL7yfphqtXgxz7qKfPTRda6LpXfh4ANZTyrFXET22vaGeXFO7Of+rBXqDqUuAjAyqHW1jLFnDjRnzelkJvKBh6P/MYBC9biVNYKEBd1DNxf+QVlw4EhFuvtoxdhcTOEMF+qyLloXrEueTF1rXIbut9oV7IqlGAZBbprnz8e2w7+BBNTho06TVbc+mdcgxOFuCDIHznu8Wq5knUcpdFlPXhw/34Q7kOh8DPkDM0u6k73bcppVdsrcpGZyWyPV0U= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The fullness_group enum is nested (sub-enum) within the class_stat_type enum. zsmalloc requires the values in both enums to match, because zsmalloc passes these values to generic functions, e.g. class_stat_inc() and class_stat_dec(), after casting them to integers. Replace these enums (and enum nesting) and use simple defines instead. Also rename some of zsmalloc stats defines, as they sort of clash with zspage object tags. Suggested-by: Yosry Ahmed Signed-off-by: Sergey Senozhatsky --- mm/zsmalloc.c | 104 ++++++++++++++++++++++---------------------------- 1 file changed, 45 insertions(+), 59 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index b57a89ed6f30..38ae8963c0eb 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -159,26 +159,18 @@ #define ZS_SIZE_CLASSES (DIV_ROUND_UP(ZS_MAX_ALLOC_SIZE - ZS_MIN_ALLOC_SIZE, \ ZS_SIZE_CLASS_DELTA) + 1) -enum fullness_group { - ZS_EMPTY, - ZS_ALMOST_EMPTY, - ZS_ALMOST_FULL, - ZS_FULL, - NR_ZS_FULLNESS, -}; +#define ZS_EMPTY 0 +#define ZS_ALMOST_EMPTY 1 +#define ZS_ALMOST_FULL 2 +#define ZS_FULL 3 +#define ZS_OBJS_ALLOCATED 4 +#define ZS_OBJS_INUSE 5 -enum class_stat_type { - CLASS_EMPTY, - CLASS_ALMOST_EMPTY, - CLASS_ALMOST_FULL, - CLASS_FULL, - OBJ_ALLOCATED, - OBJ_USED, - NR_ZS_STAT_TYPE, -}; +#define NR_ZS_STAT 6 +#define NR_ZS_FULLNESS 4 struct zs_size_stat { - unsigned long objs[NR_ZS_STAT_TYPE]; + unsigned long objs[NR_ZS_STAT]; }; #ifdef CONFIG_ZSMALLOC_STAT @@ -547,8 +539,8 @@ static inline void set_freeobj(struct zspage *zspage, unsigned int obj) } static void get_zspage_mapping(struct zspage *zspage, - unsigned int *class_idx, - enum fullness_group *fullness) + unsigned int *class_idx, + int *fullness) { BUG_ON(zspage->magic != ZSPAGE_MAGIC); @@ -557,14 +549,14 @@ static void get_zspage_mapping(struct zspage *zspage, } static struct size_class *zspage_class(struct zs_pool *pool, - struct zspage *zspage) + struct zspage *zspage) { return pool->size_class[zspage->class]; } static void set_zspage_mapping(struct zspage *zspage, - unsigned int class_idx, - enum fullness_group fullness) + unsigned int class_idx, + int fullness) { zspage->class = class_idx; zspage->fullness = fullness; @@ -588,23 +580,20 @@ static int get_size_class_index(int size) return min_t(int, ZS_SIZE_CLASSES - 1, idx); } -/* type can be of enum type class_stat_type or fullness_group */ static inline void class_stat_inc(struct size_class *class, - int type, unsigned long cnt) + int type, unsigned long cnt) { class->stats.objs[type] += cnt; } -/* type can be of enum type class_stat_type or fullness_group */ static inline void class_stat_dec(struct size_class *class, - int type, unsigned long cnt) + int type, unsigned long cnt) { class->stats.objs[type] -= cnt; } -/* type can be of enum type class_stat_type or fullness_group */ static inline unsigned long zs_stat_get(struct size_class *class, - int type) + int type) { return class->stats.objs[type]; } @@ -652,10 +641,10 @@ static int zs_stats_size_show(struct seq_file *s, void *v) continue; spin_lock(&pool->lock); - class_almost_full = zs_stat_get(class, CLASS_ALMOST_FULL); - class_almost_empty = zs_stat_get(class, CLASS_ALMOST_EMPTY); - obj_allocated = zs_stat_get(class, OBJ_ALLOCATED); - obj_used = zs_stat_get(class, OBJ_USED); + class_almost_full = zs_stat_get(class, ZS_ALMOST_FULL); + class_almost_empty = zs_stat_get(class, ZS_ALMOST_EMPTY); + obj_allocated = zs_stat_get(class, ZS_OBJS_ALLOCATED); + obj_used = zs_stat_get(class, ZS_OBJS_INUSE); freeable = zs_can_compact(class); spin_unlock(&pool->lock); @@ -731,11 +720,10 @@ static inline void zs_pool_stat_destroy(struct zs_pool *pool) * the pool (not yet implemented). This function returns fullness * status of the given page. */ -static enum fullness_group get_fullness_group(struct size_class *class, - struct zspage *zspage) +static int get_fullness_group(struct size_class *class, struct zspage *zspage) { int inuse, objs_per_zspage; - enum fullness_group fg; + int fg; inuse = get_zspage_inuse(zspage); objs_per_zspage = class->objs_per_zspage; @@ -754,11 +742,11 @@ static enum fullness_group get_fullness_group(struct size_class *class, /* * This function adds the given zspage to the fullness list identified - * by . + * by . */ static void insert_zspage(struct size_class *class, struct zspage *zspage, - enum fullness_group fullness) + int fullness) { class_stat_inc(class, fullness, 1); list_add(&zspage->list, &class->fullness_list[fullness]); @@ -766,11 +754,11 @@ static void insert_zspage(struct size_class *class, /* * This function removes the given zspage from the fullness list identified - * by . + * by . */ static void remove_zspage(struct size_class *class, struct zspage *zspage, - enum fullness_group fullness) + int fullness) { VM_BUG_ON(list_empty(&class->fullness_list[fullness])); @@ -787,11 +775,10 @@ static void remove_zspage(struct size_class *class, * page from the freelist of the old fullness group to that of the new * fullness group. */ -static enum fullness_group fix_fullness_group(struct size_class *class, - struct zspage *zspage) +static int fix_fullness_group(struct size_class *class, struct zspage *zspage) { int class_idx; - enum fullness_group currfg, newfg; + int currfg, newfg; get_zspage_mapping(zspage, &class_idx, &currfg); newfg = get_fullness_group(class, zspage); @@ -964,7 +951,7 @@ static void __free_zspage(struct zs_pool *pool, struct size_class *class, struct zspage *zspage) { struct page *page, *next; - enum fullness_group fg; + int fg; unsigned int class_idx; get_zspage_mapping(zspage, &class_idx, &fg); @@ -990,7 +977,7 @@ static void __free_zspage(struct zs_pool *pool, struct size_class *class, cache_free_zspage(pool, zspage); - class_stat_dec(class, OBJ_ALLOCATED, class->objs_per_zspage); + class_stat_dec(class, ZS_OBJS_ALLOCATED, class->objs_per_zspage); atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); } @@ -1508,7 +1495,7 @@ unsigned long zs_malloc(struct zs_pool *pool, size_t size, gfp_t gfp) { unsigned long handle, obj; struct size_class *class; - enum fullness_group newfg; + int newfg; struct zspage *zspage; if (unlikely(!size || size > ZS_MAX_ALLOC_SIZE)) @@ -1530,7 +1517,7 @@ unsigned long zs_malloc(struct zs_pool *pool, size_t size, gfp_t gfp) /* Now move the zspage to another fullness group, if required */ fix_fullness_group(class, zspage); record_obj(handle, obj); - class_stat_inc(class, OBJ_USED, 1); + class_stat_inc(class, ZS_OBJS_INUSE, 1); spin_unlock(&pool->lock); return handle; @@ -1552,8 +1539,8 @@ unsigned long zs_malloc(struct zs_pool *pool, size_t size, gfp_t gfp) record_obj(handle, obj); atomic_long_add(class->pages_per_zspage, &pool->pages_allocated); - class_stat_inc(class, OBJ_ALLOCATED, class->objs_per_zspage); - class_stat_inc(class, OBJ_USED, 1); + class_stat_inc(class, ZS_OBJS_ALLOCATED, class->objs_per_zspage); + class_stat_inc(class, ZS_OBJS_INUSE, 1); /* We completely set up zspage so mark them as movable */ SetZsPageMovable(pool, zspage); @@ -1609,7 +1596,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle) struct page *f_page; unsigned long obj; struct size_class *class; - enum fullness_group fullness; + int fullness; if (IS_ERR_OR_NULL((void *)handle)) return; @@ -1624,7 +1611,7 @@ void zs_free(struct zs_pool *pool, unsigned long handle) zspage = get_zspage(f_page); class = zspage_class(pool, zspage); - class_stat_dec(class, OBJ_USED, 1); + class_stat_dec(class, ZS_OBJS_INUSE, 1); #ifdef CONFIG_ZPOOL if (zspage->under_reclaim) { @@ -1828,7 +1815,7 @@ static struct zspage *isolate_zspage(struct size_class *class, bool source) { int i; struct zspage *zspage; - enum fullness_group fg[2] = {ZS_ALMOST_EMPTY, ZS_ALMOST_FULL}; + int fg[2] = {ZS_ALMOST_EMPTY, ZS_ALMOST_FULL}; if (!source) { fg[0] = ZS_ALMOST_FULL; @@ -1852,12 +1839,11 @@ static struct zspage *isolate_zspage(struct size_class *class, bool source) * @class: destination class * @zspage: target page * - * Return @zspage's fullness_group + * Return @zspage's fullness status */ -static enum fullness_group putback_zspage(struct size_class *class, - struct zspage *zspage) +static int putback_zspage(struct size_class *class, struct zspage *zspage) { - enum fullness_group fullness; + int fullness; fullness = get_fullness_group(class, zspage); insert_zspage(class, zspage, fullness); @@ -2121,7 +2107,7 @@ static void async_free_zspage(struct work_struct *work) int i; struct size_class *class; unsigned int class_idx; - enum fullness_group fullness; + int fullness; struct zspage *zspage, *tmp; LIST_HEAD(free_pages); struct zs_pool *pool = container_of(work, struct zs_pool, @@ -2190,8 +2176,8 @@ static inline void zs_flush_migration(struct zs_pool *pool) { } static unsigned long zs_can_compact(struct size_class *class) { unsigned long obj_wasted; - unsigned long obj_allocated = zs_stat_get(class, OBJ_ALLOCATED); - unsigned long obj_used = zs_stat_get(class, OBJ_USED); + unsigned long obj_allocated = zs_stat_get(class, ZS_OBJS_ALLOCATED); + unsigned long obj_used = zs_stat_get(class, ZS_OBJS_INUSE); if (obj_allocated <= obj_used) return 0; @@ -2616,7 +2602,7 @@ static int zs_reclaim_page(struct zs_pool *pool, unsigned int retries) unsigned long handle; struct zspage *zspage; struct page *page; - enum fullness_group fullness; + int fullness; /* Lock LRU and fullness list */ spin_lock(&pool->lock);