From patchwork Wed Nov 10 18:54:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minchan Kim X-Patchwork-Id: 12612573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31721C433EF for ; Wed, 10 Nov 2021 18:54:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D861B61178 for ; Wed, 10 Nov 2021 18:54:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D861B61178 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id D543F6B0074; Wed, 10 Nov 2021 13:54:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D01B46B0075; Wed, 10 Nov 2021 13:54:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C17FC6B0078; Wed, 10 Nov 2021 13:54:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0227.hostedemail.com [216.40.44.227]) by kanga.kvack.org (Postfix) with ESMTP id B27A26B0074 for ; Wed, 10 Nov 2021 13:54:40 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 63542183956F4 for ; Wed, 10 Nov 2021 18:54:40 +0000 (UTC) X-FDA: 78793921752.19.FC7E855 Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) by imf17.hostedemail.com (Postfix) with ESMTP id 1A5D6F0003A3 for ; Wed, 10 Nov 2021 18:54:39 +0000 (UTC) Received: by mail-pj1-f44.google.com with SMTP id nh10-20020a17090b364a00b001a69adad5ebso2608900pjb.2 for ; Wed, 10 Nov 2021 10:54:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Fk73h2bdOvtDrpJ3ztMSpAKyAIBRM59FaYEpfcq5KOA=; b=J7d9wIR4u0AVm0vPccsPD3MXm2hdMKG4t0xAyHFroI0xdH69c6AFfoJ8CmQfd4w6GE 0zdvuL65jw8u8TwTPjc7XnXJYgLBKPFqiKK4FW16ydwGp7fAUsM9oRxANSc/4nfwNypy Am1AasVYffD/qB33mrY9l6z6HIeIQoMKmnTHMXEV2OztGzkU1QQ7c1ohr1Yudv+FgdFx zlrmEDpsrw4MX6XQrHLbku6e4K/1m4BpRJSeypwJm25ai07oH8ZPCTILCKrZ+cV/o/Ai kF5M9hKU0ImXg9nI6iGLYUYJCXELmA7H7V7dEPiPKoc4OVwHKqViJ81vs9bZW1i+FsoU XyCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=Fk73h2bdOvtDrpJ3ztMSpAKyAIBRM59FaYEpfcq5KOA=; b=qIFZ2J5GJeBkNQa6iyLjPyh64+JM7QRYVhuNHnsQ2zEgVVghcJaETnwWAQ1gbq2Q9x CFBDuPguL0oCiCgTgfOXiOEzJcnKZhl2Pe+QDLVrJfQTH61rW+pua2QwQsgm1/RvlHD2 Jf/PzrUt6l5gTI1ktZnGmgxpsL0rivSezThvWdQS7tU8+eGOUze91OAEkkcOs6RlUpI0 jCaojWQNNFgaFlmNmiwSnlRxT2w6pQilwV/7CCljv493jg+5oEHHycIG99nylK+SsKJU WgXkAX5pIxomPVxBkXx+U3v+S+ptHT0eThsvIyoetLmb/xJu2DYNZSKFsOfKpGzdMWtW Ptaw== X-Gm-Message-State: AOAM5303+FjscL7qbSOtwCt/lv99qPMnLxH5432jRYDaZ0kiSWI+zfs4 xVtVl6Ffa6/qaupU15JbBbg= X-Google-Smtp-Source: ABdhPJzmiN67+rMgfz2viVeNCTKHWpXdggud8iFrL6Y4ApUCoMtITZkGo5/E39Ja7HT25X1hIwODRA== X-Received: by 2002:a17:90b:3890:: with SMTP id mu16mr19528144pjb.73.1636570479148; Wed, 10 Nov 2021 10:54:39 -0800 (PST) Received: from bbox-1.mtv.corp.google.com ([2620:15c:211:201:11d4:2de3:ab82:be64]) by smtp.gmail.com with ESMTPSA id g13sm325253pjc.39.2021.11.10.10.54.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Nov 2021 10:54:38 -0800 (PST) From: Minchan Kim To: Andrew Morton Cc: Sergey Senozhatsky , linux-mm , Minchan Kim Subject: [PATCH 1/8] zsmalloc: introduce some helper functions Date: Wed, 10 Nov 2021 10:54:26 -0800 Message-Id: <20211110185433.1981097-2-minchan@kernel.org> X-Mailer: git-send-email 2.34.0.rc1.387.gb447b232ab-goog In-Reply-To: <20211110185433.1981097-1-minchan@kernel.org> References: <20211110185433.1981097-1-minchan@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 1A5D6F0003A3 X-Stat-Signature: 87hr374rr975yraj4udc7oxi8kzrhmu4 Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=J7d9wIR4; spf=pass (imf17.hostedemail.com: domain of minchan.kim@gmail.com designates 209.85.216.44 as permitted sender) smtp.mailfrom=minchan.kim@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) X-HE-Tag: 1636570479-154358 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: get_zspage_mapping returns fullness as well as class_idx. However, the fullness is usually not used since it could be stale in some contexts. It causes misleading as well as unnecessary instructions so this patch introduces zspage_class. obj_to_location also produces page and index but we don't need always the index, either so this patch introduces obj_to_page. Signed-off-by: Minchan Kim --- mm/zsmalloc.c | 54 ++++++++++++++++++++++----------------------------- 1 file changed, 23 insertions(+), 31 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 68e8831068f4..be02db164477 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -517,6 +517,12 @@ static void get_zspage_mapping(struct zspage *zspage, *class_idx = zspage->class; } +static struct size_class *zspage_class(struct zs_pool *pool, + struct zspage *zspage) +{ + return pool->size_class[zspage->class]; +} + static void set_zspage_mapping(struct zspage *zspage, unsigned int class_idx, enum fullness_group fullness) @@ -844,6 +850,12 @@ static void obj_to_location(unsigned long obj, struct page **page, *obj_idx = (obj & OBJ_INDEX_MASK); } +static void obj_to_page(unsigned long obj, struct page **page) +{ + obj >>= OBJ_TAG_BITS; + *page = pfn_to_page(obj >> OBJ_INDEX_BITS); +} + /** * location_to_obj - get obj value encoded from (, ) * @page: page object resides in zspage @@ -1246,8 +1258,6 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, unsigned long obj, off; unsigned int obj_idx; - unsigned int class_idx; - enum fullness_group fg; struct size_class *class; struct mapping_area *area; struct page *pages[2]; @@ -1270,8 +1280,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, /* migration cannot move any subpage in this zspage */ migrate_read_lock(zspage); - get_zspage_mapping(zspage, &class_idx, &fg); - class = pool->size_class[class_idx]; + class = zspage_class(pool, zspage); off = (class->size * obj_idx) & ~PAGE_MASK; area = &get_cpu_var(zs_map_area); @@ -1304,16 +1313,13 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) unsigned long obj, off; unsigned int obj_idx; - unsigned int class_idx; - enum fullness_group fg; struct size_class *class; struct mapping_area *area; obj = handle_to_obj(handle); obj_to_location(obj, &page, &obj_idx); zspage = get_zspage(page); - get_zspage_mapping(zspage, &class_idx, &fg); - class = pool->size_class[class_idx]; + class = zspage_class(pool, zspage); off = (class->size * obj_idx) & ~PAGE_MASK; area = this_cpu_ptr(&zs_map_area); @@ -1491,8 +1497,6 @@ void zs_free(struct zs_pool *pool, unsigned long handle) struct zspage *zspage; struct page *f_page; unsigned long obj; - unsigned int f_objidx; - int class_idx; struct size_class *class; enum fullness_group fullness; bool isolated; @@ -1502,13 +1506,11 @@ void zs_free(struct zs_pool *pool, unsigned long handle) pin_tag(handle); obj = handle_to_obj(handle); - obj_to_location(obj, &f_page, &f_objidx); + obj_to_page(obj, &f_page); zspage = get_zspage(f_page); migrate_read_lock(zspage); - - get_zspage_mapping(zspage, &class_idx, &fullness); - class = pool->size_class[class_idx]; + class = zspage_class(pool, zspage); spin_lock(&class->lock); obj_free(class, obj); @@ -1865,8 +1867,6 @@ static bool zs_page_isolate(struct page *page, isolate_mode_t mode) { struct zs_pool *pool; struct size_class *class; - int class_idx; - enum fullness_group fullness; struct zspage *zspage; struct address_space *mapping; @@ -1879,15 +1879,10 @@ static bool zs_page_isolate(struct page *page, isolate_mode_t mode) zspage = get_zspage(page); - /* - * Without class lock, fullness could be stale while class_idx is okay - * because class_idx is constant unless page is freed so we should get - * fullness again under class lock. - */ - get_zspage_mapping(zspage, &class_idx, &fullness); mapping = page_mapping(page); pool = mapping->private_data; - class = pool->size_class[class_idx]; + + class = zspage_class(pool, zspage); spin_lock(&class->lock); if (get_zspage_inuse(zspage) == 0) { @@ -1906,6 +1901,9 @@ static bool zs_page_isolate(struct page *page, isolate_mode_t mode) * size_class to prevent further object allocation from the zspage. */ if (!list_empty(&zspage->list) && !is_zspage_isolated(zspage)) { + enum fullness_group fullness; + unsigned int class_idx; + get_zspage_mapping(zspage, &class_idx, &fullness); atomic_long_inc(&pool->isolated_pages); remove_zspage(class, zspage, fullness); @@ -1922,8 +1920,6 @@ static int zs_page_migrate(struct address_space *mapping, struct page *newpage, { struct zs_pool *pool; struct size_class *class; - int class_idx; - enum fullness_group fullness; struct zspage *zspage; struct page *dummy; void *s_addr, *d_addr, *addr; @@ -1948,9 +1944,8 @@ static int zs_page_migrate(struct address_space *mapping, struct page *newpage, /* Concurrent compactor cannot migrate any subpage in zspage */ migrate_write_lock(zspage); - get_zspage_mapping(zspage, &class_idx, &fullness); pool = mapping->private_data; - class = pool->size_class[class_idx]; + class = zspage_class(pool, zspage); offset = get_first_obj_offset(page); spin_lock(&class->lock); @@ -2048,8 +2043,6 @@ static void zs_page_putback(struct page *page) { struct zs_pool *pool; struct size_class *class; - int class_idx; - enum fullness_group fg; struct address_space *mapping; struct zspage *zspage; @@ -2057,10 +2050,9 @@ static void zs_page_putback(struct page *page) VM_BUG_ON_PAGE(!PageIsolated(page), page); zspage = get_zspage(page); - get_zspage_mapping(zspage, &class_idx, &fg); mapping = page_mapping(page); pool = mapping->private_data; - class = pool->size_class[class_idx]; + class = zspage_class(pool, zspage); spin_lock(&class->lock); dec_zspage_isolation(zspage);