From patchwork Mon Dec 16 15:04:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hyeonggon Yoo <42.hyeyoo@gmail.com> X-Patchwork-Id: 13909888 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C74FE7717F for ; Mon, 16 Dec 2024 15:05:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 79E5F6B00A1; Mon, 16 Dec 2024 10:05:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6FE546B00A2; Mon, 16 Dec 2024 10:05:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5792F6B00A4; Mon, 16 Dec 2024 10:05:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2B1156B00A1 for ; Mon, 16 Dec 2024 10:05:43 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C4CADA10D1 for ; Mon, 16 Dec 2024 15:05:42 +0000 (UTC) X-FDA: 82901146188.02.AE7B554 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by imf22.hostedemail.com (Postfix) with ESMTP id CA20FC0016 for ; Mon, 16 Dec 2024 15:05:09 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=NJyggAY4; spf=pass (imf22.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734361508; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gAAVT3jm//0Jn0SieEbRABow8HKrLkziLKm7kqsvGKc=; b=BfFUR2faP9lTwiBLOn/QkqAXAcR+H5OZaTJBsTAtM/qwgelWNyc0m4f6CE9Excg+NQLMBx irARuOx/3F0roDhYPQfXcW3iHCg2DUhif+o6Y2bPS7EGOhr01Ko+EaSDXB6PHqW3qbci+q D1MH3D2/Okrs+I66LGF3+WvB+vfcrHM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734361508; a=rsa-sha256; cv=none; b=znIfp5Bh4/cfBIy8sHmFV+O7VmxhG4SW+EeE97dz0PX1kU6QFvdxB3Lho18b6Z9evL/6u+ UkAk+JfSrpPYsVCWxIDPHPPzjYa2PksyuyAdqatp8hJaETs/xcve7KFCckgrenwetY4qhu 1J79Dk8z/21QZLO4P1CTOFWsRUfzPqQ= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=NJyggAY4; spf=pass (imf22.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-728f1e66418so3603693b3a.2 for ; Mon, 16 Dec 2024 07:05:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734361539; x=1734966339; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gAAVT3jm//0Jn0SieEbRABow8HKrLkziLKm7kqsvGKc=; b=NJyggAY4yEHEnnJjnqZCB+X5UwEsO2EN8irfrUqmc0F4wFKTu41OHAr942zQaI8nzi UMJ8L49UKOdDiiNU9MN/NAQazgQGDrTMutjPN3gbVLkdA3KbUrZBCKuh8cj2XgPtK3hx VyHFfC7RuA3tVrui37OqqI6iT4PAuZgWIdy3iXpXxe9Q4QOevtBm/fvwNj7h9WMUu1Ne ZHnmXKL54dW7SQltM10ebC8HpE51oPZ3rB1VUaxDVkZ2eq0m7ClvNRh03LoS6iVftiAI 2NBNTtzmbteaEdyYpYTVdfJNuiisacctFMbqtkyZoa/rvj76TYfnR7JQMwZVUykAIRDd x5Mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734361539; x=1734966339; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gAAVT3jm//0Jn0SieEbRABow8HKrLkziLKm7kqsvGKc=; b=R/6LTYCx3x+IYeRdf63dsOH8fN/heak8nciJ9CQPKdSAFlsIHVmPygP/Oz66KMZSJd kLzhRV9xRyFs0pwnMP+T+uwKPPE/cRAoVAGadHxcpJZ1Ck2TFEUGGC8M6RY49c1Dqk9T gBkTieqlEpXw6QhxUSLxdVj5r7SfNH5BsJvr4sWd1+Oycxqjq3LjXQsv0RJ0azpfg6yf w8kF3IESW6b8vZ0/lgVBQvH4Ytoq6YaHBv8nvOelc8IqjMxvk5u5BHXdGWKBoR3+osSK NhkpbEvwz7lp0zeMjhs4BT4DdflVOS6UbZP9MEf2nr202W5xZPH6M993kzcXFQkUSRN/ zP5w== X-Forwarded-Encrypted: i=1; AJvYcCU6aLBvDDkFjICwhjFKPiL5ul+OSg3AEYRRlnMCedRS5lAOKcBtwemkkmqeeMZvdSRIHGyZ1aPGrQ==@kvack.org X-Gm-Message-State: AOJu0Yx1beW2DbIyXUWqYqvKHuG1MRq8/g9hFAW+HrUObziDS59OXMJi hqoM+f3pWrNQgWn4qo0OQMq9lSPu6dzi008qIoXAhSL6nEkAhVGy X-Gm-Gg: ASbGncuuCiixsT+vsZbBfHQm3wEXCd/MM9RxRArg9h6Mbvyd5fTZeQJTUUD+izBYVdl TPtYxbHKbk7P4q08zMvv/SUNUDciCZs67ekLKv9dPHY+qaaIoyPe7hN1PeUvyRmxC66AqsBi9yT hpk8lbBhOdIlWIPaE7VXW2av3aC4Csgk9GStjBu4QJ0hS6M/o75MmYtRRZ3bk7e1Zf7GzkpxcJ6 GtC5cKHhDEnEEVUvYrdm2KLfde363b3RtXlb8UuOpIWdVCO/rfcaTCfTUY/elpm934= X-Google-Smtp-Source: AGHT+IEfq0n7VBFbKcEE2XQEuBMbK2QTTJcXcrXNXlUVilzeSIcYZLRhQ6uyisODQLBXIjnG8BOM2Q== X-Received: by 2002:a05:6a21:3287:b0:1e0:da90:5f1f with SMTP id adf61e73a8af0-1e1dfd58f6emr20096828637.16.1734361539138; Mon, 16 Dec 2024 07:05:39 -0800 (PST) Received: from localhost.localdomain ([1.245.180.67]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72918ad5ccdsm4845518b3a.74.2024.12.16.07.05.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Dec 2024 07:05:38 -0800 (PST) From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To: Minchan Kim , Sergey Senozhatsky Cc: Andrew Morton , linux-mm@kvack.org, Matthew Wilcox , Vishal Moola , Alex Shi , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Alex Shi Subject: [PATCH v9 mm-unstable 10/18] mm/zsmalloc: add two helpers for zs_page_migrate() and make it use zpdesc Date: Tue, 17 Dec 2024 00:04:41 +0900 Message-ID: <20241216150450.1228021-11-42.hyeyoo@gmail.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241216150450.1228021-1-42.hyeyoo@gmail.com> References: <20241216150450.1228021-1-42.hyeyoo@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: CA20FC0016 X-Rspam-User: X-Rspamd-Server: rspam07 X-Stat-Signature: an3riib8gmhzpkhk4n1m38gp9m9aq1te X-HE-Tag: 1734361509-698913 X-HE-Meta: U2FsdGVkX18NHy6DfIVJ7VYj4veHK6e9Wl/90eiQ2eaS+dx6OxfBuOSCNo513MN4No+oFPDpq1EoYwBYukQbHxmH/7n2L1491YU8RvBT883lQTNvhk6wips73s7Zj6lDKECC/s1ZDUM81AkS4n/hzgDkhB5S9y2sZC3eqEJ7shD5Feq686FXIErKKKRW0yk5i9acc/6xkAufckKqjfY1hs5Zce86tVAtxUUVVoCJP+GlLtVEBV8L7OXOC4s90xytZhwcHUiwLa8rDdZHoPFvz7lMo851e3zUAU15k/iqUQEbBGVun2QFhi++Z7+vE6rj6x/eQ71cpFWGei/Io3f2ZcvTpxevdvX5JTkWmrtKIBEV0E34YjyTnYEf+OsYpkTnq5gBPMy6GcK4coKtL2m0wtwWzaEghbj0Gqv4vFysUMIdk/WJF4HQ6LBztmQglRd2k9OZUpAmAamdmbOxFvAmVpjFDoXeAZgPMx1fAmTgX9khpFwtf6QGMv+yKDehoyFSbaNPgJyrCp1Z4DhmsoGJJSGEMsVc3t9nExRCVz7eEgPkNCi1Q4xc5pKD1Qe3Afe1lBUod/bevuO+d5tpPaeIHywQbeky2i3JRJLjY+jufk9gT2sVeo1ExbeH1hWDIVZUJYU6XNjinppaJjsAKvuP6UxgWhqNKP7ueedD79GyQnCdAeXzY5ek2rczKUhp5Q6Pr4Puz2su3CvLM8Xaw9EuCX7mwpfpPL+Tv50Yev7Ok4m+6cpOMO4hfP2TphLHNj4dTYOjI5++/cOPmQ6SgRrd84I23n4gPGWeJ/orwAVAD4RTQN1y8cRgSbanJnQCOuWURLI5vuS6l49I4o3BN1Bw1x4XoLghO0/SeHBdKNFap4JULrikLq6qT0fqV0qocShcpBKyXFhVqQ7qfIwKrbSWDeSonRrghILz6g0+Pjc4ToPdqhYlwbQ5afd680YW1oMtxqhTJENbXBiQk3uYuyv +DTHzzV2 eFEc+fnSLodGLMHUvac0mmujXmBIOfsiq7Q1DEt3qWuQ+mqLiKccMorIZFhB5zflAix6nzO3xLGGzh5WCvTyZy38gbh1nmT4VNbViYiAypvvslIBRLAfelpyCKuFP7O9nYjrAjftpZS4YUqX9jN7P116+AG9LxkEnlkD3kkF9zJKYyqoqHv5401R80hyRzzZwM2Pb7V+KuHQOS9YeGLsXJrhAfjN/MI8xeASkuaehtYgNWm3n5y7Yq/5sjAeH+ErbT6fco6CUZgjppj6XGQOz8k9iyWX8k+rGtrUVs2jZmqARWI3tvysjRegY0E4e+GTLNbTy9a/lWaWIe0cB6Asvp8yDP1X791x+YqDLA+3PA3saiSt4CsJ52sPThcZAiQwYmrOBXfAnf3i68rUsfBmsCPbA9p4fSVH91t6wBZFfLq5bzYKEYWQSYF2AlOX+IcZcf3wSwwbuu5pamCb+3NiZCvLY6nu0JmD5LGXDYQaONamblGYLY2h8qlFhghBMs171wfECBRyY7LptfwqVavnEi8Aj9f2YcAR0GM5fWnyyB63umjE= X-Bogosity: Unsure, tests=bogofilter, spamicity=0.450294, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To convert page to zpdesc in zs_page_migrate(), we added zpdesc_is_isolated()/zpdesc_zone() helpers. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 11 +++++++++++ mm/zsmalloc.c | 30 ++++++++++++++++-------------- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 9aca8d307796..c7c52e05e737 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -154,4 +154,15 @@ static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, { __SetPageMovable(zpdesc_page(zpdesc), mops); } + +static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) +{ + return PageIsolated(zpdesc_page(zpdesc)); +} + +static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) +{ + return page_zone(zpdesc_page(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 112603f9449f..432e78e61d2e 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1796,19 +1796,21 @@ static int zs_page_migrate(struct page *newpage, struct page *page, struct size_class *class; struct zspage *zspage; struct zpdesc *dummy; + struct zpdesc *newzpdesc = page_zpdesc(newpage); + struct zpdesc *zpdesc = page_zpdesc(page); void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; unsigned long old_obj, new_obj; unsigned int obj_idx; - VM_BUG_ON_PAGE(!PageIsolated(page), page); + VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(newpage); + __SetPageZsmalloc(zpdesc_page(newzpdesc)); /* The page is locked, so this pointer must remain valid */ - zspage = get_zspage(page); + zspage = get_zspage(zpdesc_page(zpdesc)); pool = zspage->pool; /* @@ -1825,30 +1827,30 @@ static int zs_page_migrate(struct page *newpage, struct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); - offset = get_first_obj_offset(page); - s_addr = kmap_local_page(page); + offset = get_first_obj_offset(zpdesc_page(zpdesc)); + s_addr = kmap_local_zpdesc(zpdesc); /* * Here, any user cannot access all objects in the zspage so let's move. */ - d_addr = kmap_local_page(newpage); + d_addr = kmap_local_zpdesc(newzpdesc); copy_page(d_addr, s_addr); kunmap_local(d_addr); for (addr = s_addr + offset; addr < s_addr + PAGE_SIZE; addr += class->size) { - if (obj_allocated(page_zpdesc(page), addr, &handle)) { + if (obj_allocated(zpdesc, addr, &handle)) { old_obj = handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj = (unsigned long)location_to_obj(newpage, + new_obj = (unsigned long)location_to_obj(zpdesc_page(newzpdesc), obj_idx); record_obj(handle, new_obj); } } kunmap_local(s_addr); - replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); + replace_sub_page(class, zspage, newzpdesc, zpdesc); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. @@ -1857,14 +1859,14 @@ static int zs_page_migrate(struct page *newpage, struct page *page, spin_unlock(&class->lock); migrate_write_unlock(zspage); - get_page(newpage); - if (page_zone(newpage) != page_zone(page)) { - dec_zone_page_state(page, NR_ZSPAGES); - inc_zone_page_state(newpage, NR_ZSPAGES); + zpdesc_get(newzpdesc); + if (zpdesc_zone(newzpdesc) != zpdesc_zone(zpdesc)) { + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_inc_zone_page_state(newzpdesc); } reset_page(page); - put_page(page); + zpdesc_put(zpdesc); return MIGRATEPAGE_SUCCESS; }