From patchwork Wed Nov 23 19:17:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13054149 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 893E2C4332F for ; Wed, 23 Nov 2022 19:17:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 16ADD6B0073; Wed, 23 Nov 2022 14:17:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0F45E6B0074; Wed, 23 Nov 2022 14:17:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E88226B0075; Wed, 23 Nov 2022 14:17:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D875E6B0073 for ; Wed, 23 Nov 2022 14:17:07 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A596214133B for ; Wed, 23 Nov 2022 19:17:07 +0000 (UTC) X-FDA: 80165664894.07.2D65D20 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf23.hostedemail.com (Postfix) with ESMTP id 51BFA140009 for ; Wed, 23 Nov 2022 19:17:07 +0000 (UTC) Received: by mail-pl1-f176.google.com with SMTP id j12so17496417plj.5 for ; Wed, 23 Nov 2022 11:17:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=k0SYvGjxlp7dw6JXDykKNCVFvc4QQ+pWPp2+QDRowV4=; b=kCBq9GguWJ4WHhxUlYFsFQb4CIUVrgFiCSAnYUxAlbSR0oxyreDtq45d+AovRSAQ7C 1NfMyR8KF1COiW2lb6pdV5PCD90DcINjoOox+n0IDr9+TodG62OR5Eqh4eEDGMOEt/vH vY7wlXqFnwJbDfamwmH7gEf/kTImVPaA332JgUZ+V87tkLSxrWl93o2BF+pYnMVjxhOh yveVws5/DPVjOIdLKaJ9MMvqpj0NwkgiVBHL4WqWJrVtgNaR87hIIS1+CZjiWf7VTfsW qKr+FFrLJtiv7/xEh3gmT2FvrrWxlJH1VtYA9yOj5yS1Q0N/PIPT6oEa5vCwIfrlpBCF fY/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k0SYvGjxlp7dw6JXDykKNCVFvc4QQ+pWPp2+QDRowV4=; b=iDM1vxlClsFek0SLea+tf10wa1Zr625kQuDpNn+Zhl4VnXyNA7OB3ztf1eKYool/Gx /Q2q9oLTsa2FdkVtVcJRUYrQJHO8oE8C9wTfLpD/rojgPbNHCVNY4vgbLTnzj6GX5BON bvmS4f9ys9h7rZeht3PWA40olKFBxun0f+v1uwQfnoVbIC11rBsNgdSvPKdZtFm6f71M q43aDmHS2wYZ7VkDD985FkqjU6kjxHl2Lr/F05yxtzDEhqywmO/Dia4MdwqTAm7Dg6ZP atkHVWfWST/Drl97zmcnLkaYTOrda/oGw3HR9j7Z31EQbHU21wb6T1NNY7BeVCcHGesb h7Bw== X-Gm-Message-State: ANoB5pk3UqIKl4TWT1d+hmB72nXAhlQq0r9VuvR742YUjYlpqdKg5kRw Q4rNx70q7xJYyeJbmeSfM/Q= X-Google-Smtp-Source: AA0mqf4aTF2t29dtAxRyWgY0LpBARMr/iVrO8UN6QfmAylGKzziqN6PcoAfC9Jx7jBo8WQd4tVD+8g== X-Received: by 2002:a17:902:8f96:b0:189:3803:23e6 with SMTP id z22-20020a1709028f9600b00189380323e6mr7577592plo.77.1669231026347; Wed, 23 Nov 2022 11:17:06 -0800 (PST) Received: from localhost (fwdproxy-prn-020.fbsv.net. [2a03:2880:ff:14::face:b00c]) by smtp.gmail.com with ESMTPSA id l7-20020a170902f68700b00177fb862a87sm14920203plg.20.2022.11.23.11.17.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Nov 2022 11:17:05 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, minchan@kernel.org, ngupta@vflare.org, senozhatsky@chromium.org, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com Subject: [PATCH v6 4/6] zsmalloc: Add a LRU to zs_pool to keep track of zspages in LRU order (fix) Date: Wed, 23 Nov 2022 11:17:02 -0800 Message-Id: <20221123191703.2902079-2-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221123191703.2902079-1-nphamcs@gmail.com> References: <20221123191703.2902079-1-nphamcs@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=kCBq9Ggu; spf=pass (imf23.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669231027; a=rsa-sha256; cv=none; b=TMN1lQ2YKvgAmJve8WBcWSrXdqsjlMbHQLEYCs+kETOetNIV61SifM3iHavTOu/5woWmyw 19emu9cH9PrsroY+KsnW5P5BuNFEX0Fbncrrbnl7I8DcY7eHCs0M2C4delDOAaMWxOIShr hiS/Ow4BYq1gDE4F6sp8jhCW4fIaJZI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669231027; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=k0SYvGjxlp7dw6JXDykKNCVFvc4QQ+pWPp2+QDRowV4=; b=KMdUIo7K2QGBp3XoQyhza/bI1DGVM8Sa6BP50aSKcEcTZrDSNzVHZq+ESVjcNJHFTn4rV+ AV1a6Phc9d/MZyefqFZI8//LkLHgB4DQzPp7ZAna+0aXMaMXNVjOXAs82nlmSDNhCc0S/7 XH9kwbDCNUN9Tr/mp2IK0Sa/YlmKPl4= X-Stat-Signature: nzhhbozgzjk53pybg79wih36wjfkzbt8 X-Rspamd-Server: rspam08 X-Rspam-User: Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=kCBq9Ggu; spf=pass (imf23.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Queue-Id: 51BFA140009 X-HE-Tag: 1669231027-840727 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add a comment explaining the mapping check for LRU update. Signed-off-by: Nhat Pham Suggested-by: Johannes Weiner Suggested-by: Sergey Senozhatsky --- mm/zsmalloc.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) -- 2.30.2 diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7dd464b5a6a5..d9bd865047e2 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1268,7 +1268,23 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, zspage = get_zspage(page); #ifdef CONFIG_ZPOOL - /* Move the zspage to front of pool's LRU */ + /* + * Move the zspage to front of pool's LRU. + * + * Note that this is swap-specific, so by definition there are no ongoing + * accesses to the memory while the page is swapped out that would make + * it "hot". A new entry is hot, then ages to the tail until it gets either + * written back or swaps back in. + * + * Furthermore, map is also called during writeback. We must not put an + * isolated page on the LRU mid-reclaim. + * + * As a result, only update the LRU when the page is mapped for write + * when it's first instantiated. + * + * This is a deviation from the other backends, which perform this update + * in the allocation function (zbud_alloc, z3fold_alloc). + */ if (mm == ZS_MM_WO) { if (!list_empty(&zspage->lru)) list_del(&zspage->lru); From patchwork Wed Nov 23 19:17:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13054150 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE08EC3A59F for ; Wed, 23 Nov 2022 19:17:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 75EBA6B0074; Wed, 23 Nov 2022 14:17:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6C05F6B0075; Wed, 23 Nov 2022 14:17:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4EB5B6B0078; Wed, 23 Nov 2022 14:17:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3DCCE6B0074 for ; Wed, 23 Nov 2022 14:17:09 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 091B7120BD0 for ; Wed, 23 Nov 2022 19:17:09 +0000 (UTC) X-FDA: 80165664978.24.9E2B6B7 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by imf04.hostedemail.com (Postfix) with ESMTP id 877DE40009 for ; Wed, 23 Nov 2022 19:17:08 +0000 (UTC) Received: by mail-pl1-f178.google.com with SMTP id 4so17536069pli.0 for ; Wed, 23 Nov 2022 11:17:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ruO+YinHt02cD7LDLng29WAK5pZvQbCFBqvYCl3hrTY=; b=ixSC8hDnbztU0OC/soKUCSkVcuZpNhHZrTMy73xNCHzzfaOyQag8wcAC0zxDGxBQEL cFv7auDe3Fi4p541vWWnL5tcLNc4ebxoGg0/B1hK8LUhXQmzZgDYmUKupPy06uvdhq+u pZ/qgHC8iO7qPQHIiBSf2jxM95OSvjRzimzrdE/EZR2Fz8kS07wolWVCu7u8UnPTSxL/ 8f96Cj1m0EalvWOY5NWkL9XLuCj8AWz+EPKHmBSSQ4uS1ymXh/gwxuCrYA0BxUh72CNF w9kAwhEjdjC6kPN96wiobHFCgI++KuKuykXndt6lKhDz6zmY/05z3P/IVvnwBJCzZpMr yocA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ruO+YinHt02cD7LDLng29WAK5pZvQbCFBqvYCl3hrTY=; b=5zsHyCzfueWyegciLxQs/ipJ0/kMFZ7c0WhcIYWjGaqbqSdlJusxHbY4A7OxlDQSkg 4BtaLGYxYDKRS0NvHlgecAWEAr+IBEk4Nq+f/wcpC3h+mDkrlOFxBlP3V1XbtkLdfTNZ xT8T3AQaB5XK9NlJiq4aMmU6fUaX5bPQNWBgJfqO1iaD3boIbLWXtQBIl12+G2PLkgh3 3OcJACDLUmLYX9yxR3DcPOiLcrO+bYCrE2c6bHu1sNHhct50cVqSDBALXeS1V0wWd9n2 iqBy0+JRSv04gwMliU4XWVbfFtnmLAZVuILRZgv8zF/5+MDmtZbqiZ57pjRJ19D2rRkN YsNA== X-Gm-Message-State: ANoB5pmOfKyGLfF0LPUB6WURse9w09CDNMGsHihldaBHJA+yS8e/EFCm sn5At0LAMMby0z2q76eEpZgtYs8frGR/jw== X-Google-Smtp-Source: AA0mqf4fu3wefIa6SqzYOVIVfrC/54k3vFrt9n7lwzLX39tBI4jKV6KQ5AhqX2J+InZUOp9d3J2+fw== X-Received: by 2002:a17:902:ab1d:b0:188:f5de:a8e4 with SMTP id ik29-20020a170902ab1d00b00188f5dea8e4mr11050853plb.133.1669231027538; Wed, 23 Nov 2022 11:17:07 -0800 (PST) Received: from localhost (fwdproxy-prn-020.fbsv.net. [2a03:2880:ff:14::face:b00c]) by smtp.gmail.com with ESMTPSA id a4-20020a170902ecc400b00180a7ff78ccsm14790814plh.126.2022.11.23.11.17.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Nov 2022 11:17:07 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, minchan@kernel.org, ngupta@vflare.org, senozhatsky@chromium.org, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com Subject: [PATCH v6 6/6] zsmalloc: Implement writeback mechanism for zsmalloc (fix) Date: Wed, 23 Nov 2022 11:17:03 -0800 Message-Id: <20221123191703.2902079-3-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221123191703.2902079-1-nphamcs@gmail.com> References: <20221123191703.2902079-1-nphamcs@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669231028; a=rsa-sha256; cv=none; b=zdfRmlelIqoqWB5cy0OUVYsZuuZeah9GnbL0JfIjVE0QNU/opK8rFI4CMc0dEfr3KUCCR+ L3eMFdJeFb3vXCeI/bnQFQAjtIqKBKzMzg8y/Arwrca2KMHDWFmSnmfXzpp1EwZ6+mh68V jdnGtnCXDUcngPXDz3zd+fNDtreOddg= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=ixSC8hDn; spf=pass (imf04.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669231028; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ruO+YinHt02cD7LDLng29WAK5pZvQbCFBqvYCl3hrTY=; b=6x+xAL+uM7yl8K6zFcY01cb1qU+3f+xt8NFOjLkTTcYo7mI2j4FqMQ/iGGcWhuoRxH84+t /r/Wt1OQs4CbsCWAE6zDW7ddDT1l+R34fkOiNBNFEimdj8Wed0/2kf93V8eWwg/QTH87sw Jx6/5mrClOFNrYFnKarpP8Fe6zlerio= X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 877DE40009 Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=ixSC8hDn; spf=pass (imf04.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Stat-Signature: g1cp18sc4bnnmbx6wg718j7xofnjhu3z X-HE-Tag: 1669231028-498235 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000023, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use get_first_page(), and add cond_resched() in retry loop. Signed-off-by: Nhat Pham Suggested-by: Sergey Senozhatsky --- mm/zsmalloc.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- 2.30.2 diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e2bd41f37b5b..ab4fa17a0fad 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -2488,12 +2488,13 @@ static int zs_reclaim_page(struct zs_pool *pool, unsigned int retries) remove_zspage(class, zspage, fullness); spin_unlock(&pool->lock); + cond_resched(); /* Lock backing pages into place */ lock_zspage(zspage); obj_idx = 0; - page = zspage->first_page; + page = get_first_page(zspage); while (1) { handle = find_alloced_obj(class, page, &obj_idx); if (!handle) {