From patchwork Mon Feb 6 09:25:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13129547 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D30D2C636D4 for ; Mon, 6 Feb 2023 09:26:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 752496B0073; Mon, 6 Feb 2023 04:26:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 702516B0074; Mon, 6 Feb 2023 04:26:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A5346B0075; Mon, 6 Feb 2023 04:26:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 474FE6B0073 for ; Mon, 6 Feb 2023 04:26:17 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 2075DAB124 for ; Mon, 6 Feb 2023 09:26:17 +0000 (UTC) X-FDA: 80436335994.14.BEA5C0B Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by imf04.hostedemail.com (Postfix) with ESMTP id 4FB0840023 for ; Mon, 6 Feb 2023 09:26:15 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=esixuwoW; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf04.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.175 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675675575; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6EYzqDYVAUxUcl5H7L0+RBCt9aPnrJa6Jn52QssbLoc=; b=YtSl0xy0BXMfSWVgWca/Qjy8/CcmDamC4w7ZD0U2brhrzDbjT+U/Pth8XCzc56gNo5BEbj WlRUub3N0qpAjAYGs8Z54ioPxJHTCNMp0tSAXDoNIwcexFD3SMNZ5nHbCVdsvtvv3yU0Py z0tzZkFDvIHFPUz7fzd3sTJ+efX1Hww= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=esixuwoW; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf04.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.175 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675675575; a=rsa-sha256; cv=none; b=G3F70qdlAidKT2GYqJ3e2qbXiQ2brYJGs92L3bsFiPbP6u9SHHY9RYE6Mf63UWqvxJwgEA 1RldNXRklqmlKrC2ig/jDUoiauPOTiEc2gC+kI1Q2Obdj14ePa04g3YUrV20EBTdxOhL92 Y2Vd1/9vLcmn8BiSv23AzuKo8SESdXs= Received: by mail-pl1-f175.google.com with SMTP id iy2so1346986plb.11 for ; Mon, 06 Feb 2023 01:26:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6EYzqDYVAUxUcl5H7L0+RBCt9aPnrJa6Jn52QssbLoc=; b=esixuwoW1XXvVaRSq/qeOoODQJsPCnwslpXsBr3oCxI2fa6miNpo/FU3qkmcM/DARO WFg52ztwqqeZFRoFvpPRhE8Nr3izanJ0gAkXqXQuZeRAGgnqBFQpM3XENYcwKFYJW5ev J2YKroA6WQzb4iBi3qeaWXRsDCQKVhFsZCEps= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6EYzqDYVAUxUcl5H7L0+RBCt9aPnrJa6Jn52QssbLoc=; b=S47LVqejvoyVSKnEfPYOR+CkVwW9X2HNFowq6pZ400VJugEDntS8VKvGUOdii5A+w6 N0lTi1sRqbUen99UaXbTlwwQa3ABuI6oBN3EL5gbv7gKw6KEni+e1wgXvwF+9TJ2mJKd Ibm789gL2iB4IV3Cba0xtRwK5jMcreKc5qZsZz+jl4OvEzUwG30XcwdG2xSwMXOpuoCP I9poqomHiNKyWoZjQrW4K204326prCwzCkZ/OJQPRIEyhuR5WRKolYklt6RPXGJGJQr+ SOEIV4RCIp7EHGvHmN+h0FcZG4DW/HDbGqL2/poWO3ZUjDs4nLx5s64V713l76BXHemJ 9Xwg== X-Gm-Message-State: AO0yUKUvuSOe63RgfALk/lBiKE+YxFe5Ey4Tvs6+iMwF+CwIb4YyaUT8 bM7dRDdLToMqgvkfuNF6vIIVY42urogB55q6 X-Google-Smtp-Source: AK7set82/lVo4oVJUvptLjyzdH+UDUdDHjKosk+NiYryQgYiSP/6kn1HcleZQIM1e6R8tYxqNtU+VA== X-Received: by 2002:a17:90b:3b4d:b0:22c:6bb1:55a4 with SMTP id ot13-20020a17090b3b4d00b0022c6bb155a4mr20256388pjb.45.1675675574750; Mon, 06 Feb 2023 01:26:14 -0800 (PST) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:4a1b:4fdb:174d:8f36]) by smtp.gmail.com with ESMTPSA id ga23-20020a17090b039700b00228f45d589fsm5663008pjb.29.2023.02.06.01.26.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Feb 2023 01:26:14 -0800 (PST) From: Sergey Senozhatsky To: Minchan Kim , Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCH 1/2] zsmalloc: remove insert_zspage() ->inuse optimization Date: Mon, 6 Feb 2023 18:25:58 +0900 Message-Id: <20230206092559.2722946-2-senozhatsky@chromium.org> X-Mailer: git-send-email 2.39.1.519.gcb327c4b5f-goog In-Reply-To: <20230206092559.2722946-1-senozhatsky@chromium.org> References: <20230206092559.2722946-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4FB0840023 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: oxhu3joec5e8maq7h4roppgg8g8is1oz X-HE-Tag: 1675675575-961717 X-HE-Meta: U2FsdGVkX18t/n4SuZNmNLkKJNtu6o0CezBQJOZC8I3CH8cM5Mjg0gRUxiwMLG8eRdUntW63gwjseUGX6ppbbC3mYKh+C4bmfTfrHXiSDVTFY0OplSG4cgaZKSAv236nm2lvBn4YKKT49/+85JrVWrMosNnzUnn+YErQOZfsYoT6tmrbS2pKR4RemYwI01JkZ/KfjRv6okg1c98PvkpZvIOltiNebWtMYtJi3Mjx7rttvmqXCj2wM5jJWH8eh7j22yV5a9gg8vAVTTOF7I7ehYFVjHgRyxDWYozs9rxL5eK71Fz+D4KqWJzfqStwFU8VSqe3NhurCs75csybNUmkKPXlg2O77fDXzY5A3JEjee68cHkSVvF0QhBcuWjquhnPhgzG9ezB91YJkSc2RqiEWlgmE4JV13pOad1Wu3IqXqMHiDrDbamA4GOpIyhRgHdkDPFAiKnoB9BYlT/ZintU709oCOFtSX3u1PGyeLe58HhHEiD6t79riK87Qo/AC5HfPL5xYjOT7d+4+14UmVNgg481Jav4vGH4vZw7iEvug5iMXi7NpIf35SH94CXpOxM/NEcGWqdiPDgdM/X8kzfQaMpOQQDwCYWx+zPRWwlb1FdHv49h7vRTqId5aEtCUafQhl2+lDIH+3DFyvf7NvSVc86fvjGPrSP39sF6R2wEJVMZ2WJUIMjPzGqHalLQn9EJoTN9SZzjwdwJlYyUaFm0ycLw6QghT6cJDqEworAoGdOyjdIMfHN02vRPWzz4nQpBsNdgbtgQrClP8UDhSGnJOXZvyDvWgLGs8beYmEuLOrCKKSo6/wnzKlv2y0xS7riTY72livaJNqqia4RClIJz0c1A3dMM6cxapEHTgn59lBPPxkquDuwalMe79w31u+Ov4UEzACAa4MOZqJvznl8GowQJjHLfFhbeUJGpRdmZVPD6OfhOaJpQYpIjk5DKDKgIVhX6xNDlR8sHIHj38Np Kka/1aWa 1UUhADw3PfbwY73YUF1ggDr7dSUA6NNyzG3ggFXflv9JKSv0jKY2Dy+DiBah0RB/iMMHGlvxo1STPD6NA0BAEvOuTYG+7ouwp34kNvRDvEtEzNouY4rD26feFLrxUqNdcSSKuD442Kf2v/qRUoXyfLP8AoXCoiZeWk2IuFgi/tkrDGrAnGkkcRyfX8bViZeMdnwj7uJb8l2FUuj2F7g/Cwc+3qKYHlwVz3pKjt2GZOKzzk8ePotlUeAGRWE0/8Ugb8jM4wekRd9ztntNHXueR5cTf3+r3iXx+XDjUrNzpX2tCP/6SP78Cir70oknhMBrPzF+f2WY7ALB8rkAQyF8uPAawv3Ouyqxxrb/bIBj+JYXyGWdazy/mW106dY4Ck9/nAXQmfxaFLNZUtTSCi/tpuhdSqgJXCFc9nwAxrgMvzEa5F4vJhuvilk4vTzglyOM4HfRhnoEs1woJOOVchMNsvNRxq7Ryo1R9SFW29s50RaOa3ws= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This optimization has no effect. It only ensures that when a page was added to its corresponding fullness list, its "inuse" counter was higher or lower than the "inuse" counter of the page at the head of the list. The intention was to keep busy pages at the head, so they could be filled up and moved to the ZS_FULL fullness group more quickly. However, this doesn't work as the "inuse" counter of a page can be modified by obj_free() but the page may still belong to the same fullness list. So, fix_fullness_group() won't change the page's position in relation to the head's "inuse" counter, leading to a largely random order of pages within the fullness list. For instance, consider a printout of the "inuse" counters of the first 10 pages in a class that holds 93 objects per zspage: ZS_ALMOST_EMPTY: 36 67 68 64 35 54 63 52 As we can see the page with the lowest "inuse" counter is actually the head of the fullness list. Signed-off-by: Sergey Senozhatsky --- mm/zsmalloc.c | 29 ++++++++--------------------- 1 file changed, 8 insertions(+), 21 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 3aed46ab7e6c..b57a89ed6f30 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -753,37 +753,24 @@ static enum fullness_group get_fullness_group(struct size_class *class, } /* - * Each size class maintains various freelists and zspages are assigned - * to one of these freelists based on the number of live objects they - * have. This functions inserts the given zspage into the freelist - * identified by . + * This function adds the given zspage to the fullness list identified + * by . */ static void insert_zspage(struct size_class *class, - struct zspage *zspage, - enum fullness_group fullness) + struct zspage *zspage, + enum fullness_group fullness) { - struct zspage *head; - class_stat_inc(class, fullness, 1); - head = list_first_entry_or_null(&class->fullness_list[fullness], - struct zspage, list); - /* - * We want to see more ZS_FULL pages and less almost empty/full. - * Put pages with higher ->inuse first. - */ - if (head && get_zspage_inuse(zspage) < get_zspage_inuse(head)) - list_add(&zspage->list, &head->list); - else - list_add(&zspage->list, &class->fullness_list[fullness]); + list_add(&zspage->list, &class->fullness_list[fullness]); } /* - * This function removes the given zspage from the freelist identified + * This function removes the given zspage from the fullness list identified * by . */ static void remove_zspage(struct size_class *class, - struct zspage *zspage, - enum fullness_group fullness) + struct zspage *zspage, + enum fullness_group fullness) { VM_BUG_ON(list_empty(&class->fullness_list[fullness]));