From patchwork Thu Feb 23 03:04:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13149801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2CA8C636D6 for ; Thu, 23 Feb 2023 03:05:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7B02C6B0078; Wed, 22 Feb 2023 22:05:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 75F786B007D; Wed, 22 Feb 2023 22:05:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 627756B0080; Wed, 22 Feb 2023 22:05:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 548AB6B0078 for ; Wed, 22 Feb 2023 22:05:04 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 10C20A0F1B for ; Thu, 23 Feb 2023 03:05:04 +0000 (UTC) X-FDA: 80497064928.23.A35DF5A Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by imf23.hostedemail.com (Postfix) with ESMTP id 3C92C140002 for ; Thu, 23 Feb 2023 03:05:02 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=CmpbZQ5m; spf=pass (imf23.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.43 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677121502; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7w1cKhAAphXJ2vOUap7qolI6gnUUf7d9gByR138SJog=; b=c3us1dyNXpW7uEeokLRKD70JsIQGnb2qTpP2mkMLYJp/E4GtPiGmsm0lRjTzO1c2LYw0Lp VuCuCyU0kJmsOHrm6Isj3MVC03H4fk8dygTfK4jIhFlK7AjksHbz8Vrt17zymXjGYpH5ex O4Y07o93AicDhXEeAq+i2pX79DEIqRU= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=CmpbZQ5m; spf=pass (imf23.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.43 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677121502; a=rsa-sha256; cv=none; b=adgEN21OGBBAbyCRNq6g9glfgNNstV75G2j4BY+qT2ARTiJEi0AAYgSSWxj9WmdQar3X0u GCqohycqgcsTvlSQLvfgI0iexMgQoowuHEwwrYOXTCMKT5A0thGJZnOPfdpdb81fGWupxM cwEr0zua6EE7VFIKOpKZHKzk7Xkk3/Y= Received: by mail-pj1-f43.google.com with SMTP id cp7-20020a17090afb8700b0023756229427so2350312pjb.1 for ; Wed, 22 Feb 2023 19:05:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7w1cKhAAphXJ2vOUap7qolI6gnUUf7d9gByR138SJog=; b=CmpbZQ5mXsvkYH52sH4fUdQAKboV5mAC+PL5L2EQo9+52dN8FTsKfuwkHN31kEURgt zJmYtdNdQZoFlGbZMbVJxlfCAiCY1teEF6fHg9QB2ulPwRSIVYrc1aMPXJGQEnc5XvNi xpv0MPmkAscsnxtGgUib+qZbsAdfI51mc7cf4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7w1cKhAAphXJ2vOUap7qolI6gnUUf7d9gByR138SJog=; b=j0/+GcEVLbYJZkKAgMvZ1gNYk/xrGf92tOG0KEPlVAYB4TzlhA3Fshqsq4LWDED24Z 5JRXNpHyMpR8jrEjOePwU2uKrFTabzl3kTBUwPzWxRB2D88TJzmL39QINAAYPBbpQUIZ UQDLp08iVSpHtzyOszSWQVLgvNTDfjESEuHajkRQsSiT2h4+5Zy4mYf3fsKmYJbTtPFL yimd+9ofJQfjiOT7uc+seOAAQs7/rVJqi40LSMpz7ALb12kv76rhmSHgBdyg1vpNmwC8 zQh56rWl2AkEjbD/UH6Fq27yCSe51liMiikk0NQ9yYbrQWgYkYmgsoDxcPrbhgG7NjDD VdoQ== X-Gm-Message-State: AO0yUKXBRobeRU6d3jIXFRo3kBCQ0U05H1bIEredYM7uLht3RbDdvGXt 7OXJVRREweLKXrmD71DH2ae/3A== X-Google-Smtp-Source: AK7set/dsMOb60qr+5+6Ogrh8xQz946tMFAG3MvgpCazePEMrr3hTDtUIcO7lCYLWoHiifUoUwS7ew== X-Received: by 2002:a17:902:7243:b0:19c:bcbc:a3dc with SMTP id c3-20020a170902724300b0019cbcbca3dcmr685857pll.25.1677121501195; Wed, 22 Feb 2023 19:05:01 -0800 (PST) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:6de2:9e85:b508:57b8]) by smtp.gmail.com with ESMTPSA id jl21-20020a170903135500b0019926c77577sm608520plb.90.2023.02.22.19.04.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Feb 2023 19:05:00 -0800 (PST) From: Sergey Senozhatsky To: Minchan Kim , Andrew Morton Cc: Yosry Ahmed , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv2 1/6] zsmalloc: remove insert_zspage() ->inuse optimization Date: Thu, 23 Feb 2023 12:04:46 +0900 Message-Id: <20230223030451.543162-2-senozhatsky@chromium.org> X-Mailer: git-send-email 2.39.2.637.g21b0678d19-goog In-Reply-To: <20230223030451.543162-1-senozhatsky@chromium.org> References: <20230223030451.543162-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 3C92C140002 X-Stat-Signature: 675mxdooxmj9p9yy88xqxum816w9gq9j X-HE-Tag: 1677121502-622565 X-HE-Meta: U2FsdGVkX18FeL/WDEqdKHp1+75TNFCXtvziskxIlaLy8XS8oNQoy3pNm2u0DREzdh/mMKkupL+zFRBrBl0KRdeno5s5nFMIOHjUgzRh+nUXqATvJydTfFEa+FhuabhvJS+x10ozz2DCh/nT42OMNitb875trFlvJ7teGBgMLZc8Wd/HpXLJ4S1rGYVW+jXG+r7iLAjLsYH7nWpNS4pLoIo+6DbNbhkSE7/U8LXEvQ1AruzjVw9vAIn8m3MrSJHo+yVXfRZve7Unkv0ZMOrQtaclr3LJkfMfy79Ag0pjoOQx6yhuV8bs5AeMEDdsSG8poKrVKop15gWOcTzBYwmaYukkQwcy0pBdM3ogJi5F9GF82D4esL8NiC8aQ1eo6xmN50ipmdAM2zZjB5tCBiIdK9/7ved0JTTc8Aeqc4XpfRxRT+xrqB6+SIrViRuGI07Wsu3yNlHwL1qq4wMBCBUSQN1IynKSg0wq7nwNuu1rsN9QaOXdOh0i1i7hfPgTmKfD+i0NiiZiQerAhXejJ0Ou8aGThl0m5wQuje2m/hJehG08v/7zaMGa878L+t1dPz57jccJ974n5Z7VSi/Pd1i2fBxmN0Hrztq13NF0vUdJVu1GNqIlrK6EP4YeqQGcH8LV3/VzgpWvO13lXK+aP8FxCrXrtiEGSAHExBQ8/mWSxbt3mZN8J1tufOcthT6Kz3/t11eQrvea1c6324MbHVbc3swpGyWX021fOFNFco49x3rjeFy+yqN8Ln54ekV/6bbaBLLUEPk7JAXbYNvu/bA0Q4Fw6bCp2G6h5K+0Q7x6AxwEbsqVkybNm0A5gRPuEu0cod4KQR2LdjJ9vhZgMzqHyAWyaGzcphYv59MejG1ZC87QoxftpxD53Q8DoaMLkrWX//8YF3r2jvCbOR29wMgK0SpCpSVbwt63nXm7oei7mRSNwGBtJqWMn0Wsnl8PF/aW1gBpBU4HhT/qZJQxbdW 1Pr/A2xB Bz1GYJOhkDc5YZmR51oE6fHnzlZXQO35ili5VccIGTYFL77nA9OjvjRx0BDTQd817lhygTzWk9d46lhDreKyZsJV5pXAYq5lddg/AHYXO1EoL4AjL4xBdmBGc70VgIkHpNt23gJEcGZCd1VleMKDR7KhWTtbYeWDnOmlzFNF/KpsPOzMJ0/XOa8Z3+fBdYSWX886Li7SYKzlcQbF52T0pdKysnBrQV2iFuZRG7ApeIUSf7PqLxLblBMQDZjB61GnaSop8FueK2dOT/E/ZnBHFa6qNasgOMI8jjXickA2gGvWxv3uCrbafc7WE75oTPFCZHFIAZLnCT4z9sqFC/ekOdbtRN1BZsmGAlyHyZag74/+Gput2YhnjVsbYI9KEI7j194tsX+dkyySRvZfbjZEyeU5KvO6ZQkKEZW27WGTf6zaVKM4BSnBpY837SAADHrG0fLWQXSsghkTVC4whl6CIEafiCZI92y2GAqys X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This optimization has no effect. It only ensures that when a page was added to its corresponding fullness list, its "inuse" counter was higher or lower than the "inuse" counter of the page at the head of the list. The intention was to keep busy pages at the head, so they could be filled up and moved to the ZS_FULL fullness group more quickly. However, this doesn't work as the "inuse" counter of a page can be modified by obj_free() but the page may still belong to the same fullness list. So, fix_fullness_group() won't change the page's position in relation to the head's "inuse" counter, leading to a largely random order of pages within the fullness list. For instance, consider a printout of the "inuse" counters of the first 10 pages in a class that holds 93 objects per zspage: ZS_ALMOST_EMPTY: 36 67 68 64 35 54 63 52 As we can see the page with the lowest "inuse" counter is actually the head of the fullness list. Signed-off-by: Sergey Senozhatsky --- mm/zsmalloc.c | 29 ++++++++--------------------- 1 file changed, 8 insertions(+), 21 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 3aed46ab7e6c..b57a89ed6f30 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -753,37 +753,24 @@ static enum fullness_group get_fullness_group(struct size_class *class, } /* - * Each size class maintains various freelists and zspages are assigned - * to one of these freelists based on the number of live objects they - * have. This functions inserts the given zspage into the freelist - * identified by . + * This function adds the given zspage to the fullness list identified + * by . */ static void insert_zspage(struct size_class *class, - struct zspage *zspage, - enum fullness_group fullness) + struct zspage *zspage, + enum fullness_group fullness) { - struct zspage *head; - class_stat_inc(class, fullness, 1); - head = list_first_entry_or_null(&class->fullness_list[fullness], - struct zspage, list); - /* - * We want to see more ZS_FULL pages and less almost empty/full. - * Put pages with higher ->inuse first. - */ - if (head && get_zspage_inuse(zspage) < get_zspage_inuse(head)) - list_add(&zspage->list, &head->list); - else - list_add(&zspage->list, &class->fullness_list[fullness]); + list_add(&zspage->list, &class->fullness_list[fullness]); } /* - * This function removes the given zspage from the freelist identified + * This function removes the given zspage from the fullness list identified * by . */ static void remove_zspage(struct size_class *class, - struct zspage *zspage, - enum fullness_group fullness) + struct zspage *zspage, + enum fullness_group fullness) { VM_BUG_ON(list_empty(&class->fullness_list[fullness]));