From patchwork Thu Oct 24 13:19:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 13849003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD170CE8E70 for ; Thu, 24 Oct 2024 13:23:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 45D156B00A5; Thu, 24 Oct 2024 09:23:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 40D5E6B00A6; Thu, 24 Oct 2024 09:23:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 262BB6B00A7; Thu, 24 Oct 2024 09:23:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0C1266B00A5 for ; Thu, 24 Oct 2024 09:23:13 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 355FDACCA7 for ; Thu, 24 Oct 2024 13:22:36 +0000 (UTC) X-FDA: 82708561614.02.927E8F1 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) by imf25.hostedemail.com (Postfix) with ESMTP id 1E52AA000F for ; Thu, 24 Oct 2024 13:22:56 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf25.hostedemail.com: domain of yukuai1@huaweicloud.com designates 45.249.212.56 as permitted sender) smtp.mailfrom=yukuai1@huaweicloud.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729776139; a=rsa-sha256; cv=none; b=juBSE6GOFkKSgy0wdfUFYGKDnpuruJOsEgAdrM4/vIOZnWc6m2XbCXwOdUHYXwskmZHuXO oumZ2rhiwNpu97M7BBWEU7VIZ3ckhh3uicZ+ImA8TncdRY+RgBiwBeqsC9pEsEsx0LEfIQ 4pU5RTg/8P7aEjIWDtRkGcko/SxTR00= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf25.hostedemail.com: domain of yukuai1@huaweicloud.com designates 45.249.212.56 as permitted sender) smtp.mailfrom=yukuai1@huaweicloud.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729776139; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rO2WmhiUXY6EisYSk0lLQ4XUVIZVax/1dcwjxZ89DjM=; b=nKmFx/wCXPG1izyMNp+X5V5gnvRxcY1qolj9ZVJyXBcRS8aN3SnfejrdOGANqfXWmqCv65 DQ9W0Cpcg/m0oDqGrBpqaRjCe92psuZHbb4CuFUgb59Dgt+90Bhj0w3Xou9scL5cJ4Mhla sz/xZSqH9zsVd3dh8wg8ZVn4sFLji1E= Received: from mail.maildlp.com (unknown [172.19.93.142]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4XZ66J27wnz4f3jdg for ; Thu, 24 Oct 2024 21:22:48 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id BFE181A0359 for ; Thu, 24 Oct 2024 21:23:05 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCHusYpShpn7tb6Ew--.444S14; Thu, 24 Oct 2024 21:23:05 +0800 (CST) From: Yu Kuai To: stable@vger.kernel.org, gregkh@linuxfoundation.org, harry.wentland@amd.com, sunpeng.li@amd.com, Rodrigo.Siqueira@amd.com, alexander.deucher@amd.com, christian.koenig@amd.com, Xinhui.Pan@amd.com, airlied@gmail.com, daniel@ffwll.ch, viro@zeniv.linux.org.uk, brauner@kernel.org, Liam.Howlett@oracle.com, akpm@linux-foundation.org, hughd@google.com, willy@infradead.org, sashal@kernel.org, srinivasan.shanmugam@amd.com, chiahsuan.chung@amd.com, mingo@kernel.org, mgorman@techsingularity.net, yukuai3@huawei.com, chengming.zhou@linux.dev, zhangpeng.00@bytedance.com, chuck.lever@oracle.com Cc: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, maple-tree@lists.infradead.org, linux-mm@kvack.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH 6.6 10/28] maple_tree: use cached node end in mas_next() Date: Thu, 24 Oct 2024 21:19:51 +0800 Message-Id: <20241024132009.2267260-11-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20241024132009.2267260-1-yukuai1@huaweicloud.com> References: <20241024132009.2267260-1-yukuai1@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: gCh0CgCHusYpShpn7tb6Ew--.444S14 X-Coremail-Antispam: 1UD129KBjvJXoWxGrWUuw4rXF43Cr4rCFy7GFg_yoW5WF4Upa 4DWa45K39FyF18Krnavr45Zr9Fgr1ak3yUta47Gw15XFyDtr1fXF1DAa48uFs093s2vF13 Aw45C3WUCws7GaDanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmS14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_Ar0_tr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr1j6r xdM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0D M2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjx v20xvE14v26r106r15McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1l F7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2 IY04v7MxkF7I0En4kS14v26rWY6Fy7MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY 6r1j6r4UMI8I3I0E5I8CrVAFwI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17 CEb7AF67AKxVWrXVW8Jr1lIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF 4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBI daVFxhVjvjDU0xZFpf9x0pR4E__UUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-Rspam-User: X-Rspamd-Queue-Id: 1E52AA000F X-Rspamd-Server: rspam01 X-Stat-Signature: hj3anfaenq6oebmpqspwrdegzyqq4fqn X-HE-Tag: 1729776176-765353 X-HE-Meta: U2FsdGVkX18BMxIXlUauagjfoaLPVaMfyIvdWQi1kthPVi1PSmLPmBv/s++esvrIQC2iSxkPOowS2kNAucB0Kpmb1Xjapi1D0t0biFlhRAAe9oQJog419ZbgId1TeupGolGIQ3c7O0ZY4+6CHvVcOiSxNxkNlzDlqw+gw3+I5wxoNqcvvi4A68vbLNg2CaueUkDBrPqxxlY76r88MTITrWiIlBdmz0VSLGfWQVePNy9tKw2FfCiV5hZ9CI1i1Xaw/QUko2ugPTNCv0dA9Eh15h5BGiAez4aAQ8funAum+pSuUVoP99JB5aIykoJOjV5DjmCWB3rd7BMh8dGmxjgLZBb6QQDswSsPe6vWyUHoFs4MIoM7ALnV0Jj4u++hAs7IAQP5+42WHrwk3/hUnvOPh729d+tXXkSWBSidGgK/mo8+rdw+XEk4lH7Y4igipJWDRvxV+upSbkrfdCqBWtHOT13oQAxEKHu6usa8485ObF6OK/1cTRR5daUj9L9IROxEBpgR2dD0P0x/81N9H8a3PH9ozuFTedyhX0vGpJzM9iA4VHtv4taBlW4YwlFKJ0umwbKMZxagYcm2TIZInVVdISK2LWYr5IgacAamXqg19VBxhIwo3E/ov8u0NGt33vRN492Ol2RWiLujAN/2XNvIjhC2/IbbC+7i8qIgWofrRCfC7CPtcwqjuzjyHgJxgKAEI5opiETYVCOjhcGKHIFXFJiuM7TtlpVz5E/WFQlpZ0pqjAr+y9SH0yQCOyyq523IMteo/O4YIdyeZLdRO3hN4INtxcql26GrLszhoWIcf8lBwwG4PC6HuL7zCfYjQ7h1+79T/tb2tKhKxhWKoI3no7Bp51IwFQei1KHTU2wYIZOpLbT8mn7Y2AEBIQy0LKgRnOTbcTwmh8oQHkviKw8YfJ9GIJeVctjGfAKv+qfrM9jT5rbhGyq4FRsi4AljhBfps1OZqrLKOTkqrI8w0Ry +bVvJbyH Onui7DrnfV2ptX/3YNPkAn3FJ/xlorxASPCKiVaC1U6/QSjN5j0XDKDNSrNWwh7pa22nMttlI+xOSS9+rOPoVCtrWojcMi4vd9WynXVXTYz35/uFrL8KZ0b0Co/N2GiB1Me5IqoEkfZkrONF+Yt88O15B6evDvp8R3g1ksLuZg1Dh9vyKVViRq2nFm5d9IVp8aqiNQ+D5g3cvAKh2Y2n/NzbQBcGWf7V4SBQDgIR4iuVU/vs4sQr9xvTS5I3jG/+OAwq0kFEj/8Rn8QHJir1PVMl9mKdq52WxYCIXaVzaBezMFrMRtQiFylNYNiAY4NeErPpq X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Liam R. Howlett" commit e9c52d8940cbfd94b36035bbebce7f55954e7728 upstream. When looking for the next entry, don't recalculate the node end as it is now tracked in the maple state. Link: https://lkml.kernel.org/r/20231101171629.3612299-6-Liam.Howlett@oracle.com Signed-off-by: Liam R. Howlett Cc: Peng Zhang Signed-off-by: Andrew Morton Signed-off-by: Yu Kuai --- lib/maple_tree.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/lib/maple_tree.c b/lib/maple_tree.c index d19fb14a9635..e0dcc8412da0 100644 --- a/lib/maple_tree.c +++ b/lib/maple_tree.c @@ -4539,6 +4539,7 @@ static inline int mas_next_node(struct ma_state *mas, struct maple_node *node, unsigned long min; unsigned long *pivots; struct maple_enode *enode; + struct maple_node *tmp; int level = 0; unsigned char node_end; enum maple_type mt; @@ -4591,6 +4592,10 @@ static inline int mas_next_node(struct ma_state *mas, struct maple_node *node, pivots = ma_pivots(node, mt); mas->max = mas_safe_pivot(mas, pivots, mas->offset, mt); + tmp = mte_to_node(enode); + mt = mte_node_type(enode); + pivots = ma_pivots(tmp, mt); + mas->end = ma_data_end(tmp, mt, pivots, mas->max); if (unlikely(ma_dead_node(node))) return 1; @@ -4625,7 +4630,6 @@ static void *mas_next_slot(struct ma_state *mas, unsigned long max, bool empty, unsigned long pivot; enum maple_type type; struct maple_node *node; - unsigned char data_end; unsigned long save_point = mas->last; void *entry; @@ -4633,12 +4637,11 @@ static void *mas_next_slot(struct ma_state *mas, unsigned long max, bool empty, node = mas_mn(mas); type = mte_node_type(mas->node); pivots = ma_pivots(node, type); - data_end = ma_data_end(node, type, pivots, mas->max); if (unlikely(mas_rewalk_if_dead(mas, node, save_point))) goto retry; if (mas->max >= max) { - if (likely(mas->offset < data_end)) + if (likely(mas->offset < mas->end)) pivot = pivots[mas->offset]; else goto overflow; @@ -4650,11 +4653,11 @@ static void *mas_next_slot(struct ma_state *mas, unsigned long max, bool empty, goto overflow; } - if (likely(mas->offset < data_end)) { + if (likely(mas->offset < mas->end)) { mas->index = pivots[mas->offset] + 1; again: mas->offset++; - if (likely(mas->offset < data_end)) + if (likely(mas->offset < mas->end)) mas->last = pivots[mas->offset]; else mas->last = mas->max; @@ -4691,7 +4694,6 @@ static void *mas_next_slot(struct ma_state *mas, unsigned long max, bool empty, goto overflow; mas->index = mas->last + 1; - /* Node cannot end on NULL, so it's safe to short-cut here */ goto again; }