From patchwork Tue Jan 23 18:45:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13527975 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC2B6C47258 for ; Tue, 23 Jan 2024 18:46:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 590D86B006E; Tue, 23 Jan 2024 13:46:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 53FE86B0082; Tue, 23 Jan 2024 13:46:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 408A46B0080; Tue, 23 Jan 2024 13:46:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 315AD6B0082 for ; Tue, 23 Jan 2024 13:46:04 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id C0FB21C1602 for ; Tue, 23 Jan 2024 18:46:03 +0000 (UTC) X-FDA: 81711455406.15.730A40D Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf16.hostedemail.com (Postfix) with ESMTP id 1BD2618001E for ; Tue, 23 Jan 2024 18:46:00 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=QVsUy2z9; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf16.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706035561; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=R0gfHlWpVUijxd+/zOQuAtY2BByIFtyAPQHUVgEt0ks=; b=wKh+1ZYvTtO/eUbkww5aGYQy62AYnYb7Cug7PgygvBL+eBt3GY3I1s5h9zRk3wdYEYbK0K ll0dNfQmMfIz+doE0EomzXcUuZ308H931i8pppjjUksTPOi4aoFhuutOUgZe+PaW5J3SaD dDcy3klv3eusHLB7lvnJntSvJP3EQ58= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=QVsUy2z9; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf16.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706035561; a=rsa-sha256; cv=none; b=vKyPKRrJhLa+2VvAksK9Hm1q9kvnMIHIdSC8St4T2U+AMRJ6ZUBanCtoWwrFvwedY8gKcn uWU8q+mPjLkJ5cgTqYYzzYa8822klsZ3/ShGt5RTaDaID/uP8jTNFbF5SII57894FAxejN JhoSOFl605ClkHGYOo7yFzxvqPnp20Y= Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-1d71207524dso18604475ad.1 for ; Tue, 23 Jan 2024 10:46:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706035559; x=1706640359; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:message-id:date :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=R0gfHlWpVUijxd+/zOQuAtY2BByIFtyAPQHUVgEt0ks=; b=QVsUy2z9mhV/Tnu8MRrk8xRSBIR7sznnxVYqfb9OUT0ZIS9F8kUWjVxUAlOgJf/vKp MDJyJ/Nb1jMwdTOKLZvPGjjlByeVNK56OuhPdpzgz7PLTiBYqse5Ai7oJ+8NThmZHoCE ro2MRIJ6rO7jtnT+WwcLpWUE0hhTaZC/QzZ0mU7NvFP0ltrL/NZXVMEVqYIzKdU8F+f0 5/SmQ+4J7fuH6jWwhaf3bNAo30cOTz39u6q+gW39v6bubF9AK4t/7Jh6HSZNzWt9uHTw VAGFHcI8z+85R4QobLsFP4R0xvX326mGLPj6PVWIw6I2SDD1L+tcZtHiFL3KJh91tCqe NWWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706035559; x=1706640359; h=content-transfer-encoding:mime-version:reply-to:message-id:date :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=R0gfHlWpVUijxd+/zOQuAtY2BByIFtyAPQHUVgEt0ks=; b=fZXEwCqpEvaiJD+O4QCz3QDUotRxwJkovGsiUKgGQzmwxquIW8mzk8BVnbwXfPUiuu OizvHHjV+at2tbWpI54inY/GicTORhkgzoeVkAbc9wuJzmsIsXHxrJBE51mXzTabIqgN ZrZsMXSpUEAe0pjdWLzyoLfq9F9pEknWE7NxkDiDfS9Npd++HtSozGozcsHsyaI1WBpO cAy8OzvBS1nrXjSTigHH0XWFvrGTGHFGqVQoePDC3nnplpVSjYGxtwgnYinZ2NvExYn9 6hUnKPfevT/WpAYYFnYh14QF771GjyAlFGVYH+ZlLjUyUG8/5ec2yovZ6oBkFL8Z5pXb ihSQ== X-Gm-Message-State: AOJu0Yzag5LLadX7nokl7s61/uIBS4bExxiqxG87MzYekG1/nMt8P+tH KiXddMODiVm9djiZPjmFJSw77McPUq60AqTT+FoXdG3PZzZMA4TYyuJV5L+jfirLwQ== X-Google-Smtp-Source: AGHT+IGON/iWnemM36uXYUE9iFJrD/NcRYwOXW0byNszw6qV2WjB4uiKaurl4vKO+wse/3YxjfK+XA== X-Received: by 2002:a17:902:b205:b0:1d5:ebbb:973c with SMTP id t5-20020a170902b20500b001d5ebbb973cmr3865892plr.1.1706035559165; Tue, 23 Jan 2024 10:45:59 -0800 (PST) Received: from KASONG-MB2.tencent.com ([1.203.117.98]) by smtp.gmail.com with ESMTPSA id d5-20020a170902b70500b001d74c285b55sm4035196pls.67.2024.01.23.10.45.56 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 23 Jan 2024 10:45:58 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Yu Zhao , Wei Xu , Chris Li , Matthew Wilcox , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH v3 0/3] mm, lru_gen: batch update pages when aging Date: Wed, 24 Jan 2024 02:45:49 +0800 Message-ID: <20240123184552.59758-1-ryncsn@gmail.com> X-Mailer: git-send-email 2.43.0 Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: 1BD2618001E X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: wzm6xoi34w8x5to61xtqr7gbckb5f89a X-HE-Tag: 1706035560-186186 X-HE-Meta: U2FsdGVkX18TO01xg8OvsLL2aFj3+D92IQ8zF/ltWlPHsAhR1Fw+5rGxwZ1ZiKe1DRCXXCotEX4KLNvL/OZCUHsgoIZYqIYExhj4E+/JGzpfEJxPIhuVijjj4kBr3r9g/6GHWJBkr1yv/C2iet606wHllvow6k7Fp6nLilCz98I/AjtCOxCKUHjLVpA6BBBkMVIBf+qaXbGYVuzc6g7dH0Os2SRiqL6/Ah/v5HgdV2YRbKT33FefEYJWlQLFLzONNuiR4LLWGnsehrn/0lJQK2Lua7LOQVyOHaG1Rxk0BnqgbdNqECkOsD1MPyKbgb42zqjwVjpU8xFTMnJB+f9iI1cw6oeuxLFVXBQuIho88yLVARTTj3U/2UqKKYn6hZPtLZWsp+8KgqOp3bAxVPRgEjl4K9UrwTQBE60MhAxihSYNGbA484/s5rXzzXL8gmcOuE0ROrrN4eCggGNWTFLJBJxy680X6uyCGTlK6OqK49YUH4CfofqSVyT7Vn66xiP/crTh3YS2C4obL3Rbyo8POPUwddOJe2QGAtcks0IS5nTCGozYM0WfJaQDdva0R4CYMI7OfimgaH/9MwsDVxBP6jP0VEUFonLtk7/xE4Dr+dvCmFClSDKl8FFfTd8I8e/O0TA0Ab9nguSH4jXHZOJbDTqEa9eCClPyBnWRGEPvHFEta1fFNlri8pTfDW2CWsU1ppQTkWw9duH3f3urtojmcgqbW7/BaoXIWoBunmy8lUPKxv5hQEzxeqOf4FF1xyJpvHEc6BF10RVY7172U0Hu1cWxyA9bE/ZQN8Ej9XZq7+qkLtnn6ksvwUPEMzTNpWLQtEPXN3l3OejJ9eIF5He5LoukOS7V9rpbtxSwSzLmYluXAIQ6tW/zbV+k1zwNncR7LLvDneUoB5CZ6TlolAs2TRN51RYCGeDyQZuAKdOp3DK7mSdezf0qmRFQiATiRWEWqnTlkwEAItsJSWWsdlA UaVkql7O uakZZFwWAolp5PulPiY1JpA2q/rs0P+jttR6feDn0117LANimQYLq0hFYhuLMZoFJLSv+gNt3lkodIJjCOenoNPDXuFKwS2H/CbqMx2t/vmbhdg/Q2sJnJgJT5brDS7Q8/ZgphoHwWn+uubrpFJ26q7sPrkNfGQh2gQZd4ez91QKVF2c/YfcqPcM/7qOKom5PxqcXcKByUKspVI0d/gQ2jIgh+F7462OnBfu3dL7WfTyRRC4zLumCRlK5oIgKqb8JQfa7V8nYrUXh7ZUDKeLXHCAYAp6FSEnsJ4PBgYpWwjF7J9LWCnNLwY6+1A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000099, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Link V1: https://lore.kernel.org/linux-mm/20231222102255.56993-1-ryncsn@gmail.com/ Link V2: https://lore.kernel.org/linux-mm/20240111183321.19984-1-ryncsn@gmail.com/ Currently when MGLRU ages, it moves the pages one by one and updates mm counter page by page, which is correct but the overhead can be optimized by batching these operations. I did a rebase and applied more tests to see if there are any regressions or improvements, it seems everything looks OK except the memtier test where I tuned down the repeat time (-x) compared to V1 and V2 and simply test more times instead. It now seems to have a minor regression. If it's true, it's caused by the prefetch patch. But the noise (Standard Deviation) is a bit high so not sure if that test is credible. The test result of each individual patch is in the commit message. Test 1: Ramdisk fio ro test in a 4G memcg on a EPYC 7K62: fio -name=mglru --numjobs=16 --directory=/mnt --size=960m \ --buffered=1 --ioengine=io_uring --iodepth=128 \ --iodepth_batch_submit=32 --iodepth_batch_complete=32 \ --rw=randread --random_distribution=zipf:0.5 --norandommap \ --time_based --ramp_time=1m --runtime=6m --group_reporting Before this series: bw ( MiB/s): min= 7758, max= 9239, per=100.00%, avg=8747.59, stdev=16.51, samples=11488 iops : min=1986251, max=2365323, avg=2239380.87, stdev=4225.93, samples=11488 After this series (+7.1%): bw ( MiB/s): min= 8359, max= 9796, per=100.00%, avg=9367.29, stdev=15.75, samples=11488 iops : min=2140113, max=2507928, avg=2398024.65, stdev=4033.07, samples=11488 Test 2: Ramdisk fio hybrid test for 30m in a 4G memcg on a EPYC 7K62 (3 times): fio --buffered=1 --numjobs=8 --size=960m --directory=/mnt \ --time_based --ramp_time=1m --runtime=30m \ --ioengine=io_uring --iodepth=128 --iodepth_batch_submit=32 \ --iodepth_batch_complete=32 --norandommap \ --name=mglru-ro --rw=randread --random_distribution=zipf:0.7 \ --name=mglru-rw --rw=randrw --random_distribution=zipf:0.7 Before this series: READ: 6622.0 MiB/s, Stdev: 22.090722 WRITE: 1256.3 MiB/s, Stdev: 5.249339 After this series (+5.4%, +3.9%): READ: 6981.0 MiB/s, Stdev: 15.556349 WRITE: 1305.7 MiB/s, Stdev: 2.357023 Test 3: 30m of MySQL test in 6G memcg with swap (12 times): echo 'set GLOBAL innodb_buffer_pool_size=16106127360;' | \ mysql -u USER -h localhost --password=PASS sysbench /usr/share/sysbench/oltp_read_only.lua \ --mysql-user=USER --mysql-password=PASS --mysql-db=DB \ --tables=48 --table-size=2000000 --threads=16 --time=1800 run Before this series Avg: 134743.714545 qps. Stdev: 582.242189 After this series (+0.3%): Avg: 135099.210000 qps. Stdev: 351.488863 Test 4: Build linux kernel in 2G memcg with make -j48 with swap (for memory stress, 18 times): Before this series: Avg: 1456.768899 s. Stdev: 20.106973 After this series (-0.5%): Avg: 1464.178154 s. Stdev: 17.992974 Test 5: Memtier test in a 4G cgroup using brd as swap (18 times): memcached -u nobody -m 16384 -s /tmp/memcached.socket \ -a 0766 -t 16 -B binary & memtier_benchmark -S /tmp/memcached.socket \ -P memcache_binary -n allkeys \ --key-minimum=1 --key-maximum=16000000 -d 1024 \ --ratio=1:0 --key-pattern=P:P -c 1 -t 16 --pipeline 8 -x 3 Before this series: Avg: 50317.984000 Ops/sec. Stdev: 2568.965458 After this series (-2.7%): Avg: 48959.374118 Ops/sec. Stdev: 3488.559744 Updates from V2: - Add more tests and simplify patch 2/3 to contain only one gen info for batch, as Wei Xu suggests that the batch struct may use too much stack. - Add more tests, and test individual patch as requested by Wei Xu. - Fix typo as pointed out by Andrew Morton. Update from V1: - Fix function argument type as suggested by Chris Li. Kairui Song (3): mm, lru_gen: try to prefetch next page when scanning LRU mm, lru_gen: batch update counters on aging mm, lru_gen: move pages in bulk when aging mm/vmscan.c | 145 ++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 125 insertions(+), 20 deletions(-)