From patchwork Sat Jun 22 07:12:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13708253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D32EAC2BBCA for ; Sat, 22 Jun 2024 07:12:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 59F7A6B058F; Sat, 22 Jun 2024 03:12:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 54F0F6B0591; Sat, 22 Jun 2024 03:12:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3EEEB6B0590; Sat, 22 Jun 2024 03:12:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 212506B058E for ; Sat, 22 Jun 2024 03:12:58 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 88FDA1C2BA1 for ; Sat, 22 Jun 2024 07:12:57 +0000 (UTC) X-FDA: 82257657594.12.87DCF0B Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf14.hostedemail.com (Postfix) with ESMTP id AB985100002 for ; Sat, 22 Jun 2024 07:12:55 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=j4kdspSF; spf=pass (imf14.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719040370; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8riNiWPXJqBzm2BvXgRSJ73ikaCBhCcjPOKGJpl9B7g=; b=bls/vokUs5tww2M/z2EEo/qFtH2E1IGCk3tR6J1Yg9rbPueapIaE+ZZvrhJR6yCeoMO/39 o+IgawbSmZqsumay/vzqQLNId79nOCOEynZG4hvm8fjtBmcX7+6jwC+Q2yyj4UtmPj4YKX BM6/BRtcai3IYi/7qh7gXpoS5Kawhu0= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=j4kdspSF; spf=pass (imf14.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719040370; a=rsa-sha256; cv=none; b=En3Pab6K+N8YzCsfiXWw7rByb1x/e86aKB90ioV9Fka/mtK4SKBtN2SGzeVBc2NWCQkYg7 /wFLO3yL2ZWUEARq6SSHNMPdUuvM9kysIfPWhYndUc+MwS2bVlvS7Ic9cm/HZoP0uIKZSE g6PuAvEnMdADsJ2Drc09OAmGTRMW8QM= Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-1f9de13d6baso17350235ad.2 for ; Sat, 22 Jun 2024 00:12:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719040374; x=1719645174; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8riNiWPXJqBzm2BvXgRSJ73ikaCBhCcjPOKGJpl9B7g=; b=j4kdspSF6K6y4dDyDlnQ4/C/XUqcFJgpFWrFglWZaAe/UWI2yzl/Hgp418BvgNLuY9 WlAWbmnM1SfphLeZWAhgyOWVXAcqx420UoiEfR+P1JusZjc+UT6qjzCCNWq3o5AyRgud yJwJlCKt45lIFFcI46SPDuxat4jawKoSWbX9BhxFDHu7AlyJnEwxoVtLhyn8cf4rUuIq WwYF5mZp3Pv1nU2qbjiH9gwmtSSThfdinx9fM/uGQfun8rLZy1km96z34oIs+8vFDW1i U8t0BuFk2LQjvSrFn+qZNAGABcl9FJCFL6BH23p8ZvjRhYCj2s4Re7nEkpieto/I+ldk vmXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719040374; x=1719645174; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8riNiWPXJqBzm2BvXgRSJ73ikaCBhCcjPOKGJpl9B7g=; b=WnfpHdpCMXyqLmAE0mQwvoDnneIsj8O6WRyl7UcbB4kZpI/W20oqhUeJKe0COE4aJw zZ2UK2C8nqFydT6cesoh+vckDrNrNRJy+FtWzotvJcs/iPJ3FqmT+WUZMTEN/f4tZATT YAK9WwI9LYkFK8zVu01pBWYff662d40SgnBF1rlX9izCaMBczOZ3UPwAiMa8hXyHcklt orGBXc3DC/xQY171WuN6fAOAxVuKOE8tpxAhKq3gYwaRWQPO1KEEfJgBBJYhiGNPWNvq 3fy3lp9kpfS45lhgBGVTRQgqSjTflpFkRItqGA4/QsuFdn8C28OBCB8VMM5sjew9ez07 qEjw== X-Forwarded-Encrypted: i=1; AJvYcCXsNTS1G20B76vhEz7B9NRAgogOrqX1tf0YwrVTtsbs0QQzc2xr9XnFlHAibLv1QuRm1WYGEdZ3qEMBuPOEM0T1mDU= X-Gm-Message-State: AOJu0YzsvOvY7p8x4fXQMPY5hHb1Dg3CEX3RhEeu8HEtIvkT0ki0O/5Q LcwLtGKSX+bmWegfkrb1D3OjMHyVlQgf3edvM2HbDcM/tdK32klp X-Google-Smtp-Source: AGHT+IHNJJepHyfC0lV8t456DGXOVM20KUS+z/LohhJkRB6veFSYruBi4ECW7C7qwhTJ2T4k+KcjJA== X-Received: by 2002:a17:903:22c3:b0:1f9:edf6:d7a5 with SMTP id d9443c01a7336-1f9edf6d941mr45144125ad.1.1719040374237; Sat, 22 Jun 2024 00:12:54 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f9ebbb3f1esm24831895ad.265.2024.06.22.00.12.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 22 Jun 2024 00:12:53 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, chrisl@kernel.org, linux-mm@kvack.org, ryan.roberts@arm.com Cc: david@redhat.com, hughd@google.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, v-songbaohua@oppo.com, ying.huang@intel.com Subject: [PATCH v2 1/1] tools/mm: Introduce a tool to assess swap entry allocation for thp_swapout Date: Sat, 22 Jun 2024 19:12:31 +1200 Message-Id: <20240622071231.576056-2-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240622071231.576056-1-21cnbao@gmail.com> References: <20240622071231.576056-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: AB985100002 X-Stat-Signature: w9oyshcwufxoe85f8g6i73bszerojzc3 X-HE-Tag: 1719040375-888345 X-HE-Meta: U2FsdGVkX1/8E8+3OjUCeiGbWPV8rJ4HfR1UZUlOP24gRsQDq34leDUFQu5/GwW7QThuKmigTe/Myd3PT/Pezp+CSMQQ+fc94xXKn5ppC90FlP63ZBYBtEqxJUgJOYL2Z25eOFVNlcQq0hFprN8WIhCRlfEHrF3H51bJ+yoyrjacn95JDb3iaUrD2D9/daTVl0cFTS4lVpg3sKexYPLVp1QoJpOL3bhjnY9++Ao2H/EKj16MPy1tTT+F9xRpk5cgvlVBnIMpoo4zVtr6DqtznFOd5pJa7K/B8Iah0dA3X14m6LxcaJCzUcdS8Pk+rrQ/KKmY5wpxFi8yK27NGEfbxKMTINqQdxZUjRGKuCEw02/xi/lFJynsYAxA4eAKma6WVWW12QOQbO9/MpzzH9PwsmwmZz8+rxVqKo0atPZvcU2lsXq7JKX7NM0Yq01rILTLDB8ApG9gWJskI5P+v+vRyVGAi5IQ/621LYxYPfazb2bvElM6RTmkXBtVb0r93z0mt+T4nqCudpvDQIceSf9Q6IjxxPOxd8M6/018xtnmU8BFfGNM6og8XaeK+7sDJOZwzveHcmmuy4AQ2TBdSaDGe/pwDYIaUgaRDkPdtr2CnvU0n5XmW+XjxWzTO+oiiOwERL9nABcIXFLBMV2ceUVXPER6viXsk70ZfTqo+qJJACgB5E6CyBuRi80XxMFd+VeL6Mks87AifpM7FH71fZ1wlGNWyNSSUoA+8eqb1gM2J1vHlyDkcMeTOvE4a8p0pJuaJgEDzEE8RfDaCnJt6YOkJj81Hn6jh3pVJXBeiuIKDEPpDqNxH57nQngcTvpe6IqC8TBG3FzBX1+A8dcgOvS+mLSwz7UnAsvtaHi5VEksTAeN1bzIEWTXwdohFf/eQLW9xZAtwO+cqdCfEvsMgeDe1PkK0ZlCLwskCb5N+hzgtH7Wav90+9pEyBBSBfQ6Ale6kS274Gg5qx04gIoysTO R05E6+a/ qQBA8NMvqE3/9qWg7a+yipa8sh91SFcpl4bAOI4Zd3PnjF7MLIEnjmprykDww60PjkGA8oF0Gtc9JZ90WCF5qF6GWBalchTiD8YtzjtZLOKN6Sm0LXsRKAb7XTLAJus4VONVjF62X1J3qiUx6biMNG5ebu7265qfGM6MZsW1CD2JcL2UYwQ60336zHE+EdvmhXheA3d0vMvfJZ3RvYjSMk6f7RRMBmNoaMXOe8HGVNG1QiYGG4LFSktEizae1aU82Jm6uHSOnulSeiQ+Cm4r15Olam/8Ky9cTAhbXllDSQ+82oR0JSDmeM3+sh2thGvzmwsPE1D8H6VvH5epG5Q2MatQq04hrv1HSFjRzCTpvFJP80244hTDojzaGAIBMIrZz8VjxAxFY5ENw2a9faehw0WGwULpnmUVk7n/++gjZ/JEMBdSp6gu2IHhCKkYzodXZ7eLRdY4fpUDxBLHMZT8ilDN3Ahl0zuxEftfH X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song Both Ryan and Chris have been utilizing the small test program to aid in debugging and identifying issues with swap entry allocation. While a real or intricate workload might be more suitable for assessing the correctness and effectiveness of the swap allocation policy, a small test program presents a simpler means of understanding the problem and initially verifying the improvements being made. Let's endeavor to integrate it into tools/mm. Although it presently only accommodates 64KB and 4KB, I'm optimistic that we can expand its capabilities to support multiple sizes and simulate more complex systems in the future as required. Basically, we have 1. Use MADV_PAGEPUT for rapid swap-out, putting the swap allocation code under high exercise in a short time. 2. Use MADV_DONTNEED to simulate the behavior of libc and Java heap in freeing memory, as well as for munmap, app exits, or OOM killer scenarios. This ensures new mTHP is always generated, released or swapped out, similar to the behavior on a PC or Android phone where many applications are frequently started and terminated. 3. Swap in with or without the "-a" option to observe how fragments due to swap-in and the incoming swap-in of large folios will impact swap-out fallback. Due to 2, we ensure a certain proportion of mTHP. Similarly, because of 3, we maintain a certain proportion of small folios, as we don't support large folios swap-in, meaning any swap-in will immediately result in small folios. Therefore, with both 2 and 3, we automatically achieve a system containing both mTHP and small folios. Additionally, 1 provides the ability to continuously swap them out. We can also use "-s" to add a dedicated small folios memory area. Signed-off-by: Barry Song Reviewed-by: Ryan Roberts Tested-by: Ryan Roberts --- tools/mm/Makefile | 2 +- tools/mm/thp_swap_allocator_test.c | 233 +++++++++++++++++++++++++++++ 2 files changed, 234 insertions(+), 1 deletion(-) create mode 100644 tools/mm/thp_swap_allocator_test.c diff --git a/tools/mm/Makefile b/tools/mm/Makefile index 7bb03606b9ea..15791c1c5b28 100644 --- a/tools/mm/Makefile +++ b/tools/mm/Makefile @@ -3,7 +3,7 @@ # include ../scripts/Makefile.include -BUILD_TARGETS=page-types slabinfo page_owner_sort +BUILD_TARGETS=page-types slabinfo page_owner_sort thp_swap_allocator_test INSTALL_TARGETS = $(BUILD_TARGETS) thpmaps LIB_DIR = ../lib/api diff --git a/tools/mm/thp_swap_allocator_test.c b/tools/mm/thp_swap_allocator_test.c new file mode 100644 index 000000000000..a363bdde55f0 --- /dev/null +++ b/tools/mm/thp_swap_allocator_test.c @@ -0,0 +1,233 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * thp_swap_allocator_test + * + * The purpose of this test program is helping check if THP swpout + * can correctly get swap slots to swap out as a whole instead of + * being split. It randomly releases swap entries through madvise + * DONTNEED and swapin/out on two memory areas: a memory area for + * 64KB THP and the other area for small folios. The second memory + * can be enabled by "-s". + * Before running the program, we need to setup a zRAM or similar + * swap device by: + * echo lzo > /sys/block/zram0/comp_algorithm + * echo 64M > /sys/block/zram0/disksize + * echo never > /sys/kernel/mm/transparent_hugepage/hugepages-2048kB/enabled + * echo always > /sys/kernel/mm/transparent_hugepage/hugepages-64kB/enabled + * mkswap /dev/zram0 + * swapon /dev/zram0 + * The expected result should be 0% anon swpout fallback ratio w/ or + * w/o "-s". + * + * Author(s): Barry Song + */ + +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include + +#define MEMSIZE_MTHP (60 * 1024 * 1024) +#define MEMSIZE_SMALLFOLIO (4 * 1024 * 1024) +#define ALIGNMENT_MTHP (64 * 1024) +#define ALIGNMENT_SMALLFOLIO (4 * 1024) +#define TOTAL_DONTNEED_MTHP (16 * 1024 * 1024) +#define TOTAL_DONTNEED_SMALLFOLIO (1 * 1024 * 1024) +#define MTHP_FOLIO_SIZE (64 * 1024) + +#define SWPOUT_PATH \ + "/sys/kernel/mm/transparent_hugepage/hugepages-64kB/stats/swpout" +#define SWPOUT_FALLBACK_PATH \ + "/sys/kernel/mm/transparent_hugepage/hugepages-64kB/stats/swpout_fallback" + +static void *aligned_alloc_mem(size_t size, size_t alignment) +{ + void *mem = NULL; + + if (posix_memalign(&mem, alignment, size) != 0) { + perror("posix_memalign"); + return NULL; + } + return mem; +} + +/* + * This emulates the behavior of native libc and Java heap, + * as well as process exit and munmap. It helps generate mTHP + * and ensures that iterations can proceed with mTHP, as we + * currently don't support large folios swap-in. + */ +static void random_madvise_dontneed(void *mem, size_t mem_size, + size_t align_size, size_t total_dontneed_size) +{ + size_t num_pages = total_dontneed_size / align_size; + size_t i; + size_t offset; + void *addr; + + for (i = 0; i < num_pages; ++i) { + offset = (rand() % (mem_size / align_size)) * align_size; + addr = (char *)mem + offset; + if (madvise(addr, align_size, MADV_DONTNEED) != 0) + perror("madvise dontneed"); + + memset(addr, 0x11, align_size); + } +} + +static void random_swapin(void *mem, size_t mem_size, + size_t align_size, size_t total_swapin_size) +{ + size_t num_pages = total_swapin_size / align_size; + size_t i; + size_t offset; + void *addr; + + for (i = 0; i < num_pages; ++i) { + offset = (rand() % (mem_size / align_size)) * align_size; + addr = (char *)mem + offset; + memset(addr, 0x11, align_size); + } +} + +static unsigned long read_stat(const char *path) +{ + FILE *file; + unsigned long value; + + file = fopen(path, "r"); + if (!file) { + perror("fopen"); + return 0; + } + + if (fscanf(file, "%lu", &value) != 1) { + perror("fscanf"); + fclose(file); + return 0; + } + + fclose(file); + return value; +} + +int main(int argc, char *argv[]) +{ + int use_small_folio = 0, aligned_swapin = 0; + void *mem1 = NULL, *mem2 = NULL; + int i; + + for (i = 1; i < argc; ++i) { + if (strcmp(argv[i], "-s") == 0) + use_small_folio = 1; + else if (strcmp(argv[i], "-a") == 0) + aligned_swapin = 1; + } + + mem1 = aligned_alloc_mem(MEMSIZE_MTHP, ALIGNMENT_MTHP); + if (mem1 == NULL) { + fprintf(stderr, "Failed to allocate large folios memory\n"); + return EXIT_FAILURE; + } + + if (madvise(mem1, MEMSIZE_MTHP, MADV_HUGEPAGE) != 0) { + perror("madvise hugepage for mem1"); + free(mem1); + return EXIT_FAILURE; + } + + if (use_small_folio) { + mem2 = aligned_alloc_mem(MEMSIZE_SMALLFOLIO, ALIGNMENT_MTHP); + if (mem2 == NULL) { + fprintf(stderr, "Failed to allocate small folios memory\n"); + free(mem1); + return EXIT_FAILURE; + } + + if (madvise(mem2, MEMSIZE_SMALLFOLIO, MADV_NOHUGEPAGE) != 0) { + perror("madvise nohugepage for mem2"); + free(mem1); + free(mem2); + return EXIT_FAILURE; + } + } + + /* warm-up phase to occupy the swapfile */ + memset(mem1, 0x11, MEMSIZE_MTHP); + madvise(mem1, MEMSIZE_MTHP, MADV_PAGEOUT); + if (use_small_folio) { + memset(mem2, 0x11, MEMSIZE_SMALLFOLIO); + madvise(mem2, MEMSIZE_SMALLFOLIO, MADV_PAGEOUT); + } + + /* iterations with newly created mTHP, swap-in, and swap-out */ + for (i = 0; i < 100; ++i) { + unsigned long initial_swpout; + unsigned long initial_swpout_fallback; + unsigned long final_swpout; + unsigned long final_swpout_fallback; + unsigned long swpout_inc; + unsigned long swpout_fallback_inc; + double fallback_percentage; + + initial_swpout = read_stat(SWPOUT_PATH); + initial_swpout_fallback = read_stat(SWPOUT_FALLBACK_PATH); + + /* + * The following setup creates a 1:1 ratio of mTHP to small folios + * since large folio swap-in isn't supported yet. Once we support + * mTHP swap-in, we'll likely need to reduce MEMSIZE_MTHP and + * increase MEMSIZE_SMALLFOLIO to maintain the ratio. + */ + random_swapin(mem1, MEMSIZE_MTHP, + aligned_swapin ? ALIGNMENT_MTHP : ALIGNMENT_SMALLFOLIO, + TOTAL_DONTNEED_MTHP); + random_madvise_dontneed(mem1, MEMSIZE_MTHP, ALIGNMENT_MTHP, + TOTAL_DONTNEED_MTHP); + + if (use_small_folio) { + random_swapin(mem2, MEMSIZE_SMALLFOLIO, + ALIGNMENT_SMALLFOLIO, + TOTAL_DONTNEED_SMALLFOLIO); + } + + if (madvise(mem1, MEMSIZE_MTHP, MADV_PAGEOUT) != 0) { + perror("madvise pageout for mem1"); + free(mem1); + if (mem2 != NULL) + free(mem2); + return EXIT_FAILURE; + } + + if (use_small_folio) { + if (madvise(mem2, MEMSIZE_SMALLFOLIO, MADV_PAGEOUT) != 0) { + perror("madvise pageout for mem2"); + free(mem1); + free(mem2); + return EXIT_FAILURE; + } + } + + final_swpout = read_stat(SWPOUT_PATH); + final_swpout_fallback = read_stat(SWPOUT_FALLBACK_PATH); + + swpout_inc = final_swpout - initial_swpout; + swpout_fallback_inc = final_swpout_fallback - initial_swpout_fallback; + + fallback_percentage = (double)swpout_fallback_inc / + (swpout_fallback_inc + swpout_inc) * 100; + + printf("Iteration %d: swpout inc: %lu, swpout fallback inc: %lu, Fallback percentage: %.2f%%\n", + i + 1, swpout_inc, swpout_fallback_inc, fallback_percentage); + } + + free(mem1); + if (mem2 != NULL) + free(mem2); + + return EXIT_SUCCESS; +}