From patchwork Mon Nov 4 14:30:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 13861496 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B85F4D132D3 for ; Mon, 4 Nov 2024 14:31:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E9846B0083; Mon, 4 Nov 2024 09:31:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 272876B0085; Mon, 4 Nov 2024 09:31:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 113A26B0089; Mon, 4 Nov 2024 09:31:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id DD4246B0083 for ; Mon, 4 Nov 2024 09:31:06 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 3D4F6ABCB2 for ; Mon, 4 Nov 2024 14:31:06 +0000 (UTC) X-FDA: 82748648304.19.33D5A6D Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf21.hostedemail.com (Postfix) with ESMTP id D27F41C0023 for ; Mon, 4 Nov 2024 14:30:03 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=etqRHTmg; spf=pass (imf21.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730730443; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=KS+IJ3vly6H2kPqYhI8jZizE7imRU4hRnJwqqLDKBQI=; b=GqdGPPofsx8bx3+vBA/divGjNDFfZ7QVRfbDh9/bY7TpIiwOMzHx6gbXSZU7kdDFhbM/m/ YZ7x95wjio7Kw4TF45/dNBT8VYTNKa2RnYcZDzpyZQ980Aik/cVvEDz73N0w/zMhYA1dCH CAfTQzw3JAyjvpYYFxqfR+8kusxfHE8= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=etqRHTmg; spf=pass (imf21.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1730730443; a=rsa-sha256; cv=none; b=5oF1Gphf6yGYBn8DqdPmdkYHOgPUyUCmFb6/QENqs+eycKbg8ExVW/EJjgruf1wKJTlz/J EwXS/VeJbC0mBuoOgnrmzb3U72uPM/1dN2+c58Dhita2MYeaUwOLe8oFAXWi7FqMrO0mYl MSSxd4lY41eOaqjjTvs/AhRnDMI+SFI= Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-20c9978a221so43625825ad.1 for ; Mon, 04 Nov 2024 06:31:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730730663; x=1731335463; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=KS+IJ3vly6H2kPqYhI8jZizE7imRU4hRnJwqqLDKBQI=; b=etqRHTmgQrAzAubCrikb5boC/qgU/VQ7BEU+xfBgYa4+ht/C7Y5Lo4TSTKSvD3d6MQ tM3cad7t7eNa0B8iic1VUVnAK/hsQylQlwr/cKfurbr1preRPGaCrTFAzMx6QA/KzuY9 sc9e3ySw6WOIfX6rc2b1IT1Wmqx5mJ8nl2M8WLWDod2yRiD2qxT5YVOxUtn+yFqCMsg9 HOgOmGtqHQpBgGs9UqJ4KNjn3uUw4UuzSGdzgfzSC0VEnByhFOPDYw41wQ6BfJbfdaOQ T6NRl6lQFjU2eZN2OHorkpUI/bcyXhmad9O2cXXkthOcOkSQIkuKzBcj7g7BzK2Tzk90 lTvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730730663; x=1731335463; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KS+IJ3vly6H2kPqYhI8jZizE7imRU4hRnJwqqLDKBQI=; b=WnSA4GUb3NqZrTtmppORsqrkIebGQQ57vy+fk94rFG8nz6vAoRv1Df9GBh/5AkKQUk tChKbaWCIb0fMZBJ/tb0nw5s8V9RA0fo+Vr9k3ABY0U6urW/iiky5dk5KxaBFtSJagFh zS65D2PljUEMIMBkwfottjD9wh/jTkn3lMAGvzUClqMBzoKq5Aok98+k4mYPInNorrT7 sHITUZY8OI3YggBnRs88Ey+ld61nzZOQkWvl7KNZSENTs/ksc2sxQSOt5miaIOfXyJ+i 6TH2ywy95oG4aZYrirnXZ9pTKVWVFq6/97lLQGXHwJv+425+QMhytbGHh1Lby6oyJKhB Zn9g== X-Gm-Message-State: AOJu0YwU8S8vt3Q0my6zDB+rk1vSYx1pd2UQJUUAJT9FFufzg1GKg+BN mWwx/OeypY4iQFwkuG5MM76ImP233IEGkJd19aP+bBe/XQ+ChjZq X-Google-Smtp-Source: AGHT+IEvyKz3r2BwSGBB9wa2O3GkLfaCRUQeSVMtKb5ggyueFUm+OHpJMGZEou+wyykSmvRC3WDnkg== X-Received: by 2002:a17:903:8c7:b0:20b:38fe:60d7 with SMTP id d9443c01a7336-210c6ae3d60mr444171735ad.22.1730730663002; Mon, 04 Nov 2024 06:31:03 -0800 (PST) Received: from localhost.localdomain ([117.136.120.192]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-211057c0ed3sm62034235ad.216.2024.11.04.06.30.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 04 Nov 2024 06:31:02 -0800 (PST) From: Yafang Shao To: willy@infradead.org, akpm@linux-foundation.org Cc: linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH] mm: Add large folio support for async readahead Date: Mon, 4 Nov 2024 22:30:15 +0800 Message-Id: <20241104143015.34684-1-laoar.shao@gmail.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: D27F41C0023 X-Stat-Signature: sziyb5nhbq4nmi9gtnws9md8wuqmit7h X-Rspam-User: X-HE-Tag: 1730730603-401235 X-HE-Meta: U2FsdGVkX1/Baz3kDXQuIT3nxatDAe2YJ/Yjqz17XA1eHM8P3K5of1vLGdwzRmxrSaHZPtY6cnVIfWgmPhENhimcn9uFRz+kaEpIBCU6476pNm9Cn/AFag4bsTZgTJKw1xvuCT+6HVYJjmE7eKBLtUYKRVNsjHrKiK+EFh4PQMS8cmNIT34DPEnZ2CHTUXyFhSzcBhpuq29Auwrg8hKZKf/YQtF2qU84lFZ4j6PC3pV8MK45QurE30m+2R4nWW+wX2h9wM9XxHurm2K1QjoQ4PzLCIaiMX3ImQLK/Ai4xaoRqzXm6LotKgxXEnKiApQc8sktqJrqGGbH1Spourpv+RhEO/1zuyUejLNduTOAgB7gx15PULtXblmGsu3C641St34QdP5jKubYMdmC85wHoOUP9n1hVvFqVkqUA1t2zb3BFnUyFwGJrymXI7qEOInOjkZmlMfv59UM4oJde01dGcT4x3tUNEtqqQgQ3LMPcYMJQhle1uLi9NC1GmQYJkr+IXL0Hgb2x8mYu1hgIjqDNwXBWUEUqV7ZZ5gLiNsqtj3lrqUmZxX4mrcDOoq0rXNRY/4bhUweoplcSWwJ5GRwZoXbz0d3vY9c3KJNKyMDKZUkz6YoX6CS2xp4roXZS3j+8gLifN/Z8QwC0KqPlfMBlsLAiL+RZmvIzZQDpt6NWLEwPKce/i9ju8nW6ceEELcjAMloS540kW4PX8l8aghxwj+bO3g3jYl292LHNafhZ5Ti3l63fk3/xs2lyXOXkWy+CThLCt1wsrKYvRflzy8YVxdNnE5loNZ2wU2w7n6hw6J24EpDYunQdeqjKBI2FMpO+2avw3yL0m8maiLHegxE6q4+ptbY1BWMv19hOcLJsIzBW7Z7f31RQI/BF/8SusTFeG4KO/IBQw18Zxk5iOPy0wW9FNEwJ+9NKe0k8NMhE3BU1Idd1S7GxMBCaqvZ7I6FBzKglYCU9Ocqiv4+xq7 yg9PrcE8 HJ+PtpaiMz6CPC1bpBVme5KcM9C//K9vBVfeI8QKEqr11sSamlUPsKV0aUql5dNxfzC8g5bbpxyFrqEMUQa478nSVUv8W8JYhhFrykgxiSoK90go7Qjf+vOAaaOvoJkVu5+vSdEO0410ELQUCv8R2+KU4zYkSmT4VyCmMjwblsC9g+KRTl8eB21gqVBD28BwnzDT1Hd+/m1en9Cl2H4tSzMavOHxlTgmKYSKYP5Y4MmPEAbubIPugsSQsqX0+M8emCM1m32rNyITlxWq/2cxjUiTCfFK4wfwIGZv2JRsWCz8jKXRFLO1ctXukcRONnEsA5f/mE2lqyryQjKAA2RiV16fPohQ0QY8OsRImcDEjWjh5oEKLd/lwSza2LppE3VhTMe+0DwpyiIzYNy55s7oT3bf/wKFjpjUD9ZPJnsL4HOThHAZni/EIO3RipC9X5vsinuPF+8zsGVcsAS+xIr54ZxNXg2zRriWrdbk/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When testing large folio support with XFS on our servers, we observed that only a few large folios are mapped when reading large files via mmap. This behavior occurs because large folio support is currently implemented only for sync readahead, not for async readahead. Consequently, while the first filemap fault may map to a large folio, subsequent filemap faults are mapped to regular folios. This can be verified with a simple test case, as shown below: #define LEN (1024 * 1024 * 1024) // 1GB file int main(int argc, char *argv[]) { char *addr; int fd, i; fd = open("data", O_RDWR); if (fd < 0) { perror("open"); exit(-1); } addr = mmap(NULL, LEN, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0); if (addr == MAP_FAILED) { perror("mmap"); exit(-1); } if (madvise(addr, LEN, MADV_HUGEPAGE)) { perror("madvise"); exit(-1); } for (i = 0; i < LEN / 4096; i++) memset(addr + i * 4096, 1, 1); while (1) {} // Verifiable with /proc/meminfo munmap(addr, LEN); close(fd); exit(0); } This patch adds large folio support to async readahead, aligning its behavior with sync readahead. Signed-off-by: Yafang Shao --- include/linux/pagemap.h | 4 +++- mm/filemap.c | 4 ++++ mm/readahead.c | 9 +++++++++ 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 68a5f1ff3301..b77eb5d3a3dd 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -1350,8 +1350,10 @@ struct readahead_control { pgoff_t _index; unsigned int _nr_pages; unsigned int _batch_count; - bool _workingset; unsigned long _pflags; + bool _workingset; + bool _large_folio; + bool _rand_read; }; #define DEFINE_READAHEAD(ractl, f, r, m, i) \ diff --git a/mm/filemap.c b/mm/filemap.c index 36d22968be9a..c9695effdd88 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3226,6 +3226,10 @@ static struct file *do_async_mmap_readahead(struct vm_fault *vmf, WRITE_ONCE(ra->mmap_miss, --mmap_miss); if (folio_test_readahead(folio)) { + if (vmf->vma->vm_flags & VM_HUGEPAGE) + ractl._large_folio = true; + if (vmf->vma->vm_flags & VM_RAND_READ) + ractl._rand_read = true; fpin = maybe_unlock_mmap_for_io(vmf, fpin); page_cache_async_ra(&ractl, folio, ra->ra_pages); } diff --git a/mm/readahead.c b/mm/readahead.c index 3dc6c7a128dd..8c4f95f87b71 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -667,6 +667,15 @@ void page_cache_async_ra(struct readahead_control *ractl, ra->async_size = ra->size; readit: ractl->_index = ra->start; +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + if (ractl->_large_folio) { + ractl->_index &= ~((unsigned long)HPAGE_PMD_NR - 1); + if (!ractl->_rand_read) + ra->size = 2 * HPAGE_PMD_NR; + ra->async_size = HPAGE_PMD_NR; + order = HPAGE_PMD_ORDER; + } +#endif page_cache_ra_order(ractl, ra, order); } EXPORT_SYMBOL_GPL(page_cache_async_ra);