From patchwork Sat Oct 28 21:15:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Gomez X-Patchwork-Id: 13439582 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FFE5C4332F for ; Sat, 28 Oct 2023 21:16:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC8166B032F; Sat, 28 Oct 2023 17:15:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B52D96B0330; Sat, 28 Oct 2023 17:15:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 953D76B0331; Sat, 28 Oct 2023 17:15:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 7D5356B032F for ; Sat, 28 Oct 2023 17:15:57 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 5F540A027D for ; Sat, 28 Oct 2023 21:15:57 +0000 (UTC) X-FDA: 81396127554.27.D8D9A0D Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by imf28.hostedemail.com (Postfix) with ESMTP id 67954C001E for ; Sat, 28 Oct 2023 21:15:54 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=samsung.com header.s=mail20170921 header.b=Ug90+BkE; dmarc=pass (policy=none) header.from=samsung.com; spf=pass (imf28.hostedemail.com: domain of da.gomez@samsung.com designates 210.118.77.12 as permitted sender) smtp.mailfrom=da.gomez@samsung.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698527754; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=V58CpUMglUVRv2d2n4PjwOPxGbR6MAVB1lgi43BKbik=; b=1mdUPpR4PD3acmQreFg0zYK9oLAxQIL4K/CSU5uq+WiuEA8ZEPjvG7FCj++pJkCMin3h4Y rDGHy1DmnfK/uiVAa6eT7Z4M6Qntiab16Bo0LfWnRvHXCnAFQGam/OIs5avE1LUAASdwT4 jv4O+M6d4eBJM6+Egf90ucGF6setm1Y= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=samsung.com header.s=mail20170921 header.b=Ug90+BkE; dmarc=pass (policy=none) header.from=samsung.com; spf=pass (imf28.hostedemail.com: domain of da.gomez@samsung.com designates 210.118.77.12 as permitted sender) smtp.mailfrom=da.gomez@samsung.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698527754; a=rsa-sha256; cv=none; b=ZG/t7YtUtuMtA/vMLgIXJM2qMvCnaFfoZ9B2hhPQ+PCjBGzfl+Jg4kbHp7qIwb0Isozk3v iRXHWLmlMwDhoLzncp21iz+0Fq5bThy0xHplDtZo/wGFu0zJ5+WyB5kjkni6c+Q6kjB8Z+ eaM7ZGriLG76t2otBusbxGBzOYTs+xU= Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20231028211553euoutp028e7c373ac1988a326746bcfd8f030540~SYf4KxjYJ0879208792euoutp02S for ; Sat, 28 Oct 2023 21:15:53 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20231028211553euoutp028e7c373ac1988a326746bcfd8f030540~SYf4KxjYJ0879208792euoutp02S DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1698527753; bh=V58CpUMglUVRv2d2n4PjwOPxGbR6MAVB1lgi43BKbik=; h=From:To:CC:Subject:Date:In-Reply-To:References:From; b=Ug90+BkEzSkTyQzxO6IX69s40XfH1kin4j5qyV0qRip82QwCoST7VXdZhGtirfp/C eCUwW7HRdmW9RoJGBubVMRkje6dh2LhFr2aY5RD5wPKF7i+0tilH+Z95hxaUGhQlqG PwLLloFWfV3YsuaTmdl68/3RahrJys8XbK1q3o3E= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20231028211552eucas1p2368f287afd825b44877b59399542f661~SYf3yR5P21087910879eucas1p2Q; Sat, 28 Oct 2023 21:15:52 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id A3.20.11320.80A7D356; Sat, 28 Oct 2023 22:15:52 +0100 (BST) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20231028211551eucas1p1552b7695f12c27f4ea1b92ecb6259b31~SYf3Amogw0616106161eucas1p1Y; Sat, 28 Oct 2023 21:15:51 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20231028211551eusmtrp166725da83a27a1657bdc91947630f5f0~SYf3AFyW50755507555eusmtrp1e; Sat, 28 Oct 2023 21:15:51 +0000 (GMT) X-AuditID: cbfec7f4-993ff70000022c38-94-653d7a08c743 Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 04.52.10549.70A7D356; Sat, 28 Oct 2023 22:15:51 +0100 (BST) Received: from CAMSVWEXC01.scsc.local (unknown [106.1.227.71]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20231028211551eusmtip146fad08e90fd824bcd219c14210d9955~SYf206cR30467404674eusmtip1U; Sat, 28 Oct 2023 21:15:51 +0000 (GMT) Received: from CAMSVWEXC02.scsc.local (2002:6a01:e348::6a01:e348) by CAMSVWEXC01.scsc.local (2002:6a01:e347::6a01:e347) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 28 Oct 2023 22:15:51 +0100 Received: from CAMSVWEXC02.scsc.local ([::1]) by CAMSVWEXC02.scsc.local ([fe80::3c08:6c51:fa0a:6384%13]) with mapi id 15.00.1497.012; Sat, 28 Oct 2023 22:15:51 +0100 From: Daniel Gomez To: "minchan@kernel.org" , "senozhatsky@chromium.org" , "axboe@kernel.dk" , "djwong@kernel.org" , "willy@infradead.org" , "hughd@google.com" , "akpm@linux-foundation.org" , "mcgrof@kernel.org" , "linux-kernel@vger.kernel.org" , "linux-block@vger.kernel.org" , "linux-xfs@vger.kernel.org" , "linux-fsdevel@vger.kernel.org" , "linux-mm@kvack.org" CC: "gost.dev@samsung.com" , Pankaj Raghav , Daniel Gomez Subject: [RFC PATCH 10/11] shmem: add large folio support to the write path Thread-Topic: [RFC PATCH 10/11] shmem: add large folio support to the write path Thread-Index: AQHaCePt75l+dUBUfkS9wehNfGwv9w== Date: Sat, 28 Oct 2023 21:15:50 +0000 Message-ID: <20231028211518.3424020-11-da.gomez@samsung.com> In-Reply-To: <20231028211518.3424020-1-da.gomez@samsung.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [106.110.32.103] MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Sa0hTYRjmOzs7O65WpyntU7uuJHK6mRWcwCIp8FAQXcio6LLypJWaba2L Rs0wbEMsNaFdKi95aS2mq0zMGc1srSlltrSFhjYtjFLTynRYm2eF/573ed7nfd7348NZfDs7 BD+ceoKWpUqThRgXrXn2+2Uknr6ajrrTE07qTUaMvNN5GSPb3DPI3u+5KGlxich6ix0l2+r0 GNll/MMm6zy1HLLjSi8gy38McMhH1SUYOT6qx9byKJ2yFaWKzArqXmU41daioMwGFUaZv+dz qOfXxlFq2DxvM76LG5NAJx8+Scska/Zzk765+pC0/tDTqjdsJbg7Ww0CcEisgJ5uD1sNuDif qARwqETvL0YAbHmRgzLFMIBV7V3oP0tldpu/qwJAzS8j539XzjeVX2n2KtlWFlPcBnBU7eT4 /BixFDbYzZOWIELHho2ll4BPYBHnoLFPOxkSSGyEat1LxIeDiK1QVTXsnYR7sRjmmfg+GiXC oNPowXw0j4iBFucOHx3ghR9+PsR8GBBzYc/tMQ4zXQBd7psIc8IsWKKrZzF4Npyo68YYHAFb 2t2AwVHwQVmD/2Qh9Fwq9m8phh2FVzEGi2B58ZfJOTzvTLvGPflekJgIgJ/euTiMeT0ccjzz BwfCftt9Pz8HOgpy0CtApJ2yn3ZKhnZKhnZKRhFADUBAK+QpibQ8OpU+JZZLU+SK1ETxwWMp ZuD9bY4J20gtqOgfElsBggMrgDhLGMRjxcbQfF6C9Ew6LTu2T6ZIpuVWEIqjQgEvLGE+zScS pSfoozSdRsv+qQgeEKJE5i//ql6W4dwcqNuQsPSGctH0n9kr25EDTdNb8euZUR8zsyLinzsD t29aXbBlYLdh59vfYUe1NVW2EE2kQSM595pXai/r3VXKFYWmVUeT7vsOW0ZElct9vDk/Djmo 2vteYNmwCpfsF9BNSwp7zkpuGanqGGmpq1H5+HxSrEyTaY5+ldt89q648NCRkxe4ow3TbtWb ihokkcFdktqHrWOfLzwZYiODfU8/lnXk2bIuJq1akC6yFECVZ1tdfOg60Q7HI8Hgq25Rsbtp bOOgfmFn5/jcgdz8dl7rptrglmufyDjX4qzYRnNwRb+8e+RIdZZhZob1zx7PD5FJsqjGmhdn EqLyJOmycJZMLv0LX4liiNwDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrMKsWRmVeSWpSXmKPExsVy+t/xu7rsVbapBofPmlrMWb+GzWL13X42 i8tP+Cyefupjsdh7S9tiz96TLBaXd81hs7i35j+rxa4/O9gtbkx4ymix7Ot7dovdGxexWfz+ MYfNgddjdsNFFo8Fm0o9Nq/Q8rh8ttRj06pONo9Nnyaxe5yY8ZvF4/MmuQCOKD2bovzSklSF jPziElulaEMLIz1DSws9IxNLPUNj81grI1MlfTublNSczLLUIn27BL2Md7eeMRW8kq7ovMLa wLhWrIuRk0NCwERiRftl1i5GLg4hgaWMEg0HX7FAJGQkNn65ygphC0v8udbFBlH0kVFiWvcN qI4zjBJvZp6HclYySrzomMwI0sImoCmx7+QmdpCEiMBsVonDizvAEswCdRJrns0C2yEs4C3R Nfs8E4gtIhAkcXjpUqBJHEC2nsTE9UIgYRYBVYmra/6wgYR5BWwk9l4NBzGFBHIl+tsyQSo4 gaL3v21nA7EZBWQlHq38xQ6xSFzi1pP5TBAPCEgs2XOeGcIWlXj5+B/UYzoSZ68/YYSwDSS2 Lt0H9bySxJ+OhVAH60ncmDqFDcLWlli28DXYHF4BQYmTM5+wTGCUnoVk3SwkLbOQtMxC0rKA kWUVo0hqaXFuem6xoV5xYm5xaV66XnJ+7iZGYGraduzn5h2M81591DvEyMTBeIhRgoNZSYSX 2dEmVYg3JbGyKrUoP76oNCe1+BCjKTCEJjJLiSbnA5NjXkm8oZmBqaGJmaWBqaWZsZI4r2dB R6KQQHpiSWp2ampBahFMHxMHp1QDU0CAsoTS1hAGmS/nH3i36u929eEWimr/tZDT+c+W9zdk +j7s2BBvKC2z605S8rkvx6zP7W76dXdpiD1nSUCCzzKz5+//acvJXYzLU9dRFOwRv7rpzSVr prif27hWlFeseN8dwbHrtGKr7et7BevnuVody/ztGbJti0zUyv+27/oS65/7LOU+3VBofMyU ba9bTIDa0tPsS0+auyyOkO79Fc3ie2qNyC/p6H9n7FOfzFvkdt3+P+uf9zcMHmnJeP6yecp3 L4fX6NlNa4Z0JrPtjH7Mn3uDbx282qlQsc/b0zl292vXxYLP/rz7eylyVlNTK091Z+fPjF6e pqvmcyRzN6l/04ttmFnDl7CJ8zeDEktxRqKhFnNRcSIABtzmANYDAAA= X-CMS-MailID: 20231028211551eucas1p1552b7695f12c27f4ea1b92ecb6259b31 X-Msg-Generator: CA X-RootMTR: 20231028211551eucas1p1552b7695f12c27f4ea1b92ecb6259b31 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20231028211551eucas1p1552b7695f12c27f4ea1b92ecb6259b31 References: <20230919135536.2165715-1-da.gomez@samsung.com> <20231028211518.3424020-1-da.gomez@samsung.com> X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 67954C001E X-Stat-Signature: o975ows961conjhh9zk1w5msphoed98s X-HE-Tag: 1698527754-24598 X-HE-Meta: U2FsdGVkX18ybKXPmWDsnU6wgmso9vLvUGW1GGV6u6Ww6rNTzhnAq1BvnVz/F4vbB9J7dUy+hTcg5nz2Erlwf3NmizKe7k/w0w8QcefCJdH4jo9sN9Ui0qcKBuvZ/Pl57upHs7F/9XCo+T/gCH4l2hJZCjb7tbF0yJrf9KZQi2dPbETQ80XN5hLNKbI8cUakjOL/JxjTCJhOgE1VxEcRiHyyoPVQCgaPG+uySQQdEiigWr4nVFLYFQu7+y1WFe56Dj/RRYpcXPaMTP+mCDKoJwIZcReJfxhsDEd4EiKg4lXBgHWjYe5gQDMNGqAnTZ0m9FsxrmZBlce0bY14RvVRSJW3R49FlHYg7LvSLCpQge/SP98ZPbs4fgl4/o4LZ0x+PiUvFC02VBs0lwtiuwof9V8Xf8otY5sS5IkLZEYQn/utzrFtDalCYUA0Bm80lbQOyMbbXA5xQFm4DDZtl3IvN0761VIZzsKIGwEnmmhp2Ybzi7PUJTbjEN6FUlORUtVMJFQAIrOk+ylkmKOrxmrZcvGcp8uOzf/Z2Ib5FDpob0OXKyUjb2LfZtWO8Uko4oc5D7lW2+j0hKqH7N86qk74VUxth+QiTQbGVAmK8UG713C/he+vF9t/N4EV0+MKSXhj+CrSuFjC6PtUmMYR+oUWiHWwkEyWi4rmGtnQL+xznkDCtPaJvyNktnMyogub3pzeMz7C9mG0r7KdSkolCQjQabt2cBXgMFZ0AIqsYxGHFtL5qEk01N6ztbn9BCMG0FAo29gOvVculGfMJe7X3jzDT+koSil7cOL0mv8iPXHKYC4lOnCsCQdT1pwSeNAJjv1+I7ISx6llmcApHpzBmgeBVNgJV2DZvXetEu62L92EBTXNkgxtih0lDh1OhMy6/YNtHldgjvK9kCriNbPp8OtFKeuBv7GmuPH9ao9Nytk58GJTRtLp7FyYzuSvvxHjI1mWTxikxsfW00Qo4NA29YQ AjCd8Rpy HCdOToq6BOjpIPaQbKveS5yOcno1T/ke0b3fRynTsZFJMpvUDQnt2A0p7XTfE59L1Jcl2VUe7rI0eQ2mkpVwbT73AX2LcLoynCdWIBa+lrbUvJWBFvcIIWoKA2UJMEhs9L/gbagNsDhL/DZRkItLWhaV050KNFN081j+X X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Current work in progress. Large folios in the fallocate path makes regress fstests generic/285 and generic/436. Add large folio support for shmem write path matching the same high order preference mechanism used for iomap buffered IO path as used in __filemap_get_folio(). Add shmem_mapping_size_order to get a hint for the order of the folio based on the file size which takes care of the mapping requirements. Swap does not support high order folios for now, so make it order 0 in case swap is enabled. Add the __GFP_COMP flag for high order folios except when huge is enabled. This fixes a memory leak when allocating high order folios. Signed-off-by: Daniel Gomez --- mm/shmem.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 48 insertions(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index fc7605da4316..eb314927be78 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1621,6 +1621,9 @@ static struct folio *shmem_alloc_folio(gfp_t gfp, struct shmem_inode_info *info, pgoff_t ilx; struct page *page; + if ((order != 0) && !(gfp & VM_HUGEPAGE)) + gfp |= __GFP_COMP; + mpol = shmem_get_pgoff_policy(info, index, order, &ilx); page = alloc_pages_mpol(gfp, order, mpol, ilx, numa_node_id()); mpol_cond_put(mpol); @@ -1628,17 +1631,56 @@ static struct folio *shmem_alloc_folio(gfp_t gfp, struct shmem_inode_info *info, return page_rmappable_folio(page); } +/** + * shmem_mapping_size_order - Get maximum folio order for the given file size. + * @mapping: Target address_space. + * @index: The page index. + * @size: The suggested size of the folio to create. + * + * This returns a high order for folios (when supported) based on the file size + * which the mapping currently allows at the given index. The index is relevant + * due to alignment considerations the mapping might have. The returned order + * may be less than the size passed. + * + * Like __filemap_get_folio order calculation. + * + * Return: The order. + */ +static inline unsigned int +shmem_mapping_size_order(struct address_space *mapping, pgoff_t index, + size_t size, struct shmem_sb_info *sbinfo) +{ + unsigned int order = ilog2(size); + + if ((order <= PAGE_SHIFT) || + (!mapping_large_folio_support(mapping) || !sbinfo->noswap)) + return 0; + + order -= PAGE_SHIFT; + + /* If we're not aligned, allocate a smaller folio */ + if (index & ((1UL << order) - 1)) + order = __ffs(index); + + order = min_t(size_t, order, MAX_PAGECACHE_ORDER); + + /* Order-1 not supported due to THP dependency */ + return (order == 1) ? 0 : order; +} + static struct folio *shmem_alloc_and_add_folio(gfp_t gfp, struct inode *inode, pgoff_t index, struct mm_struct *fault_mm, size_t len) { struct address_space *mapping = inode->i_mapping; struct shmem_inode_info *info = SHMEM_I(inode); - unsigned int order = 0; + unsigned int order = shmem_mapping_size_order(mapping, index, len, + SHMEM_SB(inode->i_sb)); struct folio *folio; long pages; int error; +neworder: if (gfp & VM_HUGEPAGE) { pages = HPAGE_PMD_NR; index = round_down(index, HPAGE_PMD_NR); @@ -1721,6 +1763,11 @@ static struct folio *shmem_alloc_and_add_folio(gfp_t gfp, unlock: folio_unlock(folio); folio_put(folio); + if (order != 0) { + if (--order == 1) + order = 0; + goto neworder; + } return ERR_PTR(error); }