From patchwork Thu Jun 6 11:58:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13688377 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1CCFAC25B75 for ; Thu, 6 Jun 2024 11:59:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6BE6F6B009C; Thu, 6 Jun 2024 07:59:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 66F066B009D; Thu, 6 Jun 2024 07:59:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 50E396B009E; Thu, 6 Jun 2024 07:59:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 36D956B009C for ; Thu, 6 Jun 2024 07:59:19 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id E0240C1603 for ; Thu, 6 Jun 2024 11:59:18 +0000 (UTC) X-FDA: 82200318396.22.935EC9C Received: from out30-131.freemail.mail.aliyun.com (out30-131.freemail.mail.aliyun.com [115.124.30.131]) by imf08.hostedemail.com (Postfix) with ESMTP id A53FF160006 for ; Thu, 6 Jun 2024 11:59:16 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=NdhGyyJd; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf08.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717675157; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QaLyG6fEgySNt1B9Umu3RlbPCCtnChT65+F+Wd+srs4=; b=7w/9z6ec0BCbZQFKL1nM3q06+4ngOVNrnleKBDnvzZneBFucoeopFRR0ibE9vUKEToiwIk wK6cOh/a/UiGh1JyTNpzm0q/3Qq1DjFdlWcwmPjrJUEeuXd/1SCYhxyK92E6B91Hvq7QMT 5QshyZTmYmXunO1xy+S4J3saru0aURo= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=NdhGyyJd; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf08.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717675157; a=rsa-sha256; cv=none; b=cyLAhOHOPTLgncwK+xd+I1nMJH6fdnuY4VrEKxFlIx5FH7egSG3Ub0XU1D2Oswk/XXbMVh RVCRgF1NrO+l1QvFH9aXDzkslJqGLp8tTcYGKt2ulYh/EXKZ+km2ClP5fwSHSEyDba1lPS lgI5aIxzRYfedz89yMOhpbhu226gRMk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1717675153; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=QaLyG6fEgySNt1B9Umu3RlbPCCtnChT65+F+Wd+srs4=; b=NdhGyyJdxwqunhqgviQG52l7beR8YPxvNV19Uo779mUWRcRKpkwiX55A3BRaXTp7F4wNeSxYZ/3uMCO543VTw/kmsI3aucTdA6Z3dkzAPDazoHyxkjPuO2OTQ73o0PrHUWbsCMdGuY4aAKgcQiUwIzVF9+rOf0GgviXhrwT7peQ= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R681e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033032014031;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W7y5mug_1717675151; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W7y5mug_1717675151) by smtp.aliyun-inc.com; Thu, 06 Jun 2024 19:59:11 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/7] mm: vmscan: add validation before spliting shmem large folio Date: Thu, 6 Jun 2024 19:58:51 +0800 Message-Id: <3516f4ff0242c48ad0fa000d50ac382817776be1.1717673614.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: A53FF160006 X-Stat-Signature: m6bwb7ywbz4hucgocu55aetyj1k375co X-HE-Tag: 1717675156-336942 X-HE-Meta: U2FsdGVkX1/HULgOS8ls/R9jzKIIkincpi9V0ALw/BbXknoZ+iwfFb+ILts8bw+oqXGzqOh5SDKPbXfMENIlFlz0X8jVfcMfuQwZ/gSDKyu7l89g+mnB8F700jcobBNelIgL5/HBF7/OSFV3Nt0y4lTsxTDf+45w7X9NvkR35YcyGPwgB3WjZhKH1vJY6+JvyJsgVbYp+3QbTLZ8H4k4y4ybPTaDQ6WcXkF7yssJpAYejTR5bzRgKtOmacwR9ZN1hnu4fwWY1I/nvb1Yu+OO0hThJmr46dzubEX9btFF2kXWOqLVmBC8WnNL80zXCLjQ5TnM/uHsIO8UdbjReNncjePBCzzdl0fUQ55vVZQiTSF1QH4P9Qs8bOMWtNkXsfC4unRVhDa7mkLJUf9+Akaydj8MXCvHLIhrc9P8Mqa/w16IQSi8krqEtCnASi2IhobBTnQ8U++gS93O5m0qEKtDxNXB7kGMfnMgs1DnVoCEL2AbLcvRcRHI5913aU5KVjrKXBJq4+LzkFD5Z8r96EAMIbHiae7xz/333kEKaVBH780Y0ih17+J4aaZfYthCSawF74wwJD5zBkMb0m5noeJ+dIc9hKaiEzijgCxzP/30pE/FrsgB+Mw1UhuPrs5wj2Nk7om4FGsG+7sJlzlNBOgenn5cyKWH1yY0K1qt27RN+Wt3IBA5Y4owwrLXFLmHJxpZGtvt2LxSojRu0bNyLtfebRWCv4b+rr+oSWUciI2O5ls9cP/wDFkgsJELAyxQStV4vX+5PHh99KSB7+rD82HhHrF8MvxOnkuAz6ADy2SMjPr3oUT4IUaW+wm3Fq3GgTHtMDSXlOt3vtKB20eXy7/gY765PechIw8p1SGhubgQiwSDUd/ngPe8NcY3zhZpqnbpSvVqErhxPP1pS+sWeBypjof//Er+B4LvK/K0yI6HoxJ4RMkiasg4C3fq5PEEdRVn27gdgBElt0bJ0MERtfR 7m763gJ4 AxtBKWIWnRM2dOm1xPtedpQHTQ4xt1aD+3i11O2bLarqU6oCykc/dcnvtXlU1fLUK85mqqLv1x3IUFyhuemTFkmqwjajwNp92MmVJkDQFQMEFmVoU2wj2om+CMj7YhdahnUcMrWnD8XeqM0RJyMXqSoKwdIwMkC/0NS2CQr5pQuzWtNUwZv8IjxANqHSH/h36N8eJ0Ub64ImHyYJbkZj1t9dmhXMrEqLF0oCOL1+QL22dPA91KgayyFftAW3TlKrBpCrsOiMXc9q10hg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add swap available space validation before spliting shmem large folio to avoid redundant split, since we can not write shmem folio to the swap device in this case. Signed-off-by: Baolin Wang --- mm/vmscan.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/mm/vmscan.c b/mm/vmscan.c index c0429fd6c573..9146fd0dc61e 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1255,6 +1255,14 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, } } else if (folio_test_swapbacked(folio) && folio_test_large(folio)) { + + /* + * Do not split shmem folio if no swap memory + * available. + */ + if (!total_swap_pages) + goto activate_locked; + /* Split shmem folio */ if (split_folio_to_list(folio, folio_list)) goto keep_locked; From patchwork Thu Jun 6 11:58:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13688379 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEE60C25B75 for ; Thu, 6 Jun 2024 11:59:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 622EB6B009E; Thu, 6 Jun 2024 07:59:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 583646B009F; Thu, 6 Jun 2024 07:59:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 426026B00A0; Thu, 6 Jun 2024 07:59:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0A7216B009E for ; Thu, 6 Jun 2024 07:59:20 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id BDB35140545 for ; Thu, 6 Jun 2024 11:59:19 +0000 (UTC) X-FDA: 82200318438.22.CD77BF3 Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) by imf18.hostedemail.com (Postfix) with ESMTP id B7E4F1C001E for ; Thu, 6 Jun 2024 11:59:17 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=xsYDLQZl; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf18.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717675158; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7zHkhAgm6YTME5U/JXdAOzuXxbjqoNiOEPH5oon2/bk=; b=7YW0c4liQwBEfXNFOjKh4vsgP/xfTxCn7N+d84S2c/m8fa9GhzTCDnt1c0aqDHruS1Npti +bIPTB9ZX3w60RuKnI1TLPs8bXlIoCwRYKIwAJvl86hx5JwN+nnGWbpetcj0z+fCROdB6J eXBSj3ePGr4OXl5UT3BPcaIx6GA9baQ= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=xsYDLQZl; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf18.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717675158; a=rsa-sha256; cv=none; b=LvdQQoAp2Nqgbo32jA89o4ZDpsKQpJIEeKGcEN9ajW82sCBkoPlgoufzub9y6PHT8HBbgl 9gKvZix7k1/OewjXy0XwXPixaT3ihHRQDkJXe2mpOaHfdtgLwoXE6byvJfKaO6NZSb2FV+ 0ASf6xEPpKLXxAbLcvreri1VhrBOdYQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1717675154; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=7zHkhAgm6YTME5U/JXdAOzuXxbjqoNiOEPH5oon2/bk=; b=xsYDLQZl63Y4HRu7ffDwKA0NFWjlAQRMSBQB9ac8eiLrJ4TFsqm6zCEl8way1RoOuRUJcv5FW5X9d3ANF/WUhPEJGsJi/DX+z4ckUCm5Ml3Vv2uNhn1TmYe6t3GrFeDlkQSqawBj+RBBFdnXiXHc5/qTYFVqcFpYRzxx0CbNA50= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033068173054;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W7y6tpB_1717675152; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W7y6tpB_1717675152) by smtp.aliyun-inc.com; Thu, 06 Jun 2024 19:59:12 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/7] mm: swap: extend swap_shmem_alloc() to support batch SWAP_MAP_SHMEM flag setting Date: Thu, 6 Jun 2024 19:58:52 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: B7E4F1C001E X-Stat-Signature: s6m6pfz4fdernhwrreh4fwz7ih1mtg4g X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1717675157-345548 X-HE-Meta: U2FsdGVkX19bUvhzj/G+c0oB9m5cOkJVOhz5dyPcplNq2+JfXzLQs2uDB2JqZmiMYdB6PirDEc2NYo405SyhusC7Z7NC+ZV/4up6Ud+KdwudcvRXL9AhEPfm/0bd6sTYNjvB3XexXNETEdEZ2VTgU1ioLVW/rimVip9KixOQL9PGPy0jfA2O908h8l6eLlVtdT2xOsDwV3kdt1tNX+F/1pFkLAlziuXXbk1oA0xBy8IJpOEFpTszTb/TmXQEt5xe4DIoq1IcP5AL7FbAIvWz/+EDeM9OVc8X+q4vcPk+2ThrMYvA5QN2VzIXUx1uEdzcG24ZneeXEEpc6v+nfISx2DMwjnJEzGwAEE4BwEhqErdEV0nKNg0zRYxkDMN0Q5l/6tTa3Y5Azu1z9hR15SfpD8I4jzIT+44HjAIFxMDZB74OKqhEIYWJHYk01p56M5lZKtv+0n1btMBfdkt1S/H2hDWfg7pQWUCg+k1XSKyh5e/EouRi+V+MrjlhGKmueUfAFfYCEaiKxrevX8PavvhVeyiJl04IBxjPHmH9inAxMo4jGHjPuyZwhynKesovlhx3uWqlvZpsDsOSsMdd9QjXmutlmfU7WaZtIpvknwrR0lsjfp2RrAKBkuS9ovzBK5s+h2uYomnuo3Tars1XZ+GNQhIBh4P3NSO6al2tvFro+XpobVM9GqXpoE9EWMoZgQxeHmEqRl39gVJAQVZGaBNN+sjZMDIqGbn+bSxlRWNAxUWRFoy1m4u5CAUpwPlvY6dVblkhq5olOXySZTMBiTJSWYBxxWUic8G0k2Ks+LoSx3nxyOHzjrEIc6KjrKVzqqL9yrAjsjNusfSpPIHyAABeqepKYEt7wSvcZ0s+Z4z5pIeA3+ikm2qomgjnn/shM8BmJxTW/XYHu+Kjc/fddWAHt4rSas2QIDGCjj1DE3Lxh9gcxTZ55WRMJoEjSSerTKLFSmw79xnlYFdV837WBTx RF+bweo2 YmdVy9d/xxKHr+nvB410lF4ixWix5w/nLlwWg+NaKjij0NhXaQTzr+qnDnpYrZNjXyt5jpCNcr6uTzovt0HOCyW7Eh2Ju2HcspECJR/KPtrrcUWp132lwMvmPIyBBkmZ57BirJ5dNBzCzFjUGBEaR3RueeI7DCIbL40kIEYRPvfTmFsRD8Y9oiPu3bnDpQFwt72aK4CWeFUba2/J6onBOkWoVzt21Nw6kHtNqMmLgQKoNFVwdy2njw7DQ/xb4TL/pZfINP9x++uBgObkvLd92u5KeyECiCsvOMwli X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support shmem large folio swap operations, add a new parameter to swap_shmem_alloc() that allows batch SWAP_MAP_SHMEM flag setting for shmem swap entries. While we are at it, using folio_nr_pages() to get the number of pages of the folio as a preparation. Signed-off-by: Baolin Wang --- include/linux/swap.h | 4 +- mm/shmem.c | 6 ++- mm/swapfile.c | 98 +++++++++++++++++++++++--------------------- 3 files changed, 57 insertions(+), 51 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 3df75d62a835..4a76ab0b4a7f 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -477,7 +477,7 @@ void put_swap_folio(struct folio *folio, swp_entry_t entry); extern swp_entry_t get_swap_page_of_type(int); extern int get_swap_pages(int n, swp_entry_t swp_entries[], int order); extern int add_swap_count_continuation(swp_entry_t, gfp_t); -extern void swap_shmem_alloc(swp_entry_t); +extern void swap_shmem_alloc(swp_entry_t, int); extern int swap_duplicate(swp_entry_t); extern int swapcache_prepare(swp_entry_t); extern void swap_free_nr(swp_entry_t entry, int nr_pages); @@ -544,7 +544,7 @@ static inline int add_swap_count_continuation(swp_entry_t swp, gfp_t gfp_mask) return 0; } -static inline void swap_shmem_alloc(swp_entry_t swp) +static inline void swap_shmem_alloc(swp_entry_t swp, int nr) { } diff --git a/mm/shmem.c b/mm/shmem.c index d9a11950c586..174d8ae25b9b 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1433,6 +1433,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb); swp_entry_t swap; pgoff_t index; + int nr_pages; /* * Our capabilities prevent regular writeback or sync from ever calling @@ -1465,6 +1466,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) } index = folio->index; + nr_pages = folio_nr_pages(folio); /* * This is somewhat ridiculous, but without plumbing a SWAP_MAP_FALLOC @@ -1517,8 +1519,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) if (add_to_swap_cache(folio, swap, __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN, NULL) == 0) { - shmem_recalc_inode(inode, 0, 1); - swap_shmem_alloc(swap); + shmem_recalc_inode(inode, 0, nr_pages); + swap_shmem_alloc(swap, nr_pages); shmem_delete_from_page_cache(folio, swp_to_radix_entry(swap)); mutex_unlock(&shmem_swaplist_mutex); diff --git a/mm/swapfile.c b/mm/swapfile.c index 9c6d8e557c0f..1dde413264e2 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -3362,62 +3362,58 @@ void si_swapinfo(struct sysinfo *val) * - swap-cache reference is requested but the entry is not used. -> ENOENT * - swap-mapped reference requested but needs continued swap count. -> ENOMEM */ -static int __swap_duplicate(swp_entry_t entry, unsigned char usage) +static int __swap_duplicate(struct swap_info_struct *p, unsigned long offset, + int nr, unsigned char usage) { - struct swap_info_struct *p; struct swap_cluster_info *ci; - unsigned long offset; unsigned char count; unsigned char has_cache; - int err; + int err, i; - p = swp_swap_info(entry); - - offset = swp_offset(entry); ci = lock_cluster_or_swap_info(p, offset); - count = p->swap_map[offset]; - - /* - * swapin_readahead() doesn't check if a swap entry is valid, so the - * swap entry could be SWAP_MAP_BAD. Check here with lock held. - */ - if (unlikely(swap_count(count) == SWAP_MAP_BAD)) { - err = -ENOENT; - goto unlock_out; - } - - has_cache = count & SWAP_HAS_CACHE; - count &= ~SWAP_HAS_CACHE; - err = 0; - - if (usage == SWAP_HAS_CACHE) { + for (i = 0; i < nr; i++) { + count = p->swap_map[offset + i]; - /* set SWAP_HAS_CACHE if there is no cache and entry is used */ - if (!has_cache && count) - has_cache = SWAP_HAS_CACHE; - else if (has_cache) /* someone else added cache */ - err = -EEXIST; - else /* no users remaining */ + /* + * swapin_readahead() doesn't check if a swap entry is valid, so the + * swap entry could be SWAP_MAP_BAD. Check here with lock held. + */ + if (unlikely(swap_count(count) == SWAP_MAP_BAD)) { err = -ENOENT; + break; + } - } else if (count || has_cache) { + has_cache = count & SWAP_HAS_CACHE; + count &= ~SWAP_HAS_CACHE; + err = 0; + + if (usage == SWAP_HAS_CACHE) { + /* set SWAP_HAS_CACHE if there is no cache and entry is used */ + if (!has_cache && count) + has_cache = SWAP_HAS_CACHE; + else if (has_cache) /* someone else added cache */ + err = -EEXIST; + else /* no users remaining */ + err = -ENOENT; + } else if (count || has_cache) { + if ((count & ~COUNT_CONTINUED) < SWAP_MAP_MAX) + count += usage; + else if ((count & ~COUNT_CONTINUED) > SWAP_MAP_MAX) + err = -EINVAL; + else if (swap_count_continued(p, offset + i, count)) + count = COUNT_CONTINUED; + else + err = -ENOMEM; + } else + err = -ENOENT; /* unused swap entry */ - if ((count & ~COUNT_CONTINUED) < SWAP_MAP_MAX) - count += usage; - else if ((count & ~COUNT_CONTINUED) > SWAP_MAP_MAX) - err = -EINVAL; - else if (swap_count_continued(p, offset, count)) - count = COUNT_CONTINUED; - else - err = -ENOMEM; - } else - err = -ENOENT; /* unused swap entry */ + if (err) + break; - if (!err) - WRITE_ONCE(p->swap_map[offset], count | has_cache); + WRITE_ONCE(p->swap_map[offset + i], count | has_cache); + } -unlock_out: unlock_cluster_or_swap_info(p, ci); return err; } @@ -3426,9 +3422,12 @@ static int __swap_duplicate(swp_entry_t entry, unsigned char usage) * Help swapoff by noting that swap entry belongs to shmem/tmpfs * (in which case its reference count is never incremented). */ -void swap_shmem_alloc(swp_entry_t entry) +void swap_shmem_alloc(swp_entry_t entry, int nr) { - __swap_duplicate(entry, SWAP_MAP_SHMEM); + struct swap_info_struct *p = swp_swap_info(entry); + unsigned long offset = swp_offset(entry); + + __swap_duplicate(p, offset, nr, SWAP_MAP_SHMEM); } /* @@ -3440,9 +3439,11 @@ void swap_shmem_alloc(swp_entry_t entry) */ int swap_duplicate(swp_entry_t entry) { + struct swap_info_struct *p = swp_swap_info(entry); + unsigned long offset = swp_offset(entry); int err = 0; - while (!err && __swap_duplicate(entry, 1) == -ENOMEM) + while (!err && __swap_duplicate(p, offset, 1, 1) == -ENOMEM) err = add_swap_count_continuation(entry, GFP_ATOMIC); return err; } @@ -3457,7 +3458,10 @@ int swap_duplicate(swp_entry_t entry) */ int swapcache_prepare(swp_entry_t entry) { - return __swap_duplicate(entry, SWAP_HAS_CACHE); + struct swap_info_struct *p = swp_swap_info(entry); + unsigned long offset = swp_offset(entry); + + return __swap_duplicate(p, offset, 1, SWAP_HAS_CACHE); } void swapcache_clear(struct swap_info_struct *si, swp_entry_t entry) From patchwork Thu Jun 6 11:58:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13688378 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC6D3C27C54 for ; Thu, 6 Jun 2024 11:59:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 142066B009D; Thu, 6 Jun 2024 07:59:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0FBDA6B00A0; Thu, 6 Jun 2024 07:59:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E83D16B009F; Thu, 6 Jun 2024 07:59:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id CC8066B009D for ; Thu, 6 Jun 2024 07:59:19 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 796931414DA for ; Thu, 6 Jun 2024 11:59:19 +0000 (UTC) X-FDA: 82200318438.22.968C1FF Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) by imf08.hostedemail.com (Postfix) with ESMTP id 6431316000F for ; Thu, 6 Jun 2024 11:59:17 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=aGYlBpUs; spf=pass (imf08.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717675157; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6L2s5n2sjWQ9jCz7vuGuOZ+fZmIEW+s447Fnt0+HHXo=; b=gMlOzLUhRrIphIlboZ/gcImt4WzKFx4naCydrSCczBw1Hofb8Lvgir4tHZHn3XGwkgUT3V XinpnQ3aDloStKHiMJ9DsTQdxmQ/DSFtmx50nMDdrAuQjnuBl/8VupHUH3/YbS4xmve1i7 ehwkOQykEHvxag7yDhQItbcvTegkK0g= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717675157; a=rsa-sha256; cv=none; b=Ap0Q9XBoVuYvmAvWydk77IizCywZzC1eFRr21LiVkJt9dQ4z8BFOjE39Rd9iV73ovxy1mU Gv6M5iy3mFDTtiEid/AEVIjaZ1pVSxQ+lICxzua2DerQcLaMpM/hX6P9wd1qV8KpiuhUnt FXxCzdMzLBQsj8nyXi+uIEKJmd9KH/U= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=aGYlBpUs; spf=pass (imf08.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1717675155; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=6L2s5n2sjWQ9jCz7vuGuOZ+fZmIEW+s447Fnt0+HHXo=; b=aGYlBpUs+IHwplrsJKybxO6Wt0Y/iWc5bmRfbUtPiJ9sM038kBfcjNXiAMQ8X87zEKyumGPOMeONihniOiZh7JPUZZQRpJktNabKzCfzWFuf8QHJgtSxLBEN6VhvC6pSPs5GEMzspYFa86zmwhu++1KUtPVL5ApAF7eUDvR2M9o= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033022160150;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W7y5mvE_1717675153; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W7y5mvE_1717675153) by smtp.aliyun-inc.com; Thu, 06 Jun 2024 19:59:14 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/7] mm: shmem: support large folio allocation for shmem_replace_folio() Date: Thu, 6 Jun 2024 19:58:53 +0800 Message-Id: <230e35b2fdc13c6199d8437e4cb6f68fded4072d.1717673614.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 9p4xgxnkgzxtow86st66cuby5jnt861b X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 6431316000F X-HE-Tag: 1717675157-534443 X-HE-Meta: U2FsdGVkX1+LSZEwcn0ndXyVGOwbSuLqf2fzpqqZUFWrpHsTHQwB9vcd/3uBjIy85ZKHJk/NCrH/toU88ygZODmY5m/ueD9aXZ0iozlmVH2cp1IJiR4LNuS2eY73TmeXJYa8gGS7pHPk2NM0zParRZsSm4hV/EXKaZy9b2ApBVPouu7F+GrEcbDe5AUGb7VanaWYHX86rdfaENqpu78rnGjnHKcQGt4rhrfktg7y20Bo/4FoBFdIeuyTzALriOITe5APl4CR8G0H2ryXnVE3uvJZxK7b5VfnGGraFbxTP637ZOU/Fk3+GsDfAZ4/btt/+07RBiTfdNiWAUkPTvnd4bXWLqfLalNJz6kZf7iDd0JWtk7dqQEpZfE7g1mluE50rvE0X1pKDfJl5E78laYiYn8MkE5QMA/tYsdZZ+fm4vQ230Ev2bqYTqdbVWURux3ZT7hqrSodzlPhvVab16ilfp8rBbfuybQu48iESydu6sTy0lO2ufmKuJkd40GD2w/PnZhMZYJPhd/E9Hzlezmfzn9iDNDOZ0I6qdxr6TydGmmOBS34lF4sVfwAwmmgQPWKHTmz6ZnkN8CexZbllvasBlYglk+uUXYUT6iW+Pz+RyBL0kWGN4dIsTg4/NSCeYfi6ppbV5NgHpN8pHvZwDOokeatxgZw7Wpsq0zQ/18HEfChkLY2GCFmc8RhuTlIiXdWYopHSSsaJ415RQIRCz87g2sXVMQtFy+X5WGWj06yh3eKvKLSBTNt0q0YvHCtHTGoJiiI0e16sn7ZWqhgVIGSrVFhfJJGWykQ7FNekqKxukh+tkLwcougjEgGbe0zoIP36L0Pxhmt2hmebaooStiEHiVJz/UvuuaGBAe1pr9KPpzaBYEr6bFZsQfxTIKVpZM72tXhBP0KZ1YP8+UrO+356VzeL4EMfp4y8I06iGALOZ07JLcsd8MKPQQ1s6uPkTHaI8JflTKc/hu5TR998i9 xWcnsNIG dxtuhW5kArKva9bY/u+heI0NyZ18ZCcl6Hc5NDFA/TeELDBUHlbGlEOJ61ioV/QtvCAOHzYymNFZQSabFY2yYLvnKx1wQv1ChTjm67A6M8+YqOSCK4w/dbsl6dw0RQK4TTK3mKdjlIkDGMXffCSoxvm0HzQi4mvC4dFK/zh4+W7Gp7GcYFAgQAO4dN5dMSldPLDtTSkOq9OzF1ly0n9pPXijed4wP38Ld3+7iOSBdGrTknXGcRGVc94URjzk9b2RhMU67lp2BnrlpeivpogTpqVax2lVPDXkI5qGW X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support large folio swapin for shmem in the following patches, add large folio allocation for the new replacement folio in shmem_replace_folio(), as well as updating statistics using the number of pages in the folio. Signed-off-by: Baolin Wang --- mm/shmem.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 174d8ae25b9b..eefdf5c61c04 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1889,7 +1889,7 @@ static int shmem_replace_folio(struct folio **foliop, gfp_t gfp, */ gfp &= ~GFP_CONSTRAINT_MASK; VM_BUG_ON_FOLIO(folio_test_large(old), old); - new = shmem_alloc_folio(gfp, 0, info, index); + new = shmem_alloc_folio(gfp, folio_order(old), info, index); if (!new) return -ENOMEM; @@ -1910,11 +1910,13 @@ static int shmem_replace_folio(struct folio **foliop, gfp_t gfp, xa_lock_irq(&swap_mapping->i_pages); error = shmem_replace_entry(swap_mapping, swap_index, old, new); if (!error) { + int nr_pages = folio_nr_pages(old); + mem_cgroup_migrate(old, new); - __lruvec_stat_mod_folio(new, NR_FILE_PAGES, 1); - __lruvec_stat_mod_folio(new, NR_SHMEM, 1); - __lruvec_stat_mod_folio(old, NR_FILE_PAGES, -1); - __lruvec_stat_mod_folio(old, NR_SHMEM, -1); + __lruvec_stat_mod_folio(new, NR_FILE_PAGES, nr_pages); + __lruvec_stat_mod_folio(new, NR_SHMEM, nr_pages); + __lruvec_stat_mod_folio(old, NR_FILE_PAGES, -nr_pages); + __lruvec_stat_mod_folio(old, NR_SHMEM, -nr_pages); } xa_unlock_irq(&swap_mapping->i_pages); From patchwork Thu Jun 6 11:58:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13688380 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AB8DC25B75 for ; Thu, 6 Jun 2024 11:59:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4ECC16B00A0; Thu, 6 Jun 2024 07:59:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 49A1E6B00A1; Thu, 6 Jun 2024 07:59:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 33AB66B00A2; Thu, 6 Jun 2024 07:59:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 119AE6B00A0 for ; Thu, 6 Jun 2024 07:59:21 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id B95341A0473 for ; Thu, 6 Jun 2024 11:59:20 +0000 (UTC) X-FDA: 82200318480.09.37F1FAE Received: from out30-101.freemail.mail.aliyun.com (out30-101.freemail.mail.aliyun.com [115.124.30.101]) by imf25.hostedemail.com (Postfix) with ESMTP id B04B7A0003 for ; Thu, 6 Jun 2024 11:59:18 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=xM+MAmJZ; spf=pass (imf25.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.101 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717675159; a=rsa-sha256; cv=none; b=d5+FNBfuGbvPBq+NLiSdXVgMJffbXsSgpcCzd2JZjURwPb2r7KP4hMn7WL7XEH5LkZgEv6 hKcIyqCvEbxbD5HYcXcNfR3rJGT6lmfGBbjxbjFFLOfeHZeN+USeZfcsB2EVF+1QYhzenL 1UU6zElLhxVDff5sp7jKCuLoYmtns8U= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=xM+MAmJZ; spf=pass (imf25.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.101 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717675159; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Hca3mk3oMk8MShvCe5e0YS4gCuYYhtWQR6R7w3SOo+g=; b=7XkX8Y24JF2yMXGhdspqavXWc8ojcyVy+dsQABwbLtvlynx6k0QjedTfk8VeiFNZVOGwFg wJ+DDq9CkUkTVo2Kf+KdlPwRi/zPLslF/+5hKVt21a95Dx/WNGfD7YdDWTeqrdEcLY/nk4 fzlwW7bojvAbM0dr0qgMfW/gY3VNpOE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1717675156; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=Hca3mk3oMk8MShvCe5e0YS4gCuYYhtWQR6R7w3SOo+g=; b=xM+MAmJZd/2MaO81nJ5EfVwaYed064+ao9HZyGpPISym6h0Qh+J8hEaUYpqteCDhE5D6IKq7cCxVz6Ip8aKzQVbcBu21NJIYqVcsGAxd9iMr6L19vE06garvORzou1fR2y5O757O5r0sOOHqiDsqYdhM7bKMp1PFvE7ULlH6ZJg= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R111e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033037067113;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W7y5mvU_1717675154; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W7y5mvU_1717675154) by smtp.aliyun-inc.com; Thu, 06 Jun 2024 19:59:15 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/7] mm: shmem: extend shmem_partial_swap_usage() to support large folio swap Date: Thu, 6 Jun 2024 19:58:54 +0800 Message-Id: <358924679107339e6b17a5d8b1b2e10ae6306227.1717673614.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B04B7A0003 X-Stat-Signature: fhfosmrtuu7erp1u66jeg3m7e9ekxc1m X-HE-Tag: 1717675158-609653 X-HE-Meta: U2FsdGVkX19fWks5q98Wy7V2fleJ7LaEseTycCIVeix2Cj3CpmHxt5EpgMsNEzL7ajWdBUQ2l8oBGEmscg9lL2W79m9HHQOyzhiR3X8iwvwlVe+R/GkLpW8rbMHBPZI8RyZiM9JGqbRLPzEgYhdSpiqVwoni0lMFZIAEyJ5pa81EGSp59YMDvD1cfcMfYZEnj2UO8YtDA+JbkPfaH8VZFPTX9cXmEO0G2gMiCPEVHoKdE2ELGZvZGYJWTUeNOuWDN+DtxLWCFnursxkWr/BWFb1NCacFg7GczJXQ+sO3aIuEb9ulgC5CTyzF+4tYkrG64M8SiYcH2aNl/QX77edy1fqzfbZRrqK4xfMOFlSFtANvAAXwtrRZDu7U6CkvpDVjoa07xnwk33/SXM01mvjxj9yo0YDCipju6Kb++5pykkE8dNWIHyJkyts5IK/KbfsZMWCx9nrTlrKsKE166c7Hvd4bI75HjIq3+5yhKwxYjFAcAZy0ddm7ViH222g1wipxmORyt6htxwFNfEjiulpu8VmIAwCDh9BukVXHaMOWAS26lrFhJTENEAZD/FSbx+QQOYWeKrFnpjzbZd9CmBIWGOO394MtQXC4vpBCfjDIaHDuMrXQALDySzvcCOgL50NLJSneS96jSlb9RwihMj17z+XljwD81+gwYK/AdA5vYfNv2OmqUxzHWcmIA9FUBVXC3VIH+GjlmJ4XdX6l8+urHVoV28NQIRompKsVY8C9/SVqRchd1tAzgwDKwZTu3oNdElUoV3b1GR9MTKU9ldL8HyUkZutQKtksPOCZWazMlv2uDnJ0Xz7ArNAsWQ/cw0vqtKiA1LKnhKj1/gjrRzL1w6H5+6mAM11BsGcvnddBdkmJO5PWTeQwjMjJbFudBTVDRbYJ2NCs9MFu1CjPd35jG2mGgO3aAdB58z7oNIpZbw6XrrEp9O56TSazVBjgdNUg4uQwl++OahG46FnI5/o EuVxPddM xI6012kD/JmU8ZFhl0nQ/0Aer/ye0hKgt8qgzAgMlx5OtYoePfbQ1CBiFcQpaa3+/4xLlcAr7moh7uTTz1W6UkET9jqUSZqs7dITNfglxx8Xj1U1QRIRb4q7dZ2w+7Tej8j5AL6Uw46rWbJdDEYT85GWMXXxNwmoKJ6Ll+IaivIaLwpjANK3jkb3htxWwtWaFf+9VDpM8PZ6IjNNriJJKG7E/qzcGNqnAFobqtwcSCoCjwbasTRZKNA9B8xDHNlBg/YbahHnH0MRD4rnlBE1a5i9Dww== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To support shmem large folio swapout in the following patches, using xa_get_order() to get the order of the swap entry to calculate the swap usage of shmem. Signed-off-by: Baolin Wang --- mm/shmem.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index eefdf5c61c04..0ac71580decb 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -865,13 +865,16 @@ unsigned long shmem_partial_swap_usage(struct address_space *mapping, struct page *page; unsigned long swapped = 0; unsigned long max = end - 1; + int order; rcu_read_lock(); xas_for_each(&xas, page, max) { if (xas_retry(&xas, page)) continue; - if (xa_is_value(page)) - swapped++; + if (xa_is_value(page)) { + order = xa_get_order(xas.xa, xas.xa_index); + swapped += 1 << order; + } if (xas.xa_index == max) break; if (need_resched()) { From patchwork Thu Jun 6 11:58:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13688381 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60B32C27C54 for ; Thu, 6 Jun 2024 11:59:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B2B956B00A1; Thu, 6 Jun 2024 07:59:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B03416B00A2; Thu, 6 Jun 2024 07:59:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 954B66B00A3; Thu, 6 Jun 2024 07:59:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 7327F6B00A1 for ; Thu, 6 Jun 2024 07:59:23 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 2E874A0AEA for ; Thu, 6 Jun 2024 11:59:23 +0000 (UTC) X-FDA: 82200318606.19.7886B35 Received: from out30-118.freemail.mail.aliyun.com (out30-118.freemail.mail.aliyun.com [115.124.30.118]) by imf07.hostedemail.com (Postfix) with ESMTP id C751240012 for ; Thu, 6 Jun 2024 11:59:20 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=MJejOjNZ; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf07.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.118 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717675161; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=o5pjG+XJpg8uvpOyj3LZoopMVj7n3+rVaRa69RLiHo8=; b=j4d5iRnyaBVpwdYD2/3oVrPxmpNyzaGoCU7FMEtCknjPaN5KV+YkrshVZtLoqN1Zvs0Pmu Mn4KxXXUziy8RGbIKWt/ZGuFovi2DhGzKlB+Ad5s28LHlG3bwu+Ul97n65jPFLGCBM43w8 tC2MGoE+TvatFCRa5J1tIxPp+I7FSO4= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=MJejOjNZ; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf07.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.118 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717675161; a=rsa-sha256; cv=none; b=xAgMiy/Rl1H2QZw36KES8Gj1qOSSb4WmNjM+Pkx/kCBUIS7K5jcK1RYhnwCTAWSl4Jw0+i ai1OB3hziLLK079y681mFfnFtjZj/ohuQpBsgbl4vz0YnBWUOKNMm/M5LwIEaLxsuFcilo SpYUqTN6hlRgbv0Y0P6Kf7aZQItN9xs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1717675157; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=o5pjG+XJpg8uvpOyj3LZoopMVj7n3+rVaRa69RLiHo8=; b=MJejOjNZG/SXzwGDbC5r2mj+D4SUY6IM8mgFNQtzQZtO+gnSaNddtDbXD4TaA2IOMCOEeTBa2Vi/qmughpBZDAxoyFDbursHWolksm/4u70JNlgwy5K7LCrbRZtUA3bR37Pjf0QrFNEjlaGJGhKfODZIQ+9IH6N0JYMWyVJIATY= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R201e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033037067109;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W7yDX0a_1717675156; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W7yDX0a_1717675156) by smtp.aliyun-inc.com; Thu, 06 Jun 2024 19:59:17 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 6/7] mm: shmem: use swap_free_nr() to free shmem swap entries Date: Thu, 6 Jun 2024 19:58:56 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: C751240012 X-Stat-Signature: axwni5ya6uwgbagydx8c8ofg4ncu9onq X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1717675160-784252 X-HE-Meta: U2FsdGVkX19R2XoL/CWV9DURLgs3qq8Unxi33xflt8G1Mw9NmAyajmSi18OfcNVoBmJQaZW4G95Z3q4+9SE0IfUGv8g62bbDOS4R528j3tV1BNjdRvnmvWDsG7ebLVUHe+2gaF11BDkvyOkXMnvwRwsyZf+62WU3X4Z4FyaRpCy0fT6Kcq8OUUgP+FqfmOgLMi2gbqo8eESaEVSQnmOxQykWP0x2cUvcLYnIMvQA0LkBghzI/X/yTJra8PGwx0hHCh0AhW37qkV3QGoJ8uOP8ahEUIZSrQ2CmEJG8dgEeTLo5H28RBJPFKwdlkm1ndW0Pm/hP3vbVH7O6gbSdJflCCDhBQON/Vj8gauc58k+apLJpb0xc5kD+DAmUTD5Hvfjw1e9SXyZPs3j3kK/8SaIlpQ5Tpeh8e811wfd+0qy2UwJP/Vb5Jp0AtWHHAfgSf5echj1yzhfUOpYtlyyBOI3qzPhKzDAHL0gPTZusGvuUu5iZQrNbSnKnOX8HU3K4cvPcjXOC4E10zFQCO4M4OFWMHNz6nOVOtisuQ9BIwDUyunLExm5IJTO4p3YN1WFEURwo9Puez5L1rSp30BsysFXBmfjmdCFLTuqI1XRo+0BNhS6WykAUdETCWTgVlEOSPbtS19rBw7TbIMamQDgwJBsYTTqQ1xOsrgeaf6QXMXfC/IjsYaDZxi3SCn+bWm+OaugpA4fiQ9awfj2D3lwxFgkoA6zDC9qOzrq7ID3wQ8scsimb9gX2R7Gl25y+KRdAKoAHXgQ0cocHAVMQ6tqAlmAfBty1SreOnRgzXwRN0ZCZhLQa2650AxQscdQaRN7dhoeJX2ld4Aigm0Q/FRC/DzmlZmI47REawIQWU2DHycgmykHTQVMmI/gTDLoGlJj0H3VKbx2IkcIHPBGPNgZxa20JzvA4WvFnOJZWtU4wlc+FwzNDLcW52k5mkGpHRhN5Mnc6uxKE7CJiwobO/cqxiJ WVPmaOXz vqjQklO4XQZHCjR57OI8tpUTPZ50El+9Mn8BUxLg+RhQESbE0yhDeuOnUA+5wZs5HN9loLerabyDF23D80uEECQh0JSMXTX7Qn9G3TyPltiBOL+gJ+PjVVleRway2sGjIsyT0o5p9p7L1FGHvzfkIlrmrp8ejg/2PQ3g5OpzZlwDYW+hczZ8noOfXWk+w7mWJi3zymkfTeoGWiPylQUxshZJHgT1iTixS/CTz9h00znJfN2XyHK3eDgZYyQztp16NMCbiDdAONNFk4XMePB9hUNt/71f6RlWBKr5L X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: As a preparation for supporting shmem large folio swapout, use swap_free_nr() to free some continuous swap entries of the shmem large folio when the large folio was swapped in from the swap cache. Signed-off-by: Baolin Wang --- mm/shmem.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 28ba603d87b8..33af3b2e5ecf 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1950,6 +1950,7 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, struct address_space *mapping = inode->i_mapping; swp_entry_t swapin_error; void *old; + int nr_pages; swapin_error = make_poisoned_swp_entry(); old = xa_cmpxchg_irq(&mapping->i_pages, index, @@ -1958,6 +1959,7 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, if (old != swp_to_radix_entry(swap)) return; + nr_pages = folio_nr_pages(folio); folio_wait_writeback(folio); delete_from_swap_cache(folio); /* @@ -1965,8 +1967,8 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, * won't be 0 when inode is released and thus trigger WARN_ON(i_blocks) * in shmem_evict_inode(). */ - shmem_recalc_inode(inode, -1, -1); - swap_free(swap); + shmem_recalc_inode(inode, -nr_pages, -nr_pages); + swap_free_nr(swap, nr_pages); } /* @@ -1985,7 +1987,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, struct swap_info_struct *si; struct folio *folio = NULL; swp_entry_t swap; - int error; + int error, nr_pages; VM_BUG_ON(!*foliop || !xa_is_value(*foliop)); swap = radix_to_swp_entry(*foliop); @@ -2032,6 +2034,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, goto failed; } folio_wait_writeback(folio); + nr_pages = folio_nr_pages(folio); /* * Some architectures may have to restore extra metadata to the @@ -2050,14 +2053,14 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, if (error) goto failed; - shmem_recalc_inode(inode, 0, -1); + shmem_recalc_inode(inode, 0, -nr_pages); if (sgp == SGP_WRITE) folio_mark_accessed(folio); delete_from_swap_cache(folio); folio_mark_dirty(folio); - swap_free(swap); + swap_free_nr(swap, nr_pages); put_swap_device(si); *foliop = folio; From patchwork Thu Jun 6 11:58:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 13688382 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EEEDEC25B75 for ; Thu, 6 Jun 2024 11:59:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ADD566B00A3; Thu, 6 Jun 2024 07:59:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ABFFC6B00A4; Thu, 6 Jun 2024 07:59:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 92E516B00A5; Thu, 6 Jun 2024 07:59:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 738E56B00A3 for ; Thu, 6 Jun 2024 07:59:24 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 25DB11406AF for ; Thu, 6 Jun 2024 11:59:24 +0000 (UTC) X-FDA: 82200318648.13.86BC17F Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) by imf12.hostedemail.com (Postfix) with ESMTP id 311AB40016 for ; Thu, 6 Jun 2024 11:59:21 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=Owc+0AsP; spf=pass (imf12.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717675162; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RKBAfrZjR8AbxPwpVk449LMfTdrMSV2SRdqL98hLFps=; b=XLqN6qJ/wNu0+NnVXZY3HPSbMHo4RJ2y0TSz+twrMLU69MiN+hWWLIhcVGc/3ghGfrA01/ Q9trBKb7I5oJlPsim4DDeJWLZjlbTKEPe1EH96Z28yfNxkbcZpZyc+ZqiVdrlvvyOTVMo3 bAomxlF+hUgElNEI3wa0tro0S0Owyqw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717675162; a=rsa-sha256; cv=none; b=UtEat4Xhh2ru3L62ReFqukEbvqGK3NMRwNFIi6XVHnUtpnqELCw+hms5YcAsUlthFpQzJf n6wFRQWZ/HkS6Q3i+bdzX6tX4qQTdJIDq86wQRsQ4rYmNpxjppNR131TTB1x+vf+JTYy/b uu4K7fLw6+phB+YKRO+BCgMqE2vK4aQ= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=Owc+0AsP; spf=pass (imf12.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1717675159; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=RKBAfrZjR8AbxPwpVk449LMfTdrMSV2SRdqL98hLFps=; b=Owc+0AsPHpVCRRu6x6Gg1l+dYL5zMc7HdkqwV12gBkHYHmIfqDDF35EkF6q1oTZQ4VAeAh7SW2nBqK8qqoKkVzulH2NBymlx2RMC2y9Oek9+PpQ4DsXtTJqkbqjJyaWnNtDIJaGKT4z+kEtghKQ0mnRr76MaL6n1Vc/M2e0l++8= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R231e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033045046011;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W7y5mwW_1717675157; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W7y5mwW_1717675157) by smtp.aliyun-inc.com; Thu, 06 Jun 2024 19:59:18 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 7/7] mm: shmem: support large folio swap out Date: Thu, 6 Jun 2024 19:58:57 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 311AB40016 X-Rspam-User: X-Stat-Signature: 4kk6ee491a66uyetnz8joiec98pp3gqy X-HE-Tag: 1717675161-25235 X-HE-Meta: U2FsdGVkX1+DVYxbtU8FNuEeVdiSEPM8Dliww/RFfnBPT6+4N8VMBUvQug8niVpLZd0d3arBP68eUc4T5noxM4o6P4p1qts/wY/ktxgnkfx437kdE8CBIkqUaSXkRDNRK9Fvi2+2qlTkqNlCIgGwbaXFHsJQYdTke4lK10hNGoXlSxIbPByLgvIwUA8cccBT25bfGoV+ts/vXwBqQ3wUSI5LEMmX9Om/kPGPUMsUPC1qFHbYbvo0G30jp8FPbwuTIv30iA7pQKEhdzgiVEwSNzdf7Rsa2hJcMEBGAxUhhwKduZcTMtXAkhSeGWga1r4Pp0+vy0HAyQK1qwgtGV12W4yG6WzexsKC46QQtloRKKGsnDOtGQxPGMTr2xinHYm1ZI8BaZ6PYelHTS0WORr84j9d3ZFAbNHYMhomz46SO5loWGyHf/MIog/HI408/6CsIW+hXMSm0oQnMnWWKAZ+VoUTJ3dsrBU7+QK6iWJUM5gC5ZdHNMREBzUoM3F5gd2UUg00be0B7uIfB2VUjzYfhkvW64Kj3JeCCd1pcDfxZ8BQrverXX1O49mYrPBhJMI2JxhyfuoJlMVRBtmuffIKBDaWK1IFr2Glge6q5YSRFnko2hsoutqlvn4iZ/ylRoOjVyNISfvm73cbx7RzikjUy5hi88WW9DtPmAY89OdzpiRc6Zmj32uKSAFnyazvdEYfEINyhvO2kpn4G0nN+oV8W6T6mtqYmsh4mI2TS4iv8lgCz/XTKf6o6IQk3CKNPcHYvqduB149Ct9RvPOmktgYFK1RFcXptZtYaHtw619qfXJfKBPtZFcttBn561uh4xNJ9EJNvHhkzDn0LlnMjL4XbFQY/PfETLq9388WKkpxbB1+BddZdoPJ32/Jh9OnWuRbndChTiquqHozsJ9PhqADkkQA0JxGynPVMVywZKKzS+yWsVhSz6z72HdYZSebSMOdsK/K6Kd+yun5yk/PGR4 z7ESFJFl CBRxl7aaYE/c96ps9iRQMPa+Uxm0NsxSkm2XrH2IsSGbbtyFBXS0NFkKTiKjDHa5KbvWoQcJaHiR1j80ddP6cP/t6pN1qX0a/xRgVEphc2sLsB3r3UwCnRzMmWbevGTlAROqKESOjQKyEyuGTGpW76wLlqxZLzX/o5f171tg/ajKRqN8BIlCoRHzFyKT8ggcfc+QKHwUeRmg/KvxqUkEADIoiARrTmzftwgeDAFVwh1L3EntOE9Cx5IP49XK9SMrR7WOODytS3JZ7vBgIf6P9Kt3Soem/LS+byjWhgF60R94IWXiLPpGzHAVySsEsBlou5WLx X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Shmem will support large folio allocation [1] [2] to get a better performance, however, the memory reclaim still splits the precious large folios when trying to swap out shmem, which may lead to the memory fragmentation issue and can not take advantage of the large folio for shmeme. Moreover, the swap code already supports for swapping out large folio without split, hence this patch set supports the large folio swap out for shmem. Note the i915_gem_shmem driver still need to be split when swapping, thus add a new flag 'split_large_folio' for writeback_control to indicate spliting the large folio. [1] https://lore.kernel.org/all/cover.1717495894.git.baolin.wang@linux.alibaba.com/ [2] https://lore.kernel.org/all/20240515055719.32577-1-da.gomez@samsung.com/ Signed-off-by: Baolin Wang --- drivers/gpu/drm/i915/gem/i915_gem_shmem.c | 1 + include/linux/writeback.h | 1 + mm/shmem.c | 3 +-- mm/vmscan.c | 14 ++++++++++++-- 4 files changed, 15 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c index c5e1c718a6d2..c66cb9c585e1 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c @@ -308,6 +308,7 @@ void __shmem_writeback(size_t size, struct address_space *mapping) .range_start = 0, .range_end = LLONG_MAX, .for_reclaim = 1, + .split_large_folio = 1, }; unsigned long i; diff --git a/include/linux/writeback.h b/include/linux/writeback.h index 112d806ddbe4..6f2599244ae0 100644 --- a/include/linux/writeback.h +++ b/include/linux/writeback.h @@ -63,6 +63,7 @@ struct writeback_control { unsigned range_cyclic:1; /* range_start is cyclic */ unsigned for_sync:1; /* sync(2) WB_SYNC_ALL writeback */ unsigned unpinned_netfs_wb:1; /* Cleared I_PINNING_NETFS_WB */ + unsigned split_large_folio:1; /* Split large folio for shmem writeback */ /* * When writeback IOs are bounced through async layers, only the diff --git a/mm/shmem.c b/mm/shmem.c index 33af3b2e5ecf..22a5116888ce 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -776,7 +776,6 @@ static int shmem_add_to_page_cache(struct folio *folio, VM_BUG_ON_FOLIO(index != round_down(index, nr), folio); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); VM_BUG_ON_FOLIO(!folio_test_swapbacked(folio), folio); - VM_BUG_ON(expected && folio_test_large(folio)); folio_ref_add(folio, nr); folio->mapping = mapping; @@ -1460,7 +1459,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) * "force", drivers/gpu/drm/i915/gem/i915_gem_shmem.c gets huge pages, * and its shmem_writeback() needs them to be split when swapping. */ - if (folio_test_large(folio)) { + if (wbc->split_large_folio && folio_test_large(folio)) { /* Ensure the subpages are still dirty */ folio_test_set_dirty(folio); if (split_huge_page(page) < 0) diff --git a/mm/vmscan.c b/mm/vmscan.c index 9146fd0dc61e..3523fd2dc524 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1263,8 +1263,12 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, if (!total_swap_pages) goto activate_locked; - /* Split shmem folio */ - if (split_folio_to_list(folio, folio_list)) + /* + * Only split shmem folio when CONFIG_THP_SWAP + * is not enabled. + */ + if (!IS_ENABLED(CONFIG_THP_SWAP) && + split_folio_to_list(folio, folio_list)) goto keep_locked; } @@ -1366,10 +1370,16 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, * starts and then write it out here. */ try_to_unmap_flush_dirty(); +try_pageout: switch (pageout(folio, mapping, &plug)) { case PAGE_KEEP: goto keep_locked; case PAGE_ACTIVATE: + if (shmem_mapping(mapping) && folio_test_large(folio) && + !split_folio_to_list(folio, folio_list)) { + nr_pages = 1; + goto try_pageout; + } goto activate_locked; case PAGE_SUCCESS: stat->nr_pageout += nr_pages;