From patchwork Wed Apr 24 06:30:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 13641256 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21991C4345F for ; Wed, 24 Apr 2024 06:31:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8B69E6B0208; Wed, 24 Apr 2024 02:31:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 867616B0209; Wed, 24 Apr 2024 02:31:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 707B06B020A; Wed, 24 Apr 2024 02:31:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 4E3086B0208 for ; Wed, 24 Apr 2024 02:31:03 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C3DE8A106A for ; Wed, 24 Apr 2024 06:31:02 +0000 (UTC) X-FDA: 82043452764.03.6868124 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) by imf01.hostedemail.com (Postfix) with ESMTP id 3095A4000B for ; Wed, 24 Apr 2024 06:30:59 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=bRyfQA0s; spf=pass (imf01.hostedemail.com: domain of ying.huang@intel.com designates 192.198.163.8 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713940261; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=I/uVbcXvYSr46TAilb11taod60us2/dT26t3JxjU17g=; b=QlYKzwkRZbkVy/QhaCH5ew6PRCtrLJJ7t0IVk2JLW6dkptt0mcHVsNvtegEFkMcRvLy/mY yiLRfLIQduC/51MONAtS5/hTAiz5WHpcF+oql+Hpcnty//OxF7GNi4Pe1qTKMEmjKCXUU7 +e9RoJjJCj2NX69MhdkVj1qA6GrZ73Y= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713940261; a=rsa-sha256; cv=none; b=rbQYv4ATayStMtq+0PtbRHiuboKOFxx4dCvgw5nb4/aEnSMu7PRiG2+p0mtfQyl5KyXyAk XioMHshC2saG+e7Xuw6ay4jjm3Yty9nKO1RRBOlPbB5+z069mBKqdsNeH9/B21jaxWP4r2 W3M3FyC/OAVuy3SIGFqw5Gc6C4R4Xy4= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=bRyfQA0s; spf=pass (imf01.hostedemail.com: domain of ying.huang@intel.com designates 192.198.163.8 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1713940260; x=1745476260; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=USIiH+WCOin8AaRM3HyS9rsb7dMz9eGkGM+QMZdj2T8=; b=bRyfQA0sSbW8N7qlo4zCWBdMXtRcLc2LhjWjAAoCmsxG5wtirKmGOHrr N0vLk6wR4vwWcZbyy0aNmujmhO8ljoSLnt7XNgkghHeBwsuCOkSWQ1358 ZxoY6DtTT4HP2/mMR4NvmUER0o+N4wRjgS/L7JWwTy4RyzCmb4/c501Ts Ql/nPkRtwFV7BtoccaBcOFxu8f2itZjsacDM2VgEcWcaj5eXUF19D6vtM sSqasmIo7F8iEm/3AT4UftAfOA1YW3TUqOd97xhN2v2iDUVZR3MSJX5Fl p8Fwa2QEFm77mwCxD9exui7hnRvKHw8/8faLKAc8YiNQ+yqf4ynt+Md5Y Q==; X-CSE-ConnectionGUID: h86eqy+QSsCMw8S6kzpGSQ== X-CSE-MsgGUID: z9PGR1TgR1SuHqfBPFv+og== X-IronPort-AV: E=McAfee;i="6600,9927,11053"; a="27067856" X-IronPort-AV: E=Sophos;i="6.07,225,1708416000"; d="scan'208";a="27067856" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Apr 2024 23:30:58 -0700 X-CSE-ConnectionGUID: zmYgMUzTR9e0Fgun8F19+A== X-CSE-MsgGUID: C1E3fsLkSh+nxy6LZhpbTg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,225,1708416000"; d="scan'208";a="29413610" Received: from unknown (HELO yhuang6-mobl2.sh.intel.com) ([10.238.6.133]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Apr 2024 23:30:54 -0700 From: Huang Ying To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Huang Ying , Hugh Dickins , Alistair Popple , Anshuman Khandual , David Hildenbrand , Mel Gorman , Miaohe Lin , Minchan Kim , Ryan Roberts , Yang Shi , Yu Zhao , Kairui Song , Barry Song , Chris Li , Yosry Ahmed Subject: [PATCH] mm,swap: cleanup VMA based swap readahead window calculation Date: Wed, 24 Apr 2024 14:30:42 +0800 Message-Id: <20240424063042.665018-1-ying.huang@intel.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 3095A4000B X-Rspam-User: X-Stat-Signature: hrnim9jotn7utcfwd1gx6qbxfdcfwqx8 X-HE-Tag: 1713940259-122751 X-HE-Meta: U2FsdGVkX19LaiBP8WveynHA6PcM6toutbxw3jsYG+z7ooVzIJPNlr6iuLW1x7F1a27WWshn5tgG9W4XaNn5KKzX+j263J9mV74z4rTC3EvKy/7E8wQ1WpKH97Cb8JHgc04rJmTUrJWlrljt0rLpBmAqacEbsn844sosq+yR5rGw9ZOuRWkI7mjt2TLDHYfXF91V0+Ibk9R6oFeGzYfX3QaKl1uWNvAqD3ENEWUzNsWDUuCqJI3m1Zzp6IKw8r+G+Mv8gn4zWlCppLtxm1j7/kGUzlYSp4xpM3bfRty7/D5hFDHY7OCgS9JEU1wIUkUAPB5LlkO1LMT83BGEV+txR+CsFJL733Bh2/puRjn4QQVFnFNFh9IF/RycYZB3ZA9lMXiiDZ9wnjFTaVbxY/8Cr+g0q1RVsAfObZQZcbOIwkPVBP9fYGhBTKWUw3eG0yp3kgiotDEYoK0tdfSzXCQ29xFw+o3UNvuZO/zvQkS92ghft3Dow8dW0KvWgEvkzn9AQryraFYGHM53EVunxZ8ACMzK5lNf96VVoz/rlzDQq2NpPTcNGvyfxGjKOTOkV6QzIu4SnQ7hOCSKKI5HPVVYaQijt57pLMaaRbce5pv2Swpf6pXCUy4rmq9BNizPV51ouJdwtNaCZCbNjLB0jFFhxSbBcnV5v4Jnc67XRIfT4dVSf8ZppTcHxxre2srmQOH4wDHGmbu8dXUPhXLWYICT1ba93QUYrjpI6k0mhg3CWC+Flrjvv9SQiINtVxCeUZaX/DnrhuBsA3HfRpo9Bk06OZrMpUj9FMQRhRaFc5IvGZc8Z6MDZu2daq/lAfZxdClMt6tqKkucW4BHEDchf4PJJIpFetq4BvAfK7lVjbiq4wtBgpNdaktkEngpNvQlHD7F X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When VMA based swap readahead is introduced in commit ec560175c0b6 ("mm, swap: VMA based swap readahead"), "struct vma_swap_readahead" is defined to describe the readahead window. Because we wanted to save the PTE entries in the struct at that time. But after commit 4f8fcf4ced0b ("mm/swap: swap_vma_readahead() do the pte_offset_map()"), we no longer save PTE entries in the struct. The size of the struct becomes so small, that it's better to use the fields of the struct directly. This can simplify the code to improve the code readability. The line number of source code reduces too. No functionality change is expected in this patch. Signed-off-by: "Huang, Ying" Cc: Hugh Dickins Cc: Alistair Popple Cc: Anshuman Khandual Cc: David Hildenbrand Cc: Mel Gorman Cc: Miaohe Lin Cc: Minchan Kim Cc: Ryan Roberts Cc: Yang Shi Cc: Yu Zhao Cc: Kairui Song Cc: Barry Song Cc: Chris Li Cc: Yosry Ahmed --- mm/swap_state.c | 86 ++++++++++++++++++------------------------------- 1 file changed, 32 insertions(+), 54 deletions(-) diff --git a/mm/swap_state.c b/mm/swap_state.c index bfc7e8c58a6d..19874e7bab70 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -42,6 +42,8 @@ struct address_space *swapper_spaces[MAX_SWAPFILES] __read_mostly; static unsigned int nr_swapper_spaces[MAX_SWAPFILES] __read_mostly; static bool enable_vma_readahead __read_mostly = true; +#define SWAP_RA_ORDER_CEILING 5 + #define SWAP_RA_WIN_SHIFT (PAGE_SHIFT / 2) #define SWAP_RA_HITS_MASK ((1UL << SWAP_RA_WIN_SHIFT) - 1) #define SWAP_RA_HITS_MAX SWAP_RA_HITS_MASK @@ -738,62 +740,42 @@ void exit_swap_address_space(unsigned int type) swapper_spaces[type] = NULL; } -#define SWAP_RA_ORDER_CEILING 5 - -struct vma_swap_readahead { - unsigned short win; - unsigned short offset; - unsigned short nr_pte; -}; - -static void swap_ra_info(struct vm_fault *vmf, - struct vma_swap_readahead *ra_info) +static int swap_vma_ra_win(struct vm_fault *vmf, unsigned long *start, + unsigned long *end) { struct vm_area_struct *vma = vmf->vma; unsigned long ra_val; - unsigned long faddr, pfn, fpfn, lpfn, rpfn; - unsigned long start, end; + unsigned long faddr, prev_faddr, left, right; unsigned int max_win, hits, prev_win, win; - max_win = 1 << min_t(unsigned int, READ_ONCE(page_cluster), - SWAP_RA_ORDER_CEILING); - if (max_win == 1) { - ra_info->win = 1; - return; - } + max_win = 1 << min(READ_ONCE(page_cluster), SWAP_RA_ORDER_CEILING); + if (max_win == 1) + return 1; faddr = vmf->address; - fpfn = PFN_DOWN(faddr); ra_val = GET_SWAP_RA_VAL(vma); - pfn = PFN_DOWN(SWAP_RA_ADDR(ra_val)); + prev_faddr = SWAP_RA_ADDR(ra_val); prev_win = SWAP_RA_WIN(ra_val); hits = SWAP_RA_HITS(ra_val); - ra_info->win = win = __swapin_nr_pages(pfn, fpfn, hits, - max_win, prev_win); - atomic_long_set(&vma->swap_readahead_info, - SWAP_RA_VAL(faddr, win, 0)); + win = __swapin_nr_pages(PFN_DOWN(prev_faddr), PFN_DOWN(faddr), hits, + max_win, prev_win); + atomic_long_set(&vma->swap_readahead_info, SWAP_RA_VAL(faddr, win, 0)); if (win == 1) - return; + return 1; - if (fpfn == pfn + 1) { - lpfn = fpfn; - rpfn = fpfn + win; - } else if (pfn == fpfn + 1) { - lpfn = fpfn - win + 1; - rpfn = fpfn + 1; - } else { - unsigned int left = (win - 1) / 2; + if (faddr == prev_faddr + PAGE_SIZE) + left = faddr; + else if (prev_faddr == faddr + PAGE_SIZE) + left = faddr - (win << PAGE_SHIFT) + PAGE_SIZE; + else + left = faddr - (((win - 1) / 2) << PAGE_SHIFT); + right = left + (win << PAGE_SHIFT); + if ((long)left < 0) + left = 0; + *start = max3(left, vma->vm_start, faddr & PMD_MASK); + *end = min3(right, vma->vm_end, (faddr & PMD_MASK) + PMD_SIZE); - lpfn = fpfn - left; - rpfn = fpfn + win - left; - } - start = max3(lpfn, PFN_DOWN(vma->vm_start), - PFN_DOWN(faddr & PMD_MASK)); - end = min3(rpfn, PFN_DOWN(vma->vm_end), - PFN_DOWN((faddr & PMD_MASK) + PMD_SIZE)); - - ra_info->nr_pte = end - start; - ra_info->offset = fpfn - start; + return win; } /** @@ -819,24 +801,20 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, struct swap_iocb *splug = NULL; struct folio *folio; pte_t *pte = NULL, pentry; - unsigned long addr; + int win; + unsigned long start, end, addr; swp_entry_t entry; pgoff_t ilx; - unsigned int i; bool page_allocated; - struct vma_swap_readahead ra_info = { - .win = 1, - }; - swap_ra_info(vmf, &ra_info); - if (ra_info.win == 1) + win = swap_vma_ra_win(vmf, &start, &end); + if (win == 1) goto skip; - addr = vmf->address - (ra_info.offset * PAGE_SIZE); - ilx = targ_ilx - ra_info.offset; + ilx = targ_ilx - PFN_DOWN(vmf->address - start); blk_start_plug(&plug); - for (i = 0; i < ra_info.nr_pte; i++, ilx++, addr += PAGE_SIZE) { + for (addr = start; addr < end; ilx++, addr += PAGE_SIZE) { if (!pte++) { pte = pte_offset_map(vmf->pmd, addr); if (!pte) @@ -856,7 +834,7 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, continue; if (page_allocated) { swap_read_folio(folio, false, &splug); - if (i != ra_info.offset) { + if (addr != vmf->address) { folio_set_readahead(folio); count_vm_event(SWAP_RA); }