From patchwork Tue Jan 2 17:53:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13509227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9450EC46CD2 for ; Tue, 2 Jan 2024 17:54:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 30C566B012C; Tue, 2 Jan 2024 12:54:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2BAFA6B012D; Tue, 2 Jan 2024 12:54:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 134486B0131; Tue, 2 Jan 2024 12:54:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 005236B012C for ; Tue, 2 Jan 2024 12:54:02 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C2BACA0864 for ; Tue, 2 Jan 2024 17:54:02 +0000 (UTC) X-FDA: 81635119524.08.765095B Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by imf16.hostedemail.com (Postfix) with ESMTP id D4E1D18001E for ; Tue, 2 Jan 2024 17:54:00 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=iiPuvqDm; spf=pass (imf16.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704218040; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=leW6kIEfUqfY3XAn3Ki1C0WiRu5w1qcwcxw36bp08sM=; b=GWlnkUHgMyqHzVzkkjOSZ1+z5O29dUIrAyUibHsW+LXY++PunXu2DK3WRnQ8jZHKc610k8 fhbfGYfJF/m6aI9/WgdrwbqONI2mMfFT+3kQbm/RQuHT34e8r12x0yLaUeax9hDwq/W/vf pjM5Fuzi7LirXKScKYi0vomlQFP1XQw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704218040; a=rsa-sha256; cv=none; b=hGTVcHe82nA+cJxCYFGURwRXhitZMp14sjaljjOdcXh4OmAtzet9AZ65p4HIdjpcsrDpFp y44CJMVuCFVtaDA7fDJQ1wDzmsnDHzjkW/InNlEUriikhHfe5OnpkE+3ADaoH7paNXot+t hGHKlK6fEnKRhCRHJTIz40aFwMvU8nk= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=iiPuvqDm; spf=pass (imf16.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-1d3ea5cc137so74551455ad.0 for ; Tue, 02 Jan 2024 09:54:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1704218039; x=1704822839; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=leW6kIEfUqfY3XAn3Ki1C0WiRu5w1qcwcxw36bp08sM=; b=iiPuvqDmVfJALMtlSoDAD1hK+FsTrhm53QKic8ZGuXJC5xXvSeXgJ5B2Khq0PTotLD XHqfHM4Ex9gknoyLfNzCafUNKo1L/Vh9Ei8ZAm/9c1o7L63EH1Werte20zH0PWeycdj6 QX7AneLykeK9s9jL6tpx1sP5YxnXdOJ0Lwaa8xc/2UkQngUI445ziEDoa0khcixztj+W r/miCpwtCjs7D4o9ZElad5z4coBy6AFjPcZCDA+X762DgzTlV8BaXI24fHrVlB0OE85E FBsp9LvilN+GedMNwdaIMjXGBdXwhvLOuJjl9oKMxRM4WVvtIPrjgPyKADpUcx+QIf1S ts1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704218039; x=1704822839; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=leW6kIEfUqfY3XAn3Ki1C0WiRu5w1qcwcxw36bp08sM=; b=dKjeGhvPJ7cj9G5rN0DN3gTElF2oT+eKVUMQ0GezDhrOnToHTjmxHM6WmdXYcCKPPJ bjX1T1TQiVkwEIMe8cdvCucCrzee140+lA05KjwBfDW9Uogc6zVVFAHPAZ4lFH8AARZM RBSk9ueWZ9VU+C/pyJhyNbMggQGx3ZSApMr3gfIe1buqk3M3HWqCz2e00K9c1J7qvk9x W38cTXGxW3te4cEAQacQIHgbMCdvgix3A7py4hxN08hVMIUooFST1nqHExBu0baSzD5q hshMCTMCK8lGKzYOQSNsikVtZ9M4VrQ9HrJ7XIjZ7hWE9r6P0iJCLd+CGl0oXhFlDHww LIcw== X-Gm-Message-State: AOJu0Yz37FjXt/BglEXsQHm0AoMflbIh93QudEx4EuYVX5hUFOOXQ01v gY4ousvKus82v5Vj86PYZHvvpRpaAN9HmCfa X-Google-Smtp-Source: AGHT+IHbiz8m7h29JKdOV+VIY3InvdMjldzD40RepmLTENYklXxpKThd4lbQBZ1H/dFstp+Ax17T1A== X-Received: by 2002:a17:902:a585:b0:1d0:6ffe:a28 with SMTP id az5-20020a170902a58500b001d06ffe0a28mr17151404plb.134.1704218039237; Tue, 02 Jan 2024 09:53:59 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.41.9]) by smtp.gmail.com with ESMTPSA id be10-20020a170902aa0a00b001d3c3d486bfsm22151969plb.163.2024.01.02.09.53.55 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 02 Jan 2024 09:53:58 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , "Huang, Ying" , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , Yosry Ahmed , David Hildenbrand , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH v2 2/9] mm/swap: move no readahead swapin code to a stand-alone helper Date: Wed, 3 Jan 2024 01:53:31 +0800 Message-ID: <20240102175338.62012-3-ryncsn@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240102175338.62012-1-ryncsn@gmail.com> References: <20240102175338.62012-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: D4E1D18001E X-Rspam-User: X-Stat-Signature: 6zg7etpqq1zst4pxk4fx1o7sd79zix4i X-Rspamd-Server: rspam03 X-HE-Tag: 1704218040-494312 X-HE-Meta: U2FsdGVkX18WmY4rQVd0TN8vc7Ibw59rhbiZoYP7FNU0TAl/odo3pdSkE73OaKIWX1o/pOpfYA4T9BmIVO5YHwPbmNnDmeP+kq2/2BYnKffZtsThdD3Z6C7alJI+Oz8vL/emUKrpHbmAEt2s6sRkt7JREZdP3rIZTSmHQdjSE+RCF4w6HKmyAXdo6kWpO+HV4bMzW+C2UFbRCXgXltZXk4k+sDxRFJMFQqOI13NG/1cXIpSUTO/9t7vAzUqIt9j6vuP3OrJmrPMXnXfaqkKNySd8cFG5Gz3gp/BEoVoQsiYjLl2Ux202cfFFcZxCdAsBxAWgSN53YtKZ92LIiMwRWCiv/J/uTUdo7giJvIxyZ2myONefeCP9yjHuewvmW/E+hH8SjUzJas9I1C46LAnSXHoc+MEVXsQdOJymmzPgJuBUDjcmFzJ5D2bS2C9g7gxlO76b6nJSulDin3H2G6LLuFMCk/OmJXBiIPbrfn5VoBlHAxG/9ggqnbACvH9ddmxKEPGBvHCmuAObPltxpWSciKrfJy1hKQ2DpYyTHWAFZFxYnD29auZ/9MF0tQ4wmmcGpQaCReg8x7MxOikk9Gq1AfV4AdELQEcrE8rdWgGpxijXWZOTP6hL07fudgfJRI2VfOXPm99RonQqqv10fAGlB8Bn9rFkjH3JdgY24KCgM8HSFtd/fsT10mP0ISh132RhZ9x8lhPiCGFNeVBHbMWy8lTyOVhk75JKkkjoaAOwZuMIVGLdQ8PivKyIVykR0ytqxFrhmdWkqAk9173m1Mshz/mooaHshiDXtTcPdzm42lWH+1asQ8kWGHsHmm5bTmeJuV/oj4IXfDnlF6x1b1+1MdCIT9q8s+iYwaW15sck8xBEKITzMxwFFPq2cYQYt/P3IeKiHm89tD9VdM6x0JsDwtnBFWzN6Otn/uTPYM5feyxvEM1sT1Sl+nZiYpq8aO8qZmZrqSzEp1lOnQZxI/S ObeMgfhU TEkznbPGvsddZMbrfpTdox2h/+tlt/YBcK3ZJB4i80VWqFtQ9LsnjQ8+AgR+mVfORgHtXMA4GlOxRDPNmZeXck3Q0nKVVnOHd9P+x6ib4DVz47YFQNMgNjHWZeFn1ZpXKqjL70n1RZ9D/hDPQk0b1Jr5U17D6ICGKwTaHlQWJCGvua9eIIA7M+1bw54nZLHzGxoAk2HL61HXpUYF8liiT5Nj8bRU45S/+xZ06VQDONsC9z4aTW3J+4UwDk1ZNC97Y6bIZ55EUw6AZ5MlpzAs+Nd9QUdZ5wXTYx804GgBQ9hd6uObs5V+Vb85Ew/Kg3wWumvVkTXaDI/QEh10jBl5b8WEXvNK3xJAiuVv/5tcNSVL7OAlVFCP3cvBGXNTStKDrFARB5OD+t2Iw9SiHbCcsnQ1jCMwfFNCcoXhLaP+X8wQaFk7r/OrGtPtac3Y1tqZBpvHYq7gEFf0pXTTQ3W98r4XAL0MzAuRmKjSy3Pa1BHkLMoz4JrCh18+xpV6twka3yLpEnwuOa8vClbCKf/2BVdNjt2/cWnL2tj8ePU98+b6gMiq7TmfTGBirLF5+AL/mIJeUVCFsMPT052tGrQeK5t2HBuIX9Cz27zTEkaxGnfnyVMVNzFSH9ufVHOHF4P/VwzNo X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song No feature change, simply move the routine to a standalone function to be re-used later. The error path handling is copied from the "out_page" label, to make the code change minimized for easier reviewing. Signed-off-by: Kairui Song --- mm/memory.c | 32 ++++---------------------------- mm/swap.h | 8 ++++++++ mm/swap_state.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 59 insertions(+), 28 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index a0a50d3754f0..0165c8cad489 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3803,7 +3803,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) swp_entry_t entry; pte_t pte; vm_fault_t ret = 0; - void *shadow = NULL; if (!pte_unmap_same(vmf)) goto out; @@ -3867,33 +3866,10 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) if (!folio) { if (data_race(si->flags & SWP_SYNCHRONOUS_IO) && __swap_count(entry) == 1) { - /* skip swapcache */ - folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, - vma, vmf->address, false); - page = &folio->page; - if (folio) { - __folio_set_locked(folio); - __folio_set_swapbacked(folio); - - if (mem_cgroup_swapin_charge_folio(folio, - vma->vm_mm, GFP_KERNEL, - entry)) { - ret = VM_FAULT_OOM; - goto out_page; - } - mem_cgroup_swapin_uncharge_swap(entry); - - shadow = get_shadow_from_swap_cache(entry); - if (shadow) - workingset_refault(folio, shadow); - - folio_add_lru(folio); - - /* To provide entry to swap_read_folio() */ - folio->swap = entry; - swap_read_folio(folio, true, NULL); - folio->private = NULL; - } + /* skip swapcache and readahead */ + folio = swapin_direct(entry, GFP_HIGHUSER_MOVABLE, vmf); + if (folio) + page = &folio->page; } else { page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf); diff --git a/mm/swap.h b/mm/swap.h index 758c46ca671e..83eab7b67e77 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -56,6 +56,8 @@ struct folio *swap_cluster_readahead(swp_entry_t entry, gfp_t flag, struct mempolicy *mpol, pgoff_t ilx); struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, struct vm_fault *vmf); +struct folio *swapin_direct(swp_entry_t entry, gfp_t flag, + struct vm_fault *vmf); static inline unsigned int folio_swap_flags(struct folio *folio) { @@ -86,6 +88,12 @@ static inline struct folio *swap_cluster_readahead(swp_entry_t entry, return NULL; } +struct folio *swapin_direct(swp_entry_t entry, gfp_t flag, + struct vm_fault *vmf) +{ + return NULL; +} + static inline struct page *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask, struct vm_fault *vmf) { diff --git a/mm/swap_state.c b/mm/swap_state.c index e671266ad772..24cb93ed5081 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -861,6 +861,53 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, return folio; } +/** + * swapin_direct - swap in folios skipping swap cache and readahead + * @entry: swap entry of this memory + * @gfp_mask: memory allocation flags + * @vmf: fault information + * + * Returns the struct folio for entry and addr after the swap entry is read + * in. + */ +struct folio *swapin_direct(swp_entry_t entry, gfp_t gfp_mask, + struct vm_fault *vmf) +{ + struct vm_area_struct *vma = vmf->vma; + struct folio *folio; + void *shadow = NULL; + + /* skip swapcache */ + folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, + vma, vmf->address, false); + if (folio) { + __folio_set_locked(folio); + __folio_set_swapbacked(folio); + + if (mem_cgroup_swapin_charge_folio(folio, + vma->vm_mm, GFP_KERNEL, + entry)) { + folio_unlock(folio); + folio_put(folio); + return NULL; + } + mem_cgroup_swapin_uncharge_swap(entry); + + shadow = get_shadow_from_swap_cache(entry); + if (shadow) + workingset_refault(folio, shadow); + + folio_add_lru(folio); + + /* To provide entry to swap_read_folio() */ + folio->swap = entry; + swap_read_folio(folio, true, NULL); + folio->private = NULL; + } + + return folio; +} + /** * swapin_readahead - swap in pages in hope we need them soon * @entry: swap entry of this memory