From patchwork Mon Jan 29 17:54:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13536155 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1FE6C47DB3 for ; Mon, 29 Jan 2024 17:55:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3C1376B0078; Mon, 29 Jan 2024 12:55:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3710B6B007B; Mon, 29 Jan 2024 12:55:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1EBB86B007E; Mon, 29 Jan 2024 12:55:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 0C3866B0078 for ; Mon, 29 Jan 2024 12:55:14 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 8C320A158C for ; Mon, 29 Jan 2024 17:55:13 +0000 (UTC) X-FDA: 81733100106.07.0D1443B Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by imf20.hostedemail.com (Postfix) with ESMTP id 7AE871C0021 for ; Mon, 29 Jan 2024 17:55:11 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Do3TBkkK; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf20.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706550911; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7+Pb9pW7pRGXoZWLwVdMMxRSRMr0QxM87xvWg4BF/x0=; b=auP7DJFkQAJ0oJSQ6gstcBSCb3ZuhUElkJP96W0faIZfvzFzregs4CBbxpL3kpr1IlcZEZ vjteEZW/sjLZQcQ1mYY71Y3uyUeprYJzk0/aLjOXeLl9DD1uiMQD7w3sbJ+Z9NuTbuRnwu an+GNwB/CsSS6Fqkp10imJQpAD5GbEQ= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Do3TBkkK; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf20.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706550911; a=rsa-sha256; cv=none; b=BBp+umVyClSh5t3ARrfigT82heDWi9ELn7TEO/ySkNacFbWWsHkDpYRP1+kXEL+2ZV+ryf aAUSufUpy9+tGjYkcaHolpgF5/z4WUerbHqLFcmHCJcmSrMYIwpUyty1rWs0ZaM4yyKEb8 ImyqfwUL8Ck/NempL6M9v5BxvqxS1bA= Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-6daa89a6452so1434111b3a.2 for ; Mon, 29 Jan 2024 09:55:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706550910; x=1707155710; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=7+Pb9pW7pRGXoZWLwVdMMxRSRMr0QxM87xvWg4BF/x0=; b=Do3TBkkKlfyu5zdamd7z7KeMYsU5tqoRjcRI2+qwbpBY5GMoBIPAc3/T00WM/Idtny gcjCM6gp16Qr026NsYqQclG1a8kajIZny0VwlKO0sOwmGZ4pu6dcrRFvYROZZPI+rQlZ 0cDLpFbHWsizcdJbmLLQ3dAa+FfCEULUGkXaoMHE5U9WTnfZ1Y0v7kFPbxODdoZXmfCF cvEE6RcGTuLM7rJWm37iC1/HXvUchO509XBrsq8iUvNZEnabbNvP8lkc62nfWut4zVA+ I2xlDMfSh51yVfGaNEtXo2kjOf+QEQOuoMf03ExYsVQwtoRGww/ze+pEpwaci2ChStej TmrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706550910; x=1707155710; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=7+Pb9pW7pRGXoZWLwVdMMxRSRMr0QxM87xvWg4BF/x0=; b=FKDTxi0pCHDyEKQWQej9HDUcyWakvpR23M7j/hzA6StnaW0w4/A7tj0bATRToKcTGm mLfeypS/z0asoYRro8X3T51/wojsWYkj0WioxCytYJTy2cM/ibm78XSywU6fu7VF5KQG EXvYV46CC0gzvHQmml/qbpHydQJzUKDdsPIpz2BreK1xtsF6NO0u5M9grEnzcZ4G9rF0 /s+6ahwHP5DPKaPu03qUSZfKNiDzQ/Iux2NsU+FEbowLlfiUr22XyVCwQEx7IQy64p0z 5L/bo5BgWCsPY7vRHvjjAMtN6O3VY0MIG37ykuUUSi0Eka1kgN8Y/wnIA89Nu8rXLtjK uhVg== X-Gm-Message-State: AOJu0YzHSd07GX97pwE0QZEQoBfToy5N7M3Lgk4cm8ZLrYOve7QG2LLY bmSBxwLpi8CdENVr9akkJmNvxrt23Oi8vR29vcw0tPM5aB3fshHW8IQyX/sMPJ+DHQ== X-Google-Smtp-Source: AGHT+IGbEgLtee7lvyw4iD4o6YyT7UJepc/uZ6izKXgvLtPBgmrhSNa0jHfm+bBxa5TucYAtg4TF4g== X-Received: by 2002:a05:6a00:3ccf:b0:6da:cb36:6c00 with SMTP id ln15-20020a056a003ccf00b006dacb366c00mr3552820pfb.15.1706550909761; Mon, 29 Jan 2024 09:55:09 -0800 (PST) Received: from KASONG-MB2.tencent.com ([1.203.117.98]) by smtp.gmail.com with ESMTPSA id h8-20020aa79f48000000b006ddcadb1e2csm6116676pfr.29.2024.01.29.09.55.06 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 29 Jan 2024 09:55:09 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , "Huang, Ying" , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , Yosry Ahmed , David Hildenbrand , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH v3 2/7] mm/swap: move no readahead swapin code to a stand-alone helper Date: Tue, 30 Jan 2024 01:54:17 +0800 Message-ID: <20240129175423.1987-3-ryncsn@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240129175423.1987-1-ryncsn@gmail.com> References: <20240129175423.1987-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: 7AE871C0021 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 3wqen39jwf5maa4o3nsr1xtsx9debzcd X-HE-Tag: 1706550911-262193 X-HE-Meta: U2FsdGVkX18LgwleJNA1MBJAAbWdLKc5p2975RzERkApeqhJCy1LNxuGz6ZBROPYrPoM/wVHMDd18vn9c+cdnaK/Ink5+fKBQxoS5WDX5QB98OdzVWQHpucQfDB9BaMLKbgwYljIqGtmN8JZW9vdqLB48yhTfumoJTUCmjo9bPuA8doikUcyxRaa1Rxad4y4upqWYAB8nH3V33miWMY0l6VmQhO8bLY6y8grdw18paW2bnMr2aKx+ItPiHnNaWS0mxtbRcbx2J48OzimUuS0LT0w1u4qMoicPP8d9uK9cAh3z7wPXERBtTSCjEyHpzPqwEiSXH8iiWr9wWDFgptQdLRHm3gtQzan3g7C9BHntz1eCeOOQNBeffjE+2hyDFiseOet+x83mCDhcccRouQ2QVZ0DgdEo8Uq27KoH7Cqjkj3Ex//tJQrvC/1O8Z6W2lZuTqGLCK9V+xcaYpiGTg2ciCPYQBViTugMdzoS+jOTq00KJ1pOFbprG51gkc2WiZ0EbIBWObElzPGSlKUTLnJ7i7LTmJej/csoB9UGHTXY2DGWWh4FIRa597vXhOh1QnnhYCZTEiYGLz+Sw2hXYX6u+kRm0GWwmgqFabveHM9wIWniDTWorCy8xlVoYJlpZeDjOAKwx3Qw545Ec3wjabq1ZLchkAQ1bTDh+6AKmD/sgQVWAK5KnLhq+o/EsGvrY0BPRDlcDgFnrnliLcemG8NM33hh0Ux4TsG0kRRcmyfr5qyWz6Xv0RyEXu0GzhOOKHfTszv5H0uK74easCHugPxA7Bl/kNmmFWyrrVi25785p1+T8TaT7KP9/+SoecUehPdeMe7+aHt8UHkbWhdBowJB9pzR1wMrySVoTutRCfktk6jc+TgjBAjv9MwCAzVVtWqqQ8fAPAdfBfhg4gpHUXJSx3ar4aGkR0E/G73LnrL78kXXeiXP248ayB2AUzc9H/J17IaoObL31pNVxV8Zlw hJrizgIJ 1M5QhQLSWKCZlW1CzzkFZnq1iZKZY/DfD/HWmI3Pzzscjp4j3KzXmsK1QKzo0NQuT4aYD2SpzUX0RZLPeLVCYCLL3u9GVlv7ETLMKG91e4R0HdLZ72qOxagPABmGVD7xyMAoWKp6uVgf40mfX6CkZrMPBG16O1QcKi6E3mqNxKNs8jv5p5RA+sbHqQyZKX/WLSUsyBo4ojXB75bYXnqB/5P8s8DhjmAfgO26LHEPtLKintiDncj0orC2b/f0cbAUe6fOmJKipAd5nJRACJDmaiC2d5T3aCcAAZTewKiLF5Mbq+cuw77nJIdgN1Mw8PMLW1MC/du9FXwV8d5YwttaUd7Zs7uQ17ML1UtI8A8GBWa/pwP7zWysYw+Kl+OHI7IpYs1D+ff3eAvFa21knE3qdqY25DnC3Vf+EshKDsASNy6FEcV4FriQcZ+y42gPXmItiQ0ks6a2HAM+E3qQMJlFP5g2KMFcqSsjrc6s9eIMHexbAe7IojypbbcjmQKaIDj1e3W6weDQVI91UXFLoIPneFutnh7bSZpyqYgB94k7AA7ai0w9m6un6tPNeL9/KITqbdCB/l2z8VEGFRo2lw1zmns9ipWf17DA6XQ9/7vrsIH6SeIHN6nL43i5MKzmDf413IN90 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song No feature change, simply move the routine to a standalone function to be re-used later. The error path handling is copied from the "out_page" label, to make the code change minimized for easier reviewing. Signed-off-by: Kairui Song --- mm/memory.c | 32 ++++---------------------------- mm/swap.h | 8 ++++++++ mm/swap_state.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 59 insertions(+), 28 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 7e1f4849463a..81dc9d467f4e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3803,7 +3803,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) swp_entry_t entry; pte_t pte; vm_fault_t ret = 0; - void *shadow = NULL; if (!pte_unmap_same(vmf)) goto out; @@ -3867,33 +3866,10 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) if (!folio) { if (data_race(si->flags & SWP_SYNCHRONOUS_IO) && __swap_count(entry) == 1) { - /* skip swapcache */ - folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, - vma, vmf->address, false); - page = &folio->page; - if (folio) { - __folio_set_locked(folio); - __folio_set_swapbacked(folio); - - if (mem_cgroup_swapin_charge_folio(folio, - vma->vm_mm, GFP_KERNEL, - entry)) { - ret = VM_FAULT_OOM; - goto out_page; - } - mem_cgroup_swapin_uncharge_swap(entry); - - shadow = get_shadow_from_swap_cache(entry); - if (shadow) - workingset_refault(folio, shadow); - - folio_add_lru(folio); - - /* To provide entry to swap_read_folio() */ - folio->swap = entry; - swap_read_folio(folio, true, NULL); - folio->private = NULL; - } + /* skip swapcache and readahead */ + folio = swapin_direct(entry, GFP_HIGHUSER_MOVABLE, vmf); + if (folio) + page = &folio->page; } else { page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf); diff --git a/mm/swap.h b/mm/swap.h index 758c46ca671e..83eab7b67e77 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -56,6 +56,8 @@ struct folio *swap_cluster_readahead(swp_entry_t entry, gfp_t flag, struct mempolicy *mpol, pgoff_t ilx); struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, struct vm_fault *vmf); +struct folio *swapin_direct(swp_entry_t entry, gfp_t flag, + struct vm_fault *vmf); static inline unsigned int folio_swap_flags(struct folio *folio) { @@ -86,6 +88,12 @@ static inline struct folio *swap_cluster_readahead(swp_entry_t entry, return NULL; } +struct folio *swapin_direct(swp_entry_t entry, gfp_t flag, + struct vm_fault *vmf) +{ + return NULL; +} + static inline struct page *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask, struct vm_fault *vmf) { diff --git a/mm/swap_state.c b/mm/swap_state.c index e671266ad772..645f5bcad123 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -861,6 +861,53 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, return folio; } +/** + * swapin_direct - swap in a folio skipping swap cache and readahead + * @entry: swap entry of this memory + * @gfp_mask: memory allocation flags + * @vmf: fault information + * + * Returns the struct folio for entry and addr after the swap entry is read + * in. + */ +struct folio *swapin_direct(swp_entry_t entry, gfp_t gfp_mask, + struct vm_fault *vmf) +{ + struct vm_area_struct *vma = vmf->vma; + struct folio *folio; + void *shadow = NULL; + + /* skip swapcache */ + folio = vma_alloc_folio(gfp_mask, 0, + vma, vmf->address, false); + if (folio) { + __folio_set_locked(folio); + __folio_set_swapbacked(folio); + + if (mem_cgroup_swapin_charge_folio(folio, + vma->vm_mm, GFP_KERNEL, + entry)) { + folio_unlock(folio); + folio_put(folio); + return NULL; + } + mem_cgroup_swapin_uncharge_swap(entry); + + shadow = get_shadow_from_swap_cache(entry); + if (shadow) + workingset_refault(folio, shadow); + + folio_add_lru(folio); + + /* To provide entry to swap_read_folio() */ + folio->swap = entry; + swap_read_folio(folio, true, NULL); + folio->private = NULL; + } + + return folio; +} + /** * swapin_readahead - swap in pages in hope we need them soon * @entry: swap entry of this memory