From patchwork Sun Nov 19 19:47:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460657 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCFFCC5AD4C for ; Sun, 19 Nov 2023 19:48:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 57C406B034A; Sun, 19 Nov 2023 14:48:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 52CBE6B0353; Sun, 19 Nov 2023 14:48:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3A6B26B0355; Sun, 19 Nov 2023 14:48:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2042A6B034A for ; Sun, 19 Nov 2023 14:48:09 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id EA505C0857 for ; Sun, 19 Nov 2023 19:48:08 +0000 (UTC) X-FDA: 81475739856.18.F465D79 Received: from mail-il1-f177.google.com (mail-il1-f177.google.com [209.85.166.177]) by imf21.hostedemail.com (Postfix) with ESMTP id 2800F1C0004 for ; Sun, 19 Nov 2023 19:48:06 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=l4y9msSL; spf=pass (imf21.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.166.177 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423287; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tM0GQjuVnFv7RufWXzCKq9li8NCfwru2okUxp/uvL+E=; b=TqFzOMBbqK6wGGtQzLF2U+nVkYjjuFMYRu/b98yvoQYZWZGS6laHf/LBFiATBJlYeLKRHR GzCOIYfs6dVYART+UhfL+WNGR4jbwB8yAPRiaqCaW+GAeGM1Q4Of2pIdp61ubBn2R6ZpbA 5lO2x8napQflanpzXgHiMCI+BIMhnJY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423287; a=rsa-sha256; cv=none; b=LOZJXCW0ES0vNmGnTWcDa3opMw1a2ppjfThrRqdTq+3J0LY/qzYUaNiglwzZDdSGv9SoCh GKtkpJJbFP1YeIW65h+9Oli+iZaMvExAxh6shkuyGHIQGvqqWUOc+Xcs9UzBe0bEe1ou2K p+IAZbpj3UmZiPLozNO7eCxJLV5IV5o= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=l4y9msSL; spf=pass (imf21.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.166.177 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-il1-f177.google.com with SMTP id e9e14a558f8ab-3574f99d236so14083275ab.2 for ; Sun, 19 Nov 2023 11:48:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423285; x=1701028085; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=tM0GQjuVnFv7RufWXzCKq9li8NCfwru2okUxp/uvL+E=; b=l4y9msSLB82T5UtK5v61psg8ykiHtV9zOdeIa9sB7u5d6Q+KzuxArtvg9NP4XfDGhb W/Uj2AX3eD9WEdmbMrAPLLI37vAV35ZSXi3QuDihzSHQjBNvftA/asQnAKsa0F/NVDTa cOxvbk2DI6kkPdqQu5r5vYwquD9mfpOk4Ya1BRWZkTQpy09w+tRt5n3gqAiteGnT6gS/ 2so6bs2a37vmqIcO6yJHIcop1Hu+/6J7+AYOocj2+3w5FqmtoFLLntz91SF7BKZgkpCW pzbN1YWRi7pWV/5q2jGpb5pViaQJ4SeZTUrX+TaGtuJ2hxmIEW5057Yg16GyckK7/yuj HUCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423285; x=1701028085; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=tM0GQjuVnFv7RufWXzCKq9li8NCfwru2okUxp/uvL+E=; b=Exh7Sn5LBfkHse7+fYEVxv3NlkMpw6a90aaDuQ3RgXWLKLM0xht+f34p9rSAh9DWyU aEPrAsSjuu/HJeaHt7MEj72wqwmxdFHMZyG6QuyWFzUgIJGHk4drz+cJb2u4a2pSSXOu fn0TdajmbzAjstLaF2vqHki/hJHcpvyIv4nElBH7rmsqj/ge0VHJCqKyMSfM+qR5wpip uhqUBQCQ1pWZ1OexBVXUVqYxv0lOt+bVzdHTII2cN5EzVAK5efOTYss+2O1+fpTE8hF2 vYgzhFV84/9FWB9Xo3kj3XlMO6ZsxtwJS0Xq7UUMlxO4HYRXmrlxyyqyxBoi1fzo6yyx 9Vcw== X-Gm-Message-State: AOJu0Yzy6wSMZ3E6SbQZu7jtA2I4b0O++d/1UKSk90NZLQ+mtLApTUEN +IW2hBC8WAc61Bla/AEOYabqaQZbKM/I/jD9 X-Google-Smtp-Source: AGHT+IF1UkKRv/ipdxof7nKQAoicjYZ8F2kSET5NL02/aAVKfiSk21ZO3YlPkVn68ZksUna/3gG5lw== X-Received: by 2002:a92:c26d:0:b0:35b:695:c3c8 with SMTP id h13-20020a92c26d000000b0035b0695c3c8mr353889ild.9.1700423285183; Sun, 19 Nov 2023 11:48:05 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.02 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:04 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 01/24] mm/swap: fix a potential undefined behavior issue Date: Mon, 20 Nov 2023 03:47:17 +0800 Message-ID: <20231119194740.94101-2-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Stat-Signature: 4k3k3n3thrj44b7b7ifjeue177d16zx1 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 2800F1C0004 X-Rspam-User: X-HE-Tag: 1700423286-73558 X-HE-Meta: U2FsdGVkX1/37vj/oNBjFG/S/lVCyW5pKFcv46ayZvEQNnzpjv85+gh8fZOck+g02WHyBaszOGQO9JHMqjAlGowOSWWnL8Jfk+gO+YBL3zanryPYb4ATEVdM6UN2ucTg1zadx7q+Q33wAjxKbQg+xlmtNnBhmW+pgQFtDBLySjnlRkL5A+Tk+sVfephrrektpoQQ7JJARfxLsfm1hh2Gm8520SrvZcQpC6Q4hhFkJPOLEEd6CNqZuSz9y4lULcdXSwxIatzBTDSGJle9CuG/Vu1YtjtKf5HPSs6M2W78fAY1Wdzr+Pi62c/da2a6XbwXUhvI1MIpoGHE5Fi8TSYFx5r88vzWdS7lU7PoW9+nnl22tszGe8Ewvum3PSx26odqxlDdyEmolbtxbyYAjwylT3HPFbkE+7zwQWouhSXfF+rKOsduy5wsN9mQwNYCRWy3SJKGhlZRqTrk8E7Tt+6sg7fnFFMR65Y9o9ifu4sjboIK1sSxbt4J1VzbMg0UCOXD4xCyZG9YIxfJHLAUYv+PXeaaiqWAeVtct9ZBSkT75T0t6tUlFsJG5nEuLuS9xi4O0zhOPHE6jzrg0hx7T3VpuMPjoTzT+rqM7Utx9t6nHw5m50iJkrCWi2GgHyzjI0+ffRDuRipptD7YDbTVI6IHvEtt8zwxKatmSxMIBmt48sL3GRrjqTKuB8BTO6cEyZ6dI8/eYXR4hIYQZgnIjoL8z3PQGbG7ghnzQJhPJAiM/3UNLeIwwY8V35kcncYFgm+x/rTnV2FDY/i+r1p6CxcBWKWteeCKEHUG0oq+uyUq1mHjv3JFAozXWX1PDDIkm7HoeFMIqiP19ITGtFeHiZ9L9qQPhCuW0p7KvNWG5ne7i2Ce3Jc0ymQHVXJNxB/ftVU7o+Mt3ei0ov4Xb0jvX+Wy6l8a/RWxXr5o5J8Vry20n1GLqpgl5KV5PejN9RoFJ4Lp7k9pPWW6L/ZfrHW5h1K fhXZh2YS PyxlAguj0SOqJls6Vmyln24Vm44dmKU2eKuuCwwy3SUb8551m/fOC9SMhQGMgTpuTiWrfBEsf7N7/7GkAlFKp3SmvbD1XBkMoMSFSD9yfBqYatouQoz08KOUuXyUzi3ccGA5tIdMRYfHPWsHuf3ld6biSPjWdQdt8TiGKovHdP/bdtLnHm1dcYqLOLL3oc72CqO5jYhOcsfoujPZvXIvr++bHXfXszs+U4AToEhqjPGAZ20w45eAYGc+X8+nNpgBkAEFzoFgwPORgZwI3NpXfOJZFTISMGhszOZFjlOUSbthgA7amWv+xl2hbheqsJmta3JvhEuMjlE3kaxy8r/8Y7LTwywyi3DUzEvmqUmq6JLf7Se0whvnCX+CXBZ8dyl3zlT/ykU/NTO70znjrCLeKNawtzj2iZ6O1I6nCl1aHgPLTdaiZADFn5gr76zo6trE1B3IgYGVWZ710WBniwFYLD8bytwpZGSPsuGYeV2Yoyopv5kAdYvyKLR1Zwur3DQZ3jb0j1gWtZ2Q9H/A2x7gP/1uSYdxrO9My1P1B2j7f8oxM8KEErRPow7E5rxOBW6AljrFzFUoHr3+h6sVijBTOHAswUHzotXE3BtupPELN2iBTZmQvaiUj+d0PWY4M2SwNIivh X-Bogosity: Ham, tests=bogofilter, spamicity=0.000012, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song When folio is NULL, taking the address of its struct member is an undefined behavior, the UB is caused by applying -> operator to a pointer not pointing to any object. Although in practice this won't lead to a real issue, still better to fix it, also makes the code less error-prone, when folio is NULL, page is also NULL, instead of a meanless offset value. Signed-off-by: Kairui Song --- mm/memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index e27e2e5beb3f..70ffa867b1be 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3861,7 +3861,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) /* skip swapcache */ folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vmf->address, false); - page = &folio->page; if (folio) { __folio_set_locked(folio); __folio_set_swapbacked(folio); @@ -3879,6 +3878,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) workingset_refault(folio, shadow); folio_add_lru(folio); + page = &folio->page; /* To provide entry to swap_readpage() */ folio->swap = entry; From patchwork Sun Nov 19 19:47:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460658 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 553A9C54FB9 for ; Sun, 19 Nov 2023 19:48:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D465A6B0353; Sun, 19 Nov 2023 14:48:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CF4776B0358; Sun, 19 Nov 2023 14:48:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B6FBA6B0359; Sun, 19 Nov 2023 14:48:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A0CCC6B0353 for ; Sun, 19 Nov 2023 14:48:12 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 5162B1202E1 for ; Sun, 19 Nov 2023 19:48:12 +0000 (UTC) X-FDA: 81475740024.25.1795918 Received: from mail-yw1-f176.google.com (mail-yw1-f176.google.com [209.85.128.176]) by imf14.hostedemail.com (Postfix) with ESMTP id 70C3010000C for ; Sun, 19 Nov 2023 19:48:10 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=iuGEW+1O; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf14.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.128.176 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423290; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cduqWiCMIKVDQKzalcbjrj5u/UZrGRNgI9eE9WPHu3A=; b=7nbf1htpF+i4CTLyQy05VNE2rHWbzm0ylSDJScKZrR/hz+qr7OTIQdUE3Ct7fJv4+uCd9l 9oOs9WCORTtWgUYhHNw1vgIJmGEki6mRzY+RP1hji2/RyuM8Zteclghwuiw6htDv0QXHNx 6JRMQPZNP3m3dZJSuZKd3pwhysxRu4k= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=iuGEW+1O; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf14.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.128.176 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423290; a=rsa-sha256; cv=none; b=ZyuGj6AiNk4FOGTRxQD7/agmnN9EoG4SaSXfoGNSPRBk7siunMK9UB7Tv64ybYQoBQ8XUo 3D4d31d+gtKIwd0xKDLkJpe2WnHmAB7yqyp9cWxPNZyElnzxRMU+/R18eOb+UMs+xZz8o9 y/IOvaZUlYH2kZvs6FU9S84BUL3faiE= Received: by mail-yw1-f176.google.com with SMTP id 00721157ae682-5a7dd65052aso41904847b3.0 for ; Sun, 19 Nov 2023 11:48:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423288; x=1701028088; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=cduqWiCMIKVDQKzalcbjrj5u/UZrGRNgI9eE9WPHu3A=; b=iuGEW+1OekJhCObuF4gAj5DOs/WBs4FELHna2G3aF4FD5FDSu9dTkKoi8o0OCzbfVX 6pLeUAWKCLg+HzJKwxw6US1B7qDydoUWxFMn3vYnbvK5Ev7PUNpNmttvWJOwaRvwj1w0 YF2thl8588FqBXQ9d8fwR4pHkmUfEtYE+tWhuZOJbAmjF8ANyoaBtdySCPemfrT8sclQ lYgNKat7YjA5Ncb3STLsqFN38uJ+u1tG+lBq8sqCCwObpHhndJbZkKqGIDKGQm7Bs/Hj aOcYvyhrR0fATOiv6icc2uSjWvTI3imsoMI8BE86y4YlhNlbsl2j0CjLvtUdqwwl7Bkm lCDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423288; x=1701028088; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=cduqWiCMIKVDQKzalcbjrj5u/UZrGRNgI9eE9WPHu3A=; b=YeWr1II1kiWndwaOEdhB/Yfp4D+asvP7ha6vkj81Grn+GZ1tqGOyme4ZIjTCHNowJ0 HZ8/gpFwYeCVvESaVTTe+o4QrdzKuIMIMBbwxws6Xxc60KNgrDMSH58WssRkLRVN8nH6 m50g88rHxy+gxRsqbUAbEdZb7UOUpA7TtQjFxe0ag4eFP8S74y3yAiOlPxl32DAHE/qz KJ3fcFi4NO+EdTHH7qIJkX9inYFUF1f8bnJhsYLqqh8zyjTKntmxI7neBfxNAnXJBgJE lYc7UHljZzD1uST74bzgdvWu2jjsFRPoHxTscE9Xk8pmfmU2rucV9F8Tk6ytXS7WsbhZ UJ3w== X-Gm-Message-State: AOJu0Yykj+cmG1Bay41BPZTl3qHwhyclR8vmaIJAS6Ptqx6I1ze0QG5i FYmUL7ZSE+ph0d7pPaPdytfiMMRh1sjPT6Bd X-Google-Smtp-Source: AGHT+IE1y2y3wIZaEr+VeLd45/7RB5OVC0un5QZpdb1Dva1zs1cgZibYuz6x5YXuFZ86VBTCpW+iXw== X-Received: by 2002:a81:4841:0:b0:5be:94a6:d919 with SMTP id v62-20020a814841000000b005be94a6d919mr6198288ywa.25.1700423288412; Sun, 19 Nov 2023 11:48:08 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.05 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:07 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 02/24] mm/swapfile.c: add back some comment Date: Mon, 20 Nov 2023 03:47:18 +0800 Message-ID: <20231119194740.94101-3-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: 70C3010000C X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: arunpync7birhsrqwrs761uocqpmzmia X-HE-Tag: 1700423290-561276 X-HE-Meta: U2FsdGVkX18CF3UvzckayHnJkUa0ywVMAYaGRSjmcT1pnTYuzGY8v/UAjvfysN6TMPCAE/AY59n6jJRWueMwp/ywAj3dSrs2VihD1LrLi8AOgWqMkyaWXpXdY3F4Nv2Bbwld53Spcw3BXGu2IoKDvd67JIsUfpLj9+yIQHNCDwa0K6y5YUCNNC1w81p3q28tXOqcIF3Ultx7tGlTGIf8xUSorvwxt7p3Tkw8dNmK37zJ1BwFs8OlXZrtNX+LkHKf8R5yUkB5t/FpapvI5+lNjl6s5Pi/bcqx3ZYWJoCsdoL3q778BT9xDovICaDkDxSBJLEib90et6nSQdF6QPm+oWhbRyg0MtzByDUgdNWFocMw9d+zpUdSRyZAx2FJHHeE+Xfbuz9czWr7dRmQZjnO96hA5eNsLWcEhCzyfslxFgqDI0sYEjJNGmKRYQt3H5QlSf9QV3ybUn4ZTsMSOPCtYmGUAfSixVnGvJ211EnSlzwk5W6AowQyONDbTcWF8PVxhNln8d2smaiu2/wcLEPQ4iXp30qx6GdhS2+Gjbk2akkU6ITcAO5z7uFBW/1Kpu9e4XTAjrAjcelLopwHC2YPx9gIwbFlcfR4N+VyoeK+eohLZ4mjme6APBeLrFkKjTpaZTGXSHtSNLRCBWhxmx6QA3MjwTQcaJt6wREY+0EL4ipjQyC/4hu1Q5jWXuFfud6O4ghxbTEFM8uMbFEnd9oSRfkA5KiXHXB6LtRspGVG/b1iep+/RRoOWVzOUVyH8ywBeZ7XM/ylv/f+EmiVB6XdmN+QOgEPcgl4E5aFLk9pC9Bcfa6lB4d1wP82kRidvRqESebcW9ddJc6Afi9h4WYnOHXfvpwvnitoGPRy4k/AWHlae7WLiKZGV32PElQiWmfuoYtOt91OmIgwF3FCG4+XTdIlh0kSfWkKjFt6+Kat31uPfkVHhAehNq4iPYwsJaYGVQ2lX3lus3NX2TcIQXR qCEkh3dP J350hLJBruHUdOpRptcf2vnRHYcNeULtFbnlDxFP+tyhOh+NqJuTFup/1rdewSKEd5hPfLsz0XEXzCBS/+jh/q4sT07glKwGQvr3ueKqkWs+/uYHPJh6ijk0DOOvPuvRRrBdDW8ESJqyGNWJS3sdHhsL0HPsWCQOJd7gFPVhMo5r/kBYKxlXb/OCE3Uqem+0nRjhYYmYATieNntcF27k7UCHKs2gKnTZlpi10ziF2lNhd0c98SCODHR7LSKiKxHNuBq5bXdZg/3mZL5tCw+/5siaY9NOWKE44lhNUv80DjpqRnAXMpsuTHMaJ/f11WGmUV6nQzPNZxwqd+F/eTvZkBSQnLIXemJO1N45G1Nh5otT0wlKtvu4fU012LwYbZ/tW8KE/zVbP6nQvus+L+VJJE0e29QWosAW4/jYNyPu+/T0ebkRV0kLjHNMl0twY53XJjgzL4fyNPjVJ9+K78mwMPizxkzEjHGa+VxzCF/bmVZ3Stq0wgPrJVg4vMbQOyUZNFoF01zrtPbwKk4SM21FdF//O0LOirkaPZxnO2OSqyVXT/LsLsfRhQKzjHa+gDV3jjGdt74KjJbdH+y5yVlkPu9Y3BXrkQMVf9uzVphl4vLc8OZoEDK40ds/193j55EVRz1+P X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Some useful comments were dropped in commit b56a2d8af914 ('mm: rid swapoff of quadratic complexity'), add them back. Signed-off-by: Kairui Song --- mm/swapfile.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/mm/swapfile.c b/mm/swapfile.c index 4bc70f459164..756104ebd585 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1879,6 +1879,17 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, folio = page_folio(page); } if (!folio) { + /* + * The entry could have been freed, and will not + * be reused since swapoff() already disabled + * allocation from here, or alloc_page() failed. + * + * We don't hold lock here, so the swap entry could be + * SWAP_MAP_BAD (when the cluster is discarding). + * Instead of fail out, We can just skip the swap + * entry because swapoff will wait for discarding + * finish anyway. + */ swp_count = READ_ONCE(si->swap_map[offset]); if (swp_count == 0 || swp_count == SWAP_MAP_BAD) continue; From patchwork Sun Nov 19 19:47:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D35A1C5AE5B for ; Sun, 19 Nov 2023 19:48:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 619C76B0359; Sun, 19 Nov 2023 14:48:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5C9306B035A; Sun, 19 Nov 2023 14:48:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41D4C6B035B; Sun, 19 Nov 2023 14:48:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2B09E6B0359 for ; Sun, 19 Nov 2023 14:48:16 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 09074A0285 for ; Sun, 19 Nov 2023 19:48:16 +0000 (UTC) X-FDA: 81475740192.02.B2C5895 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by imf06.hostedemail.com (Postfix) with ESMTP id 27A5418000E for ; Sun, 19 Nov 2023 19:48:13 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=GTJamWqN; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf06.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423294; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NgugyapInVThiGa6OPAw9VMpbPbUYFH6tRNCt397cNY=; b=lzj7lFznIzsAINxp+a07qo9thtJGDQ1QQpVdTJLM3YOYVUai0nd/homGtp9TUSPfGIhkad GsmvT2n3Jrln4f8sCuX1ASCU5efJwI7puZHSeU6kY4hsTJzzbzo4tl8DWGhAXEwDJgqVF3 YNaLEYbo+f4phy4WUqqc+hV806JgSWM= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=GTJamWqN; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf06.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423294; a=rsa-sha256; cv=none; b=hMl8oyv9YVdsLhoVrO0VMtCdFEA+idvxX/uDasyHLMW+5d3KJM26t+ktOBJK67ByPEAnT/ Iq7bvKIx2gj42a1cvpEmxoSGltiiHkqh58MgjLXSkd27cDCaJ2AepFwQCVv8BbJxkY4EFZ fOzXSFE5jMYfCPjhEEoJZzF1Y2uB5X4= Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-6c4eb5fda3cso3858592b3a.2 for ; Sun, 19 Nov 2023 11:48:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423292; x=1701028092; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=NgugyapInVThiGa6OPAw9VMpbPbUYFH6tRNCt397cNY=; b=GTJamWqNUrvC2o3Ho8JT7uShdEFicHjh0r3Rsun75h1WNtEmpWyiyd7w+lDOisxb8A gXmeCS/NNXK57PAbI+eq+hP9rj6LCL1ZSrX+WGHtqZ1KwdJC/hjqFx676ryv29u4kPfk RGlMagQwzAZloJZreicE45Ou5Mq3YOy1QcsJqdL+bOGZ7JFceGbzhb/FG2R9OBZUgM/B yE99KDFPPplaXTAyr/WWBYCu9TDHj0GywY8G/0ACkp1SEBaH5KIK4fugR964/uMuZPyq lz1RGpE4iWGzxnawC9NkzYRbnzTZEWfUPgJTEBKfO99Yw/q1hS1gOX+ZbyX0m634MSdm rqFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423292; x=1701028092; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=NgugyapInVThiGa6OPAw9VMpbPbUYFH6tRNCt397cNY=; b=VZtP1679tH3s1RpqqV1tbzCAxyniCxWnZBWnZNPk7sjE5zUTruZVh2oj6RmXowROtg OI2p3Qy+pSGrEgFIaY30Gqo1pPYo/tjFLz7dIUopiQXwMA2ufdjJkQlgh87rpD5tSfYc 0nNnPYL0Dx2XomfTAMXlwDkn/ga3agUcwXenNxcJ5V9npQ62uomybVvXIWELMy7TPmsE azI7IkUjCjJeA1h1MGCiQIZynkhZcpKek9hsAlKbswr+HMLkoXtwX3AHW9tx7g/qbLg6 jgNN/IuKrH4khqlXXvD2n0+LGannpJiHijSScZ9b0/I1D2G0Xuv9s6Zbkkyv/FUfv/Zh H7ZA== X-Gm-Message-State: AOJu0Yz3RaZ1BEbKKPnSrpc8VPRhg3PZDKKlMvy1PGShPRdmI76q+s9Z 3zsJLyCY4/34umbUCKe0/C+JORiIETgNERSg X-Google-Smtp-Source: AGHT+IEheipTHeYRZdBVyN0mV2ARTS7XP+zRN1OT8UJpXXSOat4LuPJfFp26youNgVdlUjeMC3A1hA== X-Received: by 2002:a05:6a20:e104:b0:18a:181b:146b with SMTP id kr4-20020a056a20e10400b0018a181b146bmr5052162pzb.29.1700423291675; Sun, 19 Nov 2023 11:48:11 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.08 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:11 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 03/24] mm/swap: move no readahead swapin code to a stand alone helper Date: Mon, 20 Nov 2023 03:47:19 +0800 Message-ID: <20231119194740.94101-4-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 27A5418000E X-Stat-Signature: a8uoc37i7mbmrztittgbc9bfjrcku4ep X-HE-Tag: 1700423293-398884 X-HE-Meta: U2FsdGVkX197q0RKbNIPZUlF9QxDSRHwtXtVdWn39LoB+ZZhOVyWgjxBmQv3kP2cnAUmpZD9YMVXppunrYBx8CaCr8LNA28yo8M51/ASdIqzFK5zKpS9JimBJm2eCIt6XC3J1TbHcBbhFXhrfvUCM1IRlBvV2fCyXj+wGGqQjD4gA4gb6076hl0c/gFLGZ9OREyx64q+ocbYZc3ztvokO5ataJaaW/FI+Xn8kfDmIkeDVjrjgq1kr0hi5AJ+y5f9yKQxkMCnYTh9hJV3w+b4TfeaufrOOy/AxJFDI1i6sKemfF3SMlCrlgYFvYVHoxyXfFqrgcwI/bLddGCe8jmHeze0r9cjtV5OA6XQJHpR8hTOu5nxnieCEshn6FpegyAGjgrSeKgLTdrPmK6oxpboRCabHApv5vYIAz+c9ZbD3a817SNFQDrIu/y5SbTFZ71KIQ4GnMrAk2gDN06x4/KvMDXmOMVdAO9xaqk0f8vDvSJ+rzq/niPzOH1t/Ifq/goSr80XjSAkdNeSKY6pJAD0Srq/9RRItMqcMiISzw552ujmI18fTSNX/BIlFn2nvZSHoQoKrvcnbg2U5vLTh1kBDwK3IHu4AdoFnOQ2K+h5h5s/+TMB/42knl/ZNsmZ8NmEUlIeEB2HEZW9stjpmkp8jvwNIYPKRb1oVIn3ct3XuyqPTnZmV5YvgpIcAyRpN8MguwdHZdAuAma/Xc8qhFtbLx0KBeIgWdwGpuob6gX6TUd6/kxthm5zTAEmhblVEIQMCOnP4vakIGLQJ/KSreXdRu+eC9nWEYpceMiIFoXjSJkUsXeRvgQnFeibuvUA3DtkSv6XuBEmlHQCoDWkKZ9J0se2Iri4HQtKVujhaprsryfBPFBuIhBA+fCmPE8KhB/EKjzJI2W+/j5b6B72R1/WpTJVOFUPV233YrMSNJh8VaP26iWOCkMXwXLuhUej2lDKXod2wGUwE1k7i/KYlbW SqkPcT5b lsk7iJiMR0k8oi8GVFw2boLkAA0OUTM9TbazlenaYXVDMdfGTyleJkqgundlBn81Yd0YtfvrICpqy16SXfu33QYga6u8BpoRlRszKLrLv0rE7DTTQX6J0E7J28CilqWc/B18GC7ymAXjcIb+4xU5srrOiOl1hYRI6vjWXfw2HCvZ4KTmB6PmqvhepchdDiRTfGE4wzK0WamK9GdHpQAJTotKTC4mKTxsAPV3n2QgwfwCHX8HaZ5sOK1jGCkZujtcArtg6xIEXAaR6eIeYDhZMF4hYOyj4U6g4E5I1u85+xPIFkHaaHXN6t9RA+5/aPl3VHYbyw+Ko2QysgddaphFoZCE//AZ+ut6LtBflB9t4isshiyKHNYy0RNUGX8kq/nQ0KD9Qhiut+TYQyZxnPWb9/6mdGAADG0JfF4/eD3MMTefi/sld+NsHYmEPJqXyu+hlQbPjeARduP3cYOB9IhY2Ut7EVZ3v2uhQLnb1pJ7yJjaiacIoekPbTkmTs/xM/3kuYUKiWbFMyffioI78TpM/ykrtBVqewGFpcGqEikA7rPKTHTOox/5vhY3ZS5JQXzEIouchDMYI8cHgEEFaitLhXREmK64HWiQJsxA0DkvNBc9kbLGNcI53T+IQab7XqQfsX2Ov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song No feature change, simply move the routine to a standalone function to be used later. The error path handling is copied from the "out_page" label, to make the code change minimized for easier reviewing. Signed-off-by: Kairui Song --- mm/memory.c | 33 +++++---------------------------- mm/swap.h | 2 ++ mm/swap_state.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 55 insertions(+), 28 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 70ffa867b1be..fba4a5229163 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3794,7 +3794,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) swp_entry_t entry; pte_t pte; vm_fault_t ret = 0; - void *shadow = NULL; if (!pte_unmap_same(vmf)) goto out; @@ -3858,33 +3857,11 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) if (!folio) { if (data_race(si->flags & SWP_SYNCHRONOUS_IO) && __swap_count(entry) == 1) { - /* skip swapcache */ - folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, - vma, vmf->address, false); - if (folio) { - __folio_set_locked(folio); - __folio_set_swapbacked(folio); - - if (mem_cgroup_swapin_charge_folio(folio, - vma->vm_mm, GFP_KERNEL, - entry)) { - ret = VM_FAULT_OOM; - goto out_page; - } - mem_cgroup_swapin_uncharge_swap(entry); - - shadow = get_shadow_from_swap_cache(entry); - if (shadow) - workingset_refault(folio, shadow); - - folio_add_lru(folio); - page = &folio->page; - - /* To provide entry to swap_readpage() */ - folio->swap = entry; - swap_readpage(page, true, NULL); - folio->private = NULL; - } + /* skip swapcache and readahead */ + page = swapin_no_readahead(entry, GFP_HIGHUSER_MOVABLE, + vmf); + if (page) + folio = page_folio(page); } else { page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf); diff --git a/mm/swap.h b/mm/swap.h index 73c332ee4d91..ea4be4791394 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -56,6 +56,8 @@ struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t flag, struct mempolicy *mpol, pgoff_t ilx); struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, struct vm_fault *vmf); +struct page *swapin_no_readahead(swp_entry_t entry, gfp_t flag, + struct vm_fault *vmf); static inline unsigned int folio_swap_flags(struct folio *folio) { diff --git a/mm/swap_state.c b/mm/swap_state.c index 85d9e5806a6a..ac4fa404eaa7 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -853,6 +853,54 @@ static struct page *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, return page; } +/** + * swapin_no_readahead - swap in pages skipping swap cache and readahead + * @entry: swap entry of this memory + * @gfp_mask: memory allocation flags + * @vmf: fault information + * + * Returns the struct page for entry and addr after the swap entry is read + * in. + */ +struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, + struct vm_fault *vmf) +{ + struct vm_area_struct *vma = vmf->vma; + struct page *page = NULL; + struct folio *folio; + void *shadow = NULL; + + folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, + vma, vmf->address, false); + if (folio) { + __folio_set_locked(folio); + __folio_set_swapbacked(folio); + + if (mem_cgroup_swapin_charge_folio(folio, + vma->vm_mm, GFP_KERNEL, + entry)) { + folio_unlock(folio); + folio_put(folio); + return NULL; + } + mem_cgroup_swapin_uncharge_swap(entry); + + shadow = get_shadow_from_swap_cache(entry); + if (shadow) + workingset_refault(folio, shadow); + + folio_add_lru(folio); + + /* To provide entry to swap_readpage() */ + folio->swap = entry; + page = &folio->page; + swap_readpage(page, true, NULL); + folio->private = NULL; + } + + return page; +} + /** * swapin_readahead - swap in pages in hope we need them soon * @entry: swap entry of this memory From patchwork Sun Nov 19 19:47:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460660 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63633C5AE5E for ; Sun, 19 Nov 2023 19:48:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE4A86B035A; Sun, 19 Nov 2023 14:48:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D91376B0366; Sun, 19 Nov 2023 14:48:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE4D66B0368; Sun, 19 Nov 2023 14:48:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A97F96B035A for ; Sun, 19 Nov 2023 14:48:18 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 7D74C140331 for ; Sun, 19 Nov 2023 19:48:18 +0000 (UTC) X-FDA: 81475740276.20.75233C8 Received: from mail-oo1-f43.google.com (mail-oo1-f43.google.com [209.85.161.43]) by imf13.hostedemail.com (Postfix) with ESMTP id B43D320009 for ; Sun, 19 Nov 2023 19:48:16 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=gtprQfLK; spf=pass (imf13.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.161.43 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423296; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xjzoOYVfaQZliXw2umUqNXWj2wEYGvE1fLa1+gbge5g=; b=SV2JcTKixnN09m5/AqvWPRz/56Hs7USpDMGfOQWXFHgFhqPPKVRHQFCydEOWyhNxxcda67 MboEYHNbUu4XNbzo8C5rgspYjKzjiXFWbJmyhtU5CJtbSFvJpTCkh2NsJMZ5Ac3DhTdLIs 6PEuvbOoiSI1U5dFdPV2pK07yn+1qro= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423296; a=rsa-sha256; cv=none; b=xfKwm17Yz1/yiDGBivoR5yHvFFyxPp/IoqXHko2LDozrYt7NpeHDMNIG6SS8SQ3NXHo40E vVIaA0Dqs8C4nhvGnxoIccuDYYKeFIhhP3qCZicBZYOAozKPLPEZHkeS9o+jT4eGDdoh/l Km/s45vo5d4fTrr7NgWVq9xlIu0OZLU= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=gtprQfLK; spf=pass (imf13.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.161.43 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-oo1-f43.google.com with SMTP id 006d021491bc7-58a737957d3so2507045eaf.0 for ; Sun, 19 Nov 2023 11:48:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423295; x=1701028095; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=xjzoOYVfaQZliXw2umUqNXWj2wEYGvE1fLa1+gbge5g=; b=gtprQfLK3WZaF4GFD9lLJkd6SKRLUqvlC/vHi0XUppJXAirr0Y1bOiFuJBWWc2+2bo IvplgFSREU2YtRM9icpFO2SPG8VU5eV6abQzJd7rHiZ80fMMZCKUSbA4kHu9Qg9PJe6f wzIGcI419pCWdaDDOYt/CY5RKPbFmdiyQmlZ0tAFYAe4Kr5hnBKR3R+jBUbns4C4hQxY TOjR1CycLwaEJpJNQyFO3yHa6u470fthi3wnFUCqiSL6mio442QLO9qozQ1MAQbDlRj+ Tmgm3bUrKmLNMPxa4M3AQpbMme74Rqy8WVH3nLOuX+XOUIjtnoYU73IB+XQcytEw8267 /7oQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423295; x=1701028095; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=xjzoOYVfaQZliXw2umUqNXWj2wEYGvE1fLa1+gbge5g=; b=Skf+7jS8nGRwn3QcprJkO+erSJKPoR9yEVSq1hs83AJlNNPRSe4jT/M6HOJ18oj6Xy B97PTkOgbzRu20NpHqe+wV+ZRUebPQ1vNhSioSdd6ylbNNKgvNttItjtNBfHAHqdN+5i dk47B8lLiF2lXWHY5S2cmZyoMo6EBW34/pxtcpvKtlAMt7oTiqYUTkY7536gLtUCt0ED fyVbeF4N/fCP0Iart7AHssSKlvlO2w8GIoMxbkftdwyTFLDibLPkM/wDQgmFwV1za0LN pBZWaIbLZAuYnbbcqgErPI2+k8w2tSnq9YDtwDk6dGOjacDHFF179QDOy8ryOjHdDPRg IqBA== X-Gm-Message-State: AOJu0YycYyz6fyZOSaETNYY5FF3ZqlqNdc8z7sMwxGQB/9w4c7mWgIUo oDqhlJYdKFhHUhxn7+y84Eiiny7waxycKLco X-Google-Smtp-Source: AGHT+IFQHR1l2ZzyKI4Xa9NdFiIiGKaudZGQta2YkNoMRhm/wHLbj2a5Jpev7P/tpTPzm9ZFNWym6A== X-Received: by 2002:a05:6870:3b85:b0:1f9:36fe:fd0e with SMTP id gi5-20020a0568703b8500b001f936fefd0emr1616531oab.47.1700423294901; Sun, 19 Nov 2023 11:48:14 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.12 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:14 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 04/24] mm/swap: avoid setting page lock bit and doing extra unlock check Date: Mon, 20 Nov 2023 03:47:20 +0800 Message-ID: <20231119194740.94101-5-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Stat-Signature: ugmh8dybc45f4aw94eimef68wnej6965 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: B43D320009 X-Rspam-User: X-HE-Tag: 1700423296-243463 X-HE-Meta: U2FsdGVkX1/+hZKdbUoD6f8w2RbT7bSALdRG1rSiyxBGQYyA3T61npBQjr27jgStuMOPzxnO46hehRJgt9Zx9TBB7cjtM4YA1nesahst504IDMmhWjFZiRJ7aUHnRNa/PQl0/+H3XfqTIigngNQbvVlojpjN+m5SXvqFS7yBORQXChhgTPlyhlpX/tfAlnB8QFPuz9WYchvza0YhkyvYs/LOhCgYP1miLzCCB00+jjxfYMYEuOVNBX1mp++FoK76OlqOF0i4PnvtOZma2cNj6+jBUsmlZ+YjF18lCA70EoLY1BMW8Yd2JjUz6fc2IuWBT41cE3cyqs7nODgv+SiSMRvCNTZcSj4Vk/llhSq8PQGXwnPNX+K/ZE5fjKpWHpPeReBetW58w928PKTzz5LPdKmhliaEhj6daTjw6KUimEuMA4sjOXYZcTZFdzz8eMIhZsEooX4gERGKhjjox78IYDcS8YsWS2Uu62XkrwnpVFyDsLSUH4G4ATC6MWKUYBkqF0P/DwyKK7s6jbiHUPnM/UhDsdDTUPKby0RmGkr0phSJn6pFKdvq9+N0k6npqRs/BzoleesmUd6oldrbKv5zWWmTSBoArP1WVUmLAGNBDPW0NEqCJYMjKZwMzHO02FvAdHGqdhMwiSP4DRWQTgCD6KuwbjR0arHwYuvoxKc12b57yHunYgctAMUbjcU53eO9lHybhQU7Pm0igBRbJE31uhs0g7jTK6jZb1Ed/HJjBnsKUK64uq36Foc3uOgfaDuYIDeEkWAS3PXRD0Cr5wab4oJiQlTPRZGTeYudigNysPIz66LD6GWcCmKQR15vflKfhfb4JJttFp83v+k1wuv1QcTWnq1+0QcyMOWkgZuYEiyQq4tRrmNXgXc2unmIMoO6kVhM4JSz/v3V8IBrPH4M0U0dw8Pb78udGLJZDGW0sQemda/PuxsSN5KgXhPbD3TSsbe8zo1cjcLmxwIyJc0 zhAYU1R4 qrQLa2nF2iiDl9w1HBlt35HPzHufCLuCZX5VpTSg7dw8oBObqfLDG5esIp3LFrzFgIXQVcKKlEBljj8KrnzVtgNTwHv27lKiCWu/oWRqtBe/FWv2wlfFMLv9w/eh3EnWxXopLVLBns6tYiOCwO5k6/nh0r5lq4j0nC6sdX86UvDkXvvn8c58zWWgMUAP0D1fdTyjPFcy0/StYQoCjHN6gUMDu1mADM4i3Yvezw+wRC1a5NKbIL3bLUwhILzB4O1MYT3T5a4CEXhDHjr7oNyRMjHVzJnpxh/qzvPci0wF1lp0tiOtF9Eoo2XDvA0mo4Em+UG8gBw0jZNSdgKf9ePHbR/BY3Euv4Q0MVsaD1ayas2Evs98aW9WeepqXRtJeg1cDkV2G41N6yuu63GxXK15RrBU9fUfOlOZ3c2nkKh8AkrJLlFWbH9g64HBNA5VhpqXj9d1KjT+OXo1f6YS6LHmnSymPpY6M+daQpk1hVU9ffC/W8gMH8IXap4xj2WmJ4A/FanGcNgNpuNbYcnpLFPyjAdTtNgDXXBtOFscQ08+1FDzgPMYMmZyP3TKaQOnf2dEqowQMuFBTbszz92rxRhzcBM1BTG3E7oHFjc32k9N/T+CwfTbpU9Fr3tTvDg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song When swapping in a page, mem_cgroup_swapin_charge_folio is called for new allocated folio, nothing else is referencing the folio so no need to set the lock bit. This avoided doing unlock check on error path. Signed-off-by: Kairui Song --- mm/swap_state.c | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/mm/swap_state.c b/mm/swap_state.c index ac4fa404eaa7..45dd8b7c195d 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -458,6 +458,8 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, mpol, ilx, numa_node_id()); if (!folio) goto fail_put_swap; + if (mem_cgroup_swapin_charge_folio(folio, NULL, gfp_mask, entry)) + goto fail_put_folio; /* * Swap entry may have been freed since our caller observed it. @@ -483,13 +485,9 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, /* * The swap entry is ours to swap in. Prepare the new page. */ - __folio_set_locked(folio); __folio_set_swapbacked(folio); - if (mem_cgroup_swapin_charge_folio(folio, NULL, gfp_mask, entry)) - goto fail_unlock; - /* May fail (-ENOMEM) if XArray node allocation failed. */ if (add_to_swap_cache(folio, entry, gfp_mask & GFP_RECLAIM_MASK, &shadow)) goto fail_unlock; @@ -510,6 +508,7 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, fail_unlock: put_swap_folio(folio, entry); folio_unlock(folio); +fail_put_folio: folio_put(folio); fail_put_swap: put_swap_device(si); @@ -873,16 +872,15 @@ struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vmf->address, false); if (folio) { - __folio_set_locked(folio); - __folio_set_swapbacked(folio); - - if (mem_cgroup_swapin_charge_folio(folio, - vma->vm_mm, GFP_KERNEL, - entry)) { - folio_unlock(folio); + if (mem_cgroup_swapin_charge_folio(folio, vma->vm_mm, + GFP_KERNEL, entry)) { folio_put(folio); return NULL; } + + __folio_set_locked(folio); + __folio_set_swapbacked(folio); + mem_cgroup_swapin_uncharge_swap(entry); shadow = get_shadow_from_swap_cache(entry); From patchwork Sun Nov 19 19:47:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460661 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 447EAC54FB9 for ; Sun, 19 Nov 2023 19:48:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C5DDA6B0366; Sun, 19 Nov 2023 14:48:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C0C426B036D; Sun, 19 Nov 2023 14:48:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A871C6B0374; Sun, 19 Nov 2023 14:48:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 94B216B0366 for ; Sun, 19 Nov 2023 14:48:22 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 73CCDC0347 for ; Sun, 19 Nov 2023 19:48:22 +0000 (UTC) X-FDA: 81475740444.24.066FEA7 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by imf15.hostedemail.com (Postfix) with ESMTP id 9AF2DA0002 for ; Sun, 19 Nov 2023 19:48:20 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=hzU2Y6NT; spf=pass (imf15.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423300; a=rsa-sha256; cv=none; b=Qe3Osz5UjRL0ViNB/aYov/hfw/vOPL7AaWWxLOR/9YJ5bWmKzvzsg/qZab2MbHhLwKfxGG VJX6r3Y2nSHfSC7GPBOvoerCRWs+BgLvI/xK4MLw9ZOo/ifN3c+3Lcuf4e+unaoYfdVYBt FinQWX6zleDNS5LBCelnZLcDrUdWX/4= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=hzU2Y6NT; spf=pass (imf15.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423300; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZPY5phyUcHn1HYB2k83NkwocYwAsatgYc16U9qxXR/Y=; b=FXPg60QIMwYrxJtRinHc7CsqxoO5bMYZElH3KcPP4YmU+Kvj4KrQIKg0k1jtqQw4pzj38j 5D7u5vpi45g6juiNpoMqBl2d4jxEpbAivjZtvnQ3hXkB3W/SbGnSniXEKKyQ56oD6+XpnP EAF6kwhpWp8aoh4L+8Nst28fQF91iuY= Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-6c10f098a27so2928167b3a.2 for ; Sun, 19 Nov 2023 11:48:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423298; x=1701028098; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=ZPY5phyUcHn1HYB2k83NkwocYwAsatgYc16U9qxXR/Y=; b=hzU2Y6NTayF2T1ZjUgzwmpQpGASxGFoeT0zHfktF919qNotItnprf//LGdl3hE4MQ3 Z0oJJAVLmxeTwyLrynz5R7neyT+8LVV5t40AKWS2BVMdcPLbxZqO+M+sQ+gt8gW+F4Ly 42usKxpHxL2dBDGOzT2FwkqPru0gnr3KGfIaXNbyNvoDdIixgcJmVfbYeT7BTwxXSqsT yN/1NONwmwj/4bwuU/nme0XC0rK+KudZ18eqGKCOTyn/az2S/AtkTEa5juiCc25n4Z9v PlIVD0wpdBIQVfrl+790THloLiQbvYtzqwMx4OpwaIiSl/4fIC+sfN4wj0RXhBtKa3w0 pujw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423298; x=1701028098; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ZPY5phyUcHn1HYB2k83NkwocYwAsatgYc16U9qxXR/Y=; b=ue8BN4XxQZonLG0DbRMGCIeMqZbVe/xJh+U5Hy389ZfzYTWnexbL0k2VefTzSsrX/I DlM+0P8bPj4N3ougcqKHbGAtiGCX7py5+EDFl92nCC/wsl52j3opy4tQvYvQ7pa6ZiZh 9dhilNS9/OwwUxsuq3pnjZxcmtLQPnSXnylr59sHejRfgbe+iZEKoSDt4TVHjGAGmR1J hiXotSX80DJ/VtjLcXnKYErtbNOz37iyvR3kOdOZ7oftAxj1WImccyiOEWOY7efmH/LP gu0PkFqLLU7gdcfvqRpOQNH4lvSqX8SlDv109mnCu3l95NU6DImj/SwHfGXWSUZMZiHo RGSA== X-Gm-Message-State: AOJu0YzSR5+s+EfSSC++fbpKL4lMdgtHC4uLueMj67Y5pNXw4OQuLyIV RBXDylNFs1CqsEpCmC4Z3MoSbS5Le708QtPu X-Google-Smtp-Source: AGHT+IFSrhZjCOK5OLW/ySLi0SEQSRf2Ywf78h0lYs0h5fiCtYrCKswNp0hMG0Figvjc/xBHlwsMUw== X-Received: by 2002:aa7:9ddd:0:b0:6c4:dc5b:5b2b with SMTP id g29-20020aa79ddd000000b006c4dc5b5b2bmr3806580pfq.20.1700423298129; Sun, 19 Nov 2023 11:48:18 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.15 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:17 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 05/24] mm/swap: move readahead policy checking into swapin_readahead Date: Mon, 20 Nov 2023 03:47:21 +0800 Message-ID: <20231119194740.94101-6-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 9AF2DA0002 X-Stat-Signature: 1ph8doetpxxrppnedm77zprd9q43eewe X-Rspam-User: X-HE-Tag: 1700423300-547185 X-HE-Meta: U2FsdGVkX19AD2E7zJVZqkfN0frLn6/Xo97YNPqxvuEXdr0eEmZOVbP9UMcfXThQck461oN1KuULBWq01Y8eah1RRK295rduKTWExDjEJKuIY0h7nx52jRRu805TTLLjQ0+3DyX9nc1zQmG8H+Sl1tTeqoscfKamwpTo7gFyRpu2ECFBZ3anJB2WqzgKPXuAtvNOZk/pVyUJ6trG0AXrY3ODEYcfFoLQok3LV8vhfvlXGPTZvkFh5yq6O4YgGI6yt/SorRHoUnD6/5YPQBGZ9P1zPfeMd0Sx5Qc4rKLCQHf7WtCow5euOY5URTr/vtjRzkQPvttYLAwn09wFe0+kL1XxWxwSZN/C5ZLbVl3QjYRLeBVBqgvhhfGKG22jY1XlRPBMHnHwuzBfbIwmDbOffmt3hvva+RMGE/96TED3KPIPv7727ZjM7vGmGJFuhTlwQhScGoKU/27Oeom95JeTsFHXAIuRfBU5c7TmoOTgkWBis4Ya8E8zuvxDetZx/yxgAlmz11Qf6+nKpfqVBWEe/SEkVzaddPfg62jscp126PXPC7m4TFeSk8fj3BGKMIOEu6fmYcXeXBeQByqS8K5VqYQPWFxb5KEqgmDIF/UAKKW4TNEX45yUhD5i7szQIGCpweAoKBkxl40sWp24GmTu/8ipTnMRRDAumgNZU9CWwtw4zo7Ho2KvthUzNZrcOIAdKU1YRexum3tdvALhz8j2LsOiX+TsoMmKZPYP9eI8T3+1b0KKodBbizv2HN+4YxoPjUi9GiNELD4ObPXoYaM1GK1adGELn4AtC5ZI2XumdollkowuFY15gdHJC9dUG8SRzqZWftm8eCEyrTJyRA2lM3dX2ptwQ8pWRbYeedWrAjCp1DdifHEXPG23eP+7M5SoKtNbdAEfW8Kg7Asda7iQIrheGed0Rdb40LC6QiAgUtFhEOkp29kFOtRWdQ32eKA5tiJgSCmLFJOrZ+X3x7V RHz6qDal a2+kWpjJAgvngElkQcdKxV/mP1e0Mtm8v00QdObDcOIGA0ePwc9YclJg7m7x6d0Q7S53hOSTCy/UGwRct++V4n/gm6x3QqqxpEHXSiDJ4kYnUfAkZJmNnHB/sZ2xS2Mszkxr8GMbHwW2azfpa3imQTkFyEL6tRBQT1MjKL+N2jNkiMYgBAhGoyw0PBzYAfsVBU1lLwcnOSLIp3Nr6nLAWVa45sO8TjsMFAzdmzN09D3nSugEL/PCK4hufZRy8m/bEOB8ki5tv1txLUoEnA7cqYvOulenZkt248qOCqGSDqcuCIetQTSXYI5jQnJF6V6T8tvYwNrTeSpFlty3z9nOTEInTDnLLfNO9zHgq2+s+7NKyzv7VSzWWh5pAzHV7BBfJXEokoaTMYUA5Fy7jeaaxJkho66mFhDR7hA67bfVrn1J4YZVZdrOuh1711/kjCbomnGKlma1X3jRGnCxk1g7pA8K+seWxUuGH8DBt0g3semT86Gv0AGiMj4cHQ22IYi6pPYOEC0rBl3yZ4oRH7upy9mxp0xUo43LtkrwcvJ2GjNv7O9VNMzCA2famq5RLg/pLFFJQkCrd4kD/3c1dvWQhf6MWOZ4uu4liincWo9uhOKs63d+LXV6mtMTskRW+H4taEe9Hj8sFnb9ZKpgmdqdHbT/IRTpMGGm1uAyl4JBYNdUSZHo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song This makes swapin_readahead a main entry for swapin pages, prepare for optimizations in later commits. This also makes swapoff able to make use of readahead checking based on entry. Swapping off a 10G ZRAM (lzo-rle) is faster: Before: time swapoff /dev/zram0 real 0m12.337s user 0m0.001s sys 0m12.329s After: time swapoff /dev/zram0 real 0m9.728s user 0m0.001s sys 0m9.719s And what's more, because now swapoff will also make use of no-readahead swapin helper, this also fixed a bug for no-readahead case (eg. ZRAM): when a process that swapped out some memory previously was moved to a new cgroup, and the original cgroup is dead, swapoff the swap device will make the swapped in pages accounted into the process doing the swapoff instead of the new cgroup the process was moved to. This can be easily reproduced by: - Setup a ramdisk (eg. ZRAM) swap. - Create memory cgroup A, B and C. - Spawn process P1 in cgroup A and make it swap out some pages. - Move process P1 to memory cgroup B. - Destroy cgroup A. - Do a swapoff in cgroup C. - Swapped in pages is accounted into cgroup C. This patch will fix it make the swapped in pages accounted in cgroup B. The same bug exists for readahead path too, we'll fix it in later commits. Signed-off-by: Kairui Song --- mm/memory.c | 22 +++++++--------------- mm/swap.h | 6 ++---- mm/swap_state.c | 33 ++++++++++++++++++++++++++------- mm/swapfile.c | 2 +- 4 files changed, 36 insertions(+), 27 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index fba4a5229163..f4237a2e3b93 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3792,6 +3792,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) rmap_t rmap_flags = RMAP_NONE; bool exclusive = false; swp_entry_t entry; + bool swapcached; pte_t pte; vm_fault_t ret = 0; @@ -3855,22 +3856,13 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) swapcache = folio; if (!folio) { - if (data_race(si->flags & SWP_SYNCHRONOUS_IO) && - __swap_count(entry) == 1) { - /* skip swapcache and readahead */ - page = swapin_no_readahead(entry, GFP_HIGHUSER_MOVABLE, - vmf); - if (page) - folio = page_folio(page); + page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, + vmf, &swapcached); + if (page) { + folio = page_folio(page); + if (swapcached) + swapcache = folio; } else { - page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, - vmf); - if (page) - folio = page_folio(page); - swapcache = folio; - } - - if (!folio) { /* * Back out if somebody else faulted in this pte * while we released the pte lock. diff --git a/mm/swap.h b/mm/swap.h index ea4be4791394..f82d43d7b52a 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -55,9 +55,7 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t flag, struct mempolicy *mpol, pgoff_t ilx); struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, - struct vm_fault *vmf); -struct page *swapin_no_readahead(swp_entry_t entry, gfp_t flag, - struct vm_fault *vmf); + struct vm_fault *vmf, bool *swapcached); static inline unsigned int folio_swap_flags(struct folio *folio) { @@ -89,7 +87,7 @@ static inline struct page *swap_cluster_readahead(swp_entry_t entry, } static inline struct page *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask, - struct vm_fault *vmf) + struct vm_fault *vmf, bool *swapcached) { return NULL; } diff --git a/mm/swap_state.c b/mm/swap_state.c index 45dd8b7c195d..fd0047ae324e 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -316,6 +316,11 @@ void free_pages_and_swap_cache(struct encoded_page **pages, int nr) release_pages(pages, nr); } +static inline bool swap_use_no_readahead(struct swap_info_struct *si, swp_entry_t entry) +{ + return data_race(si->flags & SWP_SYNCHRONOUS_IO) && __swap_count(entry) == 1; +} + static inline bool swap_use_vma_readahead(void) { return READ_ONCE(enable_vma_readahead) && !atomic_read(&nr_rotate_swap); @@ -861,8 +866,8 @@ static struct page *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, * Returns the struct page for entry and addr after the swap entry is read * in. */ -struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, - struct vm_fault *vmf) +static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, + struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; struct page *page = NULL; @@ -904,6 +909,8 @@ struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, * @entry: swap entry of this memory * @gfp_mask: memory allocation flags * @vmf: fault information + * @swapcached: pointer to a bool used as indicator if the + * page is swapped in through swapcache. * * Returns the struct page for entry and addr, after queueing swapin. * @@ -912,17 +919,29 @@ struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, * or vma-based(ie, virtual address based on faulty address) readahead. */ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, - struct vm_fault *vmf) + struct vm_fault *vmf, bool *swapcached) { struct mempolicy *mpol; - pgoff_t ilx; struct page *page; + pgoff_t ilx; + bool cached; mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); - page = swap_use_vma_readahead() ? - swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf) : - swap_cluster_readahead(entry, gfp_mask, mpol, ilx); + if (swap_use_no_readahead(swp_swap_info(entry), entry)) { + page = swapin_no_readahead(entry, gfp_mask, vmf); + cached = false; + } else if (swap_use_vma_readahead()) { + page = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); + cached = true; + } else { + page = swap_cluster_readahead(entry, gfp_mask, mpol, ilx); + cached = true; + } mpol_cond_put(mpol); + + if (swapcached) + *swapcached = cached; + return page; } diff --git a/mm/swapfile.c b/mm/swapfile.c index 756104ebd585..0142bfc71b81 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1874,7 +1874,7 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, }; page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, - &vmf); + &vmf, NULL); if (page) folio = page_folio(page); } From patchwork Sun Nov 19 19:47:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460662 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DC69C54FB9 for ; Sun, 19 Nov 2023 19:48:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ECE0C6B036D; Sun, 19 Nov 2023 14:48:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E57276B0378; Sun, 19 Nov 2023 14:48:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CF6566B037B; Sun, 19 Nov 2023 14:48:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id BA99F6B036D for ; Sun, 19 Nov 2023 14:48:25 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 92A2D140331 for ; Sun, 19 Nov 2023 19:48:25 +0000 (UTC) X-FDA: 81475740570.05.4590C04 Received: from mail-pg1-f177.google.com (mail-pg1-f177.google.com [209.85.215.177]) by imf19.hostedemail.com (Postfix) with ESMTP id BAC571A000A for ; Sun, 19 Nov 2023 19:48:23 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=RZZeIcyL; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf19.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.215.177 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423303; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UpGTVr/h0gRLNFrOGvqL9Asvs1D6zvF8lBMhGssTB0o=; b=1RBIbv0knSc5AHOHbxPPO488YkvOQZ7HoqIuzawiR8l6K/nylJEQbSzEATgTQ6hC1io2zY gNyYZOxAFDZSLkRYq81TWnUqUyvFE0otrzsZ5lTzQlhO4OnD+++Fgio1aGxkZzjhHuWAct 7gOdaBwhazfehFoXxiKuAwUVIAmuhjo= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=RZZeIcyL; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf19.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.215.177 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423303; a=rsa-sha256; cv=none; b=Koiy/UlJ3/p2Mx3pRfU277fHCT5h8Wbz/v8p5dxM4/IoHQsFqi0YoQvRmKDfZZEyX0LuMU t+2AeGc4c5L1oG/om/q43bMEZRSx+5CumcCccc1Rk5hxKy9HW2uhPTRJbF+5NuAWlm+nSY xfHyA4yb3z4B04JbZTrpOQdQUFlDtNk= Received: by mail-pg1-f177.google.com with SMTP id 41be03b00d2f7-5be30d543c4so2373383a12.2 for ; Sun, 19 Nov 2023 11:48:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423301; x=1701028101; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=UpGTVr/h0gRLNFrOGvqL9Asvs1D6zvF8lBMhGssTB0o=; b=RZZeIcyLV0ymUtYoy/VeUVk0DsOOju132g8/cAs70iSmQaeVysdtckfZvD64IRMdRQ nv/o00cJl9201sPiFycCw5ny4I6bjTC8HgpN3II/EbCSU4TS6h2JloV3InKzVKLmzboq jEMKbgF71Jd9C++dMcNeZk19niUmpCzi60UHwJL7cb8u5czJWog3ajU5aOXEGI7cWOqy fm/MO/4HNZedX6XOq82vtjS005u9Aywm8ZoRz4wjtqrQezr8xk4EOe1VVF6vHNfzideT GZdvi56G9XKNIod9iENraXXTYZeGQlmrNfhQ3MwKLgGI4/QicrjrEMvw5K9BRLPjU5Y/ iP4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423301; x=1701028101; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=UpGTVr/h0gRLNFrOGvqL9Asvs1D6zvF8lBMhGssTB0o=; b=TGB9eRi1KbuJAllu4Ntk9OsRePR8diQAXIKMzAtXEmG88lcsQq7RelwwV3jrgjumGw v3w4mEvqVxxHYKhNEqMGlgQhjNwBvWPc+i6TGCarvYf9V/r2myEkctmOimG948unypJ1 cGrM685sjybg38tmYRqetWsIYS27bl8wk3reQbPmtaSRHGVY+fjmKcWEnJrIg+rgnDqZ 72oqEuXlGYtpnGNnzMDFQTJRcJflpgjHDR55q9aOPl35p76OSx4vfZU7Ugfqbw43v+nN FqLeHYm1Yx/jQr/X+4C12AaF9Vdk0MgpAhFaLlAfPlzB4YPklXtlaW7wTRIDihE0PB1Z sgHg== X-Gm-Message-State: AOJu0YzCpJJOG0DWIEeSpTBXo6L+Da4QEAECsWnyDlw6xLZ0qONRL7gY 4wA55cQZ0D0K+6t2J2MQr5Q9TRmO/U2layxt X-Google-Smtp-Source: AGHT+IHfs1/JMlpqrq4TKTabAC0wuT2f0zXPPKJXJ4FxpJjeabGpgVADWj7Pztf6px1yswvAICZfaA== X-Received: by 2002:a05:6a20:b797:b0:187:8eca:8dc6 with SMTP id fh23-20020a056a20b79700b001878eca8dc6mr2490060pzb.34.1700423301416; Sun, 19 Nov 2023 11:48:21 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.18 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:20 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 06/24] swap: rework swapin_no_readahead arguments Date: Mon, 20 Nov 2023 03:47:22 +0800 Message-ID: <20231119194740.94101-7-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: BAC571A000A X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: isef5m3ft4yihwik71etpozpcckzfge1 X-HE-Tag: 1700423303-843922 X-HE-Meta: U2FsdGVkX18Fz1S9v9tYEI6bvVcieq7vZy0oJMxNL4VdNz1tP6pXmbYwvWjxuHAOJR3RAxWBObIImnZt0qfppQo5o1sHSBMiapDCCFHDyJyzBm7oP08+vP1J8AqX810jOJuyd39jj0ZMzmmICA9iS9MQwGaX2jW9mURv/lNbpuDj7v8jz1pCDcz2m738Bq0RkRtRnXdorZ/OPYXJQwyHcUf27d6rDtGfTuZiO5BSBT8GWsSRPBUhq0t6dWOAF1VgHlRtKg9ARkWgX8+Nm5/lwYe0mIBY5tOY/1/LyfKvpRobJSZ1CTKU/85toqrrnvDPttwpx8tC/F8T3bISDvfcGMlkJF1yMA2TtpSsNEIh7eEufj/6tzKtMbOeLY3JFiKsVLVLN1bxGjdCt6uwbUQeNuzXwXW18Aejn194nizS1dHwqj3MTRybMCLeGBXksugPDx8xFPk6ccwR0BHvIkRSiUjdzBwX3I8VEvwcjVepKbc3Ynot84zdn77zYa/uJIREIzTOOF4JO+SGv2MjMTLfKIL5q24+WuE7b/7sqSPFiWj731/nMG/CFw0MjTSbeKuMVvrAjETEQYrOverkSDHiOBD09fQ90+J+bMiA/bhs/DJPfjDCCiV3ZdzvWfwFF3k5HUAfjznnQdz6QUon/kltG/R1URarLtBaREdyvW6MHyyc/EQD2Xs6lcnss5HbFvumBEbpHG0L7KYGzFHGWeRLX4pr1TjVoOU8ReDNPOxabpo+rBzoeId+zdVuVp3rYUeEFxQesbUANL0vfV8WV9BLUtL/ZHWl8wY6wvqgYZE3EgerstldzxbV7o/ATSUI1zxD3kpHK+C1q0xL1wsN5LgSAmGd0xG2jPyZNKIoqvWtncXv91a3MAsXJJ7pZNlTM/Xp9jTvnmZqSlSj91siMmPDHZXxPKp8EXwmWHACB9NzPY3v/BBHPVjI9TfN2VQwEgvHF70LcVdDFTQ7mHgF1Cs f/MA9t9H Yo9abffjM+T0oIvopZfdEtJw6gRbGJgPhP0alYHTXek0vIQs3uBZrwM6lJ0J6m19LCqCH5doZfu+mE6/AHL6GIApb3JsUqU/IymXwXA3KTA0Zpl1OJmwRRJmGuRS4Dm6Vv4P6xrjtg+XCWYlJudb8ocfadALUzmCuUgvWKx9QnMRYP9ur4Lqfs5NsxROUJqAReNLh9Zj9a1qOX2/zlRKPV1AOSBubJoKDnXWT+4kW8HaaUXGCMZTWsDdDdOWDBDm/0sl1aFELf3oUsVwDwAqnvCdcEhrC8UAevZXWDxq5aeWSBuY6uO8KHWlfzDnj2+BrTdr4vgfyonpuZzCpEt6ywPKIwniIkdLB3+CZAkLhsiVXA1k60YUQ2giPYwikJ52Mu60DYbblkrlTUyuUqwGNs0eUXiOh/znfs1A8sLUBQfOTzPFW8MDYowF9agJBJAd1ChAf0SQMEoiDTFyMaCLYIJ7NkydN44TLt85BtP0UApHctNcerSBgyyaNK/zRKoR7IymoxkEmkL/UU9paivY8jFvI44FvuAFsbi/E8WObvJa+BohcIqTyX1DLYxD8WULFKUFm8pqTHUctcWrCaiUMm6AbQ/qZY1pnlDCRUpsMKSEgDXY/dSzqG4VuVoqsZWLSgCjREkq1kffci3O+l1KFEqem6hrAAgv2uACQSewzGe7UY2k= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Make it use alloc_pages_mpol instead of vma_alloc_folio, and accept mm_struct directly as an argument instead of taking a vmf as argument. Make its arguments similar to swap_{cluster,vma}_readahead, to make the code more aligned. Also prepare for following commits which will skip vmf for certain swapin paths. Signed-off-by: Kairui Song --- mm/swap_state.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/mm/swap_state.c b/mm/swap_state.c index fd0047ae324e..ff6756f2e8e4 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -867,17 +867,17 @@ static struct page *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, * in. */ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, - struct vm_fault *vmf) + struct mempolicy *mpol, pgoff_t ilx, + struct mm_struct *mm) { - struct vm_area_struct *vma = vmf->vma; - struct page *page = NULL; struct folio *folio; + struct page *page; void *shadow = NULL; - folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, - vma, vmf->address, false); + page = alloc_pages_mpol(gfp_mask, 0, mpol, ilx, numa_node_id()); + folio = (struct folio *)page; if (folio) { - if (mem_cgroup_swapin_charge_folio(folio, vma->vm_mm, + if (mem_cgroup_swapin_charge_folio(folio, mm, GFP_KERNEL, entry)) { folio_put(folio); return NULL; @@ -896,7 +896,6 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, /* To provide entry to swap_readpage() */ folio->swap = entry; - page = &folio->page; swap_readpage(page, true, NULL); folio->private = NULL; } @@ -928,7 +927,7 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); if (swap_use_no_readahead(swp_swap_info(entry), entry)) { - page = swapin_no_readahead(entry, gfp_mask, vmf); + page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, vmf->vma->vm_mm); cached = false; } else if (swap_use_vma_readahead()) { page = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); From patchwork Sun Nov 19 19:47:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460663 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0415C54FB9 for ; Sun, 19 Nov 2023 19:48:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 815AC6B0378; Sun, 19 Nov 2023 14:48:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 778906B0380; Sun, 19 Nov 2023 14:48:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F6F66B0385; Sun, 19 Nov 2023 14:48:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 4A6196B0378 for ; Sun, 19 Nov 2023 14:48:28 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 24894402B1 for ; Sun, 19 Nov 2023 19:48:28 +0000 (UTC) X-FDA: 81475740696.19.8219A45 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) by imf19.hostedemail.com (Postfix) with ESMTP id 504941A0003 for ; Sun, 19 Nov 2023 19:48:26 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=dRRUBVcq; spf=pass (imf19.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423306; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=65vQj2xJm5p5mk5SbjAKMeJnwT21tT7eyXmu2H+Etu0=; b=0Dc4rbMaoAt43ZZnsyx2ubzH9sMeiUfqBDD06WsZNYC2kxWb5xXgmnYo/r610HAL2skZR1 BrRN8dpmRwcwL1898IJW/EzLpKWS6wgy/r0gEM59jK5KXixQAPfnb0294eREiLa6WPOCLv Ze1OCMaP5s7ta+Yx7W1ajsFs3/UB6rs= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=dRRUBVcq; spf=pass (imf19.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423306; a=rsa-sha256; cv=none; b=TooULEyneTvVrvs+lhRDVhui9wdmSYU/cEOojq4JjqTfVAnnV43UTzWh6Gg7N4IUWJlm40 ZE/NXNWlSUENMKKnI3oz3GyO6ETadASlz3sC5DLp6aZL7BxxriLn7UmDX+Rq1NJ0+Is/0i fwij3AtWxBNRffZQDP8M9xTq0v1WXFM= Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-6cb7951d713so352248b3a.1 for ; Sun, 19 Nov 2023 11:48:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423304; x=1701028104; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=65vQj2xJm5p5mk5SbjAKMeJnwT21tT7eyXmu2H+Etu0=; b=dRRUBVcqfb5qH64M9KHloBTGpU3kiITKYEwo8qq5NkFrqgWJYDdysk5YJ+W6rNp5me H7hL1B3VUT5X+mnNeLXEsncyYXgSuJmvoLbSZfuAX+RZufvcJWN5FPsPRWzhNRPDUwW3 k3ZRg8h4SV27qPmkXu4uwP9BgmuZSE6P07HkjH2nKjcybQD5ZQRHTJlX/QuL1x0b+geC m6zynGPzpOTYK8rJQOI38WWY1rdTgkFqalQnc0rQ9ZVAkfNy1GeXdQ7vaHCUpS8d84GB Fe7Jo1s+8LXHKufn6SDVUO4HwTVLO+utYh0NYDgH47zE04saXATloJisNhADiHSNQxdb v8+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423304; x=1701028104; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=65vQj2xJm5p5mk5SbjAKMeJnwT21tT7eyXmu2H+Etu0=; b=TdM7XbXmnUzYIuFCeVXvhX+Le/yCUVE0Mv6NDHRXmkyQ950ZKby/bie/JS/MVGo7yk 45ndAAswPesaH6+TBOO8mxzCXLhmi2YC44b2JbFv+i1SlzdH25g/SKJy+5cGMAXu8mMx kbfCr3zqgoRx1wEMcWeMGJZIs0I5uFQm7t5GsmPPJmTKNnVmM8Aj08g1yuWDxPZcyOcK m182KtbPGaNlYNrF/RWg3W3o36Un3MyznjMPt8nfCrH4++dx+H9hvWyUDTmfOsy9xaPE B1ixhKAXQ22SMp5t9X/EDsa891+SEFVUvZpCrQYLAL7s0fT9zdfKpHZIxVReDmaNfLpD zjDA== X-Gm-Message-State: AOJu0YwwkwqL4zYvcBnDsd51lEjfbMA1PDtwLXielJsmF8fklgtgOy/j V2yX6mb8fVzf4dF3vebhY+DCNKjAUo9WZZJo X-Google-Smtp-Source: AGHT+IHHw1felGXoqcj71N9Zw1zsM8vWNO/fCDW6ep+Q60u+G2qJ+qT6hdQV7yWi9c9XQgH0pPdoJg== X-Received: by 2002:a05:6a00:1390:b0:6cb:8abd:39b5 with SMTP id t16-20020a056a00139000b006cb8abd39b5mr2493633pfg.1.1700423304666; Sun, 19 Nov 2023 11:48:24 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.21 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:24 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 07/24] mm/swap: move swap_count to header to be shared Date: Mon, 20 Nov 2023 03:47:23 +0800 Message-ID: <20231119194740.94101-8-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: 504941A0003 X-Rspam-User: X-Stat-Signature: qijpajutmt1exud9gq1q1wy36qai1cw7 X-Rspamd-Server: rspam01 X-HE-Tag: 1700423306-98742 X-HE-Meta: U2FsdGVkX19CPR1hEejti1ZDp4mDL2lZnCmOLsHcUMWiPxhfc5ajUHP6OrUq8xIDuntbz3lIMO/uDU9higdYscHF2el8fPJ91uvy3brG4QBXSUAGb2Xhi/aSTZZwUsoHwNTHi5MtxanvHcLzVwDjnycOKf8jKklcocLJk6WJpLbOlzPREnK6n6NUpcDaA5hm//rBUWtMZmHOjQFHObjamTSp/wfnk3a4lUsRJ/kH/qfLC6wtYtXUTTpm3vjwUADQibULaTWakSin7SM2LHoiadbYvxl4ecubHySH+lZ5nHgaXdyWAjFeyICBnG9bxovtvKEa+e5lwKjaA6EXt7Gz4ftOwi0puLA7XRXJHtNWRP2p+jJbQGp1kT+ZHKkYwo6p9Hw7OnokuMh2wNH+pJenLK/aiHSCIRHa5kJA7ykKoTzA/U5YWngNjKSJkCPYWySJL/P4PWf/YMM897dukQ1uNqFtotF7PytohkPoo+t9MpQiLTMIDWq2jYKFsO5lKmm3FS33L9OJM7tV6EM6hAd/iGRcJnWFDq7wbkuVKpIxQRbUcnsbYsys4lqmqEIneWcprEqVb0w4bYyB+XUEeH+Kmxxs0J6n+5iWM6zW6d6iwh/tuaRKMmRKO3wQ6rpkRrTR1SYjCaLCXvbsXDlMbsc/N16xSCrUlx3dm698Wte02IoJIyrfoNL7CwbuUXEZw7VkJ4VcUiFjSVmKWJax4aXvWGoeB5xrBwUS0yvP7BKs51Dy91HVXWZt9xhSLPXbhVgUnC0stmB7Q3uSvE0vSD7qjt6w5+syZVt94RRsnTbQOUpCQHdCveMDeKj9UL26HMT+12PCtymf3HDjaukIIV7jpYu/lDX3X6peQGY85ZCuWFPJGszZhnuDjB9jKNI7UmI8r6uSDmosjUMLS2yL5LJ8+ox/ZSuGJzjTrWuFfSoaDrTDCxW/UGYRcjWadbqZsZMTjC6hgeAkxjBBTUfsiJ8 09ylixqr Y932xoJDvwrj1WifYjYAAhv/nsJWUH+q0mqvSQRACMfCJd3/+kQYc70mVBvN4V67aLrzdKxb5UpYwDeZkeIpC8OrnYNYW0yhWyk2RTEap/8KUbI0Ks9aIpGRpfZA+JJtpO6YNsCNgu7inu9IM3yIoS3we1FgpDvljby+yJErU7h6B3oKTJdftdZsQGeKaZ3r96xaRd7bvFk2coe7Vc58dBVVb+wGXvyvRX+6Oi0kXGje3yv0/ELB1c6LHOzz1fl1Zp3QtJEMixQlKvR/MNRQU7mB4sXWmYZEV9BogcPWZb7t0akTiB1JGc2kEMTifasc9wXDw3PBzuFUW1P8wApCPEMQw24HR5Hazf36hVRAGYbjQHj18aS0wwb0PX0Kmfd0mCJdsPZPvB3T5KBZv2+4j4+CClbXUQIMqfr2HuIXw84zbUuW5yP+xCzOeccLTYLKXqMcWWEUQXCRnmbbK7lRIZEvlsNdpdMvc6597ucP03EvAxn20d7zreQ4SaSJLP6+fbuND/klkIt1Hn6jcHCQlmHh98sCAKptTUt+rcUvlKzy+eZWAHqH5GvnGhYdws9P4ZzKn7uJp5KJU+ctMv0B0YnRTG13sBrFajo3PIZ88BZLbbWBUWcj3quB6p6AGmZnlU268QOBXlaupvFqFH2xRl7YFM57FLT+8wQnvzjOSgJhtdzY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song No feature change, prepare for later commits. Signed-off-by: Kairui Song --- mm/swap.h | 5 +++++ mm/swapfile.c | 5 ----- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/swap.h b/mm/swap.h index f82d43d7b52a..a9a654af791e 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -61,6 +61,11 @@ static inline unsigned int folio_swap_flags(struct folio *folio) { return page_swap_info(&folio->page)->flags; } + +static inline unsigned char swap_count(unsigned char ent) +{ + return ent & ~SWAP_HAS_CACHE; /* may include COUNT_CONTINUED flag */ +} #else /* CONFIG_SWAP */ struct swap_iocb; static inline void swap_readpage(struct page *page, bool do_poll, diff --git a/mm/swapfile.c b/mm/swapfile.c index 0142bfc71b81..a8ae472ed2b6 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -114,11 +114,6 @@ static struct swap_info_struct *swap_type_to_swap_info(int type) return READ_ONCE(swap_info[type]); /* rcu_dereference() */ } -static inline unsigned char swap_count(unsigned char ent) -{ - return ent & ~SWAP_HAS_CACHE; /* may include COUNT_CONTINUED flag */ -} - /* Reclaim the swap entry anyway if possible */ #define TTRS_ANYWAY 0x1 /* From patchwork Sun Nov 19 19:47:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460664 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58B41C54FB9 for ; Sun, 19 Nov 2023 19:48:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE3C76B0380; Sun, 19 Nov 2023 14:48:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D6A806B038A; Sun, 19 Nov 2023 14:48:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE5096B038E; Sun, 19 Nov 2023 14:48:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A82B46B0380 for ; Sun, 19 Nov 2023 14:48:31 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 72CB38052C for ; Sun, 19 Nov 2023 19:48:31 +0000 (UTC) X-FDA: 81475740822.26.3B7050F Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf01.hostedemail.com (Postfix) with ESMTP id 9E15A40004 for ; Sun, 19 Nov 2023 19:48:29 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=TQvnhzoo; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf01.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423309; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/Iv3a0RkKO4tJSHfzH299Wh8nqjqW5sZox/cVI//TM8=; b=TpPTJHKDwN3DW42sIuTwyvW6MFhrTlawuUlyudaf1ZMzPpmVRayCL8dQKhKtI2D0cEg2Of RAO2FgtLZVQGCR60pxkOm7eBriQW3mKHElTR0ujHPHXh4ug1zvg64jPMekZz0nJObkRCpk Nr50WRTo1FWcj9+qr67HrkKtuSmvuWg= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=TQvnhzoo; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf01.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423309; a=rsa-sha256; cv=none; b=Y6eYYgiXvbs12gfni2n2N3PwQpwD5SF9pbXNGpY+BtbnP4tIUyttDHrWsyYlwkKxgGn3i2 aNFQ5fDMuhpYstP9n0q9undQH5kpKYV1nCNaonLm2e9R56fPnqLGY+YANDEDxQiE4XvDXp rk1AJnSTR3BRCGIz8/a57sSMfmRN9SQ= Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-6be1bc5aa1cso3858625b3a.3 for ; Sun, 19 Nov 2023 11:48:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423308; x=1701028108; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=/Iv3a0RkKO4tJSHfzH299Wh8nqjqW5sZox/cVI//TM8=; b=TQvnhzooU4ppGZmsrjtJS0ZqYRsY7GBpX+4VgrIqMscO6be9NsibhvchSnMKesDNkK HhnoXwUW08u+rtVSiyNsRYnxDxTdaV+xMhdhR/zxodGe1l5tRJeg4g1TDOLHSAXyb5hW BZ9Q3lbzwF8ZCUZuJS5d3laaW/nizDbQDHi986p2s84XEJ6bY2mE5JXS8OtMSf2NhAPb oxOVjUZPSbchWV1D07gxJKv2rw/oHsM+6U89zfr1Epc8oDLdnHMRQks/cfTlFH+4wm5g JOkS7I7b+XSzo/Z09UtNTC2vKeK9C9dpf0nPZLBWrJ6S2n2Xeki/G1EPb3x3uElkstK5 a6QQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423308; x=1701028108; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=/Iv3a0RkKO4tJSHfzH299Wh8nqjqW5sZox/cVI//TM8=; b=UqZz0csrAaWioAPPlZYM8MhqFOOncnghG+CDRWAcWREjbpn+Jm+TvFveLTwCqFVQ9T B7TgVDusPZ7rlncQErnX/DuyC1Z2tZ1qE54t2fRJx4q1muTDNMPF+9OFrm5DQGmakfzE mT2klvH8olhuJdG49FxYZ6FoSkwWu9lIcIj/FFXb674iMnNhRlB92TReEfgGgXgpMR5V NQcgbNiZUElkp5QmtGOBJ/WXcIYccOjY4Lpcd9zo1Xp7BYN4r0D/7ZW8xkzCBvTkCnNa Ara3+UwTYHy29kdQoYz+qEpG5QjNjxJ44Qjfv50uEBLjrlIg2UsjkpvXbWVAm7leTJoG zYRA== X-Gm-Message-State: AOJu0YweS2H0HX7ZFCJtU90SP+IweS0uq8RG0O8/QiZ+trpmI3+0F2/r ldZUfzvjLJgL8HA+2/ow5ADbRCOgHwyPw57V X-Google-Smtp-Source: AGHT+IHOC/pH4egukb5+EX3DwlCGnZ4HWVCMhNwTp5WbUoyceBak2FygNQZocPIPIHn77Rx48Jh+bw== X-Received: by 2002:a05:6a20:3d84:b0:17b:426f:829 with SMTP id s4-20020a056a203d8400b0017b426f0829mr7488318pzi.37.1700423307883; Sun, 19 Nov 2023 11:48:27 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.25 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:27 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 08/24] mm/swap: check readahead policy per entry Date: Mon, 20 Nov 2023 03:47:24 +0800 Message-ID: <20231119194740.94101-9-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: p3i5rrfh9yx8e1547g1w5i6tdi5jdn1b X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 9E15A40004 X-HE-Tag: 1700423309-723619 X-HE-Meta: U2FsdGVkX194b9xf2vq/Pfwtdl48OFCgdrzkokkkSscS6ykOo3EVJzTWxcjuYkqeaxotm6ee09f0X1JTGwqqUfWKKNXyoC4+qWHoks9dkIBx40KQ+YLgtr/bh1b8VKu0JrvceosbrVlE7ZVpNLtlwhzw1s36XsThffUAEMZiGoIzrT7WHpqQE6gOxuvaUYnmLjjlOiQ6xzVB+xZRERwnGO7REV4IcCAbMarvxfTCNdupCfJNF6l43ivhrIo0TzeKZYKd7LX89bVHwzI+9in8MLJxqU8BObPGg01zrpDvqqTHoxuDsKHIf4oe9n9wRlaaC5FOhoAwXBNeqsRW4P9RE+kuS9sAiYyrZLXnq+Vv6PZXl0RuaOVr31iL1w9GGQqEnWSRGVMBOd/hjhb6l5AZ8vB3PAPote43ZEwbP8qXzK93daQ/cF+AKd7oPBLF6w06IASrxFEjXCMt316g8as76xBqR6QR9PXt5Df9wAAWODUERinAU1N5sr1cvdfIpKuS3vshABZ2fsE5L20htOEyCAG4KCiTwD6Gco8gYfSr9uf1FCvjmLmFSqQhJeNYWFa1uJRuylp8MYSM+QRTJVGlKTe9GaS695CiK7k6rsDCGPG9D6smTqouCBFs1caFn2/iKWJvPjvHw9Zz4U5f8ErA5yaG/Mat0OVTxX/cDu2abgPGgiPgf4zXRb+nfABx1KgiCR8YdSSCmwonaUdRQQVw/BtUALI0j741ezfVl4a28qZdmfzxeU5RnO5ZyHgzmMeOPnACBlp43ODJhmtf9D7Ze1QPXjFO5z1S8HQRWQHAX++qSOjfsX8DpShrV2z9KaEB+boQZ7E/mZrJqlvd5rgaU2DAVN+uZA2irxFRtnQl+ARuYc2X5p6Cfs3aGbcMFiojs4ecS87ypcQOFB+MonUPNpszj0aTu+UCAjHV0EBm8eGUGkTp3gp/KL+OIzpzoiFZuSBRPl5cuS5zTGX0f6v vY8xMJWr MCGPnlxdL4VcoLal+/w+D2uB84IAfDziOkvcXHKxlSa6HkEe5uC9oSpl2RONhVoqhgOP2birvfzbEFIZNFWBWoy1S6XTsRi6yBbr71o2w7NPRHxC6NsQdg5+7z0oO4gMZAfq9ceRC4UVWeGX99yvn0BIYoA5DTj1iiInF958gnqOIBT+tUJoqqrulppf1WpVGODcqJ4qVerrIjspiltx77Y0DO9K/XAdN7utIm9idjoOfIbyH7I0sEuUIdTyg8sSLJMXiXFeQuERCdONZvGggYqqodRTwBVTIt50Kyr4h3ZALcKvY0MGc9pJMH1SprelcmJiKJhcoDoEb5SEYgyxuOamLhqf4osm6dzqV9/63n8dSKNxCh0z7enNl6e2F/KqFC1HOD+boC83sYLaJsDBEiFso49dJzKyFTR9ORYBulr7Is6R1EhPnFLmQc2878L5c7BZ28lSAQeuvQXHqyPfK3VFbGLvG51ZwMUjrtfXyB2i2rlbhIDdUGMFw0172WAOPS5vWqh1rWb4mJ12Xfb3uHliO6huNFrxcgkIj8qv8CVvDVPBd0oQmsOm42WNpbbHZTEX+oLioYe/gii8axWIZXcrA1oEIckaYe5R7iMD4CH5gMXcSkYRgYYMmmr+Forh5Tt4+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Currently VMA readahead is globally disabled when any rotate disk is used as swap backend. So multiple swap devices are enabled, if a slower hard disk is set as a low priority fallback, and a high performance SSD is used and high priority swap device, vma readahead is disabled globally. The SSD swap device performance will drop by a lot. Check readahead policy per entry to avoid such problem. Signed-off-by: Kairui Song --- mm/swap_state.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/swap_state.c b/mm/swap_state.c index ff6756f2e8e4..fb78f7f18ed7 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -321,9 +321,9 @@ static inline bool swap_use_no_readahead(struct swap_info_struct *si, swp_entry_ return data_race(si->flags & SWP_SYNCHRONOUS_IO) && __swap_count(entry) == 1; } -static inline bool swap_use_vma_readahead(void) +static inline bool swap_use_vma_readahead(struct swap_info_struct *si) { - return READ_ONCE(enable_vma_readahead) && !atomic_read(&nr_rotate_swap); + return data_race(si->flags & SWP_SOLIDSTATE) && READ_ONCE(enable_vma_readahead); } /* @@ -341,7 +341,7 @@ struct folio *swap_cache_get_folio(swp_entry_t entry, folio = filemap_get_folio(swap_address_space(entry), swp_offset(entry)); if (!IS_ERR(folio)) { - bool vma_ra = swap_use_vma_readahead(); + bool vma_ra = swap_use_vma_readahead(swp_swap_info(entry)); bool readahead; /* @@ -920,16 +920,18 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, struct vm_fault *vmf, bool *swapcached) { + struct swap_info_struct *si; struct mempolicy *mpol; struct page *page; pgoff_t ilx; bool cached; + si = swp_swap_info(entry); mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); - if (swap_use_no_readahead(swp_swap_info(entry), entry)) { + if (swap_use_no_readahead(si, entry)) { page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, vmf->vma->vm_mm); cached = false; - } else if (swap_use_vma_readahead()) { + } else if (swap_use_vma_readahead(si)) { page = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); cached = true; } else { From patchwork Sun Nov 19 19:47:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1AF8C54FB9 for ; Sun, 19 Nov 2023 19:48:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 83E826B038A; Sun, 19 Nov 2023 14:48:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7C48E6B0393; Sun, 19 Nov 2023 14:48:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 63F496B0397; Sun, 19 Nov 2023 14:48:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 4C98E6B038A for ; Sun, 19 Nov 2023 14:48:35 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 29440B566B for ; Sun, 19 Nov 2023 19:48:35 +0000 (UTC) X-FDA: 81475740990.13.931DECD Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by imf08.hostedemail.com (Postfix) with ESMTP id 4EA18160003 for ; Sun, 19 Nov 2023 19:48:33 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=GMAfQvcA; spf=pass (imf08.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423313; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7ikH9nPhVAiKzaSXcyZub9ryWf7e3smOr0BN4n2CRJU=; b=taFFVvCgxB/nipJIDsBqrcMCeV7E7vAm3wW+PdfCOiOZe8b+gplIM1T0Ce/+tiHuOQjVob 14W1Ithi+gcYeR7oOtavsi5AMF48G/1lp8n0r0I6qKxyEMc316fbEEEpbj1ZsNxZ0pSW92 gro9Da0tD5RpHFUZ+oq/2SJIRohfplk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423313; a=rsa-sha256; cv=none; b=KKo0EqaoKSSVZ3q0l1OhwD7TcYhFVk0SG5/qa+9ceeWpz3H7z+To75n/Bvu07bhzsxCQ8o itEq+fTnqHRgrbc2U9RZcxyZv/KPgzXj0u9jxSuRgJqCyC+DBCtjVeOAxybTuBO5CbeBtK hMDldrsxURRbllClARFj/rkuLRtDQ70= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=GMAfQvcA; spf=pass (imf08.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-6cb749044a2so844370b3a.0 for ; Sun, 19 Nov 2023 11:48:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423311; x=1701028111; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=7ikH9nPhVAiKzaSXcyZub9ryWf7e3smOr0BN4n2CRJU=; b=GMAfQvcA+JXERelm2lnYn6MGW6uAP+yRNulfhHZBJ0vuqbhJdsQwEavIgvDg/iY30k 7AdiKLuhgslGeb3+ix6ubWQHoXdAObOndo38lKLSg9qFMFEDWRYeyOBGy2gwiUuiUOez vKcUfsyi24Q9k/shQiI7Psax1vkaymtEwazLxIVj/biM9nrxOMQwyNBeKreDYHoYLp9h ouh1q/LIKjxbkbmxbQg3hMmmCra9u9ZPryP+EAh6wLklJjGcIgQEpY2U3PmfA8oQb/Ur uATNxSU6KeNcxrEBf5Xm59agiB5KKE2hzFMHv/+okg86lqdD88vCrvgHpZIGovRFrOCc SrHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423311; x=1701028111; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=7ikH9nPhVAiKzaSXcyZub9ryWf7e3smOr0BN4n2CRJU=; b=XNUbJZe/Ir/sT+HcTyZZQiZxOShRfH6tAfxyDYn/8Ty3ldjpvAM50EpYoB8kyKkFkR PvXSmhai9ybPZ98MxMUFsRJTE7n6NzAnCYcW5Yo+12oVHDcP4ApTKKPExCe70uOTuBaH AtLFJaK/QGlJhF9a2Pmh9a3+HXWiww/ujYYl++bBnX4o5R5dVSumWj8Ood4oSupQXN8N 8sE/ZhvnSKK8Y73yxDCP59UfFap4V4495mJP/w4P84oHwZqZxAPHu8l/gSkSKDfpdMsb 7WnNFJcDWPMoHarrFkrWswfut3yprnM4zUwE6t+ALeEc22/X4Alza03UcU8flVTeGz+4 EAUA== X-Gm-Message-State: AOJu0Ywtx0ANtSTbE7h+j+BIZJHKjNC+PXFWHnrNARDFP0WeHsYb3brL pckQHtkl2Shr6tg1zU2MJOe5rcjbff0zmPqV X-Google-Smtp-Source: AGHT+IH1zxFvtExW3yVn2ffiwyDes1ojQa3pxYWmcyzSBulOiTzqABkc4AI7t2dHHB2qRMn2sfkZdw== X-Received: by 2002:a05:6a20:552a:b0:188:1125:88bd with SMTP id ko42-20020a056a20552a00b00188112588bdmr6787703pzb.43.1700423311039; Sun, 19 Nov 2023 11:48:31 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.28 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:30 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 09/24] mm/swap: inline __swap_count Date: Mon, 20 Nov 2023 03:47:25 +0800 Message-ID: <20231119194740.94101-10-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: 4EA18160003 X-Rspam-User: X-Stat-Signature: 7nhcpeug1rx4oopkjdzseaozpmt4o3jt X-Rspamd-Server: rspam03 X-HE-Tag: 1700423313-579093 X-HE-Meta: U2FsdGVkX1/FYQ5CvjZ7M8m0OxR3WWrCqQd4uDOyV0qGDZLxvimJ+j7ivMkN+TMFHefTPlmCmjNpihNV13pB2AXtfm+5FO8i6w5SQGtDDdAh0nBCfEa1jfz7sJngDD9g9KovRz3dozFPLDuVedQoWEt6mlqlh/8Vt9faQI0yyhJYG/yom+D0QkzgCMqJkWNA0FWmNROkwLS2i+lNQHch6X94QgaDe2k42cniR4XMzU/nNHab4EYk9E51oe1lEfBZ8G6tjtpBN8A5XV9wdaQexJGgUUmEMJeGAvcIXHwlZDyI5TOFmuqDvDKcDJ6PeIeujY283BHEZxxx1LEwH6K5vz/m50a28cOzSJHzdL8ct+7s8sHQB9SFjWPTh/8QlxT8jw6tjEdAh1+/nLuxKHFHBeZMiNJl8o4nvKvf/dWADSuJ+71eEnQlDk8nxT9Hgyh8naOG/kkQPExyTMq8Mpyb6sBfcNtHajGJkDFmgl9/FHai4mSIrywEq4DwEFTAGUpGBfYHHIkk3bcfS9pHV5Dz4hGfmPwv51kFMbKnxdZ483UlhrwsTfdMWJaRmiu+yHysmAfdjWjD/vvscz8kMhFoNbG25+jMoQaGultayU/F/gFz+qey4cWn0R9SwZzK718a0WWZrYhi7bOGuoQHI3sBU78PbwIoSrgdMGD/Eh+PXqtRpDh6XWUEB3bo+T1QG4BX0eoVmMdk7c18rVlmV31HWTdUu0AcBvQXxONJ6w/GODvxDkOgxbFnbFxs94odKXY/sDdEI+kWJG79+2SPWHj2SxXHaVhJ9uJJs6zEYXsmA6DVUmfBq8kYFuE3WJ2F1UJijM2HHYgum3GaDK9DNdxQ6gRC5AnvjCCaVrXiRph9zFCRamAWQ0cBswrPqxJU54MRGOmArvz6O1o7sGSVNDTsRkkIDvzTx1gD0WF0XJfG+5NcptcbA0wsrSMCWF95hAyWf1f5f56B4Dbou5fUTxa mxwbTeos /38uwCrOOLKaie+13Q0EJMLTB58tm/5NShhGQDQh09LJQFdIZnuK5kxA58bFqaLC9U1sEjJ2/YUdCXi4Unp2vRsH/+dsf2kGeYwpNv7AluYc7H+avMVA5xR3jx5p7aIK2tYqC92Omh3qb5bXF7JhoYMx1VMivGB6hvRqyP9R6RjSNGamHYgin4p+x16si5RL8Qo/m7317jL91fY6bjSJezWLIdLLKzGtPh2eLkQgrIZuBvRKuu2XBO0mhcNSsjo4bOcaE/bCYyEF6kIvLOw9yQlXjA4UeY8XP97PWysuV3X365GfaVSXdwT9oBPHzD+gfMYB4vNHLlN5b9Lec+nxnkGKA9ZOA4MoEHody8sj2ZLu3q5nEZkhBN7P3uCHzOAygOtk4Hm2oqAk/o/DkPEMQQYBKk5WnoDnsOQSECFWAzhVwS96lkZGN6L8QRNSpZsMIgHuwj/hynxBc1v/qX1pwq6FaymoXVQWFJPx2Dmti7ztugdwFEziuSMbwvrWqJUfKvmW5QEU0RclAWaNCinWK/BFca9Ce2K1XUCGC3O2DnKJVAZ9TUrUsdzavcORc0dq0AYv9wpAm/oqs/WaTOMwIdfCrIMbnrA8jOwReIyqvFNJ7s1qojEUK3lXDjpHH0xLaDfA099XC6+TISO1IH4wS0aMgzJDWvLLYCMhP/s5NVeFDtQg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song There is only one caller in swap subsystem now, where it can be inline smoothly, avoid the memory access and function call overheads. Signed-off-by: Kairui Song --- include/linux/swap.h | 6 ------ mm/swap_state.c | 6 +++--- mm/swapfile.c | 8 -------- 3 files changed, 3 insertions(+), 17 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 2401990d954d..64a37819a9b3 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -485,7 +485,6 @@ int swap_type_of(dev_t device, sector_t offset); int find_first_swap(dev_t *device); extern unsigned int count_swap_pages(int, int); extern sector_t swapdev_block(int, pgoff_t); -extern int __swap_count(swp_entry_t entry); extern int swap_swapcount(struct swap_info_struct *si, swp_entry_t entry); extern int swp_swapcount(swp_entry_t entry); extern struct swap_info_struct *page_swap_info(struct page *); @@ -559,11 +558,6 @@ static inline void put_swap_folio(struct folio *folio, swp_entry_t swp) { } -static inline int __swap_count(swp_entry_t entry) -{ - return 0; -} - static inline int swap_swapcount(struct swap_info_struct *si, swp_entry_t entry) { return 0; diff --git a/mm/swap_state.c b/mm/swap_state.c index fb78f7f18ed7..d87c20f9f7ec 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -316,9 +316,9 @@ void free_pages_and_swap_cache(struct encoded_page **pages, int nr) release_pages(pages, nr); } -static inline bool swap_use_no_readahead(struct swap_info_struct *si, swp_entry_t entry) +static inline bool swap_use_no_readahead(struct swap_info_struct *si, pgoff_t offset) { - return data_race(si->flags & SWP_SYNCHRONOUS_IO) && __swap_count(entry) == 1; + return data_race(si->flags & SWP_SYNCHRONOUS_IO) && swap_count(si->swap_map[offset]) == 1; } static inline bool swap_use_vma_readahead(struct swap_info_struct *si) @@ -928,7 +928,7 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, si = swp_swap_info(entry); mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); - if (swap_use_no_readahead(si, entry)) { + if (swap_use_no_readahead(si, swp_offset(entry))) { page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, vmf->vma->vm_mm); cached = false; } else if (swap_use_vma_readahead(si)) { diff --git a/mm/swapfile.c b/mm/swapfile.c index a8ae472ed2b6..e15a6c464a38 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1431,14 +1431,6 @@ void swapcache_free_entries(swp_entry_t *entries, int n) spin_unlock(&p->lock); } -int __swap_count(swp_entry_t entry) -{ - struct swap_info_struct *si = swp_swap_info(entry); - pgoff_t offset = swp_offset(entry); - - return swap_count(si->swap_map[offset]); -} - /* * How many references to @entry are currently swapped out? * This does not give an exact answer when swap count is continued, From patchwork Sun Nov 19 19:47:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460666 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36C53C5AE5E for ; Sun, 19 Nov 2023 19:48:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B49466B0393; Sun, 19 Nov 2023 14:48:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AFA136B039B; Sun, 19 Nov 2023 14:48:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 974B86B039F; Sun, 19 Nov 2023 14:48:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 7906F6B0393 for ; Sun, 19 Nov 2023 14:48:38 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 55AA3120796 for ; Sun, 19 Nov 2023 19:48:38 +0000 (UTC) X-FDA: 81475741116.20.09FA10C Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) by imf23.hostedemail.com (Postfix) with ESMTP id 7345E14000F for ; Sun, 19 Nov 2023 19:48:36 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=M10LqNdb; spf=pass (imf23.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.172 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423316; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C/w+GrPz4ktaqTeTQ4DmVKbN9IqMk3W+D3u9IVSxuMQ=; b=GpaaAEMbkxlNqor7f6prD+M1NOi6I8cjsONk+/snRcpgfaJ7x+4CCPMqek5vmGghqJAUQn 314Ms3+sk283CxcUYKpxBbgCPupq7DUyhjJZKLWPxjzLM4lg81Ht8QCUwjnQVCSczg8nK6 TfD2OLAepzlkScuQMq/+MMDA4F2IMcE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423316; a=rsa-sha256; cv=none; b=LSCK+Ik04GGL1F8xVhar30NM87qMauKKByvEcH1vMJun6st7MnXrbV/OwlV2rb39EGXDIv VdqtaAmz2qcReoWIHhdFoPt7cPAyc/YkdhzfayPDwU8bsFhF7OmVs5K/TRtiuVVW1IBHM4 IXq0siouQKa0hfvlPGPaD2JWP4P8z94= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=M10LqNdb; spf=pass (imf23.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.172 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-6cb7951d713so352284b3a.1 for ; Sun, 19 Nov 2023 11:48:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423314; x=1701028114; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=C/w+GrPz4ktaqTeTQ4DmVKbN9IqMk3W+D3u9IVSxuMQ=; b=M10LqNdbRhkZQqx2fOldao6r3A/FbZmcaQnLP4rPejAHuAuyKLMJPw03hcVIEVuw05 K5ROliBQLzZ8jJ0ozUQER9GXK3GeFk+n70xCTlvk03HaTxIrWbKigxL+byj8eJ8ADmT/ 3jrky6LmAubACFEtA/J8HAy+nOczi9xcIR5jQHRj+ecwp18g+LgAIx4992oQnzo6T0ob Rk2NCl0x8MCm0IB6cAI/O735ylSbIpVLG5rCP5mPUsSceU0OajXqKE75KMGTitBQbqPV xQCJgpIXnwkzF6a/j4XOkbOAaKiM6TsbCbZsSSYO/rqW3WvmPxiddUykVijlk7L+iVfF Sl4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423314; x=1701028114; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=C/w+GrPz4ktaqTeTQ4DmVKbN9IqMk3W+D3u9IVSxuMQ=; b=m+LXz6zJ3Nc92T+UrS88U5CG22wi225Qg8fLqwkCM4iaKkfAAJgCxypUzps+OHKRv0 /7T1c0HHxPehVBCXPpFPHeSSSQp4cR/A65RgCwnEwdFT1CTp/LMhr3lHuYDKigJAyQKy /HLEM1ZFUy5n23zuW0S3CSvwj+K4XKtYymdNdXeVV1+rwXOFIvoXk2kfIPAl3mkbDocb o507cFI8vXp1oknQ8mWOnrfeU+ZkZGjeAzEMbD19K0DJo2w2MOFCUHWFNXf700INTsRO 9Zji3+S0gVOKvJhWpBD/+HjIZ7xYwmQ1QS1w2FseaGoWmKo9jwGn7RfkCUa+9j9uABc9 EtlQ== X-Gm-Message-State: AOJu0Yx6F1EP+h1xagllcUJAvHHo+QYpdpVGXvaR/BSU/hv7aK1Lte8N rfuuD7cyyJ8an0BIVi9nhOhJBTYtgG9oFpy1 X-Google-Smtp-Source: AGHT+IEspsOv+B33e2KenQSEEY708IvBCJ1YaqljuBAeYVz5GS0a0k4ga/MiJ9KapiAUwb4LUSztpg== X-Received: by 2002:a05:6a00:2e99:b0:6c9:88ee:1d87 with SMTP id fd25-20020a056a002e9900b006c988ee1d87mr19403560pfb.17.1700423314274; Sun, 19 Nov 2023 11:48:34 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.31 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:33 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 10/24] mm/swap: remove nr_rotate_swap and related code Date: Mon, 20 Nov 2023 03:47:26 +0800 Message-ID: <20231119194740.94101-11-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Stat-Signature: 7hre4h95qadg5xhicriwrwxbkrj79iep X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 7345E14000F X-Rspam-User: X-HE-Tag: 1700423316-107635 X-HE-Meta: U2FsdGVkX1+JnSR7VNBkZNGjQ/JYX2vNxRvKE6EDiF2ZENWvB2WhK/mUhJDawOV8GAd1+Kjqu38Q4lPDxWCN7RlR2R9rqBuq4htmfFXMbjcrgF70DXwDW6NoLdBwXoc6e6WvfFWwM5j6x8eFY3WPBAx8Z91Lhq8xOKm6J/AUCmRevJsxpLVpQTO41FeA+pXVRv/W7p4f6P+NEoM07hhCzHDzlMjSHZFTZfngfxO/I6VtiGkgQawGR+SoIXRhrxXQg9/hO1Hwbw+BLhgvSwVXwDeD6caXoNCUXocFgnzLQbrD+LD8cQYu+QIixJqJhN3o9FEdAwlhQXIZhxp06Z5hihVNqc4iwXxhkroEWEF54453/hr5BW8xBRgkuKHt6UBmA5w6wFTl5FLrYfscKGK6i53JNawGFulTUto0rtpjiNbpLgfq3PL0YLQbbZoLzU3YljzoCVCS5nFiaK8kVn/Iyi0TmYd8KSCFpS1Hjg1O9A2jJhnjbSHtuabF5DeVt4qV0+ZNKhv3P8taXdbUubadDGNTWwaCl7iavRqe884GYyK58yxRkfuMUCltcLcxacamfvHJ4b8BM8OHYYZhTVBL97RYsC/3Izx5qHK6wU6SkEDJpgEzwF+6WLcxfQPfN16Pi1wJDlqUtJabgEFB+NXB25pfv+26RH1dLoSdjW4EtsE7Blv43oOJBtEsyiy743cfVd+C+W2mVXcPQTcRRdusmP7yqeXDfTeUpqE5G59VfoA/1FfFQHY01vbG0kIGQH3+aZuaMczUsz77EXMJ0iGUxLy4iUDYlrxbtljNE0Rxh8Sg9K8AwKmfh2MFfemJXF1l9pXVGPQgcCdJ7zFT106g1yKo1lKfhbmxD3xCWLoZGoT+GzO9cztj0ypD0Yn/UgYUEo4EqCBbBEdJO9gxyGhgzZbHIMTYzHflEQNcyfF3r63XKBXvC3ZPCI6YHUNtFEyTN+lH221To0FqI4siBNU kFRoVon9 JS9ay3hhWMXUeiYYTCdgl38pPTRisv2HIBZt+lOnvpROLVa6QyMP3Y3uHvdOdGHDwgsNRRlKWycPJsL5vghza6nhGyk5Rdv1RAUDjcO11stLBbe7qrZlJ88O/sVncXLzcqDKuH0HtS7dLDMofyZWnrza5jVkrUb42T1OlTyfpvDqp/2P2+ZzyOu+BVBzuTRrTbL2pE5IfVc6wWpToPVI+m2dj5akEIJ4RSLLIzXeP/q0O+vkMbjk3NLuYzU0/uA105+x+USIEK+M7FYhYjoQIzUrRp2V+EQXlWZPdrV5NerghvDiSc9OJxM9ZEev+H5hkknhMc4I3WQV4g+JZEyokwpWjtyUmj4TGgK1L7V2yEugMg3acAkeQTYddjXtnYB6y02nKY/lpgnb+tewHEmmf6T9O1EXtdbWd96UM+mJZs16FJOtq4kTZA09hZiAHyKM/RNTkpR5PaZFAqjp1SU81RcKdrTyRYxB3N4WZgL6nVWOFPZxKz47NP3vuTbWy0ihsHkXbKBE7TmpqEcLar9cHckUKsTOu1eXTTA9nDMZV0n4nXa7CuUDkl67dxftTcEqlR8oxBxO1mWUYcM2q/Od44AB2AYE4dImeKH6PnSdr0nlOPAKKUI2EKiCFkW/jgapqKz9BUUaXvicbM/hqvK4ClJ1XKo1V5utWd0QJre0t4249tQo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song No longer needed after we switched to per entry swap readhead policy. Signed-off-by: Kairui Song --- include/linux/swap.h | 1 - mm/swapfile.c | 11 ----------- 2 files changed, 12 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 64a37819a9b3..cc83fb884757 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -454,7 +454,6 @@ extern void free_pages_and_swap_cache(struct encoded_page **, int); /* linux/mm/swapfile.c */ extern atomic_long_t nr_swap_pages; extern long total_swap_pages; -extern atomic_t nr_rotate_swap; extern bool has_usable_swap(void); /* Swap 50% full? Release swapcache more aggressively.. */ diff --git a/mm/swapfile.c b/mm/swapfile.c index e15a6c464a38..01c3f53b6521 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -104,8 +104,6 @@ static DECLARE_WAIT_QUEUE_HEAD(proc_poll_wait); /* Activity counter to indicate that a swapon or swapoff has occurred */ static atomic_t proc_poll_event = ATOMIC_INIT(0); -atomic_t nr_rotate_swap = ATOMIC_INIT(0); - static struct swap_info_struct *swap_type_to_swap_info(int type) { if (type >= MAX_SWAPFILES) @@ -2486,9 +2484,6 @@ SYSCALL_DEFINE1(swapoff, const char __user *, specialfile) if (p->flags & SWP_CONTINUED) free_swap_count_continuations(p); - if (!p->bdev || !bdev_nonrot(p->bdev)) - atomic_dec(&nr_rotate_swap); - mutex_lock(&swapon_mutex); spin_lock(&swap_lock); spin_lock(&p->lock); @@ -2990,7 +2985,6 @@ SYSCALL_DEFINE2(swapon, const char __user *, specialfile, int, swap_flags) struct swap_cluster_info *cluster_info = NULL; struct page *page = NULL; struct inode *inode = NULL; - bool inced_nr_rotate_swap = false; if (swap_flags & ~SWAP_FLAGS_VALID) return -EINVAL; @@ -3112,9 +3106,6 @@ SYSCALL_DEFINE2(swapon, const char __user *, specialfile, int, swap_flags) cluster = per_cpu_ptr(p->percpu_cluster, cpu); cluster_set_null(&cluster->index); } - } else { - atomic_inc(&nr_rotate_swap); - inced_nr_rotate_swap = true; } error = swap_cgroup_swapon(p->type, maxpages); @@ -3218,8 +3209,6 @@ SYSCALL_DEFINE2(swapon, const char __user *, specialfile, int, swap_flags) spin_unlock(&swap_lock); vfree(swap_map); kvfree(cluster_info); - if (inced_nr_rotate_swap) - atomic_dec(&nr_rotate_swap); if (swap_file) filp_close(swap_file, NULL); out: From patchwork Sun Nov 19 19:47:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEC9CC54FB9 for ; Sun, 19 Nov 2023 19:48:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 35DC66B039B; Sun, 19 Nov 2023 14:48:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2E3F76B03A2; Sun, 19 Nov 2023 14:48:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 10F426B03A4; Sun, 19 Nov 2023 14:48:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id EFC0D6B039B for ; Sun, 19 Nov 2023 14:48:40 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id BB3901A0759 for ; Sun, 19 Nov 2023 19:48:40 +0000 (UTC) X-FDA: 81475741200.02.79956D0 Received: from mail-io1-f53.google.com (mail-io1-f53.google.com [209.85.166.53]) by imf05.hostedemail.com (Postfix) with ESMTP id EE1AF100012 for ; Sun, 19 Nov 2023 19:48:38 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=eHgxJ25a; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.166.53 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423319; a=rsa-sha256; cv=none; b=J50sVlzUq9X3GuSseTyaMCQStUWJAHB6PUDfFQnDd1WooC/hrcQjrHfH0QiLgTB3jj0Fd1 afWYTjBtI/BQQvOnJhINSuO4KEJteNOhe653P0zAd5dTwSP165rRVzHg750CTBlbMMoqPA wGzhZbUQYp7F1wElJ1aY3B4QrFtNR8M= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=eHgxJ25a; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.166.53 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423319; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Vy77DasHH4iZw3fxXV06AQqlBWBp94UgYk7CDc75jBU=; b=oRcXBUUmTRHE1Ae874EXve14L3KJYKvHbM2113w/53ripwUkQH4xm7N+94yYqJtzTxjhkF 3nS/hxQ/0mYmvG+S5QREu9I08fmDBdYxNh193rhx7nT+y1wKxTkbxwkRsazrLXkw6cJlBb Aw2YFJhd4esofwy5vGCuLN0jPlS9oBU= Received: by mail-io1-f53.google.com with SMTP id ca18e2360f4ac-7a93b7fedc8so175627339f.1 for ; Sun, 19 Nov 2023 11:48:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423317; x=1701028117; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=Vy77DasHH4iZw3fxXV06AQqlBWBp94UgYk7CDc75jBU=; b=eHgxJ25apAMvFwXXpg7QCq+SM6OMAmoU09XTEjVIojn4UgGGPbcb3wH55j1cWbqOuv zG/adDkUW21LHbLObE3BjB4E5PCBJ53MdyJOJKoI3h7oRsNB5Wc36KrraYPxjT5Gh/UV 4udxM3QtgCmwu9pIBAL5o05grvRleHceFRA7i3qMsQcHX+jJtaVcC9clumkWkbP4erlV GrYfLjQsYY6YNxkz8PFHqj4cEWJIDU/vevxwpOrPQMtwHZmM6qwCgOO9ExWWrP3WhjH3 dd/3TGZZhyP859pexsJ4cgqWks4mtH7Q3sAEC9LX3YDMI5DaEg3HZMyEaDNtqyP8/eUg Gyqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423317; x=1701028117; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Vy77DasHH4iZw3fxXV06AQqlBWBp94UgYk7CDc75jBU=; b=eiN8T4vSaCm5wy1Zsz9JptzbUaMAYSguVHyu5h51tW7N+GUWLc6/m5I7IyA1oFw6Jg U/gEHJCJPtYtNUH+Om4XzbJc+RcKd7ppKidrWg3jtdzQCS8GssjHVyGm3z9FIG0mSYv8 28eyRa9y7vSk94mTGErrKSkNgsRjsvAPdH19mXaI9eXcMViuRak6nNY1v5MMG7Z1NVfx fjiawn9h5X6hCje3RAgpGlPxxJ1wS4/NgcGph/ADt7kK9MT26BAGzOFqkDK9h2m5zenr PXYd44YPPoig6oiG037pPIYsSECa8BXRBDn0yjqn0QyLTR65SentWAe8l4aTWdpJaNYz 68mg== X-Gm-Message-State: AOJu0YwgVzcwbN9zx7YP0x6U00K3Yr++kvPujKdZBanDegSjVzZT/aUE 0aPORq2huDlZEGanj3bQz5MPvAj2z+gruoqy X-Google-Smtp-Source: AGHT+IECybsdgtqaEHEB6jhiCW9wgMllnD9l2cqLKV8BEgk8bfpyCeOF7+1eRGrvZhfSvcvWoreVgg== X-Received: by 2002:a92:cbc2:0:b0:35a:f493:5667 with SMTP id s2-20020a92cbc2000000b0035af4935667mr5462158ilq.20.1700423317470; Sun, 19 Nov 2023 11:48:37 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.34 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:36 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 11/24] mm/swap: also handle swapcache lookup in swapin_readahead Date: Mon, 20 Nov 2023 03:47:27 +0800 Message-ID: <20231119194740.94101-12-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: EE1AF100012 X-Stat-Signature: 79tj5judroewhe84ti1e13yka7w3mfib X-HE-Tag: 1700423318-909995 X-HE-Meta: U2FsdGVkX19mfzxLV1TD0xXPbfo7z7btQKOCgPKof/LfeBhJKn1ZtVOZ+lQXdAr/rbtfNP1RsaDZ9xApjbBQKFMklU3RjNKFkD/k8OsauWsuSjCZ8symPI9tnJ27HjFSdVMxDO5NzSYlqsok2vP+kMiiAQzAwSl+uq4mIhbFrmEEPj13PPshJ8FHJRfqfN0WXT5/tfBFfHrPE2vLPTZjvP05E1rI4F0XKdf3ZVjbeyihPiywjfhVi7tPy1ITDx589I6VUnEBnIMbpYkvQKcQG80anPtgtC0H0wBP7Joz5HlEyTYY8R6a5w4hKvz3mrYjlWRrVBCxUgCnD2kUBZT6S525dw7Uw5rONd+AcyJQh96vQC5yDOgb7tpZ8Xc/06GgM/79RtO81f6ydm8L7PSJ8+9G2HsfUNYk65l3TLp3kgjaooiCcsQ6F/CkRhfFD/8PjJUl+wpH6qexTVjLtsxg1999WR4GJ6GzHk00nQEg2XmvzrNxC+A3P04aWEG8ygn+0hLsfo811x13I9bToBjBopUYerdTt2j53815Dobs786VyjXN+kPYjZSdhJfJOeOBNqodScuK8t5CI+nebvRm1CmzPzAB2n2tTLk1uwrsuvD/MMYWQSfyiP9uAcfAGbasJw99k7BTH5TT/MfC6/HFdenEb4t2mb7RMZ8qdIERFPA/cQQ0BxCFSl45n7LC5alCHWgzj8HU6dGAMtdUpDDs3q9GjkhhBOlMabUtZGquEFtkhG2TEyZT2/vu/CTrNhq4q0wegnsyNNRTVm8FePU/EVaxp09nROoY5bfwMALzTIYloZa+KBzkHngze35cHdyZ/43J9vVJoTjmc8kwMLYBIucDNL6IH1P93Ik9HFm9uVj9h2W4x28xmbJDJDRG2K7oxOBjtwFmLF+iKwZhYzTgXBVQsXcGOb4yIqhWKuJeES1otT7Hfs+YkQexGQFi/N9v5yLpZp/dVRbKqeHsH6W wruTFg1s rFBE/g/Xxg1pC90pqNIiUJwd8QItwSLsj38ol52iXGG+5CHASrUFhERo/garR5Ajz4fqzl9XlWUHkw5tNgUnE7sCVCPRQCfd1eKKUNcPsbVR8YiGtXsSUgNX3+kATvn1JhZy+EYFXKM3xw8ge2jtij3xRtwo03ICeIcItpPrmuI5GMpy4wt4hjIAmwWSegFw9eFdRXhq4VmPcSHUn5GmQRMIk4meOdn4UPIHZaMEM7s7/1JNK5CjsRIvCa/DAOrkGX8MrVoLs/0SkTfkteHqYHZ2mVz39VG369WDqbOTp3Sf/GsTJe7wDW9qoO0jdiwE9Pk2q2aLJiKkWD0PbHlnjABB63BufVQ0K8uQBXqtrztXYSVcsdAl5o6m6aMGxpD2f/3+bomdQ43Mn0LZ26Cm7A1uSgpo7fpms85pEJw8WJTs6huJAMRmtvJAKUeqMvmsic0Bi2h56+PTFH1gBXvHGC/19e2jtk83GbhVoV1Sz9cuF5U/CICrtwojiL8lUBJovVeCi/LW0kGILvSkXMnie+YnuQ8dNuoXAc2fr+iVPnK7ZMlpwwkalBGDxmq1kGPRB4est8Xq+Bv30pNbJwteFrny9mDCWdyHOW2TVUE0C65EAZBSKo5I56I7S3kobSUKm38Ckhao3a7RO6V/VcUkI0cvXHM/sjnbc9TS3ZEw2Ql/FWQY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song No feature change, just prepare for later commits. Signed-off-by: Kairui Song --- mm/memory.c | 61 +++++++++++++++++++++++-------------------------- mm/swap.h | 10 ++++++-- mm/swap_state.c | 26 +++++++++++++-------- mm/swapfile.c | 30 +++++++++++------------- 4 files changed, 66 insertions(+), 61 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index f4237a2e3b93..22af9f3e8c75 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3786,13 +3786,13 @@ static vm_fault_t handle_pte_marker(struct vm_fault *vmf) vm_fault_t do_swap_page(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; - struct folio *swapcache, *folio = NULL; + struct folio *swapcache = NULL, *folio = NULL; + enum swap_cache_result cache_result; struct page *page; struct swap_info_struct *si = NULL; rmap_t rmap_flags = RMAP_NONE; bool exclusive = false; swp_entry_t entry; - bool swapcached; pte_t pte; vm_fault_t ret = 0; @@ -3850,42 +3850,37 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) if (unlikely(!si)) goto out; - folio = swap_cache_get_folio(entry, vma, vmf->address); - if (folio) - page = folio_file_page(folio, swp_offset(entry)); - swapcache = folio; - - if (!folio) { - page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, - vmf, &swapcached); - if (page) { - folio = page_folio(page); - if (swapcached) - swapcache = folio; - } else { + page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, + vmf, &cache_result); + if (page) { + folio = page_folio(page); + if (cache_result != SWAP_CACHE_HIT) { + /* Had to read the page from swap area: Major fault */ + ret = VM_FAULT_MAJOR; + count_vm_event(PGMAJFAULT); + count_memcg_event_mm(vma->vm_mm, PGMAJFAULT); + } + if (cache_result != SWAP_CACHE_BYPASS) + swapcache = folio; + if (PageHWPoison(page)) { /* - * Back out if somebody else faulted in this pte - * while we released the pte lock. + * hwpoisoned dirty swapcache pages are kept for killing + * owner processes (which may be unknown at hwpoison time) */ - vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, - vmf->address, &vmf->ptl); - if (likely(vmf->pte && - pte_same(ptep_get(vmf->pte), vmf->orig_pte))) - ret = VM_FAULT_OOM; - goto unlock; + ret = VM_FAULT_HWPOISON; + goto out_release; } - - /* Had to read the page from swap area: Major fault */ - ret = VM_FAULT_MAJOR; - count_vm_event(PGMAJFAULT); - count_memcg_event_mm(vma->vm_mm, PGMAJFAULT); - } else if (PageHWPoison(page)) { + } else { /* - * hwpoisoned dirty swapcache pages are kept for killing - * owner processes (which may be unknown at hwpoison time) + * Back out if somebody else faulted in this pte + * while we released the pte lock. */ - ret = VM_FAULT_HWPOISON; - goto out_release; + vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, + vmf->address, &vmf->ptl); + if (likely(vmf->pte && + pte_same(ptep_get(vmf->pte), vmf->orig_pte))) + ret = VM_FAULT_OOM; + goto unlock; } ret |= folio_lock_or_retry(folio, vmf); diff --git a/mm/swap.h b/mm/swap.h index a9a654af791e..ac9136eee690 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -30,6 +30,12 @@ extern struct address_space *swapper_spaces[]; (&swapper_spaces[swp_type(entry)][swp_offset(entry) \ >> SWAP_ADDRESS_SPACE_SHIFT]) +enum swap_cache_result { + SWAP_CACHE_HIT, + SWAP_CACHE_MISS, + SWAP_CACHE_BYPASS, +}; + void show_swap_cache_info(void); bool add_to_swap(struct folio *folio); void *get_shadow_from_swap_cache(swp_entry_t entry); @@ -55,7 +61,7 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t flag, struct mempolicy *mpol, pgoff_t ilx); struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, - struct vm_fault *vmf, bool *swapcached); + struct vm_fault *vmf, enum swap_cache_result *result); static inline unsigned int folio_swap_flags(struct folio *folio) { @@ -92,7 +98,7 @@ static inline struct page *swap_cluster_readahead(swp_entry_t entry, } static inline struct page *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask, - struct vm_fault *vmf, bool *swapcached) + struct vm_fault *vmf, enum swap_cache_result *result) { return NULL; } diff --git a/mm/swap_state.c b/mm/swap_state.c index d87c20f9f7ec..e96d63bf8a22 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -908,8 +908,7 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, * @entry: swap entry of this memory * @gfp_mask: memory allocation flags * @vmf: fault information - * @swapcached: pointer to a bool used as indicator if the - * page is swapped in through swapcache. + * @result: a return value to indicate swap cache usage. * * Returns the struct page for entry and addr, after queueing swapin. * @@ -918,30 +917,39 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, * or vma-based(ie, virtual address based on faulty address) readahead. */ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, - struct vm_fault *vmf, bool *swapcached) + struct vm_fault *vmf, enum swap_cache_result *result) { + enum swap_cache_result cache_result; struct swap_info_struct *si; struct mempolicy *mpol; + struct folio *folio; struct page *page; pgoff_t ilx; - bool cached; + + folio = swap_cache_get_folio(entry, vmf->vma, vmf->address); + if (folio) { + page = folio_file_page(folio, swp_offset(entry)); + cache_result = SWAP_CACHE_HIT; + goto done; + } si = swp_swap_info(entry); mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); if (swap_use_no_readahead(si, swp_offset(entry))) { page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, vmf->vma->vm_mm); - cached = false; + cache_result = SWAP_CACHE_BYPASS; } else if (swap_use_vma_readahead(si)) { page = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); - cached = true; + cache_result = SWAP_CACHE_MISS; } else { page = swap_cluster_readahead(entry, gfp_mask, mpol, ilx); - cached = true; + cache_result = SWAP_CACHE_MISS; } mpol_cond_put(mpol); - if (swapcached) - *swapcached = cached; +done: + if (result) + *result = cache_result; return page; } diff --git a/mm/swapfile.c b/mm/swapfile.c index 01c3f53b6521..b6d57fff5e21 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1822,13 +1822,21 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, si = swap_info[type]; do { - struct folio *folio; + struct page *page; unsigned long offset; unsigned char swp_count; + struct folio *folio = NULL; swp_entry_t entry; int ret; pte_t ptent; + struct vm_fault vmf = { + .vma = vma, + .address = addr, + .real_address = addr, + .pmd = pmd, + }; + if (!pte++) { pte = pte_offset_map(pmd, addr); if (!pte) @@ -1847,22 +1855,10 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, offset = swp_offset(entry); pte_unmap(pte); pte = NULL; - - folio = swap_cache_get_folio(entry, vma, addr); - if (!folio) { - struct page *page; - struct vm_fault vmf = { - .vma = vma, - .address = addr, - .real_address = addr, - .pmd = pmd, - }; - - page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, - &vmf, NULL); - if (page) - folio = page_folio(page); - } + page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, + &vmf, NULL); + if (page) + folio = page_folio(page); if (!folio) { /* * The entry could have been freed, and will not From patchwork Sun Nov 19 19:47:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460668 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6ABB3C54FB9 for ; Sun, 19 Nov 2023 19:48:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EF5046B03A2; Sun, 19 Nov 2023 14:48:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EA2B56B03A7; Sun, 19 Nov 2023 14:48:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D1DC46B03AD; Sun, 19 Nov 2023 14:48:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id BC6AC6B03A2 for ; Sun, 19 Nov 2023 14:48:44 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 9E79B140372 for ; Sun, 19 Nov 2023 19:48:44 +0000 (UTC) X-FDA: 81475741368.11.EA5848C Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by imf27.hostedemail.com (Postfix) with ESMTP id D392D40007 for ; Sun, 19 Nov 2023 19:48:42 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=NWfze253; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf27.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.181 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423322; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ppA1yq5tSXPW7Cplga4R0sTrIzgEEoc6Te1dihtlR90=; b=Go8DyjSahnnj5dXchVs7TTf1P9+Ht2nRKvTqeEzqEGtUPCzqWJMCbhJAHejw0c/S7PSHJ+ n3ZWxMlWnf9b2UerfULXhcbtUOtw3qB//nDclp3bCPgKZgZ86meC8PsmSxOuT2AvPedccS QMnn2IidNqn9cg4Fq6NwEua1MYygbug= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=NWfze253; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf27.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.181 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423322; a=rsa-sha256; cv=none; b=kosNjcs71C4z6rLbx6vof9hzR/lX/baAV09AW8o6bVuO6gNwipCmwCxOiONf4VJtdUV7It yHsSuyXLCAYIHqgTzyegJ6RiguEmJ16yoIKNslCoe7XevbxsU7s2gvZRAXWh2bIZyT8pel i0aeVTKGctX+pneDefN8iLq2pLuzNo4= Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-6b1d1099a84so3605939b3a.1 for ; Sun, 19 Nov 2023 11:48:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423320; x=1701028120; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=ppA1yq5tSXPW7Cplga4R0sTrIzgEEoc6Te1dihtlR90=; b=NWfze253qK1uUkO/Vtlo8UYefEtw3Rakd9vCRzU/b71AV0jS2Fdn5ud7wcUCoW505t wkIWuR5sVNnup/yeJF3ynnaEmDHfolYMlrk3hBvoLABB+Xe3RpWTUfmHCJctjHNQAavJ HaccM84YbVXmspPqDJi3eoM9HFeZlsHCzo0j6GvYdQdsrtN9yx40DvWh2VM9ZgdlF2au Vq55fPkkZCHbmDJP0dvnudlsK1QBF84gV4sPvTb1TmTXP4XSVkZoS35BlG/UW+9ISPcG lgt2uUAW/ZUz8ivP1DociXxhDejQQDl4TN3CHoIzUqAzkrXkAi3IQ9DvK08yOjMfQQyY BhXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423320; x=1701028120; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ppA1yq5tSXPW7Cplga4R0sTrIzgEEoc6Te1dihtlR90=; b=htapcwKdCGUFINIe1HPEa4WculOawxqMAqhvflHinJi9cTGorg6peuhOyjsPYkwtfY R2Zzz0Y71RPvMF9xRK/fZ8PuDmgFUlHLJsQwjokO1kRUwS78nT/lsFTEJ24SMF936bjU qde9p4rHAJP21LTGNj1ySO3utc4WAFpzOsAtitb4v64/IqsC/UaCElX7SCVn1m+bQsys S7V5wjP9XM/ptntXxQiWdxZZO92iA6jB4mCgYNGhd3hbRVvMJor+/ePm2qPlRTmpyfFO bP8/CQWkEf8qoM9IZQKyoU5YjJ/+ilTnIxm63TbEiEf0kcn4WFI7enQ/ku5cu5OP68Ku yqLw== X-Gm-Message-State: AOJu0YxAXQe2+QBn18AwrKBqchaS5mRnwD5bfuTeUqe3jjLn3AfI1cx9 /JiLjemv1pbSR7hmqOG+WOVayjFOpIVyjCrE X-Google-Smtp-Source: AGHT+IEBKHSE8kWnJJK8rM1dJTQqf5O5zE26MX7FuQ337U8kKzHMmekWO97Hh8Tc9mKAekA3sDGJiw== X-Received: by 2002:a05:6a00:1ca9:b0:6cb:a434:b58f with SMTP id y41-20020a056a001ca900b006cba434b58fmr619405pfw.33.1700423320649; Sun, 19 Nov 2023 11:48:40 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.37 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:40 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 12/24] mm/swap: simplify arguments for swap_cache_get_folio Date: Mon, 20 Nov 2023 03:47:28 +0800 Message-ID: <20231119194740.94101-13-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: D392D40007 X-Stat-Signature: 4w83zan941f7gwh5gioxwou7sys68ndy X-HE-Tag: 1700423322-706899 X-HE-Meta: U2FsdGVkX1+06HIxZUgrI9BMwDDRfuej5Sl/bs//OjnTF7GogVHlMxkEjbQTdJEAuyQWWMaZ/xsTxLJB9VKCtF+NjTbL9Mlqf87WB265ntE22TaJQRypVbQJTbxpYpgYjn99lRODhyWD0wuXmS/i1029EY6TbD/5s+2w+rccAYONZdPSFFZPCq1yxskCA1MUIYZLEJFjO1e92my3WlfowwDYOJLWznZv7wLt51F5VXT5R4tacAIX6HqI1Y0utji1Yf4/bTrJOzQbrieYKrXiQ2b6mo+eRrUL17+o1AonyRV/a+5P+1jT6BriD0j+aiAggiJJ4XB5mPy2D0UiJOvHMYCpKmVd4S2gX5EKx+UWSHVYOgtF1lHpUKjBwSHeNJqqQE7lbIChb9nv1QOZG4chEdHgTIuY6hK5Ox5QOYnt0I93fT6y8kBc3++9weDGwqu41OytQ6EYdmlNPnXuLDPRd4s2NOYbygNpV0zD7jyC450nBz15MBI64Bpoj+xe0tR9K+DomOp0TazGhT/cpymCKkXxrwSBTiN/QKibXOQke5NK/lv9pLDToBsUUT2+foHjYQsDJ7ITZjFNKArxpwFk632bqGRR1cGYDqPxpKogCJAToyYSa0cehnPxqJk+K/uk6YMOmdQZZSsJGIUTN98hAkhYS8JX+iH39CQeOPqm0o8n8Mr2Ju3iWLKKwNn+4zWAgMDFZFesHgvBsP6Valt/zIK1QyhqVl9k1f8YSg3o+DSiu6tlEB3CzaKeER/yZnQ2s2D+djm6JGA0zhe26eH09uaoK6kKmENkrDaKXBfHjRPv6kEv3KOoQTzRhrWi0ixhw5KZBE1MSVfxfA+hH7B0MhEx92AH3YYBicQgqYP5NeVwyHSXvb4P5JXypHmhPEt2FYbAcAkuoLMFPNTbpNaT9Nv9bIoImzNRKBEGuhO7pQMVPA+fN+laCMUdnQv55X68eHrvQGCD6qVF6pkjir4 CKxyaZdG P2n1zS2bda3OKD26WWTQS2UXqpTe79MtyeDVXJXM1fcRicfT/xtcrmRoPvJxKeFKy0Zo1jWl3FoG6jmoPKVQYiKByVXyYAgOvpkv/JdpYAGltXbhxgVUJ1FW+cmbITznvkH2gwSFKgFvcU3krpt58Q+CZkQxVC2TW7XmPLS9LhPleITFH/7e+6DPcTFamtAQDZ2AEJL48ZyF8QRYuo2WkuqSGNeDPVpNLlmDUJUBMI65lP/Qz2VeWdcfx+XGoktVx4lHQei8F1jBDQ0jajI0JBZWIHUP4k2DIqMyW0z63V9OM77snNQLMBuZ52ka66ePZRRoIck64pDd3j4BEF7FuNN75ME/ocIb9bKpkS/E1JGXDyeJ7/Z/pLBHwsFCGYa2bXa9BBeZKIQgTYI7Ew/rSKcEcAtUrpsG8O7IaU+LQR1rfr929dn13Ngd8W0HxX9l1RHM7xCjsTw8DyWNirbTenPS61HdzPlmHjIhlYYY8C824dMhSOkmvHZJlOlJOgheyAnTa5zXVhg0z4D3xuamGUskSWsRMPH/QLPUEPCqXvdC78L8FFpcERZ06WOdDtx/64cMiUU6nzj0M7RFE8b44W/WZamKa9Ib0ZTLgAgay0lPjyP+8pbMK8QKyvJ0ZdbZW1J/ZOg2v7syb4z+eHMfRRdFOQF4zbhwoCw6zF2ZCtfSmU7E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song There are only two caller now, simplify the arguments. Signed-off-by: Kairui Song --- mm/shmem.c | 2 +- mm/swap.h | 2 +- mm/swap_state.c | 15 +++++++-------- 3 files changed, 9 insertions(+), 10 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 0d1ce70bce38..72239061c655 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1875,7 +1875,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, } /* Look it up and read it in.. */ - folio = swap_cache_get_folio(swap, NULL, 0); + folio = swap_cache_get_folio(swap, NULL); if (!folio) { /* Or update major stats only when swapin succeeds?? */ if (fault_type) { diff --git a/mm/swap.h b/mm/swap.h index ac9136eee690..e43e965f123f 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -47,7 +47,7 @@ void delete_from_swap_cache(struct folio *folio); void clear_shadow_from_swap_cache(int type, unsigned long begin, unsigned long end); struct folio *swap_cache_get_folio(swp_entry_t entry, - struct vm_area_struct *vma, unsigned long addr); + struct vm_fault *vmf); struct folio *filemap_get_incore_folio(struct address_space *mapping, pgoff_t index); diff --git a/mm/swap_state.c b/mm/swap_state.c index e96d63bf8a22..91461e26a8cc 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -334,8 +334,7 @@ static inline bool swap_use_vma_readahead(struct swap_info_struct *si) * * Caller must lock the swap device or hold a reference to keep it valid. */ -struct folio *swap_cache_get_folio(swp_entry_t entry, - struct vm_area_struct *vma, unsigned long addr) +struct folio *swap_cache_get_folio(swp_entry_t entry, struct vm_fault *vmf) { struct folio *folio; @@ -352,22 +351,22 @@ struct folio *swap_cache_get_folio(swp_entry_t entry, return folio; readahead = folio_test_clear_readahead(folio); - if (vma && vma_ra) { + if (vmf && vma_ra) { unsigned long ra_val; int win, hits; - ra_val = GET_SWAP_RA_VAL(vma); + ra_val = GET_SWAP_RA_VAL(vmf->vma); win = SWAP_RA_WIN(ra_val); hits = SWAP_RA_HITS(ra_val); if (readahead) hits = min_t(int, hits + 1, SWAP_RA_HITS_MAX); - atomic_long_set(&vma->swap_readahead_info, - SWAP_RA_VAL(addr, win, hits)); + atomic_long_set(&vmf->vma->swap_readahead_info, + SWAP_RA_VAL(vmf->address, win, hits)); } if (readahead) { count_vm_event(SWAP_RA_HIT); - if (!vma || !vma_ra) + if (!vmf || !vma_ra) atomic_inc(&swapin_readahead_hits); } } else { @@ -926,7 +925,7 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, struct page *page; pgoff_t ilx; - folio = swap_cache_get_folio(entry, vmf->vma, vmf->address); + folio = swap_cache_get_folio(entry, vmf); if (folio) { page = folio_file_page(folio, swp_offset(entry)); cache_result = SWAP_CACHE_HIT; From patchwork Sun Nov 19 19:47:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED4A4C072A2 for ; Sun, 19 Nov 2023 19:48:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7CC236B03A7; Sun, 19 Nov 2023 14:48:47 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 729036B03B0; Sun, 19 Nov 2023 14:48:47 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5567E6B03B1; Sun, 19 Nov 2023 14:48:47 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 410B66B03A7 for ; Sun, 19 Nov 2023 14:48:47 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1E7E11CB236 for ; Sun, 19 Nov 2023 19:48:47 +0000 (UTC) X-FDA: 81475741494.22.E65E18F Received: from mail-oi1-f180.google.com (mail-oi1-f180.google.com [209.85.167.180]) by imf22.hostedemail.com (Postfix) with ESMTP id 4FE11C001B for ; Sun, 19 Nov 2023 19:48:45 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Pu5CZpKo; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.167.180 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423325; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5hBTip8l+rVWgXZvgeAFfyX4NMH7GqXDBsHUYVWi3uA=; b=JCrHNN1cnGhYB2Vom8BrPL61jz4MJw8Ih9D376b8w4578jh8dFDEQhzkf5caK4EdJ32lVx 9Y+v7PsXndLLCPEe7detv2Bnj4kpA8jKQt+z0UEaOM38oqTNI/LLxh4W2YW2kAA9jTKntD 7/2o5agVOjk3q4LdaQrqLnz9+d9LVdQ= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Pu5CZpKo; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.167.180 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423325; a=rsa-sha256; cv=none; b=4feeOrP/GoYqIGZL/Hp47H0E9o6vZx5AjZ10wdde1OmkCNP4L7P587fNGsOCpqPybEUFDy dw1LrCbooXwLc/QNuuw+krsJ98uUxz+V3lnNQc8bW2qCJvP5HUbIyVlXIsu8zmg8i/zDAj 9hhTD8O9aEFGfLdH8937mCIe0m2W6j0= Received: by mail-oi1-f180.google.com with SMTP id 5614622812f47-3b6cb515917so2618926b6e.1 for ; Sun, 19 Nov 2023 11:48:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423324; x=1701028124; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=5hBTip8l+rVWgXZvgeAFfyX4NMH7GqXDBsHUYVWi3uA=; b=Pu5CZpKoZVmVvD6+nTbfGEPKYpWY0tf9vztZzPZVKCTbQ3WByAvb/xyVYid1WsyOhE KjkT/CKSOdeoqlyLoIWP1IJYrKg4GEITYUgLrrdMUOFSrWOxPv/hpzp1TYKIrw6a1rss k9c8ce1A/886i2WTeZRg3DfCVKtrlNsf2mvqzH662CBqk6YLLZ/AwgnIvb4pn+JIF0We 3rbPzyJAT7YZt1tT6LIHlO0edwUDTkuk2so57BZ1vU/2u+luL3Lt5UAXKpKg1iHuOs2L ZuO/lKUvS+5DsT1GkEQZ8Z5C5wKx4QgmdLnWyznmjX26VtBbQ7HgRV/jtMRY2KxndF4Q Da2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423324; x=1701028124; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=5hBTip8l+rVWgXZvgeAFfyX4NMH7GqXDBsHUYVWi3uA=; b=KhjplqRIW8v3+MaRDTG0JQMN6tsM//t1/SCmzz2jGM4O84HuMkpWe7zja+YYZfRZvX /XutQQiiVqrLqZL9EBvFCOjnj1GJY5cU9qjmH4kaOzR/nj0P9VCErpvMT8ZTAAK4jbDv /bAB8NjRJAK4fJfgUVyTlVxusQfbUHsw3d5qALPyq4yKPARzYDzMhfdEaPHM2US0ODc3 3bq3Hw8ydnf/fVAAjwyy+tVLYSNMZom9K21knKkrPa4kJyWtBwdP6an7Hej1MlS16A80 j8ojFce1V07fObkc33B1Ua5T5aZZ7d6PG6vSHLjP6YHbOqC5KK5UnJRChJnk07Rdtxs7 K9QA== X-Gm-Message-State: AOJu0YxRnk/pIpcQmoRlssKpWasps6GncAKv5C7uwdhiNIwbOiVes+Zt JtBtiU4y0NJ8c9vO2I1ZO7pgKx/AzI9l4gVs X-Google-Smtp-Source: AGHT+IEailrfhPaQZQEKYeEqmxWit7bC+nmAWDqCtHjGUNXtOgvTTA/qDqxaYEq3DlyE+hI13556Tg== X-Received: by 2002:a05:6808:3a10:b0:3ae:bae2:fa76 with SMTP id gr16-20020a0568083a1000b003aebae2fa76mr10473632oib.36.1700423323924; Sun, 19 Nov 2023 11:48:43 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.40 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:43 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 13/24] swap: simplify swap_cache_get_folio Date: Mon, 20 Nov 2023 03:47:29 +0800 Message-ID: <20231119194740.94101-14-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 4FE11C001B X-Stat-Signature: 8sbrk7w8mjb1346hrwphop5sg3q5mdpu X-HE-Tag: 1700423325-467313 X-HE-Meta: U2FsdGVkX18pnlissI5iy6v1dS5iIwhuqDzypkp5iOZrAyBUWL9SJrV0lYylHd3UiX7+S2is5rhdp+X8bK6dAxrMwL2q94BeKiLpBQbtCMQyEFp0eRjoLJ18NDiJOicr7XVkVIz6LD3WkW/JcDih/g+rz2s8lxV+7YzivCtHTf3P3N2m9lhzwFSWDtJHBEBgsTRFuDOWxAhkyRIIldgnIJ3t7DtW2nNVRoI64v8NreVIFtQ7WctZRoJE/wrpl3G8QU7HdWonYvR9e+Ej9CRRQxvP+Q9Tly6fXKnGaF+AsOs7CQwTOjWN+5Ku36JQkTd/RvK/QfvI7LljMrXOFbcZbTliBZqcFsOhJRaQVF4qbIRRfvt6VTu5QJzxhKqHNSzEdRuKuQgxehka2RcynIlYF+cbtafLASLTo9GjqFXplL+zlf7422Q/zA7BnnrK3wSUKcZVNd02AWkrczcohQsvkkF+ww20X3v1RIRaDVK+9KhSn3/dEpOISAHD6CId90lZ2Sc0GO/TQgyuvipPi4C4LiLa3/EGrTeH/Iy83p5BczxxzaKoo+Py41xFzxogWQcIf9E9TQjmqr3dmtzjbDaGJfZAW+/TzCn3jhcAFSr4mq94oj/wQ71lXeGyfCx2/szkc3frXRfYyqLnC1mWCNJ+3szCwQ99JgZfDzO7TE2NHGlh3M1F5M7z1wSlu2GKFA15qiTmCQ8XiVs6C2o36nAMgJESZMvIbERVEpZIUZMOFOT3Cmx3gyV223PkXxcV2xp/Z5ywo2TLm9d8TsOr17MKrxwMYWwQgDwuZy1SR/Ow/e0+k1jdoH9SW+Wd9b4It0NFjtiuabQIKGqpSowi9zrbTc5YAwcQR044YgvuqakrpTdbufMx2WOvm8bZ9nQY0sDcHI2mvVHp/drQ2gcm4eIkvDCpJg0X4OvSl1SD/yweiLefNL0WgMlp0LRsjY3cFcLZCUGvfNg0PmSi51fIWkP lTAKHIT0 jXZbved5H2loqlXE6XlALMfnUkaq0GDBBhoc+C6rM8UO1+Ig1TiuC/hdFmgc2d1eh2GVU1PWeHrqPfyFsYbGJ182oOoyru5fmjYnZG2EBSHAxbo8YC1JhzKinYgCmny5WqPOEYQqRbq0WEd8eY6vEpXblwb5NbWMI/WPkklPIoqefAqS2eIfOCpBqe4xK751FtkVc15oqjkf5lNqOQlP/ua6eMHLhE6OSXfAD1jtUmCFqjSXlEcHnUffDxBaiH98WrGCDYkGnBUoCuvGgq/u4Kp2JKCI+e/yE6SW7zYDCLUxbProkmOs1rgWDAub9NwVqU6pj2m/dsaexnZvGdulCMmSH+PVTVnxzpH4DAmjUm5ZcxL6kH1X5q0tvMyOAiOldfT0jrDd2NGsGrucyLMW2T9NS+17AZ4fsWSfERCDQngf808fBcMQH6pFqNtNp0riprxZ0WZulEFGGuOQoT0tBynnjzHZ/xjKMxPTsYozWF73ADMVAEJe848Pvfi1rs8R+a+/OMNoKvWNmQ+Al53mD+bgmuFl4Z8rxE7A4QGeTzXS9ulvlv6UkQd0U6VCbWtZEeKwHdukMDCc8RwLvclX/5Di86auBesc0MCg97JyZgWp2ATzMq3LeumXsfI2AO7Cqrk6bImqs4L2xv2Tm2q/eWYTQPeSNEJQjFmisxU0/L/f1NXk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Rearrange the if statement, reduce the code indent, no feature change. Signed-off-by: Kairui Song --- mm/swap_state.c | 58 ++++++++++++++++++++++++------------------------- 1 file changed, 28 insertions(+), 30 deletions(-) diff --git a/mm/swap_state.c b/mm/swap_state.c index 91461e26a8cc..3b5a34f47192 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -336,41 +336,39 @@ static inline bool swap_use_vma_readahead(struct swap_info_struct *si) */ struct folio *swap_cache_get_folio(swp_entry_t entry, struct vm_fault *vmf) { + bool vma_ra, readahead; struct folio *folio; folio = filemap_get_folio(swap_address_space(entry), swp_offset(entry)); - if (!IS_ERR(folio)) { - bool vma_ra = swap_use_vma_readahead(swp_swap_info(entry)); - bool readahead; + if (IS_ERR(folio)) + return NULL; - /* - * At the moment, we don't support PG_readahead for anon THP - * so let's bail out rather than confusing the readahead stat. - */ - if (unlikely(folio_test_large(folio))) - return folio; - - readahead = folio_test_clear_readahead(folio); - if (vmf && vma_ra) { - unsigned long ra_val; - int win, hits; - - ra_val = GET_SWAP_RA_VAL(vmf->vma); - win = SWAP_RA_WIN(ra_val); - hits = SWAP_RA_HITS(ra_val); - if (readahead) - hits = min_t(int, hits + 1, SWAP_RA_HITS_MAX); - atomic_long_set(&vmf->vma->swap_readahead_info, - SWAP_RA_VAL(vmf->address, win, hits)); - } + /* + * At the moment, we don't support PG_readahead for anon THP + * so let's bail out rather than confusing the readahead stat. + */ + if (unlikely(folio_test_large(folio))) + return folio; - if (readahead) { - count_vm_event(SWAP_RA_HIT); - if (!vmf || !vma_ra) - atomic_inc(&swapin_readahead_hits); - } - } else { - folio = NULL; + vma_ra = swap_use_vma_readahead(swp_swap_info(entry)); + readahead = folio_test_clear_readahead(folio); + if (vmf && vma_ra) { + unsigned long ra_val; + int win, hits; + + ra_val = GET_SWAP_RA_VAL(vmf->vma); + win = SWAP_RA_WIN(ra_val); + hits = SWAP_RA_HITS(ra_val); + if (readahead) + hits = min_t(int, hits + 1, SWAP_RA_HITS_MAX); + atomic_long_set(&vmf->vma->swap_readahead_info, + SWAP_RA_VAL(vmf->address, win, hits)); + } + + if (readahead) { + count_vm_event(SWAP_RA_HIT); + if (!vmf || !vma_ra) + atomic_inc(&swapin_readahead_hits); } return folio; From patchwork Sun Nov 19 19:47:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460670 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5317EC072A2 for ; Sun, 19 Nov 2023 19:48:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D94A46B03B0; Sun, 19 Nov 2023 14:48:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D47876B03B2; Sun, 19 Nov 2023 14:48:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BBF4A6B03B4; Sun, 19 Nov 2023 14:48:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A28FA6B03B0 for ; Sun, 19 Nov 2023 14:48:50 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 814331604AB for ; Sun, 19 Nov 2023 19:48:50 +0000 (UTC) X-FDA: 81475741620.02.E96AF62 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by imf17.hostedemail.com (Postfix) with ESMTP id A533040015 for ; Sun, 19 Nov 2023 19:48:48 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=HMb+JZrX; spf=pass (imf17.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.181 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423328; a=rsa-sha256; cv=none; b=Yb6elLmDg7sivP8BjAytYy4XzN7Yf0MHE0CE7vPHqEOjMlovcwERis8HAk4Htef4hzI8KA nOcnLCgw/JgmmbpXLqoBn84rcO7dWldRFCx5DC7hxpgf/IQbttfzEXGsO1w76xVJZxZq/w vDfF0Fb6PyBwpN2RSUcS20UJ0g8kR48= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=HMb+JZrX; spf=pass (imf17.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.181 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423328; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VCznjXM2MgFt7+7dcBajlWUiOZC/KHYcLFwrnZNwV5s=; b=pjPVpe0c/2YaLNs92emheMZHVFJOYu0pKW4B60fr1g3F/of645D4CODswBw/HhB4J8nFah dLxwgamWaUzm5MuwFqHCyz5phING9VKGZy158dy8RHsUf5MRQpQT8T9Gk81V4Mb3ScArru wc0FaxCM19ZYNanQHNRacqYkHNzt4Bg= Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-6b87c1edfd5so2937469b3a.1 for ; Sun, 19 Nov 2023 11:48:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423327; x=1701028127; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=VCznjXM2MgFt7+7dcBajlWUiOZC/KHYcLFwrnZNwV5s=; b=HMb+JZrXF3g+4usYeBAhQzCqYrX287F4RNuntWEMs9YgikCIWXO7xRKmzVZCoWgyUo zWQ+gLWg7XE0YxM+H0AkN6zo8JSvyDLbGHbhfEHjl7/Cif2tmiP3GfXJNgawbumomRyD QgyNpz4GKIxGCesDxV3V4dVkqNMfqdWpHdc082+zrKCsRdllaJJo3/T5a9XWMWZPMGRt nTJFXXnvyE1NU5OU4JfkToXDdv2TBJfnlI95/qvkY+f6Qp9fMZuKcnJRbCXyzyLfj9CF JZav1BmeEu5ci0kPhGrEr5BVA8YlWRZYoxx2PpmENdNXFflrCXpnwu0akTh/2sw8AxKn 6pCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423327; x=1701028127; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=VCznjXM2MgFt7+7dcBajlWUiOZC/KHYcLFwrnZNwV5s=; b=pWA1Jkfnqk9BgvM4k9+AguIqL5a7wY2lnn8ASlA/PVk5HdwbJCeh/Rdi4tBQb5YevH BygdDTWejxeNcMm6P98rlFgfbbJ3c/qFUHZeL352WkMq+M8gdN2O6BM6pbefoo9DzoCb Ik058Y1bxy/d5/MvK99VTMM132BBdAAXYcimXcXa747ziHlPNsUv8EsnWWsAGKwLvqjE G5G4uiatjG1tDAkZNL+aMv9Dc5RKE5lq0NZCY65cz9FvdmN84/BQU6N36VCRzj0LJJuH XFGJbtMJ6gsMYXNJ49rt10FBSnry76fg/Cxa/3F7OwvzcPHvjR/fQVeRtd2wOIqiCrVq eVqA== X-Gm-Message-State: AOJu0YxNUKsN6D5VcSGLfnsSReffVBJ8QCl/7Uyu268dcHTQfETyFeW3 e7xxcZwPE6ZZM1CArjstHtFvmSm7uiIVqor3 X-Google-Smtp-Source: AGHT+IGzuAUevQ1XHN3DEflD4nhrPqC3JAr2xo9JJHGah1/1WAAz1GSa8PG8e1ODtV8IMaqzQtkuzA== X-Received: by 2002:a05:6a20:6a1a:b0:189:df1b:6616 with SMTP id p26-20020a056a206a1a00b00189df1b6616mr3107739pzk.15.1700423327040; Sun, 19 Nov 2023 11:48:47 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.44 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:46 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 14/24] mm/swap: do shadow lookup as well when doing swap cache lookup Date: Mon, 20 Nov 2023 03:47:30 +0800 Message-ID: <20231119194740.94101-15-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: A533040015 X-Stat-Signature: xrx15ighfwztuh1gtw9c989f1etmd9xj X-Rspam-User: X-HE-Tag: 1700423328-29164 X-HE-Meta: U2FsdGVkX1+LvPGWWRt+ToAake2Wauxhqnwkr1XPzDIsM+espNzsjLl9/hhxxPSuoLcBcuJ3QuE4WwTIvYf++DWRxcOaubFSuavA6KMIwxEtf+UwBxVi3+xnVcoQdVSMJa0wqO5oUJW0+18R5xg5tsfummzOvG4NXNAFqEmbxqVVuVWYp7zLb38nP97Y7HeRPjAG8UuuUt7Vb6T01m/9XhK3hKkyriZLCimViSWs0GmQPUs6SnO89wpinQ7hY8tWXHTatlG/Dkv5bPlcCXxW+GWPWLCuxJtwLe4UgFnkggOVC/PqCcEMgiZ0RsjH1gRV3usfDPMPB07OmpieZcYvKd/ZQHJEfSh6RM3XsDI5lkTTs4EcSvtqLxhfF3m/LhRwTtm0LhK+sMEjWjJ51jFL4NhVs2LI1ETu4oyLYEkwoowFepFaPCSQuRXqNPmpkjPrEN50iuQClzgHYqZNlBn7wI4ApzwG13eypb63O7l1AHLgu7Ddc5Xzj9pGpiDYyIXqgP6O51oeFyG4Y47UdJx6qIYhVyPQXdI3+96YMCntkyEPoot8GQzs1t88Kx46jTjy+dwrnKaRL2YRiKgsd4ypxjL5AuO0znTVb+7NMKqdBFhH05NlRnyE3pavRdaTwuE9HGFYdChgOMfBg4BixGZtCIfFsWhYJrgPFzHX8G3QGS1cEt4cSvq0Pen/xWirl8JFXpT4GjtEmrZB6UGbtiOGDGhxpCgWzp/lzEXkJxZDoBXsOg7sLrzWVBrItAjSJoSGnwXgjgsAQCzsDapWkqD5248YTLPMoVHG0seKlrkld5Y1Bd35dGdmPQAfF9hspv4cb7SKE3sySbotOLSZZ6mtj385RC9ZAcFSL15i+Ecuy8o1Q9/vNm66x826zvHA7b0s251ssIjddu/QK0NIfrzjbGg6TLTIWgfmxkRuIitfGAMJqZjR0rGUY1mLDmx7ruORrkQsc41azj7dbGHUoHY OUV6CXFf p3LXNgCZLTNK9IHhPILiu6FdVJr7Ww5jpnOheJW6hik/kQCwNrBM6SgFhgGwCSmLhOwEDX3zRcU8ahotc6Rq7CH8SQ1NokcPeVo1lgT8pYA4+9NJ6Q3sLMbHUGCB5aN9GKZkP0Eep4Cd8F5Vn509LxU8dcVZIzKz3rQ7wJr2V8SIeCgeMOEvAPIvs45Ptt3QxBZ3+ymiTNnU8/6mxlqQh7PmHBfiTVbSKY+aoaKPibiSsf+3CaXtWFMSpZgQ4ER8MtFjsz+E8yUWfq7kPqcOnUgYWXVFI/pI6U8qnxIYGpUrXlab+kjMLsXxDPQOnuSroJACuFOgfmD0EMVHs9WuljyTqR6dda9ByABvDXbN5fiwhhAtP2SQjJe7lwrx6JcZbz3nHgQT04l5w2lzgtrXpVT+xXjJqj+fO4HjsEdcxJJSbf/bnesK/J7dO1iLMxKTpznlHhL6H1V0ENEqAAtLVnj8aeYjg6Mro+TXl/pSXPuurG4xlkv+uRYkT9A0M6uARz6TEYW9mNhYMN8lbheFWw8vmcphb2KwveTXsMOjKVVLuCXG9fYLFO257qOHED1tCdpXtdkLMFSpyyFbTebgnvB8rMfyZsdMrf1bG+7e4wZi7mRy+CgkkZnO9xp2uZnwLzYo/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Make swap_cache_get_folio capable of returning the shadow value when the xarray contains a shadow instead of a valid folio. Just extend the arguments to be used later. Signed-off-by: Kairui Song --- mm/shmem.c | 2 +- mm/swap.h | 2 +- mm/swap_state.c | 11 +++++++---- 3 files changed, 9 insertions(+), 6 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 72239061c655..f9ce4067c742 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1875,7 +1875,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, } /* Look it up and read it in.. */ - folio = swap_cache_get_folio(swap, NULL); + folio = swap_cache_get_folio(swap, NULL, NULL); if (!folio) { /* Or update major stats only when swapin succeeds?? */ if (fault_type) { diff --git a/mm/swap.h b/mm/swap.h index e43e965f123f..da9deb5ba37d 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -47,7 +47,7 @@ void delete_from_swap_cache(struct folio *folio); void clear_shadow_from_swap_cache(int type, unsigned long begin, unsigned long end); struct folio *swap_cache_get_folio(swp_entry_t entry, - struct vm_fault *vmf); + struct vm_fault *vmf, void **shadowp); struct folio *filemap_get_incore_folio(struct address_space *mapping, pgoff_t index); diff --git a/mm/swap_state.c b/mm/swap_state.c index 3b5a34f47192..e057c79fb06f 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -334,14 +334,17 @@ static inline bool swap_use_vma_readahead(struct swap_info_struct *si) * * Caller must lock the swap device or hold a reference to keep it valid. */ -struct folio *swap_cache_get_folio(swp_entry_t entry, struct vm_fault *vmf) +struct folio *swap_cache_get_folio(swp_entry_t entry, struct vm_fault *vmf, void **shadowp) { bool vma_ra, readahead; struct folio *folio; - folio = filemap_get_folio(swap_address_space(entry), swp_offset(entry)); - if (IS_ERR(folio)) + folio = filemap_get_entry(swap_address_space(entry), swp_offset(entry)); + if (xa_is_value(folio)) { + if (shadowp) + *shadowp = folio; return NULL; + } /* * At the moment, we don't support PG_readahead for anon THP @@ -923,7 +926,7 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, struct page *page; pgoff_t ilx; - folio = swap_cache_get_folio(entry, vmf); + folio = swap_cache_get_folio(entry, vmf, NULL); if (folio) { page = folio_file_page(folio, swp_offset(entry)); cache_result = SWAP_CACHE_HIT; From patchwork Sun Nov 19 19:47:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6AC1C072A2 for ; Sun, 19 Nov 2023 19:48:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5305C6B03B2; Sun, 19 Nov 2023 14:48:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E14B6B03B7; Sun, 19 Nov 2023 14:48:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 381516B03B8; Sun, 19 Nov 2023 14:48:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 21F976B03B2 for ; Sun, 19 Nov 2023 14:48:54 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id F0749C0857 for ; Sun, 19 Nov 2023 19:48:53 +0000 (UTC) X-FDA: 81475741746.12.7D8E8D9 Received: from mail-oi1-f181.google.com (mail-oi1-f181.google.com [209.85.167.181]) by imf25.hostedemail.com (Postfix) with ESMTP id 2A2D3A0017 for ; Sun, 19 Nov 2023 19:48:52 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=i3P9BU+B; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf25.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.167.181 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423332; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aDkLnQICnVgpq4/orPONBtwFkKKcmzZtjb3XACAs/tw=; b=kr2TA+DKCF2iLdT5+eF8WH5hY+lvBroS/rNImEDz7bVLyJNsSEXT+aavZZmZ3o5dw8jDiS yagMR1CAEwOQDBmOsEEDMC9vr40mduB0vLn6QUnc2MMlw0UW8QLk7p7S2ReYewIyP3nmir CTsxI+xGTkHO0OkRtE8K1m7B4DGMdG0= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=i3P9BU+B; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf25.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.167.181 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423332; a=rsa-sha256; cv=none; b=Ejamdyq62or8U3+MK7nTlhnFUTN0Gagpv6RLnjIgvntAjfXyLVIYCpA7p6biTXTDZGvGXX kQ7VKrR5JxRYyB8M0FqGDkMLbtD9nEgutan8Zo40r+3Jjt0KeTGGX7PMs9kksCB06XAH+Q A5I6tDS0QlL/0zJiefKjPOcyciOln8c= Received: by mail-oi1-f181.google.com with SMTP id 5614622812f47-3b82541da51so247147b6e.0 for ; Sun, 19 Nov 2023 11:48:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423330; x=1701028130; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=aDkLnQICnVgpq4/orPONBtwFkKKcmzZtjb3XACAs/tw=; b=i3P9BU+BbmO98WEYdtjSb647H5HHPK6t73M4UzpjEyQOnxOI4w4HIC/ySFVUmIVie8 LcyFMsD4HaoadIC9gYULjkaH15NoC0GyfR2Z09rYkyNIC6/PxJg4WSkVsMx9k9EEFBmH 4frNhqN8KLXwq1XhEmvEm92mrTY50aL/0wXrKRBwLcbs7lqkpFKNNla+DqiUD7MwnHk0 w3xFXrFPmphI1baCxrRwvLIYFwKuBxzb9q14nyV65+0idjK4gM3LGaC/9L0Ufw7bLNYN /A/z0b0gC4apLzK5/I393UMqDaKanaVnpPqCrL6m+LKOIy1QmUOXnhEDfqUmen5s3JuH 1VVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423330; x=1701028130; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=aDkLnQICnVgpq4/orPONBtwFkKKcmzZtjb3XACAs/tw=; b=XnqKIOXJAMof4cHQdGT63yctkEAYQK1MbG7h/9c/qq6ZxibNAce5Y16aIC6lExNgwE wCWdIac2XQFOyFk9wDqIKjrS9wCsL3F8rkH2x3Ij8d7YvW8NA9X7S1xzEr64Qp7b36bg k4dBMu1AckHYFoLNK7BRg077HIgzXBqMnlpK4clQKUd7UGghDJy/owdoLsaVGyb9WzP4 NgHwLAfOOEW/Wn1ilym0CfWbWAcLCr3BsD2S82JvqWUHZ9zw+bdfLz0xRHINTFEff9eu +BNMmBP1F4MFaw6jnAxrEexad0tDfT6CV3eVou3tBpShWTUn7/CgqiNtZ9Z5gmpV3PMC 6pSg== X-Gm-Message-State: AOJu0Ywd+NPj5pjAJspy9Kg8yPIgEBV2HT4dUlRcP6xNw0x47jMqt9D+ GgfeOHIPgi45hrlmmTIdntDfZJUd9NmG+t8s X-Google-Smtp-Source: AGHT+IHnf4WdU+15DFAUt+1f+EUXMBC9sDjxOomJ4RYRSV1AA/+YKlySoWIJz9+C0DlwqbVhK6dXxQ== X-Received: by 2002:a05:6808:1452:b0:3af:983a:8129 with SMTP id x18-20020a056808145200b003af983a8129mr7743023oiv.53.1700423330257; Sun, 19 Nov 2023 11:48:50 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.47 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:49 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 15/24] mm/swap: avoid an duplicated swap cache lookup for SYNCHRONOUS_IO device Date: Mon, 20 Nov 2023 03:47:31 +0800 Message-ID: <20231119194740.94101-16-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: 2A2D3A0017 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: fyf5d8qmgskufczwmkyj885fw77s4j9o X-HE-Tag: 1700423331-404001 X-HE-Meta: U2FsdGVkX1+uW7pmOU40dSfISjoJ5mC5atf7bhJvvh0+wvxPu+1UxRe3pDa1CzdeueR6Errz7DDdGXUcI9pmmKpYbR6RjLVq8jScSZZgy7F+tY+oFu6ypwF67lhTheXIMG/FFjp3wVlJfj72uA6SFtNNAosh6iFpgGmy4punLPw/d8GDGivdg+rpvcE/IpF5VDEvQ4TODQUy1j33UbEOuHkF7XjKQgtC23WxTPSBn7aYLqbmFInlPQhndpKGOBLO0Q7jIGg4smLOMW5vH8uf8UQ0nU1tJksr+BLkwuvsPfKNwMA73PS/rJtlbZDdevipA+DDs+eVoS27/vESbXfEwR179jWwYiIbmwX7kQHjwmqvObi+agaC7bQXNX1olb4LVo8TTL+9dZrQzt6s2NTh+tUC47GvxizRe11DCQ9JrXnDoNv55MVQYdGatgK3swLIuzuUARf8fsfwe8OYB/+EOdvDbCwhn0H6aYReechVzK3oE+a+hmhkcWPdof4SqF2zYIghyHoOux4zZI/gyyBGWND8jJ2hll18+VHhta87WAZOJTEAXGUNSon3lkkrI2FxUse4mX6cDvYAsVbbhRlU4XR8HIW910bQe8+R0w1Si0xi2QXLS18ecoK3PNrvonerrJxkWzRRk7Rwi1xEEapBFn+GXAZwk0FrYa6JRV2/w01FVqvEY7bsRgannwPokQeMjL/W07xVbMhlJR/Fgp9a9xy166XGnhlvA1Z2hiSYyyVtJvXyESEuuQrgpmcDiA6/jmZNf7xnldPc0XQgKWfbJmg8DUU8qT0tMV/SuAyzoC04afrUNLUYqNIV+T3vLUJq+R6/7fuyA9WS0HeLgygHi5CZ4SUVc0PlHJIIZEhjgqNrnq3tCkVJ4/Y+ihOOJAYTrw/9RvwXKMW9u33wXHsUKhzIGvqufXXE++w0nxohmiK4wqn7002zvvXONop6MkwJ8adoNNVJJKx0z/FR5P5 lQrt0snn TjWlNpg18Zb1uO5HxvEktD097mQoozM7CNxB1G6W5MEddx9JlLh22IntcTN9fwWdP1BGRW8TpoTZgemJbAf0Yub/FJ9HKVa2utWDs/zaHjkKnRFkhyoD2Hfy7RNdIaALD9/mUDPxIx9AAm5ZHGmsyxPoQpDrOmSNAJ7uO0w/U6h/4ivHC7V2Kdi/8J+vP9zWv+WYypp0W8F+69TRoykmr7EmuXZB10eTSSJS38YfNwOUgAwzOD8fYVC0DMDUjREG9K6K8opjAPRW1TaswwpPYHi1uNwfaxt1Sq76dRUd6UqITKTALWRq7SkG4DGcRSp6Xf5xhFnBGOqglgh/CDoEXsLZJ52S/8897g2UJaMq65e92ByclV0p5G7C5ma/l3wPUcdDwOFH4Wdyni82kaVTFQ1UT95wNamHZKhw4k2lfI9b8Z7uW8JIuD2D4BBGr+e55I6r/TpyLZ5BfkrgUkNKln3ahAhew1QMP8gHMKRjGbtYm8eNxLXwsl9P4zNbmPBxt2iTsG6Cdb9b4PnLC+XLCuLQvZEldyIQ/HhATcWgCESE/+hRjA4dQnOIkten7FKzUTN0cuNNvQJXqsoEYLlqPoKKcI7JYLeSAa3FtUAvqZ45AnV8ZBl1Q3TLVQp3/llAnt5qTxZeL0EdV7DKOORnqpVI0B/duXfxQbTFVBEMnHBj7Fto= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song When a xa_value is returned by the cache lookup, keep it to be used later for workingset refault check instead of doing the looking up again in swapin_no_readahead. This does have a side effect of making swapoff also triggers workingset check, but should be fine since swapoff does effect the workload in many ways already. Signed-off-by: Kairui Song --- mm/swap_state.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/mm/swap_state.c b/mm/swap_state.c index e057c79fb06f..51de2a0412df 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -872,7 +872,6 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, { struct folio *folio; struct page *page; - void *shadow = NULL; page = alloc_pages_mpol(gfp_mask, 0, mpol, ilx, numa_node_id()); folio = (struct folio *)page; @@ -888,10 +887,6 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, mem_cgroup_swapin_uncharge_swap(entry); - shadow = get_shadow_from_swap_cache(entry); - if (shadow) - workingset_refault(folio, shadow); - folio_add_lru(folio); /* To provide entry to swap_readpage() */ @@ -922,11 +917,12 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, enum swap_cache_result cache_result; struct swap_info_struct *si; struct mempolicy *mpol; + void *shadow = NULL; struct folio *folio; struct page *page; pgoff_t ilx; - folio = swap_cache_get_folio(entry, vmf, NULL); + folio = swap_cache_get_folio(entry, vmf, &shadow); if (folio) { page = folio_file_page(folio, swp_offset(entry)); cache_result = SWAP_CACHE_HIT; @@ -938,6 +934,8 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, if (swap_use_no_readahead(si, swp_offset(entry))) { page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, vmf->vma->vm_mm); cache_result = SWAP_CACHE_BYPASS; + if (shadow) + workingset_refault(page_folio(page), shadow); } else if (swap_use_vma_readahead(si)) { page = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); cache_result = SWAP_CACHE_MISS; From patchwork Sun Nov 19 19:47:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460672 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3FFCC072A2 for ; Sun, 19 Nov 2023 19:48:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 865BE6B03B7; Sun, 19 Nov 2023 14:48:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 813AF6B03B9; Sun, 19 Nov 2023 14:48:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 666556B03BA; Sun, 19 Nov 2023 14:48:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 523966B03B7 for ; Sun, 19 Nov 2023 14:48:57 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2EBBB1CB22E for ; Sun, 19 Nov 2023 19:48:57 +0000 (UTC) X-FDA: 81475741914.23.51ED7D3 Received: from mail-il1-f171.google.com (mail-il1-f171.google.com [209.85.166.171]) by imf07.hostedemail.com (Postfix) with ESMTP id 5B72B4000E for ; Sun, 19 Nov 2023 19:48:55 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=JqretcdF; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf07.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.166.171 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423335; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9XjpTi/p1jP45iq6KrQ/NXIY9Wo9hQ4YnGSNNoLyAOA=; b=D/X2c+iGLbdHg0i6ZttdRQgVDa5gwppvWhTt0XkCfq21wrupWjxWDb3RAq4Erj9f0eVyTh mKtMa5drz7qVt1aquELcdF3bXiq56X7r6u3V+wowM6AogfZ4FITIiBogo9xxINVJanCHRT pxCjlWyixjDvJLvXwrrTsHqweClPVU0= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=JqretcdF; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf07.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.166.171 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423335; a=rsa-sha256; cv=none; b=ybRQM/dxFNKOkMvR93lT+QSNEPKOQCMAGvKvM/9uHfMHp/4esSlXG2uncyZR3C/5780vjg V5/FLMwRDIS1WeBscdx2uxMFC+XWw+cqjkdPQ41kf+Je66v3UbM+EUQIX+SsltGrW8oXtF dj4TcbaQMzUNSVlqWx7xH2l/+Q2jZE0= Received: by mail-il1-f171.google.com with SMTP id e9e14a558f8ab-3574f99d236so14084335ab.2 for ; Sun, 19 Nov 2023 11:48:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423333; x=1701028133; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=9XjpTi/p1jP45iq6KrQ/NXIY9Wo9hQ4YnGSNNoLyAOA=; b=JqretcdFCbd26sLOZwSm9C947vWmoV4Hr03eI+coYAnOOQ9gw4wl9YXgonzVXkFPYY htW8l7Z9xNWTwSjv6RgKdzUsoFYdpT9Gc48A510Ff05fDtmR5LM0aXfSKPYrnPBmzG1T wnH6cmOzpU+Yc56WaL6jtGWQ8atrJyPjb4Oo20fjpWKSbiwTLUMosFi9GugbvgYWbUB7 4v8oeNkSIwlpvlP7WzI0sbba1lHLYt6aIcy10bjPChdKTDAcsNOmYEAM1PPr+7ka+NkF ux8RrYZXbJJvvf81fx06WvzXYaZeVu61WH/Kt/fN0WCPRIQmsmM1gWTX6usiTPrYiYJO EBlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423333; x=1701028133; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=9XjpTi/p1jP45iq6KrQ/NXIY9Wo9hQ4YnGSNNoLyAOA=; b=DY5MleMDRgV3dM/A/JX9lMbXPI/hVxIHT80QoItsi1FhtibxrOfpyw+5//SQcS4Wdf /MqX7Tdtyi4Tn3HU7npTNFjd2Jz2FCTcAkFEKP3fy04aIiW/RHYnpzu8OLNF1HrArgpc ZwZortYIz6oihYJSzBp1Npz2dR0eojOTq+pF+BrB3DbwZ7ZGA+ju+YbRJQRyYDN5nPjW 6ifZK3esQKDDVBDydLuAPLg8GsUdnNNNEg84etZev79l+9t5zEXlRb3bLtHlZbEr5nqg V+71YRFDrq435sXNcOtEDgNbNe5kKgAnfz8YDv6o490KSPz6iV/m+NQjPGgfnzN8VF+J pL6Q== X-Gm-Message-State: AOJu0Yy6lvn/gBwVNIcccPn+7m2bgrPi8daM5HvRVVMBhqBGatnHvaN+ k0+Yd2WEDuD+DiCEsxS0YLl02YrMqC9oxssD X-Google-Smtp-Source: AGHT+IFar6VHQuFklPrfy7vOCI8aqHj+K8WskagViLl4BbbPZwTP7uc43B3VnuoR6l/B/N6mqJ6dzw== X-Received: by 2002:a92:7a06:0:b0:359:5389:c0dc with SMTP id v6-20020a927a06000000b003595389c0dcmr6674915ilc.7.1700423333536; Sun, 19 Nov 2023 11:48:53 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.50 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:52 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 16/24] mm/swap: reduce scope of get_swap_device in swapin path Date: Mon, 20 Nov 2023 03:47:32 +0800 Message-ID: <20231119194740.94101-17-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: 5B72B4000E X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: p79ui1urz1ghdwkjzb1kney8hhqpw8yc X-HE-Tag: 1700423335-293519 X-HE-Meta: U2FsdGVkX1/1lYOhELNETQulmwvHKhcN5cCC2g44XQXutCfbApJ9qPk2G/IRw2F99MSJFGRAbwwuM9FIZ/H8W5SzUheexvfkYB/jhO1UL1lI5MrdB0fsDCxguxJU0bo2Vyz/GXa6IGHiKIt7NbZL6+SlNkxeIiPRS2apGrdVgaPiF21upydbiuJbzzoyKft46hnPEDUNtdLvfl/3rSSM630zrhdx1/t7BbRSH7/qoQuUKdOPsM2bGHOrB/VaZv5eMpXmEUGfQWWLSnULJXa5HwroJFuPlnauM7yHgVVcpV3GFXv42+F0SlS/H5EC5iahEYaTTZaOfyrloNGdGY9pRuYEfUPOEZBmM3sIxAsMQHIOUUI/wm5b5oV60RjqqW7GBrk41Q8DeNcEslItgH1aMRjXli9AL1kKyJ4KZyDAQ8XV3jy4pmo2VJu6POLwHKih77d2x/QBVVnkljMWWBiBTE5YghTwmS9LCAwLfmNR+ZbLa4D5Da3uFq0Yp5QWxrpqCwh1F/i1iP6mm4bp3jGu9dhrwDuV5Dp47n8EvesjdlqsO+cI/Cx4KSShCwr5tqwDhvmT5MqoIzq7wcm/wpHUYn/TXgnhq+E1gFId1tVRrAeTCzGF2jWv3WQZagiG5wKZ4AIkDvyNqe6qvLBdZBEttdImZ09vKesy92XRQVH+S2SvnBu3M0nrOZVdvQ6HV7zBtmS93Jo/Hh9ZNValxjMIQ/wEUI8M303EdDyc0DFxvN5G4pgP8ITw+XIl9SirGPl/HM26Z2VwezGeS9Fg3czgZLrkDXlldcZogMRNm+IFB+paUV0swBRVlhU5nAgxgMVFHh9DaSFjgrHWY5EI6eFyEKc0FIN2NOMgwNQctQnU43H3EO2rHfRQrQI37pO65hQyNTBVjO7ve5sThU//r/2lu8IMYTURuPhTnyeLOXFdn3/XyeHwpsCmTNrnBqIsr9HB1EuZB4RHAREQkLc/ufp rmVkHpL5 AcwmLo4P6cHq4vRZcCTXRL8wkdEzNcwcZaoaNhM3I3WK3TTMcfQ3isIsJtkqIlf3W7msTiFTvUKSRP5cY/oXzY0Gae1T5BzdlkpJ/k8xMVU0m/XU9ovP5goCp+jXTrm2LJGi1X+Cp1nE8Hh8tjJ+1jZcfz0RKB5zvqwsVRhGGvKrEUb7qbMr/pmIUF5lOgWLqLOyvUYxqWg5+5NL5bHZOd6OoByenWtwVIhzNo56bSwF9rkLZijGhx7u+D6wtpkmBhvTl424ElMibSjwmDcVtOJoboKH/RsIpIJSWHehrYEym/B27GAOiXKaPTv/4nfl2bSJ7ufocrPlS3ptMb4/6JiEbYQQ61BxB9IAtUd3Qm619AQKuu561Q1LoVyltSCAoIASM8UEZt/dBTAikTOgxkHvlLMl6+iEFHCo0uqZ33O0sfDgm2ZBGzPF1HjopOatM3zO7o/48Zaxj05jiMcNIoHFYYQyi7Qug84aFg3u7029343xKOKegF3kzTfGi1/6UodozUKLmJGg7U8+DSAFv8GY0RXbSya371rvQWh9JFhmwxlggohDHZ2/0TDF4Lf4WedRD4eo2UF3qYKwqaxwimc22Zxhv1yBTvGGFbbZX+4MNGx1tW1qW29fEIKy8+YdijFluRy/d6uiZyUcTOjNQ5zleoRP7XCwOZ3+FzIVAQs69N4E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Move get_swap_device into swapin_readahead, simplify the code and prepare for follow up commits. For the later part in do_swap_page, using swp_swap_info directly is fine since in that context, the swap device is pinned by swapcache reference. Signed-off-by: Kairui Song --- mm/memory.c | 16 ++++------------ mm/swap_state.c | 8 ++++++-- mm/swapfile.c | 4 +++- 3 files changed, 13 insertions(+), 15 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 22af9f3e8c75..e399b37ef395 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3789,7 +3789,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) struct folio *swapcache = NULL, *folio = NULL; enum swap_cache_result cache_result; struct page *page; - struct swap_info_struct *si = NULL; rmap_t rmap_flags = RMAP_NONE; bool exclusive = false; swp_entry_t entry; @@ -3845,14 +3844,11 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) goto out; } - /* Prevent swapoff from happening to us. */ - si = get_swap_device(entry); - if (unlikely(!si)) - goto out; - page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf, &cache_result); - if (page) { + if (PTR_ERR(page) == -EBUSY) { + goto out; + } else if (page) { folio = page_folio(page); if (cache_result != SWAP_CACHE_HIT) { /* Had to read the page from swap area: Major fault */ @@ -3964,7 +3960,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) */ exclusive = true; } else if (exclusive && folio_test_writeback(folio) && - data_race(si->flags & SWP_STABLE_WRITES)) { + (swp_swap_info(entry)->flags & SWP_STABLE_WRITES)) { /* * This is tricky: not all swap backends support * concurrent page modifications while under writeback. @@ -4068,8 +4064,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) if (vmf->pte) pte_unmap_unlock(vmf->pte, vmf->ptl); out: - if (si) - put_swap_device(si); return ret; out_nomap: if (vmf->pte) @@ -4082,8 +4076,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) folio_unlock(swapcache); folio_put(swapcache); } - if (si) - put_swap_device(si); return ret; } diff --git a/mm/swap_state.c b/mm/swap_state.c index 51de2a0412df..ff8a166603d0 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -922,6 +922,11 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, struct page *page; pgoff_t ilx; + /* Prevent swapoff from happening to us */ + si = get_swap_device(entry); + if (unlikely(!si)) + return ERR_PTR(-EBUSY); + folio = swap_cache_get_folio(entry, vmf, &shadow); if (folio) { page = folio_file_page(folio, swp_offset(entry)); @@ -929,7 +934,6 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, goto done; } - si = swp_swap_info(entry); mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); if (swap_use_no_readahead(si, swp_offset(entry))) { page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, vmf->vma->vm_mm); @@ -944,8 +948,8 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, cache_result = SWAP_CACHE_MISS; } mpol_cond_put(mpol); - done: + put_swap_device(si); if (result) *result = cache_result; diff --git a/mm/swapfile.c b/mm/swapfile.c index b6d57fff5e21..925ad92486a4 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1857,7 +1857,9 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, pte = NULL; page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, &vmf, NULL); - if (page) + if (IS_ERR(page)) + return PTR_ERR(page); + else if (page) folio = page_folio(page); if (!folio) { /* From patchwork Sun Nov 19 19:47:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8791AC072A2 for ; Sun, 19 Nov 2023 19:49:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 15FFB6B03B9; Sun, 19 Nov 2023 14:49:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 10E446B03BB; Sun, 19 Nov 2023 14:49:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF2406B03BE; Sun, 19 Nov 2023 14:49:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id DB8516B03B9 for ; Sun, 19 Nov 2023 14:49:00 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id BAF091604AB for ; Sun, 19 Nov 2023 19:49:00 +0000 (UTC) X-FDA: 81475742040.04.229F568 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by imf02.hostedemail.com (Postfix) with ESMTP id DB25780017 for ; Sun, 19 Nov 2023 19:48:58 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="gi+U4qw/"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423338; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FCPW0dI33gWre1bmPp4wG3A/kpp2Dtpn+fZQd/yP8/o=; b=YSN+97syxJcbrK+Azl06Ai36LHwPDaj+lcIHBojYDVSDSt8zH/FUnKIoMvQYlECAeIToTf ZDQHuLjZYqrSayBGUN98qYD2BTkMqupltoc+ZLWZLh6AK0v624Xzbef2yTzQjwbe2OsdzI 9XXjntFjMSdzG9+hrrnw02ONZd9XPJ4= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="gi+U4qw/"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423338; a=rsa-sha256; cv=none; b=CDQ1I8S3BCI+4TRZXnjQWtzGt5bqv8wiaRFcl6gdOFslBTLoFUrld0U8ijUOfKvJoAcZof 7F51oZ3q25O3tUkcVa+UWBQm4aYOnXnYZKZSPd1YiTVQmo0y2wdvQH4PMgN54hItpzpssO 1ElLIv7L8ia9QCq+VrrJlr7L10jZQEY= Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-6b2018a11efso3881502b3a.0 for ; Sun, 19 Nov 2023 11:48:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423337; x=1701028137; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=FCPW0dI33gWre1bmPp4wG3A/kpp2Dtpn+fZQd/yP8/o=; b=gi+U4qw/5tZAF53CaXVqhJ2hOviCi6bU7ZwkcSeE12bw1MsntwCxdS7VbEJBFnw6EZ xBSoB7OwjhQcg5muYNV/6RNszjU/V71cSmar6owKOaCvzcpu16HgHuxc7YLryas3/jww WOAw8Yx+H1vOMjVhqnhcl3ccCGuf66WgaZG4w7pqEnkM45q7irDiXgvv94vuPB2P6JTT ZHSvGY+qdQUmqMj4buGyrNYeG8vc6JTCZz44UQsTMDfzKOxFJHtiMHcMRVF19mYmTulH +K12sRlobTkWqeBR2qsEFnUVoBPpEJLlCCfkb+0qPv/DzdOX/mCR7a6nNHJOFBrUi1UL +hkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423337; x=1701028137; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=FCPW0dI33gWre1bmPp4wG3A/kpp2Dtpn+fZQd/yP8/o=; b=gkTRjWaHCHF+1w65i8vHtSnQx5rnWqpa3PHBZIEf+/Xv+txfTq7RC0wJRM6WeaCyWj fCtk5rAkX5fdaihqBK98mg9DDo33zEgSbKOwxOqJyK0Dy1DT5mLC9YPKIZ2ePeDFjIsE gBjb6BudNx7ru0COR7TkYPTn5+j2rjZNih7kA/BGmrSGLyfk0WCHRZwKQO/18oZMpWu2 HBr+l207fy+onc0xB1WbKceNPHRvhlZdNi32NPcICtfm886xjoXA19pQVDY4dyKJiVji t4ZhHPNJVP9jffSGeB7Kv5s5Lx8ysKJ2hp/s7rzzC9VKK6kq0HprcXJXlXfwkKPPA3Ex 9JeA== X-Gm-Message-State: AOJu0YzNiOa6vJyknpexaZhSdo54EVaOb8FTQo7fCnKYekhFCN/BCOLa 50zPck9iAAHEncI29tj33wwcZKcNDGzoZhKB X-Google-Smtp-Source: AGHT+IF9QoAWouGwWEXuyCQ9NU0AgZDxQWALIyGpxr9BM9RgNBGPUMqyqjGgp7YBTc1CUFiZCcylkA== X-Received: by 2002:a05:6a20:244e:b0:187:f23d:f9f2 with SMTP id t14-20020a056a20244e00b00187f23df9f2mr6715469pzc.58.1700423336700; Sun, 19 Nov 2023 11:48:56 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.53 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:56 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 17/24] mm/swap: fix false error when swapoff race with swapin Date: Mon, 20 Nov 2023 03:47:33 +0800 Message-ID: <20231119194740.94101-18-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: DB25780017 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 377xcpi63umsiwngymsd5dyuo9x4phsk X-HE-Tag: 1700423338-955692 X-HE-Meta: U2FsdGVkX18yKcwAqGkRpj+4+YcOrmam+SGDKUN4CT8OqMmZuhAjAeO3wLR1qNLpMfYNRy0sHriQNweeNS9tsMXdoJzVSFasvWFFnn7vRuCX3KNDKPDCnjjGE+HtMfydvD2PuVFIbJpMEgoVr/F6o7/KZblqRfTMJK3DNEUK7vi1o8IorTP2cyVgoKqYnaDNucgFfXBc+AaDAPE07+pLjr6WaFVw4JTOkRTIGAN2f04awLSeKIa14+5sTHQuQtSDeofVYRCe4o0y9rXZIb8hpf7hoEcJwTzfvWPgrGmZTA5KJih/KS+L6gYxF+yXKusVMj+SpMsWwXbuoXEN/a4YA/kCdn1yxGH4u15k1Y4rMe8irAtgckS3lKBI3l+0WeBgp7ZdjJ/C+cyL5xNAag87Jx0iKzLBlukqu8aCBTbat2EcnWWZ1eVqjOasHblIcDRq0ysnvE5kJH99iRlQG1b0DUkdMKLfnRExJyDi5knPCbnovDZcuTYFOmwp99blajkse9RpkYafqIbxJCneNRaxAl0nzw028uXE9xXK3qY7vXc+Wu4yQAM0uIc4w5HiMmg88OV5LrZyHhhwNwrw7/0Bm7QtH+WzgQSLHMP4bduSPc/l4cl+3I8rLD/A8rFRxR6gNbdfgzggqC+l6J0p5f/Cr+2KowLTLfvG8gi5V8VSkkoZoC8KJNKos/t4SvdOQE0lx6OAg5JBAh/kn/1O5Wo7RReTq7btx0Jt9uiR2m8Vv9XeSDd6S38DTPQSL1fDVTLhk42iPJzK7bwif73QJtgkeulkarOClg7TUdelUD+bEFP7RFV2T/XEoVlMxxCdsaaPJ+lpleBpLjShL9CwrKzu2DDH7FQe9n/8SjI7rXlgpoPH5H1LF47+mQORDInqkgxbhFMo9YwTJFSF/Uw31Bf2F4v/Wb1ynkvoQEtzouMD3fFXQnNN8iaJ6B1n/lh34rDqxbOCEpvCtAveWasLJA/ q3DjTUrr hEHQJqGwz4c/ai9wv36qYfqw4/UoabW+QyIV01J0iC2ouup38wRj/jPAPz8pnTgFfWCmUT9YsmNz+aseHRDBEmw+gM1r9GEIQBnnozQMcPLwLuhToUVpbLBM8P/D8en2ncjJIKW5hEvCNv+yMmJpztmU0jvn2Npjc1XYN3jOoQ/d8vcNtlM6SaguG+a2r52VMbqziF85iHFzVSpoZcG4wlodKzIyDjLQTKfUPADUgsK1Hz7f9Ehyv85YxZpu5HF4MqIchOHCvCBGABweueFpnz20uH+aiAy+/jR0mM8l1LhWmHyn3Yo2DuJ0Dg8VkciLct2Vu1LOv+YLvgVvbgWFCq6x4Ql4om0MLKzIweAhyR6gvkaTnRppmGNtTwIEJkfuO4r/KZ3qdnuEmqCyPolEDwAh5cxwKN1kcLNUWYEoegZT/a71YWwkVFgnIWgI3gSNA0w8OCqH0kOK3Y7Gi9/KZ+aIJIJhvbykgYMwVe2ifHrnLC7rXg6l0ZeZUzS9nJ13YAjY1dfKhci2NeunUiSveowMyPLcO+UCus4njY5F03ik+sKeNHaw6XM0IVFdjCqK2IwAx9HCgU+M5kf6RrC+nIHzAykcoeXEVilOU/o3l4i86yRZYZc0OcOiiEuwI3sozLMRF X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song When swapoff race with swapin, get_swap_device may fail and cause swapin_readahead to return -EBUSY. In such case check if the page is already swapped in by swapoff path. Signed-off-by: Kairui Song --- mm/memory.c | 29 +++++++++++++++-------------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index e399b37ef395..620fa87557fd 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3846,9 +3846,21 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf, &cache_result); - if (PTR_ERR(page) == -EBUSY) { - goto out; - } else if (page) { + if (IS_ERR_OR_NULL(page)) { + /* + * Back out if somebody else faulted in this pte + * while we released the pte lock. + */ + vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, + vmf->address, &vmf->ptl); + if (likely(vmf->pte && pte_same(ptep_get(vmf->pte), vmf->orig_pte))) { + if (!page) + ret = VM_FAULT_OOM; + else + ret = VM_FAULT_RETRY; + } + goto unlock; + } else { folio = page_folio(page); if (cache_result != SWAP_CACHE_HIT) { /* Had to read the page from swap area: Major fault */ @@ -3866,17 +3878,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) ret = VM_FAULT_HWPOISON; goto out_release; } - } else { - /* - * Back out if somebody else faulted in this pte - * while we released the pte lock. - */ - vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, - vmf->address, &vmf->ptl); - if (likely(vmf->pte && - pte_same(ptep_get(vmf->pte), vmf->orig_pte))) - ret = VM_FAULT_OOM; - goto unlock; } ret |= folio_lock_or_retry(folio, vmf); From patchwork Sun Nov 19 19:47:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460674 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEC10C072A2 for ; Sun, 19 Nov 2023 19:49:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4BCEA6B03BB; Sun, 19 Nov 2023 14:49:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4455C6B03BF; Sun, 19 Nov 2023 14:49:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2BED56B03C4; Sun, 19 Nov 2023 14:49:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 10E616B03BB for ; Sun, 19 Nov 2023 14:49:04 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E4CA9120569 for ; Sun, 19 Nov 2023 19:49:03 +0000 (UTC) X-FDA: 81475742166.13.FE7D92E Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by imf10.hostedemail.com (Postfix) with ESMTP id 160D9C000C for ; Sun, 19 Nov 2023 19:49:01 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=YF43tdsq; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423342; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ogKmBoCop4FjzMsXxzq+m99gJioFg3IUepp9Cnvtmi4=; b=Xxz9FuwunKnCeeNllqtGEObzhi8iu24xKW/Vq15DTACcSfqRGX0+QMrOdUxWHA443Rcc8F WIus5L77Z6xuc3VHe2c4KDvlbtzhWC53fsuBOz7z4bH/dCHcVWveum8CnH1pLucGRLsfCu jWc5NWbdq+rGFTgdGvEpBqs7NXWEfSc= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=YF43tdsq; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423342; a=rsa-sha256; cv=none; b=I/haQXivQcaWGnj7/SaifeUnnvoVm8kGZn6ipzGhh9L7TF11RSbkcwykUJbJzU0RzEICCP KDzH3rFtlkibDQS3L9BCwy0aKKsjJqWHFoqdP/f17J/H//EYOz5o2IZNAAUblGfhN8LsY0 L6JPlGazrobrc2/7isbGDgu+5A5qtiE= Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-6c4d06b6ddaso3055878b3a.3 for ; Sun, 19 Nov 2023 11:49:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423340; x=1701028140; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=ogKmBoCop4FjzMsXxzq+m99gJioFg3IUepp9Cnvtmi4=; b=YF43tdsq2Dy0YJIraJYvPbQVn3dytDT96SfETw7iDyp6onSftfq8jc4NH4jMrk3tfS mtyBmux2m1PKc3gVtCj360hR+ed8Ygr2K6yjPVYT7Ph7E3vm9Uz6VCLOeJeZViNwSLiE u0VyqzNd+v6i/JYqppu99z5HljniexH+AXG04snogGGL95QjqMNjpusTNcKnRNuFSkqa BKgHQAu39/S+mKshtnJ8BO+/ehaUH+VJ9IGA5thaYQB/Ia0sBsE9xFW6Su/KlYel1qek AzOHWtevQM85HeJS3N6uYf/0pQVo0Ai7IY5+Yg4SSRNSBqqznKcwzEj9OxNL+veL00Zd Kl3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423340; x=1701028140; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ogKmBoCop4FjzMsXxzq+m99gJioFg3IUepp9Cnvtmi4=; b=ctJGtvOL7/eGQO/0U/VVj76VV65iuyXJxr5Q+gzKjreeDIxYhRJv1n0sLdCAq28Tro vfMmf2O1oRg83LJnV0HkbWWClH6d/ZoFfIMIQDMsJNakZtMss7HkLKEM2uCPJ3Sju8sp jQg1yhKngwxK+38xSAuy398C6aYmnyIjfI3kczo4DiFl6D6m7Rjlnrllz2AeoOdnpGtV hbgqB8pl+nzSuUyOm6tjuh4ZT/lzCIY5JC8oZbjCDaCD+dt5fGl4X14Xgbm4vRWMk/2I loUIH52/ekLrwJLuWPoQETxLxW/yG43DSbOOMLWjjRTfpHVkB1RaStAK/VKs9ttn9e3c e5AA== X-Gm-Message-State: AOJu0YygHEM8oDGkyuSr4ofomlPXhH7IXah5UJv4ODRA4K1gluVcscGF 8fdYU6sMZJhnwEJyluGOlrBwwfRlut5wt7/T X-Google-Smtp-Source: AGHT+IEtFX0YlsrOdaCPyUWZ1WjriHhJN7FSpvj0uTb3jwG62CN+QG7TWo6/q5IUmUrNJVLup3+uwA== X-Received: by 2002:a05:6a00:301d:b0:693:3c11:4293 with SMTP id ay29-20020a056a00301d00b006933c114293mr4840721pfb.14.1700423339887; Sun, 19 Nov 2023 11:48:59 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.56 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:59 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 18/24] mm/swap: introduce a helper non fault swapin Date: Mon, 20 Nov 2023 03:47:34 +0800 Message-ID: <20231119194740.94101-19-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 1kiabr8joo99g73f9tpwfc3k9hdh4nsc X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 160D9C000C X-HE-Tag: 1700423341-344533 X-HE-Meta: U2FsdGVkX18k9XGoM7Fae+Y0mCOCvtfh2zhfEue9uY8JxepHFka3x1aX2aBALSqxnasO658YjmT+b6+paUDd/rBffHL+Flf9npTy3fqu29WVPkzWkZOUwogaCPm4LiQk8PgjmzObnKZ/wDhyh0yCJ0VTmkQWlYkzKs52ZmP/DKGQC8bYxui7f7Vt2XzQFS5jfb48JW5yTp7T/+3ybVvXZ0eMyVFYMpgcm82GPaZtZfL8GT5Qvizr0XGYfkbkEipFkseb9c6tqft3OoPzInVXDlymdIfmGq1VokQWddSHlGGWH2b6IoFRBsXDo23wRTu233Y5B0bhumk0HlL0SWuvTtORTN8Wg0QSS1EUPuP/IN2fsHz880G0dZa73WKWmv4Px8i577NwamEQHkc5LbcEmWSJKPqB8XXsMMVbdh7D3ABWnfND1uj/3pXfZJeGOscT42RuPfKAumRAk9Pw1Tg76I8MRBToPY5NYuHWE5Up+r+jyMHEKlAkZefQbqQvs09gw74u/5yA7HUTkbK4JcOYr6MKOQPRql2bC0m4k+QuTD45I/2C60njxdmiZplXvCk9jAnkoB87aE0xUZy0tvL0xaOsSG36yd3QKQWhc3xrAIKW6h0yY0881tG3DUbTBqf8UIFqogG6A1Z1PpKSwPVGcRx46RqiLrAoY2QNzDXt+nSzAHIir5qBJ0QO1257wjI1Z/KjIZyDPzni+ukC7iub2B2ntu26daWyXfFlUa+wYQV43zTw5/i21YHMn9XqXSfO803jd4wDGbt+qZ7zBy3UiIjsnyseJsnDXa5VKHrNu8leZvCsTFaCY5j0flV3nCDa04t5cyHnGYDHqvMTrn3Rbg8+enYynWgPu36bdaKTiLDx9M/rJRE1Ds4kEYRL5n24vjBiOr8/YjJSBZ1QDQgyt2Pety16ULOfQfR54rd5RyqKUQcLzZB8y9PFPcdH+AQGOqFcICmFeuYVjs6fKsi ZMfV7pp6 guiYLrZ/PMq3keDQ65vNKYco2yzMwgcENfxEBTuMRZFoxLzSiagGqgdcqeq5iiI6Rq3/ib+3YMmfe7enQGe/PaGVYzvxiLidt1wFcgMTe/CoY+oaDG7/FS2RSWQcOoxNkxORqq/Yiwby2Br/O/L0vwgOWW3hDEDs02FCT299/7eLOnL5i80HELxUaey5OYtN2ROxjB362U4+fCDD9hLcCGJr9xGT/DJhB5SWhOn5mHxXyqXcieDnE33SO2X6ZwGd1fzPq8A703g+I6B0Gih6F0Rh80d79PI46JrGIxTZrD3VYu53wURd1xgUIvP0fEzTBKl6SQsEVjIDwfRIxkmIL5PVXnk5JgMEJ2eAGLYd5sKTkmnvi/pVAIrBLj7D+Dr4yQUNM37+qc+eo34DNGrayRff7cE+IRDf8sr35VJ/hZ4qw92uKU0s+JQfjkQTqOB9WdXmawscHd35E4o7XUexck0CtxlK04USKyZ7zgtad1KVgodaPMwQ4PFM6VlizUTxA+g6rgRyCCOd/tR5CDdR4wW/YEMrmGoEgYy0W3EMhCOx5ufJx8spBkOKEuGwnPsJoz424x2FSmwaSX5re48bTC6DyxL72rLYbOvvw/En+uByeSr4JeZqwWWtOashNcLPgYV6k X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song There are two places where swapin is not direct caused by page fault: shmem swapin is invoked through shmem mapping, swapoff cause swapin by walking the page table. They used to construct a pseudo vmfault struct for swapin function. Shmem has dropped the pseudo vmfault recently in commit ddc1a5cbc05d ("mempolicy: alloc_pages_mpol() for NUMA policy without vma"). Swapoff path is still using a pseudo vmfault. Introduce a helper for them both, this help save stack usage for swapoff path, and help apply a unified swapin cache and readahead policy check. Also prepare for follow up commits. Signed-off-by: Kairui Song --- mm/shmem.c | 51 ++++++++++++++++--------------------------------- mm/swap.h | 11 +++++++++++ mm/swap_state.c | 38 ++++++++++++++++++++++++++++++++++++ mm/swapfile.c | 23 +++++++++++----------- 4 files changed, 76 insertions(+), 47 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index f9ce4067c742..81d129aa66d1 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1565,22 +1565,6 @@ static inline struct mempolicy *shmem_get_sbmpol(struct shmem_sb_info *sbinfo) static struct mempolicy *shmem_get_pgoff_policy(struct shmem_inode_info *info, pgoff_t index, unsigned int order, pgoff_t *ilx); -static struct folio *shmem_swapin_cluster(swp_entry_t swap, gfp_t gfp, - struct shmem_inode_info *info, pgoff_t index) -{ - struct mempolicy *mpol; - pgoff_t ilx; - struct page *page; - - mpol = shmem_get_pgoff_policy(info, index, 0, &ilx); - page = swap_cluster_readahead(swap, gfp, mpol, ilx); - mpol_cond_put(mpol); - - if (!page) - return NULL; - return page_folio(page); -} - /* * Make sure huge_gfp is always more limited than limit_gfp. * Some of the flags set permissions, while others set limitations. @@ -1854,9 +1838,12 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, { struct address_space *mapping = inode->i_mapping; struct shmem_inode_info *info = SHMEM_I(inode); - struct swap_info_struct *si; + enum swap_cache_result result; struct folio *folio = NULL; + struct mempolicy *mpol; + struct page *page; swp_entry_t swap; + pgoff_t ilx; int error; VM_BUG_ON(!*foliop || !xa_is_value(*foliop)); @@ -1866,34 +1853,30 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, if (is_poisoned_swp_entry(swap)) return -EIO; - si = get_swap_device(swap); - if (!si) { + mpol = shmem_get_pgoff_policy(info, index, 0, &ilx); + page = swapin_page_non_fault(swap, gfp, mpol, ilx, fault_mm, &result); + mpol_cond_put(mpol); + + if (PTR_ERR(page) == -EBUSY) { if (!shmem_confirm_swap(mapping, index, swap)) return -EEXIST; else return -EINVAL; - } - - /* Look it up and read it in.. */ - folio = swap_cache_get_folio(swap, NULL, NULL); - if (!folio) { - /* Or update major stats only when swapin succeeds?? */ - if (fault_type) { + } else if (!page) { + error = -ENOMEM; + goto failed; + } else { + folio = page_folio(page); + if (fault_type && result != SWAP_CACHE_HIT) { *fault_type |= VM_FAULT_MAJOR; count_vm_event(PGMAJFAULT); count_memcg_event_mm(fault_mm, PGMAJFAULT); } - /* Here we actually start the io */ - folio = shmem_swapin_cluster(swap, gfp, info, index); - if (!folio) { - error = -ENOMEM; - goto failed; - } } /* We have to do this with folio locked to prevent races */ folio_lock(folio); - if (!folio_test_swapcache(folio) || + if ((result != SWAP_CACHE_BYPASS && !folio_test_swapcache(folio)) || folio->swap.val != swap.val || !shmem_confirm_swap(mapping, index, swap)) { error = -EEXIST; @@ -1930,7 +1913,6 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, delete_from_swap_cache(folio); folio_mark_dirty(folio); swap_free(swap); - put_swap_device(si); *foliop = folio; return 0; @@ -1944,7 +1926,6 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, folio_unlock(folio); folio_put(folio); } - put_swap_device(si); return error; } diff --git a/mm/swap.h b/mm/swap.h index da9deb5ba37d..b073c29c9790 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -62,6 +62,10 @@ struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t flag, struct mempolicy *mpol, pgoff_t ilx); struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, struct vm_fault *vmf, enum swap_cache_result *result); +struct page *swapin_page_non_fault(swp_entry_t entry, gfp_t gfp_mask, + struct mempolicy *mpol, pgoff_t ilx, + struct mm_struct *mm, + enum swap_cache_result *result); static inline unsigned int folio_swap_flags(struct folio *folio) { @@ -103,6 +107,13 @@ static inline struct page *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask, return NULL; } +static inline struct page *swapin_page_non_fault(swp_entry_t entry, gfp_t gfp_mask, + struct mempolicy *mpol, pgoff_t ilx, struct mm_struct *mm, + enum swap_cache_result *result) +{ + return NULL; +} + static inline int swap_writepage(struct page *p, struct writeback_control *wbc) { return 0; diff --git a/mm/swap_state.c b/mm/swap_state.c index ff8a166603d0..eef66757c615 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -956,6 +956,44 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, return page; } +struct page *swapin_page_non_fault(swp_entry_t entry, gfp_t gfp_mask, + struct mempolicy *mpol, pgoff_t ilx, + struct mm_struct *mm, enum swap_cache_result *result) +{ + enum swap_cache_result cache_result; + struct swap_info_struct *si; + void *shadow = NULL; + struct folio *folio; + struct page *page; + + /* Prevent swapoff from happening to us */ + si = get_swap_device(entry); + if (unlikely(!si)) + return ERR_PTR(-EBUSY); + + folio = swap_cache_get_folio(entry, NULL, &shadow); + if (folio) { + page = folio_file_page(folio, swp_offset(entry)); + cache_result = SWAP_CACHE_HIT; + goto done; + } + + if (swap_use_no_readahead(si, swp_offset(entry))) { + page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, mm); + if (shadow) + workingset_refault(page_folio(page), shadow); + cache_result = SWAP_CACHE_BYPASS; + } else { + page = swap_cluster_readahead(entry, gfp_mask, mpol, ilx); + cache_result = SWAP_CACHE_MISS; + } +done: + put_swap_device(si); + if (result) + *result = cache_result; + return page; +} + #ifdef CONFIG_SYSFS static ssize_t vma_ra_enabled_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) diff --git a/mm/swapfile.c b/mm/swapfile.c index 925ad92486a4..f8c5096fe0f0 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1822,20 +1822,15 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, si = swap_info[type]; do { + int ret; + pte_t ptent; + pgoff_t ilx; + swp_entry_t entry; struct page *page; unsigned long offset; + struct mempolicy *mpol; unsigned char swp_count; struct folio *folio = NULL; - swp_entry_t entry; - int ret; - pte_t ptent; - - struct vm_fault vmf = { - .vma = vma, - .address = addr, - .real_address = addr, - .pmd = pmd, - }; if (!pte++) { pte = pte_offset_map(pmd, addr); @@ -1855,8 +1850,12 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, offset = swp_offset(entry); pte_unmap(pte); pte = NULL; - page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, - &vmf, NULL); + + mpol = get_vma_policy(vma, addr, 0, &ilx); + page = swapin_page_non_fault(entry, GFP_HIGHUSER_MOVABLE, + mpol, ilx, vma->vm_mm, NULL); + mpol_cond_put(mpol); + if (IS_ERR(page)) return PTR_ERR(page); else if (page) From patchwork Sun Nov 19 19:47:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460675 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79574C072A2 for ; Sun, 19 Nov 2023 19:49:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0500F6B03BF; Sun, 19 Nov 2023 14:49:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F40966B03C6; Sun, 19 Nov 2023 14:49:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D93AF6B03CA; Sun, 19 Nov 2023 14:49:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id C49836B03BF for ; Sun, 19 Nov 2023 14:49:06 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id A74F91CAF4D for ; Sun, 19 Nov 2023 19:49:06 +0000 (UTC) X-FDA: 81475742292.27.C5C9162 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf26.hostedemail.com (Postfix) with ESMTP id D763F140014 for ; Sun, 19 Nov 2023 19:49:04 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=IeFcDWJH; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423344; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CM+OPz8sAX9/+pckE36Q2lCrK7XvSIfXWi261rZELtQ=; b=DsJArpAM88Tnt4jDis4JIs0/BrG610iW/zbclpBOe72DU8p2gGlxbKiBf9b7WlzQiBz5/B eXy1HZyTeGaFZ0p/MWYKRa8LQUyeKUvFLbQX316SFhESfmAVbfzaO3cqVuLe8v29li/shC NOiOik3lbvd6+wbOkWuAGNdv2PFL5MY= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=IeFcDWJH; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423344; a=rsa-sha256; cv=none; b=Nx6jcTZ2kRy/odI4Ss+B8mW21E1lYDR2UMU4/4gf0FekjpWQzbHaypjCvfecI63cnScDFZ P3D3FYPyObaQlIxbYcmwSZFsyHZLtbdfnhbIBTfyxF3t0eOOFnm00PeGC9cVqRj6l4nKKj H1oGGsiI+fU10J2owjL6sHd5nltIVeY= Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-6b7f0170d7bso3605658b3a.2 for ; Sun, 19 Nov 2023 11:49:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423343; x=1701028143; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=CM+OPz8sAX9/+pckE36Q2lCrK7XvSIfXWi261rZELtQ=; b=IeFcDWJHKFixFZKnGcZKlveIDvC1LIocxukfp7I0FCFUA34bOHj9RlAQL8q5azxHf8 irgbwD4YhHThG0I3hlCtAdkO604Bomaj1oJydYkzwBvVCmaasBHrqCbdDawXsulpe6/n CyhPdlZapKQ3cXWuvSrl5s9kZ7NkEdU21eVs2GluvzTMwv/KE4yReZ4mi6synsXaAb6T tusLR6pLyDLvCYhkYFB44gfASSMmqSm+6mdj15pd3saWb1gyBLuvrUOPBKOhFt5ekx2G RCRq6LIfSdjXJNLUuRlV6geEo9OHLSlsNCsQpAFsYPVX/zaQYxAcVRFWcTV1LH6plJMu Vi6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423343; x=1701028143; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=CM+OPz8sAX9/+pckE36Q2lCrK7XvSIfXWi261rZELtQ=; b=dinXHl5+uqkEuSRj7mEyJsZEK0s+F7gSxTs2ByQwxfYljjMVQEonFU4FQZLPKdzhjY 0ysuCXDmfsnH6h/KDBT7TG3IzclO5HWIu+Q0rDmqaBNKJZfd4KXmh8Vq4dBRDUdGZoxN FTqNcrQ/9Q9cC1k4tZSbXr9/U3tMrrgUSw8CBzmyl5+Ke7479/iNT4Ze48J7kOi+Tv2Z 9XOY9nZDJndHTuddQDfXMueJB6sxZ+mhOzmRk7pJinRhxUzNznZjaUCAjC8qVUiMIB1R F4NyLUqoddz2R8oVJ01o+TYfvPo0Inmlm1HiCBIICyXR3BwRFU8JnsaLEWqR52fLN9De fNrQ== X-Gm-Message-State: AOJu0YyrFvfR2GMsLclReO1Hj3EGrtnd72LGwQgQ23igWuJnYC4zza/I o4AgDp4LqmHXI6mwVM6PWDRt7wiw0ZjzQNN3 X-Google-Smtp-Source: AGHT+IGkCTfkx1rp2FzBZeJz7DuMjH5OX9xB0kVWtfaGQTv9o7aHxqdL3R9OOEYrRIDUL1WVW8rBzw== X-Received: by 2002:a05:6a20:748e:b0:186:58d6:ca65 with SMTP id p14-20020a056a20748e00b0018658d6ca65mr7241893pzd.32.1700423343119; Sun, 19 Nov 2023 11:49:03 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.49.00 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:49:02 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 19/24] shmem, swap: refactor error check on OOM or race Date: Mon, 20 Nov 2023 03:47:35 +0800 Message-ID: <20231119194740.94101-20-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: qb7yjnf5fd8w6azq9gq888ted8gk6g88 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: D763F140014 X-HE-Tag: 1700423344-910708 X-HE-Meta: U2FsdGVkX1/je2Sk1VApub/RT+07Kp0a2alR/gRg+kLxc80XwuWw+mQxYJvFvLL1aRrUm9fEZYMmUqS27jAWPpAo55mevdsgJGr+AAQF4tyNoNxEpiuuCVeZHJSv5TSB27H/WLbhtf4tLWI4Jeq4F7bMaV3XDD2fXBniPAB5rLH2pMRgjHG9y4zFlJ4bZx7cocj7N+rSq5LaGCbQ61FW4qbJwsgnDbCyxvmG/ONgrYzOlqoiwh44dx40L1IkPpStOZWr5WHJ3IGHKRrhvFucc+/0CANerNXagdOaL9qnIiZf6UZ7Uh6lZObysDmfMK9A1h1/1uaJVYqmQrec1Gk5UqJr7uGd+icGNGwKap90fAtxjOmyB5mw0ICJlfwFZHUHN8yzv5b9w2fXVpMlirMgX7G94Hz2gVFGScG6I7ptJFLC2XirXWcukHPw5S25Y1ExfyJEqHmx/LnHuVG1TlC5mNodg00UFTbXh+laLlgunTWcpNPPGdtmLfH5kA6oSHLlJIZuUb0KgnPErjieoCfgu8ellSw2l2dEygh3OnkBejYM4Um/g2XaAGvsAfxY1HY18MrqmIC3WzijqURukj1O8nJklbxHZWotJUS3jOHbYKSebwoGb8gksE8DltKRejtN2GDIn63AREGyPA4IDZKn8GUfBMS2xJBbfMct5YVzSGOmzL/k3aym4kq0SxowWJxxSTfUV/O8F+XFtMHvVko6hEcv7JDeTECZKhBdpK0gk5gYIlNJhff4j+N5fp53gih8Ih169pVoX6TGOv//iJFWebX5Su9nSQujRCmHyxRgq/hfoML10aX5SQu3lASTJqSBwxeLS54Ok59HtJfuYCTqdXWvA0DfS4pysDUewu/gk+KaJRMsriXJz/VydYn747XBBN1m85yRpmLjFKvwDuUY8YX45BKjIbV0prer7JPYn5yPRiTQ7qcTyfQYVM/HpQWjw7J3AchFb8h0dY4Eakn 1kmYC9Fd z8nR6y1vHau8OgrEe6ovPrd6pHtC9qncSUepOIQf80xAL6bPVNroP7dikGfXYGJ1pWs3pjP8HFVSx6Brgv08VLD/GBZ2hN5p2zhN580C2OZ2awa5pTfvCgTw4ZgDnC33rEYYMi6a5ObrTjS3F2fEh5fzLiklGYeunstvKmZzFjCOC/dhADDv6PKdE+YGRMHn1vL5j3UyyJ1aLVKSwNw1NjCqZgqV3LPrOLZ2EZIrHQj87eTgUyNBjHfF8D7aXUUjJohJ5PayIKJb85r9YNpQ7ElBRodHVxOS7oP8Dizx6gqkWtGCYhwTPevEtEvLHLgYAkfUH1/YMtWYK3PEG3FUIX9xAsTqoQF6lWaCD5YqiewxpHvp9ffUKhyOqIXNej8Y5nSwC1Vp3FvmcyZ7w0o6h2pSCudsNC8IJXJItTb9N+QaOZqT5OTkTwerMHPkOSEXXTVMobh8eaLVcgX4/FX3FVeq0EXfx4z9oLDB/2LqXz5WWMMumTVUvR0hlTFmfhzmvoPVjC38Fv5cSIWD5732ZROcxf6ZrXjQDy8gYN9Bbw/bSm0Xy+5zho6vF3eUpday2kTitEd5x5Cl1WPCt8Tchs6+PYAE4L1sx9ezALp31aOrYf8dEWDwSjqIMA6Zav333JA0/cQ7vOb9m2bsWe3wlmymqyQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song It should always check if a swap entry is already swaped in on error, fix potential false error issue. Signed-off-by: Kairui Song --- mm/shmem.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 81d129aa66d1..6154b5b68b8f 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1857,13 +1857,11 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, page = swapin_page_non_fault(swap, gfp, mpol, ilx, fault_mm, &result); mpol_cond_put(mpol); - if (PTR_ERR(page) == -EBUSY) { - if (!shmem_confirm_swap(mapping, index, swap)) - return -EEXIST; + if (IS_ERR_OR_NULL(page)) { + if (!page) + error = -ENOMEM; else - return -EINVAL; - } else if (!page) { - error = -ENOMEM; + error = -EINVAL; goto failed; } else { folio = page_folio(page); From patchwork Sun Nov 19 19:47:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460676 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC74EC072A2 for ; Sun, 19 Nov 2023 19:49:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2C0046B03C6; Sun, 19 Nov 2023 14:49:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 254796B03CB; Sun, 19 Nov 2023 14:49:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09B086B03CF; Sun, 19 Nov 2023 14:49:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E699F6B03C6 for ; Sun, 19 Nov 2023 14:49:09 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id BD8911604AB for ; Sun, 19 Nov 2023 19:49:09 +0000 (UTC) X-FDA: 81475742418.08.ACF5D90 Received: from mail-oi1-f173.google.com (mail-oi1-f173.google.com [209.85.167.173]) by imf04.hostedemail.com (Postfix) with ESMTP id B318040008 for ; Sun, 19 Nov 2023 19:49:07 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=VE21LzHN; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf04.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.167.173 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423347; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ze1Psd2Ie5OovBPk7SmUTsWgEFqDeyPvhXbgqpZhXY8=; b=3HmcqYX7NjNiXXJKy496DdtZ7LDc/vp4Eh40D8F9Yjq1zxC8BAloq4EpY+Tn00PB9JG2Uy EOuEwMSZFvggR/HEkY/apcp7VlNk3ocbIeT8Nck2nPGs/PHsiTX2PcV9yNCxvCEUYeOsN7 vzeJs/7O/XDw7+SmMkpEmG0jFTUzqz8= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=VE21LzHN; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf04.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.167.173 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423347; a=rsa-sha256; cv=none; b=4N+82dWQiSB0AOOuQPTpTIo9+eMy2xREM6xODgYzFF8ZiV788LF0Uy7RDHFEXyo607jh2/ fmeGbttPp02q/M8aP7DDEa/JBNzp1NpQGJ23Ty6GVyeGnDcJrDqAp3Vc0+LbBWD31BVIa1 NyL93Cm68KFBWZeGIn3jE31B3eNM97Y= Received: by mail-oi1-f173.google.com with SMTP id 5614622812f47-3b2e73a17a0so2542981b6e.3 for ; Sun, 19 Nov 2023 11:49:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423346; x=1701028146; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=Ze1Psd2Ie5OovBPk7SmUTsWgEFqDeyPvhXbgqpZhXY8=; b=VE21LzHNwvpLhDJuCuWzR+0zGkq2DB7Gt2qulsmiHYyoYkKlLUcb5QmJimckbmWLn5 ckGjUreicUkoQP28TJvojjRk+EPfSxwFJJalAzIDvbe8XTa+x6L8eMj2IhXRKWl+6S8i 8OiUleXura6T1D439ODlNiCfosVFVqDtagXDfvKRiVaf2rALWXpkyhoeVPC4lR2WoOHx MRBA6sriKTOGm6Vg/vCb0V4aDKZYmpjW083TsAcVKKAsINUlzIMeRFjPrjnvFDWFHICy 3lmZ2wqP8mEYecmP5Jo0E4xJA6gcaT09RqOsoH0lRJxjsOzX6r58lhMX/VUFjeo3aR1O jDzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423346; x=1701028146; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Ze1Psd2Ie5OovBPk7SmUTsWgEFqDeyPvhXbgqpZhXY8=; b=qmHQjHC4N3tf0lBB7y5GAokV5+EOvTzc8lM5Z3HiUPWVxLD9W3zwjsvKwKfG1dV8/u UrW6BbqgHJQ5+/lC63fkp5bZW5QS949ZTIm3puzTZ7bph1Gnd1me9xOWEGAH5LIenKba XXhMD9NT+DFzgVn326FbpLJLu1AHdD9zJoZEtHZpc4BjWx4mjrVs/WWMm2mPGlJiwC0L F8y3dG8b2KLDIVr8uoPSTXlFXoJAzSHFB/9OGrBNq9QW+mUhHt3+d2GdkxGEdqMPiYj/ 0PGNpfnj1uGlYHgc9f/Lpgrgd9ZYgkep2Sct/NhvxyJEXlmfeJZo+HAZziw5vl/5Cb44 Pndw== X-Gm-Message-State: AOJu0Yzg90q+C2JwfpCXUYmCJ+qOWAitodLk3CXuHQVO9LN3YjXh7Eig HMWoX9UIWfbGjnZz53F4Y6y/0b/DxBL9kOZ7 X-Google-Smtp-Source: AGHT+IGTlsOhMDU+nbF9BiKSPMYiu1dnBRu6RvVsGq1/cPrbFHBN7F+llm4LzYDPkkR6tdAEcGAOgQ== X-Received: by 2002:a05:6808:1451:b0:3b2:dcff:9e54 with SMTP id x17-20020a056808145100b003b2dcff9e54mr8319167oiv.24.1700423346359; Sun, 19 Nov 2023 11:49:06 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.49.03 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:49:05 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 20/24] swap: simplify and make swap_find_cache static Date: Mon, 20 Nov 2023 03:47:36 +0800 Message-ID: <20231119194740.94101-21-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: B318040008 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 5d55utfepoco37s5f9q97z7bui91yjjh X-HE-Tag: 1700423347-635675 X-HE-Meta: U2FsdGVkX19j0VYnQIuxgHo5Xk+s0PA7pBlY6zt2RSoVSB9vcr1tY+2131J8Wz0zdCySCwE1Xz4XoZdGXlAR4PthTIc/9T8i0Fy5nNexlc/C6ivIgRLO/60OD5Zfz4/hhNM3wPCmGpp+s9WoC2ECF5QbaKymtqDIa9rBwC96dUjVF5n+P9mazHdP54twgi36B9NgyDidmXS1CrU5u+IfFVGepvodDpWAdlL5QLZMoLH4YNEfyE2vGal/bWYSU+4Db9+/6XiW02ZFk5EurCgJSRwHT6wmHExDRrlUuJvbaDHtBlaVmfk0ahiyQ3+sfKEUP5jj0Lg/tBcLVdrrOkpA1223bvbI1SN41Ld1sCaefGz8gpNZLcQ0kDHMrdC7VAdyKlKBELA8q3Fos1a+8Ybkd+NojMKgZjHocYy3eTrmTYZ5TpGBl1vdg1x5FMBw5+BnMCzdtEkwAEs0ZpQgSfTESOapIkMk6zDmnyJcVGGNVBWW4pn57k2aw9IZBLizuuvwEHMKJikPAn7JgVeRRKGiymMzh7SzO8AEsMEob183pRe91rfmVtkNzJlVBIezmn+2BMNjUZbjrvaUFJF3Mx+y8wx3+KNFO8Sv90GUxJCRMYbywC9hQHjHeKbUKd2TQzkdLQW9duSn12/xIMRDAl80sNo8Bywux1fKCBaFmoJgL3vREj2iRAME+tHBQ4dgliYKlbjW2uY+vZy0Yb2z4ZzUBVKIvsILoB1NkuZgdf0j0WP2pN1FBOr9eh53lbD49sJBAny/nYdd4EbrjA0DgzA50LxE5gwqO0lmC74gpdVBgAh/7r/UUvGikcj41uaGyakVkqcqb/H5rYen4/K06L8u3U4qnpxYTANj0mug7z7t6tIJ3B71JxirgCe9yLv/Lbv6TO4ITBp28bxRknjxI4qNcpanljpmjK3Kg1F9TsBtM8bZ1uu2sfu77KeRfPJ1Yq6OLLxM1vZ0/IOnudxttuj 5E7Dv3xk Ayqh0d43pByTxX3zFo0JstYoJuKwl9ws9jBVKcbfyYPUDAv8+E/Noq+UZb6A9DZLLjoa+4B6njnzXDRXzU9SClqNM6mpDEojCUb7kW4O4M0F6MR6HTS/8FKo7cV3Qr9c//UfqW/16nEQYL9KpnlCjP4poRs4Ll0qHfXmmAtqE9AzJSfu8Zq0EfNqHhPeFlxIlUKKqjeb4cKaSI9HuzPIeN18JHibD+73SRt7zZQaGOdL8IXUyhMXVcRGGAFukxtp2OopupUyQHVKsopX6GDP9FdYBe6baVw4FSRXnlqpWZgi/hkXzHLWYmySLPCiDBNlnnEqxPbVnKJX65+FQv4vauAqYCY6ghsKZiaRoqfJGyUj41WRIHulWG4COsEtY2AkT22EUfatS23J8PgQ+gK/eK6tp87LBKShvboJwm4purtYaNqvqZtXxaF14NEle4r2tHFG3yK5fCzIYuC9ZO0M+ykmbikyCqfHG1cdx246POAuLiexuMgAjKzj29TogjPpYVMzPryXPWQvmCIA9r3uu2D1iXQ+6iuSN0COQf/qNuqHRK7MjblPFZVYzbP+MtdRJPlhMMqmT1jDDFKpTxviZtmOYoM52nyzci/g/g9K6WqJlbPvKCic4bpOk/o9Qe27iW5kKxieat4Zu5VFVpmomPnln5M3SwrjGFn0YWOuXxDRVNN8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song There are only two callers within the same file now, make it static and drop the redundant if check on the shadow variable. Signed-off-by: Kairui Song --- mm/swap.h | 2 -- mm/swap_state.c | 5 ++--- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/mm/swap.h b/mm/swap.h index b073c29c9790..4402970547e7 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -46,8 +46,6 @@ void __delete_from_swap_cache(struct folio *folio, void delete_from_swap_cache(struct folio *folio); void clear_shadow_from_swap_cache(int type, unsigned long begin, unsigned long end); -struct folio *swap_cache_get_folio(swp_entry_t entry, - struct vm_fault *vmf, void **shadowp); struct folio *filemap_get_incore_folio(struct address_space *mapping, pgoff_t index); diff --git a/mm/swap_state.c b/mm/swap_state.c index eef66757c615..6f39aa8394f1 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -334,15 +334,14 @@ static inline bool swap_use_vma_readahead(struct swap_info_struct *si) * * Caller must lock the swap device or hold a reference to keep it valid. */ -struct folio *swap_cache_get_folio(swp_entry_t entry, struct vm_fault *vmf, void **shadowp) +static struct folio *swap_cache_get_folio(swp_entry_t entry, struct vm_fault *vmf, void **shadowp) { bool vma_ra, readahead; struct folio *folio; folio = filemap_get_entry(swap_address_space(entry), swp_offset(entry)); if (xa_is_value(folio)) { - if (shadowp) - *shadowp = folio; + *shadowp = folio; return NULL; } From patchwork Sun Nov 19 19:47:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460677 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9F07C072A2 for ; Sun, 19 Nov 2023 19:49:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 690666B03CB; Sun, 19 Nov 2023 14:49:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 640026B03D0; Sun, 19 Nov 2023 14:49:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4925B6B03D1; Sun, 19 Nov 2023 14:49:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 33DD16B03CB for ; Sun, 19 Nov 2023 14:49:13 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 0D4D3A01E4 for ; Sun, 19 Nov 2023 19:49:13 +0000 (UTC) X-FDA: 81475742586.18.A651319 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by imf16.hostedemail.com (Postfix) with ESMTP id 38388180019 for ; Sun, 19 Nov 2023 19:49:10 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=K+BDN5ir; spf=pass (imf16.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423351; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=w2h0VsFq1SfFmbRfFbDLLmtclse5HYqoYk4CcDA6qDI=; b=SOti0/c23aEgrSjuL7EwJKehAXSWwoM/jXkYz3oT9+FY5UbnXHA1mYL7jK8NafudifTHbb 71B+ZWIhXLEHMnfTkBXBEAW6JjP1wOPnopuMabFC+A7oQxf32Kc1yhwZ6DrcgBm8xICMS/ cQjXlO6CFscZIVcIVC+RrB5y2wriTzk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423351; a=rsa-sha256; cv=none; b=tajcWWYmCz+0eJ/WYMXSZvz619EORBf7xKAuEPxUJKkfCsIbHTSr6ml7vNjl2dFuNPPri1 pO9cOXk2O/BppBq5YfooTnLrmR5J2mYx8695y4lzQlgpr7Qhxu8S55EzmF96etd8P1u4cg Nhgd2sy0CQD2DAE2EeromqT/6kNjtqc= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=K+BDN5ir; spf=pass (imf16.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-6ba54c3ed97so3871114b3a.2 for ; Sun, 19 Nov 2023 11:49:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423349; x=1701028149; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=w2h0VsFq1SfFmbRfFbDLLmtclse5HYqoYk4CcDA6qDI=; b=K+BDN5irpayOSZaoP5h5z+t1k2IVuXlGyPl5WuTQn2kJWx0RC/AuR75FDAXqxH+M9K ujsA7ugjzHbZR3LdOE14lHYH1Q7gK/PrJY+y/bToQbMTkGqqMxnW6+T4gqqi0wvNxdl1 Krq5skWG+JmllHE0ng3yeJ3AyJFkFGMwd2RipCb+/8MUYVgFVFFDCLZSYE+uB8HXqgGU iM427eJ0ZX5j8NPDy9jWnDSE+iC9nCBU/mjQUt1K4TMpmQt9vU0U8VqzBp9ncnP/8hL2 zAaPHgTZ64y4Ha3Yn+Z2oRRrv8Vrct2GxXW19iA58XhygiLlTENixQ02OpTisCACLL6v Wldg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423349; x=1701028149; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=w2h0VsFq1SfFmbRfFbDLLmtclse5HYqoYk4CcDA6qDI=; b=uoTnbTJJTxC1RvGTmWpaclxHf5QCFpXsL8LuYsmOLvOpdVW+vYgoYRZTrlpgsm39G/ jB0JGsiIz9N8Yl8ebHqGo3mX13/hU25ZzCxlz55z/w/Hi7OoJ6Pu/OIarZIa2WXDTRXW LHU8GaJoY6xz91wwDFOsLfyW5Yw2I40UPzbvMLvvd4Z4Sw25vTobfPq/kREVgh7c2zb6 4w0iXUw6C2QcH4EYUJlp7HceP7dYuMM5ctv8NHeoUdvpqqpyezVBrJeSEpSHC0JlCnN6 adJbXUYAP92lwTB8sISeM3VEI8w14mXNGsYQES3cuhu7f4SzW0KUJ4edHfHXcvdcLQTO 4X6A== X-Gm-Message-State: AOJu0Yw0u2XSgC8uhZ9qwsdR03omM/VQMzwB0ABESbHNLZF/FcybZ3bH yQTa6s76WlY2WBTMSKmI6HIpHzMMyWOmnBog X-Google-Smtp-Source: AGHT+IH7Ef3T9tYhjLf81++NfU6x692Tm7IbNx4ET6n1TwJLp+72WBRurfMYbX3sMqxL34SXbWHpQw== X-Received: by 2002:a05:6a00:1988:b0:6cb:a1fe:5217 with SMTP id d8-20020a056a00198800b006cba1fe5217mr787020pfl.16.1700423349552; Sun, 19 Nov 2023 11:49:09 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.49.06 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:49:08 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 21/24] swap: make swapin_readahead result checking argument mandatory Date: Mon, 20 Nov 2023 03:47:37 +0800 Message-ID: <20231119194740.94101-22-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: 38388180019 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: kprz1kqonnu76khdbmfeqtt1c4r9uwcp X-HE-Tag: 1700423350-792556 X-HE-Meta: U2FsdGVkX18B8x2tt9rSY+l4M6iAf5NaPWId/OlHKhSWlCp5XsQnzEy715qxqhEwpdUS63Jk1UHLtYieLs0PHFovLNEZutVaSfql7nEoianMD4evY5Sto1xXg0wKiQh++iF3fORezkhgiS5DvVUsczHar8sorrAh4yxo/S/7JjWn/v/o/VhNzpcBu+THOxl8ECq5ie+HVIZh0bur76QrvceM3XRjLXhX/zoltwgB7bNRKluErHO5szrTckuymZH8n44o0/TdnoukWFqTHtVuhWs3Y3lAkBdGE/8USNoydgwNNYVsDRnf5UsWWX9bgyRTtHQTCu57kBnrqBU57cP6Df3GoCYKGD+5LIHSrdjOz96HRR/vxRT4lvKNQAG44KU8sA4EgK314hsFX3BC6DcR1z+YGCLpL1W3FJd0AgqGy9Sd1MI8tmtD82Cua5c0UCAM1UzmGnLOKc+azUG5MDdSNcFta9ciSENsrhK0+2mYVb68p+PgWJCMGc47U/W8xl5cvMxTdTwL2cDpNP6ASX07kBI6IylHjJvwnIY7DpBAMKu1pRSmSW04sxaG12uSupAYC/7nIEccQnTECtbirFkDfneCTG4MXpfsaEcCyrgteT5qHa6yTX1Gjp0j4D41Fqc9wy2kJhd7nWA1tNvYwZGYv2+u+GWvkgGvWwcVU1UMZItZGOWq5YREyv75w7XOOz4yXHIwYUHMaXGKXZjmsAx3mv4q94XsaMud5NLdNuKQXZWnsY4LlxGc0DXMHEtUg8leAVEz7ihNk7/EL1SvPOOJ9kfkMPr8iNBbzj5hEf2KWxHB92/aENa8IqJB9tm0M6MzGU/lL8x5N/xvnhseUPAAXtcIEVLz59ygxtABP3yj53vdoh9tcBSgli+4DrnPNPU5cs10UW8Kk/SWmwVuOqTHkPExOOF0ZOR2uflduEOkotWMtzCRRXGruynI5SV7HQnIXGEOg41THUcObg6BZnN zPZPVEIl d0fVAcCQXwupU7VUQSMqWDOCim+9Yo9XaeS5gsfzJ2niSRe4agcbuAwzgS3WtTLm3ZfJI3bY2efuoFoCC3ocN1enhuSN/Qk+PEzc0Wj6UPM+b2HK6M6F1KcQfl91Ty3mIZwNSmcumcHtqKW6et/ua/7IctAdwuDQ2d3TWMJqRA25UW+Y3xNMqu1KWfkawql9tbMHYYGYSWh+CK8ltHd6lrHNzAPZmc07UyKCTg8Ey8M/UEMBsKEXQ90VNQUtZeRwx4l7BSbYluvFU1wa8310PsTYWS3z/Q2CreO99tqG5CaTb+2WbMMw0neqLUAaIUcibzbSe+2QIs/Uy1p6oYeZvAqMxgS+vMSqOzaGoRJ7xZPHkawP2XzANm3yvOT2HZFxd75if6mldDnfLh3D2XyUZ/Z9vxVqXxvGCkFQyViove/Uj7cJW+kbGVnbYY230zrBLJiLztKNYFY73eHuxRailaAxLeQHpTxrdRZfsSsdkEyP9DF4QvYSt9yeaR+n40iazstV29zzuXke0qLM4/NLHQzQMxC6SwTu1TAGFXtkU6Gy8I0ayrvmHn/V086MsSgKS8zHBXovDF5cJHZ07upD3FGPA14029uQacWV3vJ2biupGXQVa1a2Q/QXzZDz7uDKhfBBceyYCvzrVwmlvz1RXmbfbeS7dL/cjt1VMFxWx5ZjDNgA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song This is only one caller now in page fault path, make the result return argument mandatory. Signed-off-by: Kairui Song --- mm/swap_state.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/mm/swap_state.c b/mm/swap_state.c index 6f39aa8394f1..0433a2586c6d 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -913,7 +913,6 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, struct vm_fault *vmf, enum swap_cache_result *result) { - enum swap_cache_result cache_result; struct swap_info_struct *si; struct mempolicy *mpol; void *shadow = NULL; @@ -928,29 +927,27 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, folio = swap_cache_get_folio(entry, vmf, &shadow); if (folio) { + *result = SWAP_CACHE_HIT; page = folio_file_page(folio, swp_offset(entry)); - cache_result = SWAP_CACHE_HIT; goto done; } mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); if (swap_use_no_readahead(si, swp_offset(entry))) { + *result = SWAP_CACHE_BYPASS; page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, vmf->vma->vm_mm); - cache_result = SWAP_CACHE_BYPASS; if (shadow) workingset_refault(page_folio(page), shadow); - } else if (swap_use_vma_readahead(si)) { - page = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); - cache_result = SWAP_CACHE_MISS; } else { - page = swap_cluster_readahead(entry, gfp_mask, mpol, ilx); - cache_result = SWAP_CACHE_MISS; + *result = SWAP_CACHE_MISS; + if (swap_use_vma_readahead(si)) + page = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); + else + page = swap_cluster_readahead(entry, gfp_mask, mpol, ilx); } mpol_cond_put(mpol); done: put_swap_device(si); - if (result) - *result = cache_result; return page; } From patchwork Sun Nov 19 19:47:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460678 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21C16C54FB9 for ; Sun, 19 Nov 2023 19:49:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A14DB6B03D0; Sun, 19 Nov 2023 14:49:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9C3546B03D6; Sun, 19 Nov 2023 14:49:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83A9B6B03D9; Sun, 19 Nov 2023 14:49:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 6FF926B03D0 for ; Sun, 19 Nov 2023 14:49:16 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 532E6402B1 for ; Sun, 19 Nov 2023 19:49:16 +0000 (UTC) X-FDA: 81475742712.02.121FA08 Received: from mail-io1-f52.google.com (mail-io1-f52.google.com [209.85.166.52]) by imf27.hostedemail.com (Postfix) with ESMTP id 85AE940007 for ; Sun, 19 Nov 2023 19:49:14 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=FKQZVLaK; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf27.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.166.52 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423354; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YaE/AcfmvaBX4nhUPBOl45o2uIqec3EidHpbbgh18/A=; b=yn+nLvz/w9/E2niAlamSCHZAL0crKELwPIkBQrY3jsUP1AG4MbTi3gHfQirwJRdtPNifSW 6+XyZ1Lm+tBzOsaiTj5Hyv1aYCRUzr3MsZowSrd47RuksPc5PPo84VLwk2igWtYTCqNoRY //9zm1ujK2zW7QohG5Q9sb7erox4XF4= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=FKQZVLaK; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf27.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.166.52 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423354; a=rsa-sha256; cv=none; b=SgH6mAK0CilR+7xbNY0ZAfyA0bsI2LfytP4il1dgSD9VKi0G1jc4HUNsZQ54vw8+mFr2xk grA9DftLT4S1BWEBWqKdbRAouzePnQf8yMuY2agSSBx8NolZKU2tx65Fv0fEEVKYVBKu/X bSCw3xEvu4JMJJPoIFAHxXodc+wDZDc= Received: by mail-io1-f52.google.com with SMTP id ca18e2360f4ac-7a93b7fedc8so175649839f.1 for ; Sun, 19 Nov 2023 11:49:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423352; x=1701028152; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=YaE/AcfmvaBX4nhUPBOl45o2uIqec3EidHpbbgh18/A=; b=FKQZVLaK/oKhjU/S14i0v+1cZp9875AFtVsHou0w/ui+wDkCViFxVtlXVrSbtsF55K AxhXTqybFNcRpiE7WNRw4dBePek/qiZKAj29srPRfhNl2nnfORPvRtutmCkPoM0GC8Gx lA1OwcJ8v84J+fQP1jXcjvav7iKv3XBNA2j/Y3Q4Spa2ngZ/Muis4jgm/GDCOFkmvZu6 ccnXvKarCeVXH39oP87jyGDYpSVfWMfjRRiRhQwKkKF4kAsv9j0VH1VaKXbbmS/NH8QR QNOvgptDYzNDwOjlgla5Uki4s/AUbUjYyadCY9agC8FZsyU9OhK+hfeWB9jizZUlIfb2 Xcvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423352; x=1701028152; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=YaE/AcfmvaBX4nhUPBOl45o2uIqec3EidHpbbgh18/A=; b=VUXnG8DlkHbjFdACHys/nZCslgJnwzLV3k3kB0zP/3VXfAb/ug1c5AzCWMwFWIqv1m JW02+iK1aPdADikq6tE8EmDhzJLN6KbmW8lVKDASFI262DvIUeabpa5mNf6i4q3cehe5 MGnLuWSlFezUZ5J7TMgUKMnNXJnzY1+gp3P+YCgVYiYgn4CwlhCPMK7cx+VbRmufVtUY 151rPBteabkkasLQfntisgLooi3WR/+dQ9Eucx5UzyIJ6/m/TJqpzJbbP31O181LhhMb GDi3V8P3MT5HoFkJm7Uw5pb+OkX/YKlh8BOxhAbzvkxKdspmS7oNgMRMJXjylkcopYEY uyrQ== X-Gm-Message-State: AOJu0Yw31o6BjBqm4/0qut7yLxE9BzfDgksGpZVgU5zVxGWlO+6gfGpO BdObyBNtxoYCYoQT35EbX3aXHWl3Aev6oUL0 X-Google-Smtp-Source: AGHT+IGe3yrZIEULCUKtOSeAd+ADt5bek5FOv3ZnPoX4w0FfFxyEh6EumJmpzCHvfJdKyxdaJe+uQA== X-Received: by 2002:a05:6e02:2607:b0:35a:fae1:1358 with SMTP id by7-20020a056e02260700b0035afae11358mr5301806ilb.12.1700423352719; Sun, 19 Nov 2023 11:49:12 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.49.09 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:49:12 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 22/24] swap: make swap_cluster_readahead static Date: Mon, 20 Nov 2023 03:47:38 +0800 Message-ID: <20231119194740.94101-23-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 7exkp63x3nomxa7xt1ai3dnm161iis9r X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 85AE940007 X-HE-Tag: 1700423354-692890 X-HE-Meta: U2FsdGVkX1/SaNGMixtSr4sbPzlJLMFoGcWOCywNwBoD0hyfuDdNJQCytgW0JD9FiqEkCYPvXIS+yneypTF8JSX4i2K4yBnNtnIouSg8uWnIneH92GQxgz06w6/77hEgLlPIrcgvN+Pbu2IHe8oGgkN886uKEWhQKDTDwgMadAlL+3P/bnW07eFWEeN+6VwYsWTqNiTs8ZRpSkghehlJajfNPSot5M+GeiAqsVshtHzh47ORkgB14TsVUffqEyPiMkYlfd/3cPnix9/yeSCe2U4x0fM0XCtAChl6B7nGuMTnwiWYVcSmdfKS2jAIWvRhywUAprAZiMb54hCH8OrpFNtI45y3UFexUS0IQ1T+/qV799Z8/NIHlkz1h7VzaIKfbwCTqB5dXtQgoV9fa9PZJfm6QFiN7m6LsJX+OLrFbyHwB6wSA3fQR7ySVMUpB/77P1KKnJLuHCBNagMsdTURitYbXr21RwSu8pH4UPnJAJYmL9HTsI/Mk9/ZgE3mQpFAGNPZZ2f3TJFQ0Jvm6RwZaocoaxScbcaWS18kBiS1p/gQJSoTFtGXuUSaCpbCkNJTb0td4oznNFNsAnEng8RhMPyabOpQlsfj9i+/Ogn8bnkfdh2SKb8MKis4QPdaSS7EeBpb1m82Kluu5HV0TtHRyHp0tdyzxNCDIY60qH4cBcuBRe9on+1h9JVLeE/D7aorK7UjWydXelX0Q2y1UcQp55T2YcMqFiIVBFyA8C7bY/XH1HHPW5xCnxUscjX9ozrr/08QyxEY8Brijv6Xkda81SgEVwFg909hR6J/sQlMGuztuaJqvRNSkbB3K+WjlDEW2avVepxTpK9bJsmBz9nlPTqYgKClTbbg+oXbcOhWGb18vIYiGczzM3O+uclu6n9bdbki9QzaqmkkjDYfWEtmym+g7rHXtDNn9tLfkZYJSGB2jLXxCYgFT+Q7dVGLRJiVv7h54NxuDLAYXHVliFU zi84pdyQ AC0GFkldzKgDIA/9m5z8uOCkFdUxpENkVmm5Mw10Xef5WThCk/cXwzhZ5nkScHTBs+PNRItfNkDcdg74/9gZauEoKVcwOLBxUiDn7lOjhg/sudx3gYsT1xtN1Hb+ZLdRAqyUtuI6B/04NEW0ecjvg73K0QGgVw6z1kkDZdWqiKF/0YF231NZ0AOJZ4zdpLRZcfpL7/q2hF08hPzSz5DGse8XewpWG7d54Bp59iXBdfVVmTXEcf+81S3CpGHR0z90bGbW5K7rK2W3aWnN4gp59Iul64JcbIc1j5g3OGc3bC7Wx2HIWcmtltPm+VfHrTerVIZyajirNXlGELFM/+LnJfH4yMU4OLlWVv9e8QPh+Njqz/wEKobs358+dDhgDgNSUL97om2SoJAXLpfVUOyJct8HY1XDPufRHipvPFUy64ZtP71OCnBOOeWocR5gq5Vv5UcMa8jLFBj+u/WmTdk3c5rcRl2dmFBXCtDvo940E7hiL2yE12RO1scidsF0KOZgQX4hNctCoZmxXjAmUC23agTvAFZcHOSBSamYtRoDJZsXCjpKD4iTHVSUPLW/jGyDiw8enOblqLabv4kSobjxEI03/CzcnalFo6OML7QzM1Ll66fNRB27zqfv6i/pnPlOKF2haaU8kXXd0ARZgrWL2g78iicJ+dr6KrvsTVGlHEsqzJac= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Now there is no caller outside the same file, make it static. Signed-off-by: Kairui Song --- mm/swap.h | 8 -------- mm/swap_state.c | 4 ++-- 2 files changed, 2 insertions(+), 10 deletions(-) diff --git a/mm/swap.h b/mm/swap.h index 4402970547e7..795a25df87da 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -56,8 +56,6 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct mempolicy *mpol, pgoff_t ilx, bool *new_page_allocated); -struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t flag, - struct mempolicy *mpol, pgoff_t ilx); struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, struct vm_fault *vmf, enum swap_cache_result *result); struct page *swapin_page_non_fault(swp_entry_t entry, gfp_t gfp_mask, @@ -93,12 +91,6 @@ static inline void show_swap_cache_info(void) { } -static inline struct page *swap_cluster_readahead(swp_entry_t entry, - gfp_t gfp_mask, struct mempolicy *mpol, pgoff_t ilx) -{ - return NULL; -} - static inline struct page *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask, struct vm_fault *vmf, enum swap_cache_result *result) { diff --git a/mm/swap_state.c b/mm/swap_state.c index 0433a2586c6d..b377e55cb850 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -627,8 +627,8 @@ static unsigned long swapin_nr_pages(unsigned long offset) * are used for every page of the readahead: neighbouring pages on swap * are fairly likely to have been swapped out from the same node. */ -struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t gfp_mask, - struct mempolicy *mpol, pgoff_t ilx) +static struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t gfp_mask, + struct mempolicy *mpol, pgoff_t ilx) { struct page *page; unsigned long entry_offset = swp_offset(entry); From patchwork Sun Nov 19 19:47:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460679 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DDBBDC5AE5B for ; Sun, 19 Nov 2023 19:49:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4BA726B03D6; Sun, 19 Nov 2023 14:49:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 41DB06B03DD; Sun, 19 Nov 2023 14:49:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2BCAC6B03E0; Sun, 19 Nov 2023 14:49:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 197056B03D6 for ; Sun, 19 Nov 2023 14:49:20 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id EEFE4B5628 for ; Sun, 19 Nov 2023 19:49:19 +0000 (UTC) X-FDA: 81475742838.07.4DDBC99 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by imf21.hostedemail.com (Postfix) with ESMTP id 290CA1C0002 for ; Sun, 19 Nov 2023 19:49:17 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=G6RgB0+o; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf21.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423358; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VWrBT21AdvUolArI+Vamx/Z+QXIBnOosXdhjX6fE5gQ=; b=QsOTqYpLd5wYnRwTaQvWhhWbwd5dzWvSSBELtdOKvAJ7ZXkwfGUMkIUMyaEZqmYPwPJZBm kBwQ86/RljqON03NImAj/a9KadxG3fgcOkjV9338Dwl9ease9adVANmUFUYWCACdccK4nK Q2YkIr5DoChm2SDRJtaLFxoAge4AQc8= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=G6RgB0+o; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf21.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423358; a=rsa-sha256; cv=none; b=grk6DiyDc3dfHcy3gRe2Z9m/pw8DU4kIID9OrgYIITlAQDm+otWqRCLV3dLnBQdXruBDlB B0uC/H9TrqLEghqmfv/4YarQwYf5kmLx0hnSbLBw3Y2WBuQHhciCGcEJ5v2ZMjqXcZuO0K 9IWNvvKy0OgSWAa+2DfalyV7e7cwmz4= Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-6c320a821c4so3100307b3a.2 for ; Sun, 19 Nov 2023 11:49:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423356; x=1701028156; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=VWrBT21AdvUolArI+Vamx/Z+QXIBnOosXdhjX6fE5gQ=; b=G6RgB0+osWEJMjIBHy0EXZb+YbOIubyo/6+bPF3oeMzaJWDTvmfaIyW1z1Z0kez8a4 tUw603V7GW5GR4jDmk5Rm6ClEBRs9MUBwbNHoTumqUCVisE5YzZLuREPZ1kj8GDFMsFV yiJ4mnh0AL5s8uEVGejUtBpS1+T++sfgv6iltWKqToQau553ljTs9POx6IfMFLAG2V7b /gcGoCODu8XBVF26gJTy39RFf7Vx9hUUIj45FySWn0CYrdI6RWQbwdwJpAdXVbKa05Sa 4uVM+TA3bhF9IkpMwj/o6X/VofvcmlqgVzQyaqu85ztgm/xko9AAkPGe4HuWWZN+AwN4 H2DA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423356; x=1701028156; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=VWrBT21AdvUolArI+Vamx/Z+QXIBnOosXdhjX6fE5gQ=; b=PFxgftVfQ7Ov9Pov7Tqr70kwoKcG1M7Wlp8wtYL1i2V2F0EiB5EDDXiMieFehXhr3d YY+XHOhvhiBnOU+siT512XpV4/WPYSZo0oDVKBj0w3pH7bGDf+2CvmGVhh24dxJhJq5B IHMk67cUkBkd9iwLBXqtdILVwxajvG3jpJDkzDEpbaC0VJ1u+JIeOFnPmldAVB9u417H CJVg6NacaL/lV63sPZ87nQ3DbmSbUpLVn9W+sb9LfU4prqrsL8lE6kCOby9SCrVtXwup eJYRAKFmZUcfVK4HlRAoMIculsZ7gw2huHpF5lbkogSIrEgfESaeND4rRtuZzgLtBDwd qyIg== X-Gm-Message-State: AOJu0YypKv1sWmPvq/Q6yfFKuXlm5mUT6+W667YbHkKknURXB+f55ykY MLuOrIRCZgbGelRzIAFkZwOlVHEqieszGgPq X-Google-Smtp-Source: AGHT+IGWIkDwN2GaIA67iopevmfTqkLtEBcX4Ge5hWNRIHGYUTrFSZBXVCdyc6Dj2MoCIbUQoxa2mw== X-Received: by 2002:a05:6a00:3492:b0:6cb:a0dc:3d56 with SMTP id cp18-20020a056a00349200b006cba0dc3d56mr660136pfb.0.1700423355969; Sun, 19 Nov 2023 11:49:15 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.49.13 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:49:15 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 23/24] swap: fix multiple swap leak when after cgroup migrate Date: Mon, 20 Nov 2023 03:47:39 +0800 Message-ID: <20231119194740.94101-24-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: 290CA1C0002 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: ph47ttxwq1tmso1jbmme4oijmsztrfza X-HE-Tag: 1700423357-635506 X-HE-Meta: U2FsdGVkX193dHw38WEm9jwSdCBnNHWc3b/VJ0421F6wUqq7D3Y7MpTI+06b9ysIYQFbb27Sj7o8zFhJjZWSNqPX69JSlK/pPEe2ZlPqbqNlbUTnQXJEHRX92BqbzjTtWMeGUrQ0I9v5xD7dBSzD2fqFiLqAPc1Mlrq+TqPp4QfI7ijMCzwmfOm0XhW1FVM68YP86hd9kPiLle3v+Bmi4N4OP8S3ffJHQSKiJdmHq6F8m2l0/aKcfBFqO4WLwAlNhrDLXoxHQWL1t+qwD0UFQaE9QV3HBG3RBkiKnXyTU/aKktV7AHlpaOauUGIqI1EgVvpiRLBZvdfXhk/bgCx2lyGJ7bcz74lmjIQ6694iDgAdNmKZqUDsh38GDkjiIuGwkDVCsgJ/7VtZj6Qz77RC2w/Yex8h1krbYVlTdHdtjOmNJ5GQaJl1DIRDo1x/GltwdbxI6rPGrP2yS6Ykfwj1U29cKGf6Zy5mD1MNGzTS7vX3dudL1fYz8A0dj//7k/0yikI5j9/T2WADwFboK9lq4C6tBeoQnklAiOwroJOTFwM6oUEkbHJAP1Q26bQpbFl7lrrcqmvyxfRBtaIq/ABWi/11Ptg/lxrua0J2hYc+6wBMt511NtjQC+lE0NXXLKHfe1Vs59g0z10oFk6aE82BWIR026ls+zm0tVoenqFxKmYTxabR2kiDbnYGINiGJzzEBzJ8q1A/Ec2XwpGodDvkccbrhcaK+/3dt+JD8hN2Jp7mdsdKl+5LXXGz8mkXj6i7PAhPtS2kXuyow/ULURZ3kdQG14ZZhcXVZJPviOUbOfDBNEOFFSOs2mVBFWozyuIdjdXGzZFP9JA3a7Y+Jd3wFcuPTQ1Bji30unf+xayQZL8DskebSqEvsG44HJn3UAIp4eX9Dggdi2zJvcWUrHuTsKlVO0pAM+V5nI/YXYfo5Nddaf84aC26PgbKJ3HhjmMctKyjmeT3/xk06Am+vjl zLbQbdyc ZsTdK6IRxydFPU5SnZ7o43CNEHmC2dtQa0wDgtgWb7m6ZywPGtOlRxV18Q8Tba5VU5uX8xi07422b9TvJF6AMzoEXi6ZpEVIrABTwzPjgZrSnSNQlB5i6O6st1hOlutKh4pUS0D0z/F7WhMLRyy/2OV6fGMAKDEL8vTH4Ko65lTS8c0sBqTnsJ2/v2Rnl3stZuTaf7N1BEUFW7bR8VN4f1RxQTW9iNBmxfw413L8SkXJkobpCvl8weSK6lVIZTIS+hO3CSDVBCf3nPz84wfwk2YYAtDo5Fi2wcoBJmxqjN/egoYSlZqMKlcdoFrJUCpIK5UVQKjmnQaF50aGAU5IA5ABIvqhSgI9R15KaAgfKKu8TdslY56xjibfgYHRyc2OiBYpOEo5KCf6ar/Lr2acxweEM3RUjYgd0WE1fykVEm0JPxiDUIyIZbD8i/3CzijFZtDqME/zUmsvZ+ZnUWmMdCeYQKDJolPhv5/QMgWXm26N6q1Uu9BYez+gxVYcxJWbv0j8jXNS8DPTink6ECPitfwzH+Eot4TWT64FZi+hvmNFFyM2OEw/kgHLH7DT4cAVzApFLNGQqybMavXHSiqFQXlYXZudIB/muruqdeo0JG38Twjln3e8WwPqFpetqT5XnLOWy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song When a process which previously swapped some memory was moved to another cgroup, and the cgroup it previous in is dead, then swapped in pages will be leaked into rootcg. Previous commits fixed the bug for no readahead path, this commit fix the same issue for readahead path. This can be easily reproduced by: - Setup a SSD or HDD swap. - Create memory cgroup A, B and C. - Spawn process P1 in cgroup A and make it swap out some pages. - Move process P1 to memory cgroup B. - Destroy cgroup A. - Do a swapoff in cgroup C - Swapped in pages is accounted into cgroup C. This patch will fix it make the swapped in pages accounted in cgroup B. Signed-off-by: Kairui Song --- mm/swap.h | 2 +- mm/swap_state.c | 19 ++++++++++--------- mm/zswap.c | 2 +- 3 files changed, 12 insertions(+), 11 deletions(-) diff --git a/mm/swap.h b/mm/swap.h index 795a25df87da..4374bf11ca41 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -55,7 +55,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct swap_iocb **plug); struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct mempolicy *mpol, pgoff_t ilx, - bool *new_page_allocated); + struct mm_struct *mm, bool *new_page_allocated); struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, struct vm_fault *vmf, enum swap_cache_result *result); struct page *swapin_page_non_fault(swp_entry_t entry, gfp_t gfp_mask, diff --git a/mm/swap_state.c b/mm/swap_state.c index b377e55cb850..362a6f674b36 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -416,7 +416,7 @@ struct folio *filemap_get_incore_folio(struct address_space *mapping, struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct mempolicy *mpol, pgoff_t ilx, - bool *new_page_allocated) + struct mm_struct *mm, bool *new_page_allocated) { struct swap_info_struct *si; struct folio *folio; @@ -462,7 +462,7 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, mpol, ilx, numa_node_id()); if (!folio) goto fail_put_swap; - if (mem_cgroup_swapin_charge_folio(folio, NULL, gfp_mask, entry)) + if (mem_cgroup_swapin_charge_folio(folio, mm, gfp_mask, entry)) goto fail_put_folio; /* @@ -540,7 +540,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, mpol = get_vma_policy(vma, addr, 0, &ilx); page = __read_swap_cache_async(entry, gfp_mask, mpol, ilx, - &page_allocated); + vma->vm_mm, &page_allocated); mpol_cond_put(mpol); if (page_allocated) @@ -628,7 +628,8 @@ static unsigned long swapin_nr_pages(unsigned long offset) * are fairly likely to have been swapped out from the same node. */ static struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t gfp_mask, - struct mempolicy *mpol, pgoff_t ilx) + struct mempolicy *mpol, pgoff_t ilx, + struct mm_struct *mm) { struct page *page; unsigned long entry_offset = swp_offset(entry); @@ -657,7 +658,7 @@ static struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t gfp_mask, /* Ok, do the async read-ahead now */ page = __read_swap_cache_async( swp_entry(swp_type(entry), offset), - gfp_mask, mpol, ilx, &page_allocated); + gfp_mask, mpol, ilx, mm, &page_allocated); if (!page) continue; if (page_allocated) { @@ -675,7 +676,7 @@ static struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t gfp_mask, skip: /* The page was likely read above, so no need for plugging here */ page = __read_swap_cache_async(entry, gfp_mask, mpol, ilx, - &page_allocated); + mm, &page_allocated); if (unlikely(page_allocated)) swap_readpage(page, false, NULL); return page; @@ -830,7 +831,7 @@ static struct page *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, pte_unmap(pte); pte = NULL; page = __read_swap_cache_async(entry, gfp_mask, mpol, ilx, - &page_allocated); + vmf->vma->vm_mm, &page_allocated); if (!page) continue; if (page_allocated) { @@ -850,7 +851,7 @@ static struct page *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, skip: /* The page was likely read above, so no need for plugging here */ page = __read_swap_cache_async(targ_entry, gfp_mask, mpol, targ_ilx, - &page_allocated); + vmf->vma->vm_mm, &page_allocated); if (unlikely(page_allocated)) swap_readpage(page, false, NULL); return page; @@ -980,7 +981,7 @@ struct page *swapin_page_non_fault(swp_entry_t entry, gfp_t gfp_mask, workingset_refault(page_folio(page), shadow); cache_result = SWAP_CACHE_BYPASS; } else { - page = swap_cluster_readahead(entry, gfp_mask, mpol, ilx); + page = swap_cluster_readahead(entry, gfp_mask, mpol, ilx, mm); cache_result = SWAP_CACHE_MISS; } done: diff --git a/mm/zswap.c b/mm/zswap.c index 030cc137138f..e2712ff169b1 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1081,7 +1081,7 @@ static int zswap_writeback_entry(struct zswap_entry *entry, /* try to allocate swap cache page */ mpol = get_task_policy(current); page = __read_swap_cache_async(swpentry, GFP_KERNEL, mpol, - NO_INTERLEAVE_INDEX, &page_was_allocated); + NO_INTERLEAVE_INDEX, NULL, &page_was_allocated); if (!page) { ret = -ENOMEM; goto fail; From patchwork Sun Nov 19 19:47:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460680 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40C27C54FB9 for ; Sun, 19 Nov 2023 19:49:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BE7656B03DD; Sun, 19 Nov 2023 14:49:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B6EAF6B03E8; Sun, 19 Nov 2023 14:49:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 99C366B03EA; Sun, 19 Nov 2023 14:49:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 835CC6B03DD for ; Sun, 19 Nov 2023 14:49:22 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 60657C0347 for ; Sun, 19 Nov 2023 19:49:22 +0000 (UTC) X-FDA: 81475742964.27.F756FFF Received: from mail-oi1-f170.google.com (mail-oi1-f170.google.com [209.85.167.170]) by imf17.hostedemail.com (Postfix) with ESMTP id 81FEE40015 for ; Sun, 19 Nov 2023 19:49:20 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OV7DYibq; spf=pass (imf17.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.167.170 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423360; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ujyhaBmVKqUCILaG0//cQCFqqP/+VsPiP59qcIbNEbE=; b=FS22OOS5KOyPSneKbKnEylMtGXBv9nQXW3z/zs6fKkGhmiDFMXPUVmF2raIpPIXBJLd9UM vGZ8WynSUzz4MXytS4bB+ZBXpqpI7ITMiVP3ZARgX1QpRsY/mqboLDf/7KjCrCHBYQibSq WHYopN6LN7obD/Wb/uAhLF8x6CY2OwQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423360; a=rsa-sha256; cv=none; b=ozJiE1SXXPjTwuSwGVpGnKnqaOEiq1Kk1W2rMZ1NbGF2ehx6MOT2aLUNUaVxZWdjGU73rg oe5uhKIr5VKJcge/w+A41HdzaloT2WrxyuaTh7MRblgj7bZvXCnSSNMM9a6HFvgGqOoBFB WH4B+Q9JlnXCLzKvPloHaO674HZATPM= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OV7DYibq; spf=pass (imf17.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.167.170 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-oi1-f170.google.com with SMTP id 5614622812f47-3b2ec9a79bdso2778165b6e.3 for ; Sun, 19 Nov 2023 11:49:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423359; x=1701028159; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=ujyhaBmVKqUCILaG0//cQCFqqP/+VsPiP59qcIbNEbE=; b=OV7DYibqsK13uyakyh7Y66eMAhMYMAwNwqCZIXX5+Z+GXMnHbh3+JwnBKJ4AmmbQ67 jAi9jxn9tCSkuC8WMWT6b9hpkgGD0BJo2TVYhv38By356T6EUXqpdG88gNuS3kllSUtm WYpdvx0/UDNI4yvuLGsDCeCkZNgOSxTgBu+YoddtqqaypiCP8mFaEqt5+9Zyx5L+CJYK FYvtAsnnJsPb4wcSgTEIqKMw1ZkrwR5lIjMqjxs+bgkXbltbnCYEJZn9eh8auMrrkZ7F bkSvtPXD0H4Gvb+jn+5IMVK71rwoQQuosFl629d6uyHtXv3mgA26ZK32pCGAEpFcBPr8 ObsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423359; x=1701028159; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ujyhaBmVKqUCILaG0//cQCFqqP/+VsPiP59qcIbNEbE=; b=IJwJm63eoSIgDT94QAC8mdPv63MEmnYCozGMaUJCDHxnt3wunq5Ma/NtIiiXYOVTxW 25JzA6gL6+BYdZPcaGxyIdeBcIQwtUopmzl7vUnlu8bugwEa/hJ+AD8W3lwV3DP+iSa4 /r19OVnLfMqmSSDQIkoyxYVsC/x4JkU4vzuC459cDSAc0c0FomAmD5e0MbTeBmsG+WP3 Xo48hfCGoOWT1JkjiIM8jjfACqQ7Xowh0vQH4egD8EN2Kahm0jdwrk4RdQIesNzyPydn t05chEn1znoa4CC3jYbaM8qLpbaxyYzeZj1v0yZTNKE9tPcQjHuU9hdxjYFFt06yuyb0 4ECg== X-Gm-Message-State: AOJu0YxsbQ3XBvkd8tujGQsodDAF9kDY+aNdeXOk1LxzhNSTrG+k+WUs qdM638VXU62zHft3qX6cS3twdUh7/hy3bZJX X-Google-Smtp-Source: AGHT+IG9ZMvtFIPRmfdYGQxFIrUp0PVcy1icQvi67iQPNdmmyu8YtKZdvx7dJGBXf+Hyuvvv17Q3Eg== X-Received: by 2002:a05:6808:2389:b0:3b5:75d3:14aa with SMTP id bp9-20020a056808238900b003b575d314aamr9092741oib.25.1700423359201; Sun, 19 Nov 2023 11:49:19 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.49.16 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:49:18 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 24/24] mm/swap: change swapin_readahead to swapin_page_fault Date: Mon, 20 Nov 2023 03:47:40 +0800 Message-ID: <20231119194740.94101-25-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Stat-Signature: hdjsxjs1em79w5fconyd7yzjpw736wtw X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 81FEE40015 X-Rspam-User: X-HE-Tag: 1700423360-94624 X-HE-Meta: U2FsdGVkX1/IT2AW41edbk8ALQGaBP9bGVM6Thy3vQW3V+oOMGFj229Z81qOzq8Tm2UxneMgLE+BLVL6v0fswbWvd2seqHqmeIbOxwXNLz21Ej+nbWCIKv3sxWcLaENqkGZSHIRrWTZhMrZ+am0HKQaeyTdYhHbR1wFE4I3HY6VUW1qe4e+uA9hmklmeA2v9pN/tzFzQBhcfv341/9vj3ITwxozBYPdZbgzZCnch4IAODDxIhMMLKp1NIe8cCv6Wa5TVfT8/WbYic7V9E51yH/AMRgAb2Pi37HaDCx87fFt4ZgzCUJ0jGg421gVMS3t75V1Ox6unoNgYQxwgcA00bjIbOG/viX5rkxkOsKl4xlXwBGgL2yvMbEwF18w6r03Tbjw/GcXzJaF3+TpsAiXuU75eaAo3/y31f2qa/LSXiyiFc1L1hwI6jp7tm2d8lz+VjXJPYOQKiAXDtuIPM1xrBe+uN8ioBgK8kfn+9zGRK5aCNoB53LeAJnnJlipBxVRSlDDBktPcqUkaYvbsvzWfvXzIKygBzES5P5IviRlEoT4DWHSpWviUo60zrQR/kA6UNjvTqrpcwyztPCy9Rwo8+JoSxq7YDHDgixtYjvw5/5gr3HtCvqgQHlASboawI25tWk/Vjer1d+Oy6zw7OC43seoYjLbxxNFmuy3ZhLknFKbWexbR7A2A93CCP8FqiITNueKQdV/eobdCA0hOq6G+aCn1nqzflyFoogRlmnE5tARkukVvLCzw++i0q5r1xT8FeY0+8MTILadAcMw9jqkXxjLyCnGtTTYhmgB70SFeHq48yZ984AaIsY4lRMeC5Fvb9ri+u8yL8UxDdqqXtCowfQAcAlePnTHxjqsa8OxsFL8kuLmlnW4ZP3Jdvv/4ZcdpM4454PJVOG5/P9bVHCVu0lhW+UVJo9FKYtcFh6oxWd15hzQu7e1mK0rCdrTWggczKTIRbN8c8s7H2Vw1N8W 7qAkP3Dz OBxLdt3qpE7MIarv4uXEcxdWMFBQbL6AMTOG8tlzctwMdC8rwabCO9zCHRYSdsleUtDFBj/kVUprTjOTgAkw8nHF6rkm2rZQF2hBQo56TYWi8+XoZGDXOfJ03rTAC7JXvR5aBSHVusmv7fricbEVeY4zyYYGaBZKy285wEFYCtaPVTMbyoLRZFk2+Nb8gQJ0W5VhyFZhDYL/S+DXJMKxrPkLlZnZ+S14r69urwNZrF1bHz/VZ8TqoqAsTWF6ZJrjqVk8Hacn6tE+C1DhPZRI2weIsxWsCj/30i1eugpOeGqPtbf9llAvbiasyix3GJD72naYAgjdw74FOtKlp0WkHHA8FUqe6ExZ6bPDZAGnYzy9Nj4QnsTmrKsCr5ryMCsRHZh6yq8rzOMTNMrHgJkokc6XfvOVV4j57341ag7sIpsUKLEvwdtCAa+YrCCYpjNQmQXFpojbhCBeS5u9PkEgh5Z5WZKc/YxxEo5akL3W1IZ4fcsJLKCFPHDdmVVmVe2YKiphQLE6NagsGymJl9sO7nfclskrb1h53lDVo61i4mX62+kYWPlD53yU09n4cnSlfOupRhCjZigwN80c= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Now swapin_readahead is only called from direct page fault path, so rename it and drop the gfp argument, since there is only one caller always using the same flag for userspace page fault. Signed-off-by: Kairui Song --- mm/memory.c | 4 ++-- mm/swap.h | 6 +++--- mm/swap_state.c | 15 +++++++++------ 3 files changed, 14 insertions(+), 11 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 620fa87557fd..4907a5b1b75b 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3844,8 +3844,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) goto out; } - page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, - vmf, &cache_result); + page = swapin_page_fault(entry, GFP_HIGHUSER_MOVABLE, + vmf, &cache_result); if (IS_ERR_OR_NULL(page)) { /* * Back out if somebody else faulted in this pte diff --git a/mm/swap.h b/mm/swap.h index 4374bf11ca41..2f8f8befff89 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -56,8 +56,8 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct mempolicy *mpol, pgoff_t ilx, struct mm_struct *mm, bool *new_page_allocated); -struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, - struct vm_fault *vmf, enum swap_cache_result *result); +struct page *swapin_page_fault(swp_entry_t entry, gfp_t flag, + struct vm_fault *vmf, enum swap_cache_result *result); struct page *swapin_page_non_fault(swp_entry_t entry, gfp_t gfp_mask, struct mempolicy *mpol, pgoff_t ilx, struct mm_struct *mm, @@ -91,7 +91,7 @@ static inline void show_swap_cache_info(void) { } -static inline struct page *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask, +static inline struct page *swapin_page_fault(swp_entry_t swp, gfp_t gfp_mask, struct vm_fault *vmf, enum swap_cache_result *result) { return NULL; diff --git a/mm/swap_state.c b/mm/swap_state.c index 362a6f674b36..2f51d2e64e59 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -899,7 +899,7 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, } /** - * swapin_readahead - swap in pages in hope we need them soon + * swapin_page_fault - swap in a page from page fault context * @entry: swap entry of this memory * @gfp_mask: memory allocation flags * @vmf: fault information @@ -911,8 +911,8 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, * it will read ahead blocks by cluster-based(ie, physical disk based) * or vma-based(ie, virtual address based on faulty address) readahead. */ -struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, - struct vm_fault *vmf, enum swap_cache_result *result) +struct page *swapin_page_fault(swp_entry_t entry, gfp_t gfp_mask, + struct vm_fault *vmf, enum swap_cache_result *result) { struct swap_info_struct *si; struct mempolicy *mpol; @@ -936,15 +936,18 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); if (swap_use_no_readahead(si, swp_offset(entry))) { *result = SWAP_CACHE_BYPASS; - page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, vmf->vma->vm_mm); + page = swapin_no_readahead(entry, GFP_HIGHUSER_MOVABLE, + mpol, ilx, vmf->vma->vm_mm); if (shadow) workingset_refault(page_folio(page), shadow); } else { *result = SWAP_CACHE_MISS; if (swap_use_vma_readahead(si)) - page = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); + page = swap_vma_readahead(entry, GFP_HIGHUSER_MOVABLE, + mpol, ilx, vmf); else - page = swap_cluster_readahead(entry, gfp_mask, mpol, ilx); + page = swap_cluster_readahead(entry, GFP_HIGHUSER_MOVABLE, + mpol, ilx, vmf->vma->vm_mm); } mpol_cond_put(mpol); done: