From patchwork Tue Apr 9 08:26:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13621995 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99E30CD1284 for ; Tue, 9 Apr 2024 08:27:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EDD976B009A; Tue, 9 Apr 2024 04:27:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E8E356B009B; Tue, 9 Apr 2024 04:27:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2F326B009C; Tue, 9 Apr 2024 04:27:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B09266B009A for ; Tue, 9 Apr 2024 04:27:13 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 78ADEC02C7 for ; Tue, 9 Apr 2024 08:27:13 +0000 (UTC) X-FDA: 81989313546.13.CF3D9B0 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf13.hostedemail.com (Postfix) with ESMTP id 9E60C20005 for ; Tue, 9 Apr 2024 08:27:11 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=WG9nMih4; spf=pass (imf13.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712651231; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9ZXg/AhadeS891XoBBBMkIgTpnMXwbdxyyz0Yc4z5x8=; b=D8ukdxalbrgUONxZFQixYUGOLQMv1K7QVJ+QoHXdexLmlsQK6GWBH4XCrc969i/VAJUH+j AaFnHBX076RiJAq3iUKtHFS6gY+6P6uFOp25B+gHH5wVrnaEg6sAx6smSp+tnEy6YWj9HW Q3H9E4jkeilQFW97lp44cxFjngbCJgQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712651231; a=rsa-sha256; cv=none; b=pgmxHqMID/gpJYsZjDK7a1NBjW5DaG/qfuLolBXrMZctgYW/Cry5ewEvPotm1usTRmCIXE MfAreetK3uVRo5VwR3J3VmXtS2ZMvlJdxhWyO90aDWxWqhVXhb8uGnf7SOWMKxgv7C3TXa aOMWCyQRFm53cY2tAwZcYDBGyAUwz8Y= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=WG9nMih4; spf=pass (imf13.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pj1-f53.google.com with SMTP id 98e67ed59e1d1-2a52c544077so1216801a91.1 for ; Tue, 09 Apr 2024 01:27:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712651230; x=1713256030; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9ZXg/AhadeS891XoBBBMkIgTpnMXwbdxyyz0Yc4z5x8=; b=WG9nMih4kxZ2dM6X98GQXSAuurlIhbkCtAGL7qcrKWjpCRd++aQnRzPlHW4rTbOGAt paM2tWD50L2ytQN+dhAvGdqkHx5/VXIo6xWunG0Tq9We5NaefaAcqGm57LYRVavm/SD1 +E/OHyAZuA3icBOD7MBMPuH/YGR/WFz3QYAzzgAfIMz8BJqlVRx/ruuGa7CRfsnxkueU PccstpM6RSzv8rIfsHefKd2Mgrcb0aCqC9H6jbaNMGjkED2LqgwTTNTlsbsLS4i4MFIP Z0fU3NY+tuWlhPDPz/rlI8mQ+jchtLmjEcurPJbXZyhrNp5CdBLkZL8Mv/bjAwd2uNCI EmzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712651230; x=1713256030; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9ZXg/AhadeS891XoBBBMkIgTpnMXwbdxyyz0Yc4z5x8=; b=YGxW/mnNw5hE9Zh7twyakldsOzQsNHgImvk4wU37eSKLJCKhrv3RxqAGG9nE5tG92V n7Yu/sbkhR1cpni9lJxtLgnOGaljzWdRyx+dZPoEOZLma48ycXg7jk2WUqdz6epIh1hk 1Ki3pQWRK5oz4n2z3uLZRMlcdcSY9x2vwJUF5f+cwGUrEbTAldNmQnkOOL9BN3pc41s+ eZNVAx4eFTMlx1/vXXE/GKQCnJsJdVnm0r17hCFyAgnQcL3waecXl+FB3s/z+aZHcPgA /R7Py/HQ4uPiYHbhMQI0ADr+sLC2GZCzU+0bBiZxaqfTjLSPiFpYyC/Aw6Hks421H0j5 xFjw== X-Forwarded-Encrypted: i=1; AJvYcCXyXEsZEHG1hD6rulw/QKOjT2xIrRk2wFeHjrLANynD6c3YFlaeNMv5hzwjuPZy0C0udoL2N+hrx4lfyWgNYjqB0wM= X-Gm-Message-State: AOJu0YxWELrjjNpezhSmkh8pOZdUuK1xDBvefU0yGwjJiMewX3MnCAqo 35y4k30j8utvfMEB4FXKvP5GUaEwZGe+L9j5in3ymCXBtiUFRS3U X-Google-Smtp-Source: AGHT+IFg5AbIygO1G/4Sf/xhmF54Nd1zJbM7cGgBTgS8FOC9fn0UWuAxj2wjNfCbUfjBKgj68N+xMw== X-Received: by 2002:a17:90b:3587:b0:2a4:e962:b62c with SMTP id mm7-20020a17090b358700b002a4e962b62cmr4228631pjb.49.1712651230311; Tue, 09 Apr 2024 01:27:10 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id r16-20020a632050000000b005f05c88c149sm7594238pgm.71.2024.04.09.01.27.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Apr 2024 01:27:09 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: baolin.wang@linux.alibaba.com, chrisl@kernel.org, david@redhat.com, hanchuanhua@oppo.com, hannes@cmpxchg.org, hughd@google.com, kasong@tencent.com, ryan.roberts@arm.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, xiang@kernel.org, ying.huang@intel.com, yosryahmed@google.com, yuzhao@google.com, ziy@nvidia.com, linux-kernel@vger.kernel.org Subject: [PATCH v2 3/5] mm: swap_pte_batch: add an output argument to reture if all swap entries are exclusive Date: Tue, 9 Apr 2024 20:26:29 +1200 Message-Id: <20240409082631.187483-4-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240409082631.187483-1-21cnbao@gmail.com> References: <20240409082631.187483-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Stat-Signature: hqtnxsup38mk9z38i7sje58ocrboi9qg X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 9E60C20005 X-Rspam-User: X-HE-Tag: 1712651231-856992 X-HE-Meta: U2FsdGVkX1/XBwazRswzB73dN5RD3HVxL2WCR40yZYGtvKo4FmjleGqtvbni1gpcRHGSHPA1T1DlyOH1q5JZKf8VrYRzTXwSxsT8dw6cPujWDT7iqiT6KpmZ13HHGODL7VcYlcdJNd+eRFINCteb+XgTyWQMIMi4H3ZB9EkAdGLQ/dp1MGngTbh8oxgxaip2QyrjTq8n4RalkSSwLAVYShPvL4yYemAYYCFqdMTq7j1rpr+qQMW4vHWQ3jvSewfVK5Uj+oE5MBKzic3EC/eoS1cUbcxbIvE22+T8hTkV8+W5o9NrfS6O8W5QEhrcyWg8WJIGRH0w0k3Qvj2wbU+z4NxCCDcgUqZu664mVfYOJoT9Y6XmjuQKwGp6IJmA3cRZlWjXU6aCAXIjYz3z3dSlWnAHSDXdfMu7sOHK7+Cp44IS1vyecsj5I1i27ez3qJE4Q4ElxS45xLcR8DtkoZE5No3tDUkHUZDZvuFY9Z0xRJ/9H/AuX3KxzAQmlv9kmf7pr+5cMiz0TabZBrPpUvvoOaIBxTf1S2at8nszcY/WwugEPOYREZOwmysibCmQX6sB2D42GgrKsfXSfwxLAjujLEKYsArjaapmOg4WXoXhSTNa04/YRYOY08DO5Bwo7AgJGtK09JT8yDo32ICIMBAVRuu3hC2L5Y/MejslDPBVFkkSJ+2FcRrHGB5lzaUvBh6FJ8hLu6xB57SJYTD3w8ymZYBZ6Mr8GvJ/OXgsvL+DbTCs3XHooJpSNvJvuKo3ydNL720FCLbDxW5C2quACbVvvfqKRFGpTaYfX8R7voqxfRydpaKbBOYMu6r9yZABWVzPiTTssbaofPaJBjboa9pFugJx+YKmJqm5OtXIaQTaT6jpc97eUy79r24sKhq3Rsoy68OoJOU9Sz6gsTtQ8eKvDysg8ldZQ1uSG26+vtZrcA2KUXTD2SzSCk2RT1izehtTqSN2JyV8K317157q+Fl pkUTzpVl sdT0H9ccHW8eUQjz7zCjLOqEoP13pWiOSfkOngCt3Sj/VH473UZwN9azrKV506bxlI0uwIQ6acBIyDrIA0b8L+OFnuZ77BQIvJNNKvHYJAsqUgfdRs8DODlbfGO85A+8YXbQ+utPFgES/Fa8JPeunwl6lIdmqOMxAIGaOnaGiE91GtfA0/X1kdtA+t6HgMWUkxWCgjQao9+ziuCHVVDf0J9O85NmiZu8x2oWYoyy5TI2UY/wQaGlerpF98oEmLRrFxFDN8TDzajO4E+upMJR+dUm1h1ZZ4aM9QToTMhy6iPvsypOpmBfgmH4B5ZZqTb3ionri1R230cAlPEAMxcgdkgUOuLX5Z07M1VUnCbL1Va6zOZ3z9PomRXSzCE05VgSERxpSE0b2TWq6yMlfAzyiWUWX+5Zb2VkyQ01HTtO4a/GDI0G1XSl7ntnJX4z42hMJV5kd8H8/BdUlySYQ741Ww7tq16kB2cF1S/zN9VRVgYOzQQeIASf843rtEfg4g9pGTxA0GEJdgqtnyWoE2wgH6+p1RF4ZPJCw3pNOrud8vxWeVTmmjLR3/bBcYxf6Dr3OyYs7 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song Add a boolean argument named any_shared. If any of the swap entries are non-exclusive, set any_shared to true. The function do_swap_page() can then utilize this information to determine whether the entire large folio can be reused. Signed-off-by: Barry Song --- mm/internal.h | 9 ++++++++- mm/madvise.c | 2 +- mm/memory.c | 2 +- 3 files changed, 10 insertions(+), 3 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 9d3250b4a08a..cae39c372bfc 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -238,7 +238,8 @@ static inline pte_t pte_next_swp_offset(pte_t pte) * * Return: the number of table entries in the batch. */ -static inline int swap_pte_batch(pte_t *start_ptep, int max_nr, pte_t pte) +static inline int swap_pte_batch(pte_t *start_ptep, int max_nr, pte_t pte, + bool *any_shared) { pte_t expected_pte = pte_next_swp_offset(pte); const pte_t *end_ptep = start_ptep + max_nr; @@ -248,12 +249,18 @@ static inline int swap_pte_batch(pte_t *start_ptep, int max_nr, pte_t pte) VM_WARN_ON(!is_swap_pte(pte)); VM_WARN_ON(non_swap_entry(pte_to_swp_entry(pte))); + if (any_shared) + *any_shared |= !pte_swp_exclusive(pte); + while (ptep < end_ptep) { pte = ptep_get(ptep); if (!pte_same(pte, expected_pte)) break; + if (any_shared) + *any_shared |= !pte_swp_exclusive(pte); + expected_pte = pte_next_swp_offset(expected_pte); ptep++; } diff --git a/mm/madvise.c b/mm/madvise.c index f59169888b8e..d34ca6983227 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -671,7 +671,7 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, entry = pte_to_swp_entry(ptent); if (!non_swap_entry(entry)) { max_nr = (end - addr) / PAGE_SIZE; - nr = swap_pte_batch(pte, max_nr, ptent); + nr = swap_pte_batch(pte, max_nr, ptent, NULL); nr_swap -= nr; free_swap_and_cache_nr(entry, nr); clear_not_present_full_ptes(mm, addr, pte, nr, tlb->fullmm); diff --git a/mm/memory.c b/mm/memory.c index 2702d449880e..c4a52e8d740a 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1638,7 +1638,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, folio_put(folio); } else if (!non_swap_entry(entry)) { max_nr = (end - addr) / PAGE_SIZE; - nr = swap_pte_batch(pte, max_nr, ptent); + nr = swap_pte_batch(pte, max_nr, ptent, NULL); /* Genuine swap entries, hence a private anon pages */ if (!should_zap_cows(details)) continue;