From patchwork Mon Jun 3 03:31:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lance Yang X-Patchwork-Id: 13683226 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62E9FC25B75 for ; Mon, 3 Jun 2024 03:31:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8026C6B0098; Sun, 2 Jun 2024 23:31:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 78B6A6B009A; Sun, 2 Jun 2024 23:31:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 604D56B009E; Sun, 2 Jun 2024 23:31:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 3EF3A6B0098 for ; Sun, 2 Jun 2024 23:31:50 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 8EAAE40656 for ; Mon, 3 Jun 2024 03:31:49 +0000 (UTC) X-FDA: 82188153138.17.81B86AC Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by imf12.hostedemail.com (Postfix) with ESMTP id D5C2740019 for ; Mon, 3 Jun 2024 03:31:47 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=dGHHhQ87; spf=pass (imf12.hostedemail.com: domain of ioworker0@gmail.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=ioworker0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717385507; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=wnXzlpt+ewq8VcN12LI23XP9wCUpW3QP7SwmLBsuIIs=; b=WXo3hACaxzjOd+aaaLo3Dnlpt8dhU1c/DVYiCpzOq+eQgOAfuf++esg3V+Sa3gFfEZFe35 RBPqT1eaAQcXMspbQ6wQGFujycXubS9SEyjinJEBMqiXBTZ0rBHiWp2Ni+khLbFyphV2gl CpZu3QfLnB0cR77SW9jH7eq5OcoAyew= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717385507; a=rsa-sha256; cv=none; b=AauQ/AM14brmoTFC8SmpIBueao74UERQ+LOjAu9EuPdiblTr6fArKI5n8IY3c6Msszbzhv FN/NGVyHAACCAZLERqcQu1wrshhT1r/B9zBadaDllPNJd1F+QC6496Bn8RssiRVKEz4bS9 rHYeMNSWI2yymLEdaGHinWqpl0G+yks= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=dGHHhQ87; spf=pass (imf12.hostedemail.com: domain of ioworker0@gmail.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=ioworker0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-1f630e35a01so21581825ad.1 for ; Sun, 02 Jun 2024 20:31:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717385507; x=1717990307; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=wnXzlpt+ewq8VcN12LI23XP9wCUpW3QP7SwmLBsuIIs=; b=dGHHhQ87G3PboKzLgvSd6WkaPfe9OL70e2r18tVL+adOiEZeIHx4V6J1842cCYKw2a SxVvHXtBJdzx5RxiHs9lAOYl+dDL8vKXAHLUEP7s6X5ks5pPyjd9dwxQDUXjXiaHkESW RJmZ6msKFfMwa6f96X0GF5cgDDrScX2NUnBAb3YIvRkGq/biCwB8UhWfnpBdTEeL+XUD aMlDwAC6FFB31uXfGNtMe+EfVYMQmS0+4tGbmrHiPzkfztyO8Y2Z8GRGuMRraiQoUcSD m7uOo5Z2WesxQ7cHwG9Mbfy3zTjpgNNY6rwD2tKxOpf8Mcyxgbxmriu5cIwBr/bNq0qC U/kA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717385507; x=1717990307; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=wnXzlpt+ewq8VcN12LI23XP9wCUpW3QP7SwmLBsuIIs=; b=sC8cF6OTIJqg0DUU8LsAatE0T+Y9c9I9nvkjTrJspJVgf575sxwPIAaSiTDathM1dU f7lDZa8qK/T6bpJvSi391/kszhciRvXtCEKrn5YXYDiPdCNUQGr2BYUCDb6/U0hCOpD3 k2tGrlPmPUdguj6TcFrteVCKPpPlkFSeky/ZUiDyy6VZI/W7AL1UHkyjkryZobC7hhIE vjvcyOlCFWjadkCpLcSXEeAWYEZ5/r55yhjSNN7brZKjDnUGDnz9EvpgGt8h6u3myQiI 7A5/oYajjkPvbablfEiXk/9rENOqhVF8EUVX6u9Lf9BlvE1YKHRF2itwHLNGqaOohgE5 Yu/g== X-Forwarded-Encrypted: i=1; AJvYcCU9EkQK3Nml5eGxFOuq6PPCgp9c+ze7Yyb1JO9Ujc3AEg8W1OdKMBCQANcbqoG5QJJM1zEM6A6Bmx+n/7rK2DKLU1s= X-Gm-Message-State: AOJu0YwKvS53RNs76QMQxbjol5QapVPv6a+9+833xEwp0QZQEnbepBDY cWq5PL+YWqrrgAqLhz1cUxZ1iC78s5oIUEsfV3B/mAylwe6R8Q2D X-Google-Smtp-Source: AGHT+IEKzX/tqYS1avrxEP31ld9oCkHDAefsyiUM5IBHFIPplNgcj6wzYu7/aa2RlIzfrp+zBiaX5A== X-Received: by 2002:a17:902:f689:b0:1f6:7e02:6af2 with SMTP id d9443c01a7336-1f67e026dfbmr16656045ad.22.1717385506595; Sun, 02 Jun 2024 20:31:46 -0700 (PDT) Received: from LancedeMBP.lan.lan ([2403:2c80:6::3038]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f63235f88dsm54568635ad.88.2024.06.02.20.31.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jun 2024 20:31:46 -0700 (PDT) From: Lance Yang To: akpm@linux-foundation.org Cc: ryan.roberts@arm.com, david@redhat.com, 21cnbao@gmail.com, baolin.wang@linux.alibaba.com, ziy@nvidia.com, fengwei.yin@intel.com, ying.huang@intel.com, libang.li@antgroup.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Lance Yang Subject: [PATCH 1/1] mm/mlock: implement folio_mlock_step() using folio_pte_batch() Date: Mon, 3 Jun 2024 11:31:17 +0800 Message-Id: <20240603033118.76457-1-ioworker0@gmail.com> X-Mailer: git-send-email 2.33.1 MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: me5j4zeg9pez4y4dwbeszchyt4nyx1zj X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: D5C2740019 X-HE-Tag: 1717385507-388996 X-HE-Meta: U2FsdGVkX19hrJcu+aUACCWiohfKk06eDlwvM/Ehw2CxzSy5VU1jOluHKCdvvas0egpl9rO1zTPJGwzMHxeSM/20nFhqXUH2BSrYEr5BzndgIfddZGtY2CRhpPKFYMgjbQwgjOMbEPPPV9opJ/Yr+XTdAZkQEeZ1Ggy1E7VeaumaLd0Ns1f4zKvdv5/v74Aib1eWlAHVpN06OaHacpsfGufP8ZI6BHqRbqB+9hJlVcILlhBS1Cx/comv+DyjhTur3tYXeG/Ht8BwXc5WkXl+3BIknj5NJUo/xhjvX/EwzVkClTQ4OD725Q0azjW1+4DnwBR1m4DvdkcNPknPRD+92fnVRnAcuOQycD806Cou/hLdL+Ojp+yNiaCRKWdOK0/JkuTXPzvHFeLUSKT8KlLBSNP3Bvxm+fO3AK2rm4hoQ8i/+CR9PKIqAssWzGCQgfWhsQY7prblriqFoiADRqfKF32TGCLXg5WUiUi34umWu+GBTP8vo/C+8frdi5ZIQ7ugsD/8FehbrNKLAZz4XiqTYCZ02OhCxgiEYqgUf4JkC/yimu1Hd7kKHoGn0mKwOpqdz3bibSnyj79DP6s78nynYQGdiupXxYc9pzoTC6C2LO1Cx4YAYCKfwNWICSzzlYxCbjY1tVpsuU1bin56uLR+5Bbx/dl2OHYEq8gFZO2vnL2pwdTyVXzDCtzie/DqVIZCgZVxG4zBfrR2NdMr2v18Z5LhByHxX6pzm3O4LPVH5+Gg6dGioxrWpjc4rUOv0X+a2WyFrH4jODRrt/Hbnn8pl+DXCXH0g9KqZD7rUmuO23zCg2QtWyV5Z8RXjA/FEpuc3LP0FG9deHJVbkWRFH9tgej2Q03mnuAnhAdzZqbvEk8absnbplMjXFHz/Oquj40SID/Gx38W+ECH9ZYQoK9PIoZ0lL0BATvNF/P87nsXLfPGzU9awwI/KT+QhwOa9X4CYY6gFFPmmTq4rvz71op d6EqzPPv t6elg/MWF1QQ7LGs1Spx7Vt4XcyFaQVN/y271f5xGHZVuR7STUGI+Q1cN9ErCGWXk41JG088k7U81sqVC8KBfHV7Vb1MFkmjqMNu/6g+gIFDm93tL+EAI49uFPrmPLiB2GLyiT/pioQGXV8imVqGpEmzmo9AIhJvtYYcaChEgWdYJyVRV0MMY3UGxVv4+Y7vCxaBNs5l5G5Vrs412aw60f0E1y9tmzB4XHCnui+d+vlNF4TX4PqsjVp8iNmuOUW9nfqJNUVHMIyS3PCNBLBmWN2Hq7nvP5yjOmHDujYKyjQQtmCDHzSTPKLXe56JGzULFjtXRZpDkYVqPu6w7wE4qVGZHgGy61em9XCZFOgl+6GwSJOCVb2I/pesb5X5Rea4hmxbB3seTtrlJhVeEO8kc5uKRTmDGr6u+axcO/yAPL1B4063RIkoQ8zTNFWZNmnkBYrN3wvLwAoXgfTc1UEnsPXGZZlK2JSuUDTY/2k04U11AeojYgsoIydbNJbwzE/tXazY+YHZzdst2IoXAMl4s2kUPUf5yxjqYxxdB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Let's make folio_mlock_step() simply a wrapper around folio_pte_batch(), which will greatly reduce the cost of ptep_get() when scanning a range of contptes. Signed-off-by: Lance Yang Reviewed-by: Baolin Wang Acked-by: David Hildenbrand --- mm/mlock.c | 23 ++++++----------------- 1 file changed, 6 insertions(+), 17 deletions(-) diff --git a/mm/mlock.c b/mm/mlock.c index 30b51cdea89d..1ae6232d38cf 100644 --- a/mm/mlock.c +++ b/mm/mlock.c @@ -307,26 +307,15 @@ void munlock_folio(struct folio *folio) static inline unsigned int folio_mlock_step(struct folio *folio, pte_t *pte, unsigned long addr, unsigned long end) { - unsigned int count, i, nr = folio_nr_pages(folio); - unsigned long pfn = folio_pfn(folio); - pte_t ptent = ptep_get(pte); - - if (!folio_test_large(folio)) + if (likely(!folio_test_large(folio))) return 1; - count = pfn + nr - pte_pfn(ptent); - count = min_t(unsigned int, count, (end - addr) >> PAGE_SHIFT); - - for (i = 0; i < count; i++, pte++) { - pte_t entry = ptep_get(pte); - - if (!pte_present(entry)) - break; - if (pte_pfn(entry) - pfn >= nr) - break; - } + const fpb_t fpb_flags = FPB_IGNORE_DIRTY | FPB_IGNORE_SOFT_DIRTY; + int max_nr = (end - addr) / PAGE_SIZE; + pte_t ptent = ptep_get(pte); - return i; + return folio_pte_batch(folio, addr, pte, ptent, max_nr, fpb_flags, NULL, + NULL, NULL); } static inline bool allow_mlock_munlock(struct folio *folio,