From patchwork Mon Jun 3 14:07:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lance Yang X-Patchwork-Id: 13683875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 037A2C25B75 for ; Mon, 3 Jun 2024 14:08:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 720386B0085; Mon, 3 Jun 2024 10:08:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D0846B0089; Mon, 3 Jun 2024 10:08:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 570766B008A; Mon, 3 Jun 2024 10:08:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 3A9436B0085 for ; Mon, 3 Jun 2024 10:08:11 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id A4CE8161186 for ; Mon, 3 Jun 2024 14:08:10 +0000 (UTC) X-FDA: 82189756740.06.7A27386 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf11.hostedemail.com (Postfix) with ESMTP id E5A9440025 for ; Mon, 3 Jun 2024 14:08:07 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Cs8xQ305; spf=pass (imf11.hostedemail.com: domain of ioworker0@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=ioworker0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717423688; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=j58eNeNH7rsPU3MjTHYc+gexQAkNzCD+uWaIEfNn8EM=; b=zMxvuh25BAxhEqLRrsv++ldFy4l6C1yHwMrQ/Rep5RTDvvvUCdMnQz8mXb5oWRolzOVoM0 3sAdK6YuZtLBfBx0N63Z6Xx12tcthohU+/WJLPTgpeaiYVXS4pPZGVtpfQ3MfPSuR3YF/1 D4bW7FRmkwuj2pj8Wf6J/pN5aLFQzj8= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Cs8xQ305; spf=pass (imf11.hostedemail.com: domain of ioworker0@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=ioworker0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717423688; a=rsa-sha256; cv=none; b=heIQ5Mk6LtfElJQPeMJra4usG8Pn8MEboBws2hxnqEEr9pKYGPU3FEgHLAE+TrkD4pzGDC awtdCb4Ix2+nHxUqOH40hBeI+PpPMxK+wQIWhj2tohE6W0qmH8HelqKAfP94hPSt90QJev 8ZVzjhIIGB1zA1nnPwW/fn/rlXEkOWM= Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-1f480624d0fso32713285ad.1 for ; Mon, 03 Jun 2024 07:08:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717423686; x=1718028486; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=j58eNeNH7rsPU3MjTHYc+gexQAkNzCD+uWaIEfNn8EM=; b=Cs8xQ305OtviRomzO6yZnYJDONyiWSHcR6YjFcxp8o3non+c3Y1Rz5RTfP843diy51 lXmuQ8BYIzGzbaBgPKC2WzqlryoG1L2zDTIRyuSC2G1TH4VqEubakGWyUlF17Ovay7qS mwulaYMNAMpADYXsGYhHHbVVCaNjXv/nN/a+BKrVe7E9T6WadDVIi4Bo1EokqfEsWzff I81fUcoMUH0b4LbJIuzVQkOlmBkBPtm+yDfRSw9ddnogRq28yIVU/vx+ehIEgU9lYoPN MlDqECdPOS/ffZe/77LwLdBHpomW7L548gBbnOIso4AtD2BhXd6XdTo8/bHV3kNSn1ys Pl2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717423686; x=1718028486; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=j58eNeNH7rsPU3MjTHYc+gexQAkNzCD+uWaIEfNn8EM=; b=hu9nloI6dsOR9xQjhYxSrVHCFOsyPUAl6arXAk2ijOQv0azr+XL5DFD7JGlw7kJJdw 7iEiAjoq3SbNO8liLe0L3Yrj2WlsbHMUGOjBUJCVT1OuCLL1OtUXo4Fg31jYzt86vwy9 tkHdUwwYxJ7qBkKIo97J5q/0yf1110e+ZyUeIjQhSKVxaurQZlgxwnOBGL8bQe9g7m8i rcBFAkAnMBDbZt0NDf03E69VmglTYqNLsjGNqOr/jMUj3Zp5zhZvXFnQW6/XTAkFCiiK NW0teerYoYYHpReU6ZC6bGxXZRPYTm9xX8Lfy3TpAVj7gtGLNqUQk/E8EWsOdgNr8tnE BonQ== X-Forwarded-Encrypted: i=1; AJvYcCWjvNrI/nIOMvlhgMkAsJ7GvoqazlF5Zf8iUtO2ErB80oUin3dmPU8HlASLW6nFe2BbC88CAu35BA0/KZnaIpabQ14= X-Gm-Message-State: AOJu0Yx/lcrJl1avlJLCLi25NmzCqPsxKafoM8IqESXtgjOLeXAjz5JJ ELp/h/PUfsi5BDKB41yR4Oh1K7WjIBleI31g2Eq+pU4VLkoQdX6d X-Google-Smtp-Source: AGHT+IHlLzn+EplsEOgPQIlc0aZVcgcXn/VkhWI9rGprxUKyJdZQeE82lsMwbmTU2WwZstwh/3UlcQ== X-Received: by 2002:a17:902:e54a:b0:1f4:a52a:dca4 with SMTP id d9443c01a7336-1f6370eee8fmr107153315ad.69.1717423686295; Mon, 03 Jun 2024 07:08:06 -0700 (PDT) Received: from LancedeMBP.lan.lan ([2403:2c80:6::3039]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f67aa290fesm20624485ad.289.2024.06.03.07.08.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Jun 2024 07:08:05 -0700 (PDT) From: Lance Yang To: akpm@linux-foundation.org Cc: ryan.roberts@arm.com, david@redhat.com, 21cnbao@gmail.com, baolin.wang@linux.alibaba.com, ziy@nvidia.com, fengwei.yin@intel.com, ying.huang@intel.com, libang.li@antgroup.com, willy@infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Lance Yang Subject: [PATCH v2 1/1] mm/mlock: implement folio_mlock_step() using folio_pte_batch() Date: Mon, 3 Jun 2024 22:07:45 +0800 Message-Id: <20240603140745.83880-1-ioworker0@gmail.com> X-Mailer: git-send-email 2.33.1 MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: E5A9440025 X-Stat-Signature: wfj5doeiuxonoenexcq4rrgdqfuc7uup X-Rspam-User: X-HE-Tag: 1717423687-659108 X-HE-Meta: U2FsdGVkX1/pdqzz4k27tkGZoeAPbVor2NJ7eG0K+RG+KRZGxWOvFEF3YLdujF8syUiQekCNiMupeNA24e2BHiTRfQa3FO6NuYva3w49Ggg/g9pjQbL+K6bGzRbaTMlBBaLCuOEg80IYlGg5IILkdqWhg7sAzPSKFE22NrJivPzm1ZWAy7yf3rsBqYw4YHF351zqpdbGZbK27BB33+ljwVn78Dkcu0+Y4F/AGAvNQ+4AmWZbmp7f/62tVzdWfrqZ5IQe84jV8anZ90GrBw8ivNYiwwcMoqWF1CiFGfbSsNJztx8T+Ct+d9iJ1GlywBLSyP3sdYeJ0tLbxxtI/iYKKtOUGHr9ACsWrNft0wjNtd3KlaEOq5/RSSth8JkvJTO8es/jV7HDT5g30k5qViqmNR8HpMmo/At/afVtbzDuVnnitcZfpqJsCAJ3cIdoT8sYq3ccwsYJb6hkHLeT80XC3K+tP4d2pwveObD9mqfSQUm4/tO92VgN0y8tRphIvYYlYNrou2IQdtcRnTxE9UA6lD3WNkyAMxgNHjkIn/h+k6gQxjKU2bHRyRTx7IL6BgZ34lnVLjq6nZu0WMhrYo8sZkUT5oXCQOanOY09Iq4UFGimd0Attbfl0QAig+qlNlZVUcEA1awXCa6P27f8ETAeBmcNXovUWjyOWfwoCUhvsJKH8fyDziO/Djyg76kcQcqrQ0yMTuv/QAa4Yxx6jBhC6AJ9vOVbDpHoCXjVoXM+9OQlwcTs0MjEDXMhqRqAr0gAspCJWt+a44S5q9VuLRAR216sf5MiE/Y2Hspi8HptJ+OqecoRQlv2oxJ3V2pcuhJiXqmmS+IcT4M0ojk14EXTZ6xEErsnnKJYdj+oaCqhVX5HxbN8s5bexgmLgqnkY/5Op69U1rU903BCkhnDSuZFz92tOoYHPJjTTcxTRhfAn5lDBkB8jwHjKBbegR4991ai8jMFBA737QXWr4U3Lqk +j8eBymT XiYaSzKE7jHQUGzk3xEMgnAa5GzcnZl77a/QemVDrrUpQmdUTKtNlypP/p3BAqjQBkWwwe68dyjpl73rUgXVnrfxm0HtyYwdp98XI/cEhaP+3bE00VlwJvHr6Qmd3zibN+ikF1J7BjVmmzS1lTA+AUUc1IBu1YQaz/6OvsJheptJgBFTzoi/cycGDt4oYSvZW3ROkQD/JgFMdb3d0ny7NVX4QY2YuuDG7o+oaVCFWAb4RzNMmHPFvhvAHPZiqeU7gReQ5kaemGBs72W88KwrtiJeLayQeub37k5xL3Ci5pW2GjZKYP51qlQZp/ilC9kbN+aK5urmwtjqU1mY94KQf9kMt4qejtX7kGTWcZVY78ajNBk0ocsdBcb/BlrYMrJ2CxRRhZUZR0YEIHpfy0YB71UG73yqgtCOu4u3YH76ceXKErCwx+7uWQehNPXTvsrVsYmdAqPgqJt88JZsVXKs39O020rId8FK/NpFQWkVjvmzDbjTCdmYh/3yZUrcWSaOfkDHDKMLQCNClW7yw2PkFx0lv+Q191ik3dTLjpakaWAqUT52+0dY+O4ymJZu64rYmMEvzr4haohMuQarNimV4ngVMAys3DCgLmjV/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Let's make folio_mlock_step() simply a wrapper around folio_pte_batch(), which will greatly reduce the cost of ptep_get() when scanning a range of contptes. Acked-by: David Hildenbrand Reviewed-by: Baolin Wang Suggested-by: Barry Song <21cnbao@gmail.com> Suggested-by: Matthew Wilcox Signed-off-by: Lance Yang --- v1 -> v2: - Remove the likely() hint (per Matthew) - Keep type declarations at the beginning of the function (per Matthew) - Make a minimum change (per Barry) - Pick RB from Baolin - thanks! - Pick AB from David - thanks! mm/mlock.c | 19 ++++--------------- 1 file changed, 4 insertions(+), 15 deletions(-) diff --git a/mm/mlock.c b/mm/mlock.c index 30b51cdea89d..52d6e401ad67 100644 --- a/mm/mlock.c +++ b/mm/mlock.c @@ -307,26 +307,15 @@ void munlock_folio(struct folio *folio) static inline unsigned int folio_mlock_step(struct folio *folio, pte_t *pte, unsigned long addr, unsigned long end) { - unsigned int count, i, nr = folio_nr_pages(folio); - unsigned long pfn = folio_pfn(folio); + const fpb_t fpb_flags = FPB_IGNORE_DIRTY | FPB_IGNORE_SOFT_DIRTY; + unsigned int count = (end - addr) >> PAGE_SHIFT; pte_t ptent = ptep_get(pte); if (!folio_test_large(folio)) return 1; - count = pfn + nr - pte_pfn(ptent); - count = min_t(unsigned int, count, (end - addr) >> PAGE_SHIFT); - - for (i = 0; i < count; i++, pte++) { - pte_t entry = ptep_get(pte); - - if (!pte_present(entry)) - break; - if (pte_pfn(entry) - pfn >= nr) - break; - } - - return i; + return folio_pte_batch(folio, addr, pte, ptent, count, fpb_flags, NULL, + NULL, NULL); } static inline bool allow_mlock_munlock(struct folio *folio,