From patchwork Thu Oct 31 08:13:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13857706 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F68DE68958 for ; Thu, 31 Oct 2024 08:14:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A5DE36B0099; Thu, 31 Oct 2024 04:14:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A0CC36B009A; Thu, 31 Oct 2024 04:14:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8606E6B009B; Thu, 31 Oct 2024 04:14:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 676636B0099 for ; Thu, 31 Oct 2024 04:14:29 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 1318612131E for ; Thu, 31 Oct 2024 08:14:29 +0000 (UTC) X-FDA: 82733184114.13.8C157CB Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by imf19.hostedemail.com (Postfix) with ESMTP id B93151A0005 for ; Thu, 31 Oct 2024 08:13:53 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=ApHWNwbX; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf19.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.50 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1730362348; a=rsa-sha256; cv=none; b=g3ew4AFXT8nywsh+6g5NNxGDqcTC+8dgAdj7342ZbsEVhYy+DjNvHyISusof0MThmheQb2 02rvx0fGymcG52IUMk/r03pwbbsThNG+1yH4WRnNhImFTqQe+4eBxAUk6AfzJw3pLmOTXQ 61W5Cm1imUNqsKf9WE4fcrINddTnNek= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=ApHWNwbX; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf19.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.50 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730362348; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zhaSq+JiwY2eXMq9iHjNOMwdUtMvBCWvXjuj5WTX7eI=; b=jJiILgI4zVCYdPXRAvM3FdYiAsVGU2hcRNKHs1QZoQpJYbZ2G+bvzAcdUglJU3DnIX05Xp hmoT0pv42eztDbbUqDo42okZJCUPXe11igj6+EvZ+btvp5uN12/M+cDXryIzQ2LufvJrPW +7i8OA8iQV8PFnXHbeUIvy9xG4hDSlI= Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-2e30db524c2so502761a91.1 for ; Thu, 31 Oct 2024 01:14:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1730362466; x=1730967266; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zhaSq+JiwY2eXMq9iHjNOMwdUtMvBCWvXjuj5WTX7eI=; b=ApHWNwbXZkS5tKJQyOL9LPCB57azynTA3F7ckNxwm6YcqWEMnDR4yhrDlPL8/94MZ6 0RdZO0IiQa+pM4Bgy/4vNmgbj/VjgIyTz7NS2C8Nv9Ea4gxO4gexUORTrZ2U6JiMIyJ4 telskKcwxMlQU6LFrCoZ0NHb9pHinhTcSOQ0jA+EkhjglgqpAGvHjwlLu6nYQqtss5X0 1JZg+7GRuvX8gaVCYa8isUeaEznC6KdVKcWY8ClJQ5khwk1R4q2OtJ++TNjg54U6y38J YAAjqNBoVQe8dft4yVk5faNKA34EtFN1TjFJcqC6Oo/YifFnJCqEd/H9VbVRTkJUbnL0 BjsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730362466; x=1730967266; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zhaSq+JiwY2eXMq9iHjNOMwdUtMvBCWvXjuj5WTX7eI=; b=uEy2xpMh8hm9rpWL9DqZvzhxuLWzDbcKvV7CtCYWJTep7HeowAtBVcteol+nt9XfDf JOotKu4JvrdqP043o6HK59QwOTzXT3B5EdmAETKtMYFo8B71OnL+L1HX0OPKNzfRMeCk GxHH1X7R9cyE5ux+T6FwAYIZQh50dTqVosbXbHdhvh4jL+qaN4AH54hBhDRtIbnFHXl6 CgKB3ukpJjiULnS54NO2zhoKJkeyy9Aa/P3mOE/AbEr+I59+TJvCPEmRuccqgsReQ6FZ qD8DyneHQjqb75j51USp8Hy1zFN015F31p7tMsd4PZx3yoikerIS+U0aJhT/zqKY2KqV LvaA== X-Gm-Message-State: AOJu0Yyve9rOrUpqK/AySu4ALA2OkZxhAY7Ud4F6TX95lDnrlwR4ka1t WXjjsX3nGX8D+/iHK8YM98qe/+fTLAk/pcHLepxbuXCzRuUYAXTSrRM2V+UoOhA= X-Google-Smtp-Source: AGHT+IHK1eQB1HIOVcDML9/U+qgad2lrGxmYCIjO8Kr0cktQfTqjB9fcEN6v3D+sRskXXQsEGQyaZA== X-Received: by 2002:a17:90a:bc83:b0:2e2:d17e:1ef7 with SMTP id 98e67ed59e1d1-2e8f1054a67mr19337853a91.3.1730362465831; Thu, 31 Oct 2024 01:14:25 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.149]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e9201b2586sm5163996a91.0.2024.10.31.01.14.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Oct 2024 01:14:24 -0700 (PDT) From: Qi Zheng To: david@redhat.com, jannh@google.com, hughd@google.com, willy@infradead.org, mgorman@suse.de, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, zokeefe@google.com, rientjes@google.com, peterx@redhat.com, catalin.marinas@arm.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, x86@kernel.org, Qi Zheng Subject: [PATCH v2 3/7] mm: introduce do_zap_pte_range() Date: Thu, 31 Oct 2024 16:13:19 +0800 Message-Id: <1639ac32194f2b2590852f410fd3ce3595eb730b.1730360798.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: B93151A0005 X-Stat-Signature: 463by7mr9ieryabod33ij9morn13zgsi X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1730362433-646319 X-HE-Meta: U2FsdGVkX19pk8NS7fL4k+UK5B/1nliSS42bYkEZa4Icm5mACSW61aWJ/5D0zwh0PTvwr/Z9aHKvVDmeSoPZl1VoD3uO37ObTI5Ku0EKDlSF4RDGPJTMZy2Xec0NcOMfKUN/EENSmvh2+3n0owQ6SqEa89LqS7nnFxA6dhe2MDQjmUQTGpG4kIhL+NSpOWf1+Dm6VTtbfsMxnRsqB7jBfZKUcoAthXAjShl56UXk59Cpin3Qla3X+Vd1/YxFSJOFv07MrFRyErSjQPaUjU6jidnE36YVByK8qagfVwqxMpl8+S7m6ZFDVS0zMJMdTL9GPamcunf0EErrrGyE0h8jq1oSJu8bm85SyCbruCaMWZ3Iq6smNawZ4KiptyPo4MMyjy/wPNiGEUOa8GYiNBhpZ0yg4hXvtuL8Cq7Acol4/Sevp9mRcTQUD1lW2zjOOH40or2ICqTJKRvjkwfwv7XTAaPGVouCHrBDfj7AYmBIsL7WTyq0wa3YvCa4R/pgvnlLGo5unNkaXWimCdAMScPvzKVXnQt6gdC8vIwDU7rlzal6TLa9rCgXbGRq5b9OXqBRNF9e6sbvcAJdfGwCgpCe2cUNriU1gqVk6fZgBbJsXsKjHBx8RL08RwdSV50x5C0HAUGPW7AEOBlnr10smOOfFoN5Vs6Xw0ThoQeX1dp1HGK9EgwfnwzNTJLRhIE/CEWnfc/PMeAan95+CBWhArFJOI3wtlbJw9fVirO+yEy++oJlHkeHn92GEF4BbT3PMXzmncCrsnYxEMC6uEQZPOYRTn916L1c5GxzS/pnoTkewgLNbEHCJz6pIB1qpcFQZfYRviF8gtDe8u17FN31Wvv+gle/StqsPkabcrpj0jS/yiaKglfzMft+ZsTMXKHzwLgFkGGktVloPbocpSdh3+EhhOsFvUGddgMTKlcgg4vzeHHq9KaWKQa11fubbdJa0t+QAZ/wNXe/W36BsFIRMh8 3oEo8LOq QZACPNwUPWHrjRFwh2nnFMWXx1E4oYNaNOwlHGCluEtc42rzUKbH4PxrickP0LA+wX4KbOYxcJdo/4uzCaI/FOcYhhFBp/JtURgzwrjlF9TgfamIIM5ytl5Vjt5HI9b0qHoGE6k1YtOVYkbOssLdfEcvc+V1VkdQvOkUBOJelMjhbRPH8QL+jWm1y874u9nCS8GCE9pCEDbGZdG15fcg8l0PW/jMCO6KWKfw0QW+O2hss0r/gykHoCF4qQNjyHSLIl0VYQ9hbiHKayvukSIOVWsbnGYO2QEbmwme7x13hyYb2l7zNxiNoRxWuNeBbWERArX2zwUKPzR1vH78BK0lpSADIRipOuh171XJV/RR9HPn3Aao= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This commit introduces do_zap_pte_range() to actually zap the PTEs, which will help improve code readability and facilitate secondary checking of the processed PTEs in the future. No functional change. Signed-off-by: Qi Zheng Reviewed-by: Jann Horn Acked-by: David Hildenbrand --- mm/memory.c | 45 ++++++++++++++++++++++++++------------------- 1 file changed, 26 insertions(+), 19 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index bd9ebe0f4471f..c1150e62dd073 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1657,6 +1657,27 @@ static inline int zap_nonpresent_ptes(struct mmu_gather *tlb, return nr; } +static inline int do_zap_pte_range(struct mmu_gather *tlb, + struct vm_area_struct *vma, pte_t *pte, + unsigned long addr, unsigned long end, + struct zap_details *details, int *rss, + bool *force_flush, bool *force_break) +{ + pte_t ptent = ptep_get(pte); + int max_nr = (end - addr) / PAGE_SIZE; + + if (pte_none(ptent)) + return 1; + + if (pte_present(ptent)) + return zap_present_ptes(tlb, vma, pte, ptent, max_nr, + addr, details, rss, force_flush, + force_break); + + return zap_nonpresent_ptes(tlb, vma, pte, ptent, max_nr, addr, + details, rss); +} + static unsigned long zap_pte_range(struct mmu_gather *tlb, struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr, unsigned long end, @@ -1679,28 +1700,14 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, flush_tlb_batched_pending(mm); arch_enter_lazy_mmu_mode(); do { - pte_t ptent = ptep_get(pte); - int max_nr; - - nr = 1; - if (pte_none(ptent)) - continue; - if (need_resched()) break; - max_nr = (end - addr) / PAGE_SIZE; - if (pte_present(ptent)) { - nr = zap_present_ptes(tlb, vma, pte, ptent, max_nr, - addr, details, rss, &force_flush, - &force_break); - if (unlikely(force_break)) { - addr += nr * PAGE_SIZE; - break; - } - } else { - nr = zap_nonpresent_ptes(tlb, vma, pte, ptent, max_nr, - addr, details, rss); + nr = do_zap_pte_range(tlb, vma, pte, addr, end, details, rss, + &force_flush, &force_break); + if (unlikely(force_break)) { + addr += nr * PAGE_SIZE; + break; } } while (pte += nr, addr += PAGE_SIZE * nr, addr != end);