From patchwork Thu Oct 17 09:47:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13839730 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD68CD2126D for ; Thu, 17 Oct 2024 09:48:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E35E6B0099; Thu, 17 Oct 2024 05:48:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 492CA6B009A; Thu, 17 Oct 2024 05:48:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 333236B009B; Thu, 17 Oct 2024 05:48:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 120936B0099 for ; Thu, 17 Oct 2024 05:48:10 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 95F90141154 for ; Thu, 17 Oct 2024 09:47:58 +0000 (UTC) X-FDA: 82682618046.23.96DA612 Received: from mail-oo1-f49.google.com (mail-oo1-f49.google.com [209.85.161.49]) by imf23.hostedemail.com (Postfix) with ESMTP id CD2F2140005 for ; Thu, 17 Oct 2024 09:48:01 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=jJhEySTI; spf=pass (imf23.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.161.49 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729158414; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ns8LrZyEwJOJHznjnD7PlHQ9MHEMoW624Fd3QWpp04I=; b=CnI74dX3K/rvBzNIrPHzScHomN0wAwkNLFC7i4zA7r881KdXFO71DU+Rv7/EyCUpwf7pRA f5+sbtcQ509X6OamqGqiDEP/YY1kJVtI/LsXNlVSqJEmSR3jQYgmdWnVdXrlehDGnEH34T SSW1REtHcg9B3uOtUaWmutNdYIk1Q9A= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=jJhEySTI; spf=pass (imf23.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.161.49 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729158414; a=rsa-sha256; cv=none; b=a5G6Fy/kgfZjHfPCxWkYG3rXpD2PDoLx7TnEzHxaWQ87WBLeVfKg+JTsIkKP4DzqQyzc+W RmIatTGId68YQp/A+4DQeuCLH7T+kb+9UiEIZZu5elNkv6tcbG0iT9Vb1z7NbG4Rd8xtjJ lms+pFEitiQ33z3cXi1bsYr3HEDck/M= Received: by mail-oo1-f49.google.com with SMTP id 006d021491bc7-5e56759e6d7so435891eaf.3 for ; Thu, 17 Oct 2024 02:48:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1729158487; x=1729763287; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ns8LrZyEwJOJHznjnD7PlHQ9MHEMoW624Fd3QWpp04I=; b=jJhEySTIIJb6GrlZnKz4Gj3qxN5vTFEXF1DgSwm/MOxDiplQSftLGQoW1LaClWttT2 l72/GhCGBGMDF+JOQkrA+qyP//5LqcQSs6VL4iGm1AVnkpWsISk0TCietV7oPLf+dTvN IY+6NoWqNs6Hu/MZtP1VFYy7vFWiex2APTGFUhVTBEbqS42fWN+1LdShSesLpWITrMOi QVkbP7moyWpXSdztWOrnmQ8acjc/e8H8MmtQ7o+jlbZW1g/YrkJURCX/8LGeisde8qT2 JSPx94PkKk85HgnRoJMn/d3/Orcv8TPEPQSV1HOcMwzc4t1ZNwBJRKAY1K6nUfJr1ech XUIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729158487; x=1729763287; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ns8LrZyEwJOJHznjnD7PlHQ9MHEMoW624Fd3QWpp04I=; b=AflWEm50vehRRuViflu0BwEAPFNl0SebAgll/i3BVBG0JFR0P927cbwrOy3c0dBI8H 1MZsJ536v17KevskIW/bR2xrI+uOmJh1GWlgyjz8DnPFr7iEN2LZE5QvDmz3+bKn7Fcz kQmWjyI1kMV71hLG9+OurGKdyXchTSJKFGKFCIHYhIdm0rxvxn/sS+gr0e308mUYpfyH pWR8Uy+DyJ1vIl0JlPelqHsleg7RDad8OFvzIZSYcT1Q6BhmrS1Pho04eUUZingV9KD5 XFbah1QmTFGqUHGkHXAJPBKVa3AUjtYjA08PduCZ3WodWjB5cwU4nFXyrK7zD4Yz1PHa gcvg== X-Gm-Message-State: AOJu0YwI0UdBVCHxcNvhVGs9wqbxWwxY7opwwY+EvX+pZEXD941lJJf1 /XTkJ3P69byCAYE89YvIzS/CFR30GoTGqkI91fvO0TBZ9TawF2TFUZ1DP2PYX7w= X-Google-Smtp-Source: AGHT+IHvYEBhNwHNitO8wA559PhHl4cuzZKZHlF0Sd6OJ2vGi8dRLztGVbwRe2pbe9tQOZtPvKzwMw== X-Received: by 2002:a05:6870:798:b0:287:ce7:7c4a with SMTP id 586e51a60fabf-2886e00f920mr16405829fac.38.1729158486665; Thu, 17 Oct 2024 02:48:06 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71e774a4218sm4385365b3a.120.2024.10.17.02.48.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Oct 2024 02:48:06 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, mgorman@suse.de, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, zokeefe@google.com, rientjes@google.com, jannh@google.com, peterx@redhat.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, x86@kernel.org, Qi Zheng Subject: [PATCH v1 3/7] mm: zap_install_uffd_wp_if_needed: return whether uffd-wp pte has been re-installed Date: Thu, 17 Oct 2024 17:47:22 +0800 Message-Id: <37983544cb4fbd27c244b68d3e37d170ea51f444.1729157502.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: CD2F2140005 X-Stat-Signature: nt8pwdj5amjryxwitgnisux8onzjspzj X-HE-Tag: 1729158481-804596 X-HE-Meta: U2FsdGVkX19M2ctG+jrHu8gaPh7pYkBrSWJUB0/k9GHWYhnQDZQpQUWAMCcreS3lv8GTEj9QofSPWPMSKsZsRXhNcRMGQ44qfKygc79btM+Jjizoq5EhMigabcbCrLbEYWqRdsJ2qxY9E6jDjdT1kW6sEZihXpndeqEorPYJW30wjvGZrf7WK5cWsguKs8NZ/ouDB3PrZBrrzGzX3L/fdrnBbMnD6NW8C+hfmQrdj9KOFYmvYHl5ifjy3Opy/EJZjCgAkwNy9dsV9PW8mLfUEyS845SSKs8eyMJ+7aObL8nDa074LLloCsd+VIqq2ViqPXl0sbF/KIT5zF7qpW+mWBUe4agb5g4yFGaWibkEUAHou13o8PU4UU8Ij8ewq0taAc0aCDkptb75q86xKqUOxhuGyujXcrweyoGofuUEu93lj1M8bLUtoC28Kd+jir2ipn+QUYDWXiXs3t6+8zTEzoz5A4oqpcn4aWu/pAjcnYq7KfSSYHWVnJGA3CNI5z/ppTjEi4p68n4IjzgjuvozpSq2K4nHO9pzryd2IqfRWIQ0sdw/bu0rBBzutf4+JxmajzzzoqoJTMeIDeV7fPaMsFpqCI7CEbRen9NLLiQM/IORNbgtA4GzZ4ESuwQMMayJw97yezahsSMg79uEt1lzVNpEWN425ZUDvm5GOAWEaY1mORFhzb4MbDGXefFHs3UifQveR2sXXz4F4nofVMhCtP6nqI3c4p5gJ1W8DAwrmfT8cdoQ60sg75ZxM/sN3haCXms+l0oNMHGVcya+kh/ORoxQAfXlmORJFWOQ5OGzTXAvmvfyoaix3NEJ4tRVeVIH4mgwuOUeqlKrh+D7uNfsNw0GzemTcTNEuKyBIkjHLgD48YYnNytOQW/j/gOQ3XkTj69SM4tVJDHAEFI/8Y0Iln5l/+cCIRMntEJ6DdRpx5IHvWVER/Hie/g+pKbxUBk27uaJohwU4WZFcCvnOLq ISH2XnS/ KK3qDrxdV6bglyZQ6JxGMxvLciY7QQkaWZWA3846cUTgdsu5kmHGUVzvNKZSbVbBlLRNRH/WXGA2koPUyGMOqA0Gfp/WI0Qj37nSeOS0jgvJ019AoZdg9G+gOd2WcV3o4r2dIIH/yZy2jtk27S+ue1/EXkFj8ADWXqxmg4Z7Qiz6p/nCTr6EkaHGLZZtfkbJfiXL6EwbtL6dibYVBJtu5j6MbhudHH7zp1sVZ+Owf0F9osTUYriHQ2EPalZLe5b7UuhaurWela2wJ7iromNKvtauUng0yYh7foyaZY6lQBWDc/Rj0esa0kOczIeR3FHUk1YHPD7cpRmA1f4lycC8oVoIwd10Uk8Yg/HOqhQnHAtl/pRc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In some cases, we'll replace the none pte with an uffd-wp swap special pte marker when necessary. Let's expose this information to the caller through the return value, so that subsequent commits can use this information to detect whether the PTE page is empty. Signed-off-by: Qi Zheng --- include/linux/mm_inline.h | 11 ++++++++--- mm/memory.c | 15 +++++++++++---- 2 files changed, 19 insertions(+), 7 deletions(-) diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index 6f801c7b36e2f..c3ba1da418caf 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -552,8 +552,10 @@ static inline pte_marker copy_pte_marker( * pte, and if they see it, they'll fault and serialize at the pgtable lock. * * This function is a no-op if PTE_MARKER_UFFD_WP is not enabled. + * + * Returns true if an uffd-wp pte was installed, false otherwise. */ -static inline void +static inline bool pte_install_uffd_wp_if_needed(struct vm_area_struct *vma, unsigned long addr, pte_t *pte, pte_t pteval) { @@ -570,7 +572,7 @@ pte_install_uffd_wp_if_needed(struct vm_area_struct *vma, unsigned long addr, * with a swap pte. There's no way of leaking the bit. */ if (vma_is_anonymous(vma) || !userfaultfd_wp(vma)) - return; + return false; /* A uffd-wp wr-protected normal pte */ if (unlikely(pte_present(pteval) && pte_uffd_wp(pteval))) @@ -583,10 +585,13 @@ pte_install_uffd_wp_if_needed(struct vm_area_struct *vma, unsigned long addr, if (unlikely(pte_swp_uffd_wp_any(pteval))) arm_uffd_pte = true; - if (unlikely(arm_uffd_pte)) + if (unlikely(arm_uffd_pte)) { set_pte_at(vma->vm_mm, addr, pte, make_pte_marker(PTE_MARKER_UFFD_WP)); + return true; + } #endif + return false; } static inline bool vma_has_recency(struct vm_area_struct *vma) diff --git a/mm/memory.c b/mm/memory.c index fd57c0f49fce2..534d9d52b5ebe 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1467,27 +1467,34 @@ static inline bool zap_drop_file_uffd_wp(struct zap_details *details) /* * This function makes sure that we'll replace the none pte with an uffd-wp * swap special pte marker when necessary. Must be with the pgtable lock held. + * + * Returns true if uffd-wp ptes was installed, false otherwise. */ -static inline void +static inline bool zap_install_uffd_wp_if_needed(struct vm_area_struct *vma, unsigned long addr, pte_t *pte, int nr, struct zap_details *details, pte_t pteval) { + bool was_installed = false; + /* Zap on anonymous always means dropping everything */ if (vma_is_anonymous(vma)) - return; + return false; if (zap_drop_file_uffd_wp(details)) - return; + return false; for (;;) { /* the PFN in the PTE is irrelevant. */ - pte_install_uffd_wp_if_needed(vma, addr, pte, pteval); + if (pte_install_uffd_wp_if_needed(vma, addr, pte, pteval)) + was_installed = true; if (--nr == 0) break; pte++; addr += PAGE_SIZE; } + + return was_installed; } static __always_inline void zap_present_folio_ptes(struct mmu_gather *tlb,