From patchwork Thu Jan 9 02:30:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13931811 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACE7BE77188 for ; Thu, 9 Jan 2025 02:30:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4C3576B0098; Wed, 8 Jan 2025 21:30:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 44C536B0099; Wed, 8 Jan 2025 21:30:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24FAD6B009A; Wed, 8 Jan 2025 21:30:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id F35E86B0098 for ; Wed, 8 Jan 2025 21:30:40 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A8470140B1D for ; Thu, 9 Jan 2025 02:30:40 +0000 (UTC) X-FDA: 82986335040.24.0BDC8EE Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf02.hostedemail.com (Postfix) with ESMTP id DD86D80011 for ; Thu, 9 Jan 2025 02:30:38 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=UaY6A9bn; spf=pass (imf02.hostedemail.com: domain of 3zTR_ZwYKCHYmolYhVaiiafY.Wigfchor-ggepUWe.ila@flex--surenb.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3zTR_ZwYKCHYmolYhVaiiafY.Wigfchor-ggepUWe.ila@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736389838; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tU8qdV00qxcThV6AIuoPhomYC//sTigTQ5m8fxahbLA=; b=YT3CZsrGyqm+D8aGvhLhZg+wgDBb8+MQZ3jCDGv+HA7PFX3wonsWvq+KIheAY3rpbs8MFN yGHjJqy1227lJn6y1Ace9/L8oCf/79gxbdF7bZnaJWs/8vikc2LtOAWdR+5Jgujn3pK6Fj D3lCdCazQTU85BoR+OofUk+08tWkSWg= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=UaY6A9bn; spf=pass (imf02.hostedemail.com: domain of 3zTR_ZwYKCHYmolYhVaiiafY.Wigfchor-ggepUWe.ila@flex--surenb.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3zTR_ZwYKCHYmolYhVaiiafY.Wigfchor-ggepUWe.ila@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736389839; a=rsa-sha256; cv=none; b=DNUFqrtap4rMXDSl129hO4SU9EPYV2Pm3kFQ0PfQGMIKEZx8Hq5IYXlxY/xJjK8WDUhinX XvYOpDsKtMjytMNfaQU910KVvLbaA5QAv/L35eH/s0GLXXhtVphwp+p2nP1K5vWpWnURm7 oqPZpyGATDYuLFq0veGnXcp4VzVz8sI= Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2ee9f66cb12so823950a91.1 for ; Wed, 08 Jan 2025 18:30:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736389838; x=1736994638; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=tU8qdV00qxcThV6AIuoPhomYC//sTigTQ5m8fxahbLA=; b=UaY6A9bn8OlunaGtEOwgoVcSoYkfovle1JNCYLp0h83bQ5DNunYxYLoucmw9/BouYF c3A0oDiWYj/npyygYbq4SkVPP160v6yOeyhA0cvHShxm4s/l/AgXD9ihoY9ulBOU3xRY z55t/kurmp58wtOdaejEdcv8ikwg4o6C0crHKIKA1rvHpZj2oLLia50N9hDKWX98NSSI 7IBoflPdqSAyIPCN63bmiITBcew/uUkYl8JZA5UUJaXjjWEJ1ZKHgcCUEgFbifAiDoEy lcoMeudFwqKTgLQ+g+zz/XPocb983bb1fMKdekTbUX4lp/s9q1xs+IMdXjCpv3WpMfiB goPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736389838; x=1736994638; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tU8qdV00qxcThV6AIuoPhomYC//sTigTQ5m8fxahbLA=; b=D53Aa/9dVfB4gVxw165oMg1R4uVjjzMRN4PUm1J1vBEAyyINvcuIcyqWmC9noUQXlT zmlKuB+2BhEfIiAgkhxNGUqRFCveSEx8gALfIaO9U64ga8K0nLymNUZF+1+X5L+e9GMQ rO0J+U35mct2vrVNVYyNF+Gs15r0oXhyCqPn6pKN/Z8aSG0jLUXUx8u4SUSb1dLzkaFM Tz2IJ32iWSPylhgLVLmJ21W4FDQ/XUUJcRfSbJCBa6bZYGKF4X5UH3bVpSh+WvBc2eNT xaRHK9EdHqeak/Ndr9rJ8WDuBUiKyWo4V1RSD4p3SVALlYJOp+8SlTTiFqgfS4yAdvuX lOXA== X-Forwarded-Encrypted: i=1; AJvYcCWRmDdX86W92bFIB0A0NnMNRbijpn7u1ILo6Poxw3gQ0/EQ7m/2Ie3ku/wBOUo2SR5zFH3CFln0Ww==@kvack.org X-Gm-Message-State: AOJu0Yy47dz30WovcB8DHpm7SxV0bCfb/Yljv9Hw8pRwTchSe7zu7o73 lounHOVbuWmU0glaUTy/GFpGMrdByTFeY04ZNJF+FvGWJ/n+y07YdMfWTInl+tYMx+5jferKR8r nSg== X-Google-Smtp-Source: AGHT+IHe7fElKYNkHiMi+Ss9Fbmy8ZuO1qg0Iy4wm9l48QbX7o9iXLM3T+s6E3Ey0ObPvizJE76ByK8fFls= X-Received: from pjbsk16.prod.google.com ([2002:a17:90b:2dd0:b0:2ee:4f3a:d07d]) (user=surenb job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:dfcb:b0:2ee:863e:9ff6 with SMTP id 98e67ed59e1d1-2f548eae2ecmr8238330a91.16.1736389837604; Wed, 08 Jan 2025 18:30:37 -0800 (PST) Date: Wed, 8 Jan 2025 18:30:13 -0800 In-Reply-To: <20250109023025.2242447-1-surenb@google.com> Mime-Version: 1.0 References: <20250109023025.2242447-1-surenb@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20250109023025.2242447-5-surenb@google.com> Subject: [PATCH v8 04/16] mm: introduce vma_iter_store_attached() to use with attached vmas From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: peterz@infradead.org, willy@infradead.org, liam.howlett@oracle.com, lorenzo.stoakes@oracle.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mjguzik@gmail.com, oliver.sang@intel.com, mgorman@techsingularity.net, david@redhat.com, peterx@redhat.com, oleg@redhat.com, dave@stgolabs.net, paulmck@kernel.org, brauner@kernel.org, dhowells@redhat.com, hdanton@sina.com, hughd@google.com, lokeshgidra@google.com, minchan@google.com, jannh@google.com, shakeel.butt@linux.dev, souravpanda@google.com, pasha.tatashin@soleen.com, klarasmodin@gmail.com, richard.weiyang@gmail.com, corbet@lwn.net, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com, surenb@google.com X-Rspamd-Queue-Id: DD86D80011 X-Rspamd-Server: rspam12 X-Stat-Signature: te4t565ixczqrqpxg96u188ktdmj6m1y X-Rspam-User: X-HE-Tag: 1736389838-490478 X-HE-Meta: U2FsdGVkX19uxg8zrkyW14UAdYn1P9NY0X1sfE9Eszof6S/IucKI0vIBrK4C2ujL/LBtS8mcCf+YZy2r3sGNXkQBRH4i/mFetXRY8p/9Ol0jcb+5zyvrBkLtCgQrEHxlOvNqqV4FXXx9mDYc7ijH7r74oyA92yPFrzS2c+fvUakxdmBz1i0jdAo8A+Uy7LdHeX+yNbrmYNYPbPSyHojbxXW8dJlq437AnJo8GhigyPA5NSNVfJ91XCLOADnKK3nJmGkRY2N5hf7DhDOvb1VrErF0HpzIzfr8G+w3Yo4YPE6q4XtM4XThUagYGrrr4SApvmyuLukQgFEVxFNIF7qZR9rrtMDdyU+L37BOzANpexZkCAw6UaqCdDtHRsKRZq0QT2g6VjBKylW6++IHSI3iW2WR/nIL4aI8ZNV/lJCcdCOb8KAIF7QY+lzSl0EPpwPUoFafjoO4A31wZKoc/iTdZX3Z9B+Yw1DXxKaoVEHjwb2ifRDiP2DnU9zuDILuGXP/lMzMHaVzrSfiasv8jC9icNkrwvF47F9dzNNT8jWGVrYTMp8L+RQfJWySiE5XuuLiAvo8wUXe5Cv/pKQ8G8SBLWcHkYrdB5Ulo+jsBhtEaWaMeezaC5N16rtpEX0QGygbeXDQsxIZ5oKaTZNkiMdD4khihBKTq2qv1aFZ/BtTXdpe4kJd35QgGo65RnTMMsXMD4QgLuGdbb4QlxOHjRYy19zpeNDsJSnVYiLXtE+lGDkOCS6u/XA4OslzmE/34sALQTVgZZDPWZIoDw8NQLbq5tQ116zcdumy3Yw9o1SHBT9GdS1Z6qSI1IeMgeQ1WAA2W1Spz+p+1JFoK8bO+8hRAAMMhJcl4qCEJCFS3M3ul5oVQAMP3iJAHNm5jgz2HDL+ynmD29MYHrVy1deRDMQ7hQJm+CDJISiUyFdeOy3hdno/YhP7JDM7cKytQxhCZ78HEr8wmKQv89O5s3tJThR 5pdOXkpx MrL/LVz59WAiUQOAm9hmoFpoozYHMIF6tvXD6/jq6IdP+NlRuh97lLbODoIoajK51qipjtxWUBBny+SoDMdigJmwXmw6HXz4ptmESbN84/UXA/xltTfLdRUNQ8HYSNhXkJ0CE5JxNAyU2bBs8QZcZc1zuJXwN97LoVTlOlR12YMrS9eljNJJXGIuvA7hCVZTaEjdEG8XogQ3tW7IXiIjJgeD5DL8B5eaoy3PaCL5vtUwPEnuDuloUC5FfLPCi39k4GqMCSZY9hzVXTOGec1rmFE2jDMq1buniwKk3rO+ZcoqQkbb2GaS0bHP+73I/aXviCTrvpnIj+9pO+iuh5d/aEBX7fTimtPDRPY1OROgrBdu0462R4D125meXoosJq7e9GKG4Ga3aENeRxm02ss2vJWzNpQIoOOvypCt/xMxVmIdTkLH6CCYZkcbAwCpcGubV2pTmiBJYAEbK1JbHxmuFp1CwL/Tfol9krLLhwpGJyiHV0oen5pQ1m6486svep0SJRj1NGfqAbNJefYdslCciF7KN9G529ts0GQW59HVWkCskWlRQnKl7kcZnyB19+qcuzr10vCS6TGv+LvyvXfceJ4PblW8ibS4V3Qbenal6DoaRchd9iem8FZ3R3Db5UAMcI207RI9rbGsxb8PFcJCYjXNNblDXMe2v1lZX X-Bogosity: Ham, tests=bogofilter, spamicity=0.000064, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: vma_iter_store() functions can be used both when adding a new vma and when updating an existing one. However for existing ones we do not need to mark them attached as they are already marked that way. Introduce vma_iter_store_attached() to be used with already attached vmas. Signed-off-by: Suren Baghdasaryan Reviewed-by: Vlastimil Babka --- include/linux/mm.h | 12 ++++++++++++ mm/vma.c | 8 ++++---- mm/vma.h | 11 +++++++++-- 3 files changed, 25 insertions(+), 6 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index a9d8dd5745f7..e0d403c1ff63 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -821,6 +821,16 @@ static inline void vma_assert_locked(struct vm_area_struct *vma) vma_assert_write_locked(vma); } +static inline void vma_assert_attached(struct vm_area_struct *vma) +{ + VM_BUG_ON_VMA(vma->detached, vma); +} + +static inline void vma_assert_detached(struct vm_area_struct *vma) +{ + VM_BUG_ON_VMA(!vma->detached, vma); +} + static inline void vma_mark_attached(struct vm_area_struct *vma) { vma->detached = false; @@ -866,6 +876,8 @@ static inline void vma_end_read(struct vm_area_struct *vma) {} static inline void vma_start_write(struct vm_area_struct *vma) {} static inline void vma_assert_write_locked(struct vm_area_struct *vma) { mmap_assert_write_locked(vma->vm_mm); } +static inline void vma_assert_attached(struct vm_area_struct *vma) {} +static inline void vma_assert_detached(struct vm_area_struct *vma) {} static inline void vma_mark_attached(struct vm_area_struct *vma) {} static inline void vma_mark_detached(struct vm_area_struct *vma) {} diff --git a/mm/vma.c b/mm/vma.c index d603494e69d7..b9cf552e120c 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -660,14 +660,14 @@ static int commit_merge(struct vma_merge_struct *vmg, vma_set_range(vmg->vma, vmg->start, vmg->end, vmg->pgoff); if (expanded) - vma_iter_store(vmg->vmi, vmg->vma); + vma_iter_store_attached(vmg->vmi, vmg->vma); if (adj_start) { adjust->vm_start += adj_start; adjust->vm_pgoff += PHYS_PFN(adj_start); if (adj_start < 0) { WARN_ON(expanded); - vma_iter_store(vmg->vmi, adjust); + vma_iter_store_attached(vmg->vmi, adjust); } } @@ -2845,7 +2845,7 @@ int expand_upwards(struct vm_area_struct *vma, unsigned long address) anon_vma_interval_tree_pre_update_vma(vma); vma->vm_end = address; /* Overwrite old entry in mtree. */ - vma_iter_store(&vmi, vma); + vma_iter_store_attached(&vmi, vma); anon_vma_interval_tree_post_update_vma(vma); perf_event_mmap(vma); @@ -2925,7 +2925,7 @@ int expand_downwards(struct vm_area_struct *vma, unsigned long address) vma->vm_start = address; vma->vm_pgoff -= grow; /* Overwrite old entry in mtree. */ - vma_iter_store(&vmi, vma); + vma_iter_store_attached(&vmi, vma); anon_vma_interval_tree_post_update_vma(vma); perf_event_mmap(vma); diff --git a/mm/vma.h b/mm/vma.h index 2a2668de8d2c..63dd38d5230c 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -365,9 +365,10 @@ static inline struct vm_area_struct *vma_iter_load(struct vma_iterator *vmi) } /* Store a VMA with preallocated memory */ -static inline void vma_iter_store(struct vma_iterator *vmi, - struct vm_area_struct *vma) +static inline void vma_iter_store_attached(struct vma_iterator *vmi, + struct vm_area_struct *vma) { + vma_assert_attached(vma); #if defined(CONFIG_DEBUG_VM_MAPLE_TREE) if (MAS_WARN_ON(&vmi->mas, vmi->mas.status != ma_start && @@ -390,7 +391,13 @@ static inline void vma_iter_store(struct vma_iterator *vmi, __mas_set_range(&vmi->mas, vma->vm_start, vma->vm_end - 1); mas_store_prealloc(&vmi->mas, vma); +} + +static inline void vma_iter_store(struct vma_iterator *vmi, + struct vm_area_struct *vma) +{ vma_mark_attached(vma); + vma_iter_store_attached(vmi, vma); } static inline unsigned long vma_iter_addr(struct vma_iterator *vmi)