From patchwork Sat Jan 11 04:25:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13935770 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 650CDE7719A for ; Sat, 11 Jan 2025 04:26:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2F9606B0092; Fri, 10 Jan 2025 23:26:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2815D6B0093; Fri, 10 Jan 2025 23:26:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0D2436B0095; Fri, 10 Jan 2025 23:26:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id DB47F6B0092 for ; Fri, 10 Jan 2025 23:26:19 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 9E397A03DF for ; Sat, 11 Jan 2025 04:26:19 +0000 (UTC) X-FDA: 82993884078.03.B03A1CC Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf26.hostedemail.com (Postfix) with ESMTP id D0DA314000A for ; Sat, 11 Jan 2025 04:26:17 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=azIXF1bw; spf=pass (imf26.hostedemail.com: domain of 36PKBZwYKCBkHJG3C05DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--surenb.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=36PKBZwYKCBkHJG3C05DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736569577; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ADhTMbqXYt1v8ykOXAPhOs0iBeUY/Z6oTrjn3gyIBaM=; b=zXS66hkDTVSbEmXvFfH647BHFITOHVOehFe2hIXcPUpeRnBjvIY8NwKHEU7HYu90tvVJHf 7JMpcy/gUAv5aRGtA4Ygv0j4kWEzu+MJO9CwSlp5uHFdjCb8FwFRjRQaRsXY2HQn0PGzWW fJf+A64fmQmJN+aQs7l/hA45GCJQD5s= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736569577; a=rsa-sha256; cv=none; b=FjQsxXgCzLJlB+725PaqvXoxeAmdu+s+q/QprsbPHLNeIn1kkMBwonvEKQWLwy9+/X1FdF y/uiiacJJPfBRz4AtpDO7v/fYbgADTv516SvQjZAp3rgbcJEsvXW/8Z0a/dyz1MmVINQP7 7ikLDKOOycfJSL8fD5w4DR11yFaP8vo= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=azIXF1bw; spf=pass (imf26.hostedemail.com: domain of 36PKBZwYKCBkHJG3C05DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--surenb.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=36PKBZwYKCBkHJG3C05DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ef7fbd99a6so4773800a91.1 for ; Fri, 10 Jan 2025 20:26:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736569576; x=1737174376; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ADhTMbqXYt1v8ykOXAPhOs0iBeUY/Z6oTrjn3gyIBaM=; b=azIXF1bwp03Azo39Pz8lKfJ8vqXyXfs+9YtUyIBwVKoEla9TMAzVTKJAeV7/ul83A2 +luWTJfdxgpmbkbggo0vWRspgD5JcNpZJ7+geVkaPJl6iyhjPTJD28YLJ9JFcPkCgrrD mzn6MeMVQnN0l8Q6xNZp7HMSWkxFahNOfT4OplbTOWLE+w35gTeK17rdDyHFRhPnMHDj ecoVp6rHvXQi/uDMbmlrdniDVqCUb51U2ZpbBEY5PNrbeRU6oBCroBnO/lfEgTNFlglK WkykAO9eXVFLAmi5MxQng2kqF7ZnbhYTsSC7rT5d9aahxPyjXXxyTplp89kCQgPXZzzP 2l6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736569576; x=1737174376; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ADhTMbqXYt1v8ykOXAPhOs0iBeUY/Z6oTrjn3gyIBaM=; b=ASLcKcJPjDR8pn/YxbvuvpSnL5Hu7YYIoRYSKtbrsRdBA/XJaaYx/bq+hNCcjajj1G 9x5TqX/1BWt/f5wCkZZrJFykIifGHYbs9CXfYi7ZRzn8P8G8zB4p1bkPM/vTvuO1EBQE DZc+aQRI/GWp6giMIZvNjFCEhmBHOwKOv8cszdBjz6HVRsog9MyqiNvLwNQvxFXSCdGx GA5s0PmqVXJH3Xwz0TOIJvPk2oTs2g7syBlSzxWriaDWfWFrjjNRe64BfjobpCM9zMS1 WphkGoA0uq1inWl9rCIZcsr3y6d3+eBK4jY5CjaUmZ6Ji11rxd2ixiVooajHbRBFhxdc N83Q== X-Forwarded-Encrypted: i=1; AJvYcCX8O0HUeP4qV/ovAT3gnCUUXAhwEWUQhlKRAeZ21IkXmdIa/yc+JwHvf1+E134nAerYqk/5xvt3tQ==@kvack.org X-Gm-Message-State: AOJu0YwHKrkb1YPwnMWAg9UNwfS1w8XchDfjXc9FknPcvS7s2+SjqBYv CNyJXiQYtohX9h09spyJzP8RURRKul/a/EEPduW0YSjuIMwzLoeZQVqMXqjscIV0SrV3VG4/O67 Sdg== X-Google-Smtp-Source: AGHT+IFORDPp3F2kDYS1t6EeTCyG81wOk/69Q07YfVbBJQQfUoW8CkvQ1ejcetFlgpXNkE764lNprKA+Q/U= X-Received: from pfbfd28.prod.google.com ([2002:a05:6a00:2e9c:b0:727:3b66:ace]) (user=surenb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:3287:b0:725:456e:76e with SMTP id d2e1a72fcca58-72d21fb1d3fmr18712375b3a.6.1736569576461; Fri, 10 Jan 2025 20:26:16 -0800 (PST) Date: Fri, 10 Jan 2025 20:25:51 -0800 In-Reply-To: <20250111042604.3230628-1-surenb@google.com> Mime-Version: 1.0 References: <20250111042604.3230628-1-surenb@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20250111042604.3230628-5-surenb@google.com> Subject: [PATCH v9 04/17] mm: introduce vma_iter_store_attached() to use with attached vmas From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: peterz@infradead.org, willy@infradead.org, liam.howlett@oracle.com, lorenzo.stoakes@oracle.com, david.laight.linux@gmail.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mjguzik@gmail.com, oliver.sang@intel.com, mgorman@techsingularity.net, david@redhat.com, peterx@redhat.com, oleg@redhat.com, dave@stgolabs.net, paulmck@kernel.org, brauner@kernel.org, dhowells@redhat.com, hdanton@sina.com, hughd@google.com, lokeshgidra@google.com, minchan@google.com, jannh@google.com, shakeel.butt@linux.dev, souravpanda@google.com, pasha.tatashin@soleen.com, klarasmodin@gmail.com, richard.weiyang@gmail.com, corbet@lwn.net, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com, surenb@google.com X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: D0DA314000A X-Stat-Signature: 7idewtbae5ebin7qd87ntw5mc5fzext7 X-Rspam-User: X-HE-Tag: 1736569577-344435 X-HE-Meta: U2FsdGVkX18m1Zpd9VjbYP14heIAVGXodoBGBMDXyu5qpBORb4CdLFRp4z2p4M0TnzPREr+UG+fqxuMfybqhq7Jgx5BmUrpbtm6kEscSoJM7SYdP3G2TcaP5dcfiM6ksB2+T/QuFnXCy6DDl3QAPMSQBpxiU8sFfuAKZsZv+nbZodud/5l64y1CJjhNtIHXMu45ti77cMGEjX6lwBuH+9YCB4PlNBp9tXnPOydMIvvhRYHuweYnu5oLaLKVvxfbZ+c3v1BBEudn9Q29ChOMQxodSWD/QqAliwrzOKYJNL5lcyWcFLhiITQHny+IbXsF7FsRODSbuC7nmkk2ZbIV2M3vfiMHPb95H/zGYZxErB41DFJc2PGjgJCSE7TPaXBJF5rD/yzcyAejcs2X3rizqLd+kDrxw2wnB96htwe9/euGT11DTeWzOtUU18D4IhdDJOLpc5ne3p/OqZP7q1+BFqzXi8mPv/4h3V2yNQw4G/grFX+gRX9tE92ZBVnlkeCtkulw9xfwR2ntXxklcAwn1Oos+/4FxoHU2PtP1B8TFn//8lwhBMtv0P+/ITraVFf3lVO5ntqoLR/Q8ttHNleES1VM1nj+Y1TjYSJqQ54snzlsrDlD9jyVvglTkkhOAhulippuXSkQx8iMwaZF7Ifu8jgrnv43TUt1RXCOA6Qg/c7W2iraLq0IsySh4yhjbJbGjpwInWO1uY2NA5VuLeP8MWmvfNQQly01gPljjJkJRQM286OuBT+bDZmdhoZ5n8bS1XgyEcKOs4+B/nlTosmr4/6bDoPtIiIkqH6vs5miDNpNCe6IH26ml1d9xyIJi/8t8+6E+WEPh3GSO0GI+nfgLqpKFAD133RKUjnYP2HFZnNVrNCo/OaHIJFLYbNeHd5LgAPBU3Bg2TGgYKv7SE5TPRp4cWW7cbHGh4leSbZAqadY0jdXoEzPXIU3dLNHBnODdKwxfvueyTIVl9s8JxMC CWwaY7L2 j7Ah6C5uZ/xGbii2sjhFytxEof3s9KcOZCD5yd/loudcE6+cTJ6boTXBz4Ccc8YHxckk9HKdN4ZYsfqzxpaxI09LZlQehiO/9KMXhj+6k/i0hy6jMVdfLSBR193Eluj4GuNkwddFGLdK5Q7yb6BghQd/OE8JHZ58zudCGZ51ozG2kD6J5ZBejPcwBESt+C8fv5YpgLekhJ8ciLpo++wQPUcIRoZXOqba4TPVamyBXVgrUUAKwCDsh/wYp1+QHRHknKjZVb8yq0NuM9576gx4RfoEFLtq9+HwCIRSBz0mlmJHMK4X4HOQv5eJ7j+BKNUyVF0554fqyutAo/SKeQnVdTAmslG5JWnvRTUQWJZXb5lCKkyzNACL7rTsshRSROvjabtWdXW26F/ElLhFOUulsCrTUSA1Truc6lyoo2KSDlWOln5lBtv3ubONnRFlwsgYlaJK48tdsmrALTPCjtMW9j4zPtvjSKYhE0JEFTOYCyj5UbCLMT4qhJpYb2Vzt68qMxbtHGCIH0Porl/59LqiGUzO0pOgWmSBZcnr00u8zXIT/PuHhUrVGWMR8rrbti087HoXfYZB/r/lLFPz9Jw9JUPjAzpfMeJSQB6AVj+zLQfVBUbSHd0r0BmdS4tKnv0St4K8pQ1eAj4HIJ6H3Ti6JSCgwTkuauePDe2MQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: vma_iter_store() functions can be used both when adding a new vma and when updating an existing one. However for existing ones we do not need to mark them attached as they are already marked that way. Introduce vma_iter_store_attached() to be used with already attached vmas. Signed-off-by: Suren Baghdasaryan Reviewed-by: Vlastimil Babka --- include/linux/mm.h | 12 ++++++++++++ mm/vma.c | 8 ++++---- mm/vma.h | 11 +++++++++-- 3 files changed, 25 insertions(+), 6 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 2b322871da87..2f805f1a0176 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -821,6 +821,16 @@ static inline void vma_assert_locked(struct vm_area_struct *vma) vma_assert_write_locked(vma); } +static inline void vma_assert_attached(struct vm_area_struct *vma) +{ + VM_BUG_ON_VMA(vma->detached, vma); +} + +static inline void vma_assert_detached(struct vm_area_struct *vma) +{ + VM_BUG_ON_VMA(!vma->detached, vma); +} + static inline void vma_mark_attached(struct vm_area_struct *vma) { vma->detached = false; @@ -866,6 +876,8 @@ static inline void vma_end_read(struct vm_area_struct *vma) {} static inline void vma_start_write(struct vm_area_struct *vma) {} static inline void vma_assert_write_locked(struct vm_area_struct *vma) { mmap_assert_write_locked(vma->vm_mm); } +static inline void vma_assert_attached(struct vm_area_struct *vma) {} +static inline void vma_assert_detached(struct vm_area_struct *vma) {} static inline void vma_mark_attached(struct vm_area_struct *vma) {} static inline void vma_mark_detached(struct vm_area_struct *vma) {} diff --git a/mm/vma.c b/mm/vma.c index d603494e69d7..b9cf552e120c 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -660,14 +660,14 @@ static int commit_merge(struct vma_merge_struct *vmg, vma_set_range(vmg->vma, vmg->start, vmg->end, vmg->pgoff); if (expanded) - vma_iter_store(vmg->vmi, vmg->vma); + vma_iter_store_attached(vmg->vmi, vmg->vma); if (adj_start) { adjust->vm_start += adj_start; adjust->vm_pgoff += PHYS_PFN(adj_start); if (adj_start < 0) { WARN_ON(expanded); - vma_iter_store(vmg->vmi, adjust); + vma_iter_store_attached(vmg->vmi, adjust); } } @@ -2845,7 +2845,7 @@ int expand_upwards(struct vm_area_struct *vma, unsigned long address) anon_vma_interval_tree_pre_update_vma(vma); vma->vm_end = address; /* Overwrite old entry in mtree. */ - vma_iter_store(&vmi, vma); + vma_iter_store_attached(&vmi, vma); anon_vma_interval_tree_post_update_vma(vma); perf_event_mmap(vma); @@ -2925,7 +2925,7 @@ int expand_downwards(struct vm_area_struct *vma, unsigned long address) vma->vm_start = address; vma->vm_pgoff -= grow; /* Overwrite old entry in mtree. */ - vma_iter_store(&vmi, vma); + vma_iter_store_attached(&vmi, vma); anon_vma_interval_tree_post_update_vma(vma); perf_event_mmap(vma); diff --git a/mm/vma.h b/mm/vma.h index 2a2668de8d2c..63dd38d5230c 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -365,9 +365,10 @@ static inline struct vm_area_struct *vma_iter_load(struct vma_iterator *vmi) } /* Store a VMA with preallocated memory */ -static inline void vma_iter_store(struct vma_iterator *vmi, - struct vm_area_struct *vma) +static inline void vma_iter_store_attached(struct vma_iterator *vmi, + struct vm_area_struct *vma) { + vma_assert_attached(vma); #if defined(CONFIG_DEBUG_VM_MAPLE_TREE) if (MAS_WARN_ON(&vmi->mas, vmi->mas.status != ma_start && @@ -390,7 +391,13 @@ static inline void vma_iter_store(struct vma_iterator *vmi, __mas_set_range(&vmi->mas, vma->vm_start, vma->vm_end - 1); mas_store_prealloc(&vmi->mas, vma); +} + +static inline void vma_iter_store(struct vma_iterator *vmi, + struct vm_area_struct *vma) +{ vma_mark_attached(vma); + vma_iter_store_attached(vmi, vma); } static inline unsigned long vma_iter_addr(struct vma_iterator *vmi)