From patchwork Thu Feb 13 22:46:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13974137 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B788AC021A0 for ; Thu, 13 Feb 2025 22:47:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 90C5228000D; Thu, 13 Feb 2025 17:47:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 89560280001; Thu, 13 Feb 2025 17:47:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 699B628000D; Thu, 13 Feb 2025 17:47:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 43546280001 for ; Thu, 13 Feb 2025 17:47:21 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id E9B2116180E for ; Thu, 13 Feb 2025 22:47:20 +0000 (UTC) X-FDA: 83116409040.06.FAEAC20 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf12.hostedemail.com (Postfix) with ESMTP id 1877240009 for ; Thu, 13 Feb 2025 22:47:18 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=wT1FZos9; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of 3dnauZwYKCGERTQDMAFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--surenb.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3dnauZwYKCGERTQDMAFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--surenb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739486839; a=rsa-sha256; cv=none; b=qMnSVwHqZQtn7ZGqMnN8nUSISbVM3mxP0dcJFHyi1AHI9V+C4XoXryhuu14Lf7VRX54LT+ HkkL2SehCLiEGjggNm+cqfGuOi0u3u0lFOa6nNOgcnNt06tCpcpyos6cXcj6tDCacOnlx4 6KO9uJT18bbrYHzaHuPcyk9XBZm/kO4= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=wT1FZos9; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of 3dnauZwYKCGERTQDMAFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--surenb.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3dnauZwYKCGERTQDMAFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--surenb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739486839; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YNoCxF31vK1w52ANaPw+siaL/yNI9CD2wHk/1nIOSYo=; b=IvR6lf1XO8jEkyA1bla+QyjKpJQWeL8XXzJIHnteeBB825+uZ13ktEQlVJVsv8qDZYXJ3X IHPNRx+Udi6ZJg3MpkOT7gnjB//W7cR4223lyYAe0iWcTq3U7TwE/8zfVlwfvfNzB8zoLZ 8OLoIFvLKrlM8qvpywjbLvUty6hgGlM= Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2fc17c3eeb5so2141682a91.1 for ; Thu, 13 Feb 2025 14:47:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1739486838; x=1740091638; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=YNoCxF31vK1w52ANaPw+siaL/yNI9CD2wHk/1nIOSYo=; b=wT1FZos9dXWeNj7TPBp7GSXcgOiuREwGgajP3D7KYPFxyNZU263OvoiHAgx3V9jX7D ibS5bciseDddmlyq9siBI39mukNS7f53lHnW7NeS4HIR4lS7JCITjjXzVF2m+Pgzin7E /ZFYP+VAyJqg2QgNbsRPenDisWRyy1TSCraXN1Uh+N8JPY3gS3A9MBTg++9FHjoeTiVx cwJXXbzA1GPUCejen5rIEhpEc2/um4O1Myvd/8Fc+1MLyypoI9KE+94MU4jvRuuTbQTG fGZDK1dBzeAXcro57LRRYC8fsWXzD343WYlryicLgpvbKsgprWTe9um6xYHbAvsxe562 sUBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739486838; x=1740091638; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=YNoCxF31vK1w52ANaPw+siaL/yNI9CD2wHk/1nIOSYo=; b=mIhdLWV/kM7acPoiyJMSIywNlzyDM3ZpvYkjHUKMmv43uf99vmcOQ37Z/7CoYz3yD8 axYs0EZmq8+CbhIy43iR2Mj2+opvAHKFzbK6vLQ8V9u45QOdbAb6pJC0bQwqCakldE4C AC4bTDrwzIJ4KM1aD3u4DasQc8DTHD7tVwNVNe0VWvmVBNnQCXXRT/Kb6tuiJtUm2FVd N3jZBqjE9ljzLTyy5Xl8TBAL9qEbhAIRymDOqVHMyjaOueUWunht9rESFlkfGWc0UOpA F76lVtSFrx0wl/dfj1jG01AysLYf09noKyAIQpMwRmeIkmNek5dboXbo4vDReI6V4Gtk lbsA== X-Forwarded-Encrypted: i=1; AJvYcCX/0lxwbvtaGFFPQuyHuZBgWtxGnBS53nG675FfrBvVAmuQZJ8xGXPetBVnoYIgUaKdmK/5rpZ8HA==@kvack.org X-Gm-Message-State: AOJu0YxDsImroiGztpEQrQ20VVecsX3BdgV5kQEQDJc9dBJU51DR2TZQ dkKybFlD/RI8q3Praid1Go9zWUhKe8JlwCIZ0frJRrPCAnSCc6fqi0Y1kFp/DTeHYTEZ67c089v L+w== X-Google-Smtp-Source: AGHT+IErQJcr8UIFZH8RFb6dXrkV59J3DVPaTleZ6Gt1R/Rh5Le0xTpOCinKGJEwYZoy9GO+2bPedgwmiK8= X-Received: from pjm4.prod.google.com ([2002:a17:90b:2fc4:b0:2ea:5613:4d5d]) (user=surenb job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:54c4:b0:2ee:ab29:1a65 with SMTP id 98e67ed59e1d1-2fbf5bc07e4mr13915594a91.4.1739486838053; Thu, 13 Feb 2025 14:47:18 -0800 (PST) Date: Thu, 13 Feb 2025 14:46:46 -0800 In-Reply-To: <20250213224655.1680278-1-surenb@google.com> Mime-Version: 1.0 References: <20250213224655.1680278-1-surenb@google.com> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog Message-ID: <20250213224655.1680278-10-surenb@google.com> Subject: [PATCH v10 09/18] mm: uninline the main body of vma_start_write() From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: peterz@infradead.org, willy@infradead.org, liam.howlett@oracle.com, lorenzo.stoakes@oracle.com, david.laight.linux@gmail.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mjguzik@gmail.com, oliver.sang@intel.com, mgorman@techsingularity.net, david@redhat.com, peterx@redhat.com, oleg@redhat.com, dave@stgolabs.net, paulmck@kernel.org, brauner@kernel.org, dhowells@redhat.com, hdanton@sina.com, hughd@google.com, lokeshgidra@google.com, minchan@google.com, jannh@google.com, shakeel.butt@linux.dev, souravpanda@google.com, pasha.tatashin@soleen.com, klarasmodin@gmail.com, richard.weiyang@gmail.com, corbet@lwn.net, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com, surenb@google.com X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 1877240009 X-Stat-Signature: y48nerxrgpw3gkuxoy8cd81wf78g6mwu X-Rspam-User: X-HE-Tag: 1739486838-157597 X-HE-Meta: U2FsdGVkX19D6DuvElYbygNSIwxtITKPXUsX5YE2/H1BrM02z0cw9vcZLpdZMGB3seT2NSlB3g7UMtyq/wiVHICZWgPEhr1bRrd5I18gJy6ifQWGBXAXZS551bePysXhaJb+CKdUmOTv3yTIOdpee6HZfJlRlYkmssJoRKdcs+eNKSCz+qxOZCT5qfetDZTpM7Uaep/c2LaZVfmkgDt4RtsKc5ArxKKLcPHtboeOh1qhMrP/MTTDQhJgMOu24g89hUtUwZuXqoY2AglnuluUaHC21bY5EffuJVado9fE/KC93VniSAJHRlvHiFUDmox+trbGnCJ0g5ScHmqp/TmISkIhQt9I3EeuvtRqPwTyuK2iFGLapH3tkfwX8m5KuRZ9Nl+KI8r1NPje7ZSEQIzc4/65q58yOx55jqxVKM5nOcApir9g3aO/z9MN3piN4OUiaBniARXrO0U0VsRvyn/jGvc6fY/tHD0lAJPJtVsuFQSlWf5XcXJO9P9cdIjkRlmW/lRsNkRVTwSl9ApXsFT6PxAnKojW16ZmcJ2HEfEdixES7KzT9GiFX+QB7O8yPz5mqtgiIWWwCMTzCuWoxWx+uiir+w9dsYMqtpSxh12EG/01ampRcBsL4TC1bKJsA6w4sMvjNX4UaIeUCiWICGxYsNedPYJcOgmgP7x6xPuFXze/m7iY45MkfsPzFjnZ65Lrzk2Yntlata2q8QKhpGyk5WCCKpPcErCReeCeSNskPlQ0YQSA6BJPDDqncwcO2ROfNLnI7SvNd0To+DEZcRaUSyAl2rssWZX93HNwOZIOye+gSnFqKrbS7lYi5EI7H6zGXZRvNHbtIvCOV79eDsjcQnnsIR5efj0rQIpq5lq+oGEmJb7QwDlrbOailm6yk9UgGGWAZuYdwnvPiAfth9pJ6eC5rUCNhXIVSC1J6D9xcvF8+9wx6qZqJ+LWC7A1cC6AVEVeeVMlrlobHjI4zXg vOmyrNuT 0t9E4+leNjwYQGjKb2ykNQeK3AkD13G6ZxvHUBgQFx+3rcx1pbt1hFjuedBT3t2FcMyiXxXxAnWmaOZ1mGIvw03jb9ZWHjruq/M5NAL5NyWZHtfBheaiShPM8tw5wxCqnRVaVH+RhmT7QNo0LB14u+/TdYuhbzBpvgNW4ICkGLYHABGAQHmb2rPuIkNTTJwpk+SaSbTvT+6T1jvdcaKMrD/a+d3XdvHkt0hgYZs/eCwbDRGCGfwfBO9qeyV+PVAamtSipH+1ymK90wFLkJfcTXzy/2D/SXsbIP/YB7iMfaBGEZVQAF6Exe86TJdh3YmVmnzK7T71g/cw2d5P4AiCm+Cu1mkWPfAUF+zo8/7DrGRZz1QcaxpKv86yKh1vCfspfD3hFOjfJkF7rVYAGMjwDEFYbO5o+YdJMQGFFcB9pMx9EbrfLT+SM9YC5Iroy42RARH+LKARPTp4y+urK18EE6exSxd6ej/jK2OnVWKo39RWJbloRtfKpyN0vpMDu+prWZWuvwoNS9CVBgmhvDcEtqQqoMHjxPIzdW8t/WALPgkkMxw0jc2Dv+726A6cXguIMYBaOtxepJFoJNjRWKUPQ/7CnJqPJRfkJfJPHjpVPlBUrNqA1yT5/R+yJondi5k/F7baEqMoutbT55pyyw5bjO2fJvW7/Zc9ySCLClakB25+z2h0tDV9CbjLZH1Ef9TZJZ5WETJurguYmbhI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: vma_start_write() is used in many places and will grow in size very soon. It is not used in performance critical paths and uninlining it should limit the future code size growth. No functional changes. Signed-off-by: Suren Baghdasaryan Reviewed-by: Vlastimil Babka Reviewed-by: Lorenzo Stoakes --- Changes since v9 [1]: - Add Reviewed-by, per Lorenzo Stoakes [1] https://lore.kernel.org/all/20250111042604.3230628-10-surenb@google.com/ include/linux/mm.h | 12 +++--------- mm/memory.c | 14 ++++++++++++++ 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 7fa7c39162fd..557d66e187ff 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -787,6 +787,8 @@ static bool __is_vma_write_locked(struct vm_area_struct *vma, unsigned int *mm_l return (vma->vm_lock_seq == *mm_lock_seq); } +void __vma_start_write(struct vm_area_struct *vma, unsigned int mm_lock_seq); + /* * Begin writing to a VMA. * Exclude concurrent readers under the per-VMA lock until the currently @@ -799,15 +801,7 @@ static inline void vma_start_write(struct vm_area_struct *vma) if (__is_vma_write_locked(vma, &mm_lock_seq)) return; - down_write(&vma->vm_lock.lock); - /* - * We should use WRITE_ONCE() here because we can have concurrent reads - * from the early lockless pessimistic check in vma_start_read(). - * We don't really care about the correctness of that early check, but - * we should use WRITE_ONCE() for cleanliness and to keep KCSAN happy. - */ - WRITE_ONCE(vma->vm_lock_seq, mm_lock_seq); - up_write(&vma->vm_lock.lock); + __vma_start_write(vma, mm_lock_seq); } static inline void vma_assert_write_locked(struct vm_area_struct *vma) diff --git a/mm/memory.c b/mm/memory.c index e600a5ff3c7a..3d9c5141193f 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -6393,6 +6393,20 @@ struct vm_area_struct *lock_mm_and_find_vma(struct mm_struct *mm, #endif #ifdef CONFIG_PER_VMA_LOCK +void __vma_start_write(struct vm_area_struct *vma, unsigned int mm_lock_seq) +{ + down_write(&vma->vm_lock.lock); + /* + * We should use WRITE_ONCE() here because we can have concurrent reads + * from the early lockless pessimistic check in vma_start_read(). + * We don't really care about the correctness of that early check, but + * we should use WRITE_ONCE() for cleanliness and to keep KCSAN happy. + */ + WRITE_ONCE(vma->vm_lock_seq, mm_lock_seq); + up_write(&vma->vm_lock.lock); +} +EXPORT_SYMBOL_GPL(__vma_start_write); + /* * Lookup and lock a VMA under RCU protection. Returned VMA is guaranteed to be * stable and not isolated. If the VMA is not found or is being modified the