From patchwork Mon Dec 16 19:24:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13910242 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83043E77180 for ; Mon, 16 Dec 2024 19:24:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC7016B00BB; Mon, 16 Dec 2024 14:24:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B77716B00BC; Mon, 16 Dec 2024 14:24:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A13ED6B00BD; Mon, 16 Dec 2024 14:24:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 803E76B00BB for ; Mon, 16 Dec 2024 14:24:43 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 328861608EE for ; Mon, 16 Dec 2024 19:24:43 +0000 (UTC) X-FDA: 82901798028.27.F89FE70 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) by imf22.hostedemail.com (Postfix) with ESMTP id 41BE4C0004 for ; Mon, 16 Dec 2024 19:24:10 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=UUrek2xm; spf=pass (imf22.hostedemail.com: domain of 3d35gZwYKCDgmolYhVaiiafY.Wigfchor-ggepUWe.ila@flex--surenb.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3d35gZwYKCDgmolYhVaiiafY.Wigfchor-ggepUWe.ila@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734377053; a=rsa-sha256; cv=none; b=gdNY69I5fuokhcYiR3vP5gMxu8QBLglUh8noRl0Zj9lyzLqtMM8zIdhb69kEnxQyqHb88R opx+N1ntUrIMF/zpNZK2wCzRfOPswB99n69QUKsQflB/JvfNSzoWIQtN/RZrEVdgbL+rcV OHKNU+Xz0Y2Trl4U0ZJT6QoDqjhlE4U= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=UUrek2xm; spf=pass (imf22.hostedemail.com: domain of 3d35gZwYKCDgmolYhVaiiafY.Wigfchor-ggepUWe.ila@flex--surenb.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3d35gZwYKCDgmolYhVaiiafY.Wigfchor-ggepUWe.ila@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734377053; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IlSIABx/zxxF1ijBMV6mDoRgzimHB9QssQ8SIetRkVU=; b=vVkebs/XRF9Z76cMmqkHdvHBmVsBcSrulxXsvWKhpCaRwmULXNekzr3A5ENOyqby+1YhGJ Mqdip/vOd0ZO5qSktnFkYnA4L/Km1TRfF4iwTl1kn6DQsxKxRBPLFy8uy0ueDwMnNXWRCg VpYPBRDAyc8GsL95/OgvmuBKLyrK8Lo= Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-6c8f99fef10so4980635a12.3 for ; Mon, 16 Dec 2024 11:24:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1734377080; x=1734981880; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=IlSIABx/zxxF1ijBMV6mDoRgzimHB9QssQ8SIetRkVU=; b=UUrek2xmE6eXiAnoSjOt1+8VeqRyJKojXydaCr6I+tvlkZ+aw6fcxAZYgIfqu98BcE ly4I33YloPRanruInoguqiGHiW3A5J2pdI4rRafXW/Ka2C5kfPj6YubGG9Sjx3eKiYhn IHOyVe+kfdmxPexeiRRU0QWHKUb3LYMklKDAbyNJiE10xH1SbzaRR2MAublShY+Y7snI IgJhiDI7kU8bmS9ru1wKwUuPtChfu+sFYAq+SXNBpchOVwyxEwkUYtsP+gx5PMDYdW05 gnVpSidXY1ZRXTTHgWNY5ZAKg4Y6e7TJRc1jeZ7EDKpFviese3vNcNe7jvThChkoXPuG lGwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734377080; x=1734981880; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=IlSIABx/zxxF1ijBMV6mDoRgzimHB9QssQ8SIetRkVU=; b=VEq5ZaGUnm8cfPLcGTlH1KoxXics0qSbbXR9H8SoJ4r7OOMzHfYxGBe1HwJzWd6uoh BbVZTfl3ydIOjJkek5lrDJLABrrapwQe0tligdiA15N5BuHXVV2LYCLh+yStzGJbu8TR QOQCB7O42mkXErxw1uk3B0Lx6LLqO0d28xOMW1TvpwIiQIGdFgcF9NBnmeiGyouMtHOF 6KBRjFOmIJMV17pKIEkaFJ9fY+QdGHYdYM24puSKZEfeC01lXlzzIwetanSJspPOgNjw 8iLzuJ0GxZflJ1vZhT0AFl7IVDa0wqRRln5mup/m038HKu/2ZN/KJ6pt0IUKemzbfyWp 5wMA== X-Forwarded-Encrypted: i=1; AJvYcCWfOEqo+J56ip25OmJ7QA9IhCJCj4jgDxnWGICRTL12fPiyL27Ban48ZlU1prie/oJ3/F0CS2P+Gg==@kvack.org X-Gm-Message-State: AOJu0Yx5bcMOt+ECfICwFD1FTR3kQpHmCUFYTUHNSqE9w9YBviMeOcAs MSuQn84mO94ogP4RiwM/Z43gVhe+kbLLzuTRpso05I/4j6zWyvoayqIHM83cCtQeA7cS4BgDqfG Krw== X-Google-Smtp-Source: AGHT+IFInU2/c8WUI1v0xjpwtORb6r7NqVkTkq8OC0u0JtieghImrnI2AnpvFTBaqoW2DCB2chLg/yy/KrA= X-Received: from pfblh8.prod.google.com ([2002:a05:6a00:7108:b0:728:f1bf:72ad]) (user=surenb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:9c8d:b0:1dc:790e:3bd0 with SMTP id adf61e73a8af0-1e1dfd3dbb6mr22228664637.15.1734377079970; Mon, 16 Dec 2024 11:24:39 -0800 (PST) Date: Mon, 16 Dec 2024 11:24:11 -0800 In-Reply-To: <20241216192419.2970941-1-surenb@google.com> Mime-Version: 1.0 References: <20241216192419.2970941-1-surenb@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20241216192419.2970941-9-surenb@google.com> Subject: [PATCH v6 08/16] mm: uninline the main body of vma_start_write() From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: peterz@infradead.org, willy@infradead.org, liam.howlett@oracle.com, lorenzo.stoakes@oracle.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mjguzik@gmail.com, oliver.sang@intel.com, mgorman@techsingularity.net, david@redhat.com, peterx@redhat.com, oleg@redhat.com, dave@stgolabs.net, paulmck@kernel.org, brauner@kernel.org, dhowells@redhat.com, hdanton@sina.com, hughd@google.com, lokeshgidra@google.com, minchan@google.com, jannh@google.com, shakeel.butt@linux.dev, souravpanda@google.com, pasha.tatashin@soleen.com, klarasmodin@gmail.com, corbet@lwn.net, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com, surenb@google.com X-Rspamd-Queue-Id: 41BE4C0004 X-Stat-Signature: ahzjqj379o33gmruoaadumsbjjqry9sk X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1734377050-741476 X-HE-Meta: U2FsdGVkX19jPM9HJYaaS9t8OCB2ia1Kp6V3VW0qEMeZ1/NPh37DDA0oQrw3EJVDgpOLyU67dsDyAZUP2enCgwLsRKEnaIIeivdo+7x+8Q2tjWrBxbYqkyuoPX+1Wv7+dTON18jNkcdLH4pmddKOesv5Kw2jOifbGYvMIuQT+4W8isDyAvC6MGFDyinp0xIOvGkynI0JRW672nH/f5rfmvGY7bYMvFedBa651laQjEwxP9YQ2H/DTnZaNz4diYWZNABq1rywMLrZkq2Loh3FLUD7F25sSFn2UJWZUMMti27T2YQUV9YUP8CFkpMIXa0kwL/WUpx+97hKnlBImK8zkT6jJEMTCIGzvmXjTpKE4XpuALrdycXSYwwfw58Y/tmRx2Dr+WI3ulgqNVobPaMFPNX/8UxWPd4veohVXpGu5hhbxyjmguf/tqUqScKyTC7FsCiqLXbZ2/AMmWiES1zROJ8uGIibglbcB5yrRPw98DIi+cUiIjVJNQh3qKT6qDK2rrOzqfWr+uacJ4BfAAv3fees+4pQjUWhfXsw5+/voguYEAnVLyCH+kYG0rf5zYebyAEI9mLzEnXhEwoHrl3a+YezgRiCC58moPoJFy/BmsnI17DFLjxSV4OabzOcIGHJMgqP+FZxUQ+6jECBKyEdlLgM+8/CeuQeT9rwPSd+Aj0nBUe+zcnf29KmYiPSgAEsp4BZguCMILow09LVAaP8ZNNeeFFAY7xDV0CNayV3ZHEd/T4k9py/pqtzpRF9iDyFo7Oz5P5KVFi/4mEvOc9r8FMVkwo/b9/UU/IVT8wyvTCDPl68IOcmbhr4Ye7SlzwYcCllY7/XqfUD0Nn3SB5+yufnlL/UCfQb99bO9CY45Kz5wohkRMKUaoGJdJSI6T7k6+gXUMPAY0MoQA/iwFwE/7Prg9aAq076l0GmhF283S9i+IcDbuoerapgA3knZn1ZDDGxEA9g4nNwBU8/YW4 rS9/fUBq Gg53tlxZxHE/BgUpmq6o94MnvwcGgxWWVOv8Lm/Bd7jcCLZM769kOOwox1VvOSLY3v02LTpZgNPzj25490Z7DKfbo6La7bxVqtDsl5i8Hd9b70wHDO5USabxTDHCOVUkjzMgNaT4hT/XUVKscIIS49GX6trEfzocYFuAAY5pODN4ZglixG9v+4vBeFcEtsQzNUfR/6hxMy+rzoO1BxF686jagczjVdPmcjUMYr7RhWe8Mxk8SJCKs0hsnkh459a6C1BslMPX/V8YsE1h/t4hDDaaYkngrCuJJ1gknGpLJpTkHUwG69rkqmsl1dc5Xj9FZQaVFCw0eWCHYij7Xy0tOl+egE/MmBjZ5Zera1e7bQWEiHDAXgjP2NtrG/8s6SNXlF2vtJKbJkSG8kFQNrALTYBL2s9PHnhKmLbwbj8O/kB2accNOBNL1hLee2GGE3uFxLS5RpxNhW0xqxvGWiQu8GK4AG8nDijoVVvRwPzdEesck5dfZQJ5HeQoZbyYvABpM11B5VPdag7ku0vBRIOk429e2u0n7eK5I+DUIHKt/FIl+7SNbaK7CmpBqt/wPhtwozkCZYjagkefb6y7Y4OKNGaQvXw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.058830, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: vma_start_write() is used in many places and will grow in size very soon. It is not used in performance critical paths and uninlining it should limit the future code size growth. No functional changes. Signed-off-by: Suren Baghdasaryan --- include/linux/mm.h | 12 +++--------- mm/memory.c | 14 ++++++++++++++ 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 0ecd321c50b7..ccb8f2afeca8 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -787,6 +787,8 @@ static bool __is_vma_write_locked(struct vm_area_struct *vma, unsigned int *mm_l return (vma->vm_lock_seq == *mm_lock_seq); } +void __vma_start_write(struct vm_area_struct *vma, unsigned int mm_lock_seq); + /* * Begin writing to a VMA. * Exclude concurrent readers under the per-VMA lock until the currently @@ -799,15 +801,7 @@ static inline void vma_start_write(struct vm_area_struct *vma) if (__is_vma_write_locked(vma, &mm_lock_seq)) return; - down_write(&vma->vm_lock.lock); - /* - * We should use WRITE_ONCE() here because we can have concurrent reads - * from the early lockless pessimistic check in vma_start_read(). - * We don't really care about the correctness of that early check, but - * we should use WRITE_ONCE() for cleanliness and to keep KCSAN happy. - */ - WRITE_ONCE(vma->vm_lock_seq, mm_lock_seq); - up_write(&vma->vm_lock.lock); + __vma_start_write(vma, mm_lock_seq); } static inline void vma_assert_write_locked(struct vm_area_struct *vma) diff --git a/mm/memory.c b/mm/memory.c index cc7159aef918..c6356ea703d8 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -6329,6 +6329,20 @@ struct vm_area_struct *lock_mm_and_find_vma(struct mm_struct *mm, #endif #ifdef CONFIG_PER_VMA_LOCK +void __vma_start_write(struct vm_area_struct *vma, unsigned int mm_lock_seq) +{ + down_write(&vma->vm_lock.lock); + /* + * We should use WRITE_ONCE() here because we can have concurrent reads + * from the early lockless pessimistic check in vma_start_read(). + * We don't really care about the correctness of that early check, but + * we should use WRITE_ONCE() for cleanliness and to keep KCSAN happy. + */ + WRITE_ONCE(vma->vm_lock_seq, mm_lock_seq); + up_write(&vma->vm_lock.lock); +} +EXPORT_SYMBOL_GPL(__vma_start_write); + /* * Lookup and lock a VMA under RCU protection. Returned VMA is guaranteed to be * stable and not isolated. If the VMA is not found or is being modified the