From patchwork Sat Jan 11 04:25:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13935775 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E62BEE7719C for ; Sat, 11 Jan 2025 04:26:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 086606B009D; Fri, 10 Jan 2025 23:26:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 012456B009E; Fri, 10 Jan 2025 23:26:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D7ED56B009F; Fri, 10 Jan 2025 23:26:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B40F56B009D for ; Fri, 10 Jan 2025 23:26:30 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 679CE456D0 for ; Sat, 11 Jan 2025 04:26:30 +0000 (UTC) X-FDA: 82993884540.24.864DBC7 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf12.hostedemail.com (Postfix) with ESMTP id A227F40006 for ; Sat, 11 Jan 2025 04:26:28 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=qPndwAph; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of 38_KBZwYKCCQSURENBGOOGLE.COMLINUX-MMKVACK.ORG@flex--surenb.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=38_KBZwYKCCQSURENBGOOGLE.COMLINUX-MMKVACK.ORG@flex--surenb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736569588; a=rsa-sha256; cv=none; b=K7ILVmD59+E5/E3jWrOGbXIjugrX+VkOYS6Ijvbx3Kju7FD9ZNiFItksSgfes1+aSllj+d +Wc1DWo9+qaMaYwIEx1Mam6B1ZlguXFpNFoyTrPs2lOsDeS3BRxxLtdzIqG2YXKFp+yE3d 9t5bv/jwQHqebm0DDAX4cUO7NEhB7yY= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=qPndwAph; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of 38_KBZwYKCCQSURENBGOOGLE.COMLINUX-MMKVACK.ORG@flex--surenb.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=38_KBZwYKCCQSURENBGOOGLE.COMLINUX-MMKVACK.ORG@flex--surenb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736569588; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=STfGBHDxpr+nljSTXZBRe7awpLRa2H0ex1QXNY89OAU=; b=HbONVYlDkVtnn/N9exz6fqbxNbVjNSMjbBJNaKoYfazbTPYpP/fXRG+ShGy2Sh4vRWIQOL I2Qfg7pprAoIYOZzoOtX+chPohsqPjLG8V6HTMtxEZeYazoK85RiMdqklK9nGWwGNiUOk3 ZKSfW5CqPEUeSlVm/E7puTBVMjx5pSs= Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-216387ddda8so52048915ad.3 for ; Fri, 10 Jan 2025 20:26:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736569587; x=1737174387; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=STfGBHDxpr+nljSTXZBRe7awpLRa2H0ex1QXNY89OAU=; b=qPndwAphVzuRyn8cH68Jz+GfGJMpm8AhKru9u4luDQvp9CLnVUoNdcn81kL6X5Fr2W iqbb0y0/0E9Obj2r6tzHx3329N/3bg5fcB4G8qRm2G5wiUe+xHbF7Vkb5n5Se7iv0fsE kOmvel5UZGtwnnty0kv34Ed+BfQ6OZLxggNYMg6X3Lx31UvGPIoAwQeJFnvB7Voi39/F oqq8RUXbPzkL1+s749/GOormlaFppnTpAnvq2mtz6i87N9O7+bcamWMdRRhkFuAaW638 ISyfcg5KYkNERO/qK2p8nfLoDloCMum3PnVq+yN3C1MPt6hRVYX3/qB2cMDOYLhEVIy7 wXBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736569587; x=1737174387; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=STfGBHDxpr+nljSTXZBRe7awpLRa2H0ex1QXNY89OAU=; b=Xn2K1XLWXYKmsOdE+9Yp0WbC+uxDW2jj/tBGmRUhzvfhDwBKpg3EKRiQ38U5a4XPeH l0TttnAXs++1CIIAlTg+tgT83etJCwZTeIcAsiq3mA10CGRK6x2BpaAUbZh/nAmlqey1 +rN5ff8i4wJOho6XFZ9vqsVvc0KRsRiiR/4YHSHlGI9hYR2cfE55kq4GvUraxp3bfKPT peuKNfM6NeeU8zb2kqq+97oJLTsH1iU6OLnb6bmeE3V5LZ0Nmnk2IPjid4PjIS8LFcCX hG3l2uxTLd3M7NxRBHYi0IgccRsrx1c/vXQ2dmnoJjYK82IG6pOPqKjF4aH+eFovl0x8 4nNA== X-Forwarded-Encrypted: i=1; AJvYcCXYgE8sK3Exu1CmCbAR6M/ScZrhvFDumJ2rnW181Pfny4T7Lt38zPRfiSQ9XiShJ4KRWHf2PMX7xg==@kvack.org X-Gm-Message-State: AOJu0Yz+bkk/BT/NCaEOx15tU9bGgnCTNbunNlXo4S66fIRzr8VJDvoW dR6aL0NDARA4frZ39k72yOwuFtpLpXEFKWxjC8RMk5xeEeokP5BSo20kSGbTSUQuxBexGvq9PYW cPw== X-Google-Smtp-Source: AGHT+IHvoBVJY80+e+qdsj3aPRMB2UfI+za8DGuxRrXd9nOwFJGkON2pj+F8zNabSAnJMgvKZtfO1mpbjyw= X-Received: from pfbkp5.prod.google.com ([2002:a05:6a00:4645:b0:725:e6a0:55ea]) (user=surenb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:1489:b0:728:eb32:356c with SMTP id d2e1a72fcca58-72d21f459dbmr17649276b3a.11.1736569587302; Fri, 10 Jan 2025 20:26:27 -0800 (PST) Date: Fri, 10 Jan 2025 20:25:56 -0800 In-Reply-To: <20250111042604.3230628-1-surenb@google.com> Mime-Version: 1.0 References: <20250111042604.3230628-1-surenb@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20250111042604.3230628-10-surenb@google.com> Subject: [PATCH v9 09/17] mm: uninline the main body of vma_start_write() From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: peterz@infradead.org, willy@infradead.org, liam.howlett@oracle.com, lorenzo.stoakes@oracle.com, david.laight.linux@gmail.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mjguzik@gmail.com, oliver.sang@intel.com, mgorman@techsingularity.net, david@redhat.com, peterx@redhat.com, oleg@redhat.com, dave@stgolabs.net, paulmck@kernel.org, brauner@kernel.org, dhowells@redhat.com, hdanton@sina.com, hughd@google.com, lokeshgidra@google.com, minchan@google.com, jannh@google.com, shakeel.butt@linux.dev, souravpanda@google.com, pasha.tatashin@soleen.com, klarasmodin@gmail.com, richard.weiyang@gmail.com, corbet@lwn.net, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com, surenb@google.com X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A227F40006 X-Stat-Signature: pc6h8oyekqaiz8prey9c8b6aa879kia3 X-Rspam-User: X-HE-Tag: 1736569588-621418 X-HE-Meta: U2FsdGVkX19ZfUfq4B+5Rlvh5WPFcxtoA74RjRCOhNNumATlX6f82hdQRfs6xbcue/JVxqeycu52LheE8Srgm0e3CghjpuSmj/Sewm/yTJXQn0TX+WD1L7dLzcJe/A0zt+E0Y1udpHlJz+FPkbZFDHuA4zrTP5mEsotPyk1AhPKyF7l5+0aUWqnZyeWwWPbnW3kPZyOTGtBhTqVcWylElMtgAvPAJ4XlEmWk7UwIqa1j4SfPEZoPegJxhTpsXgJk56Bnah2opCrA/NWmhGatUOfi2/afKGRlANnLRdKaVP8TRMOUK1eOgAULmIw2r6tiEsT235QkkWudW1wRnPvZ/Tk4zBNNYx/NgbDCebGgfbN9+omNU7/ZMf4HAM5lsWkTgemtVIVNYpkMeJe0RQOvgBkWiM7barq3rC+mYM3tALKs3Xade3lqHsVWx6J7viftblXAz5p0t+Vm65zBAYpDA1In6Vmh2EgMQto3mpKaTkILxsuPG+1v0yEpH+liXuG67UttEllOqaTP4lv1Mq3FCbr5e2A0oO9VZ0yAsJiPVjNW1arj/KHZLu73mvTqvaEBPh0yL2yrVoBS9o2WUtPmMFEWN0q65QH2VpERBue226f9e1wknH788tuCrS8Ihzw2Shyu/3bS8gMktIBMNPuehZdsKpCe664cLZ09VfFsMaWDLvfRlUFxRaPTCKw2bXp9yqUC8N3ShF7hNg5RfcAWxgBqunILdM15mJYQBC70zbverNrThUBQ0shwn+qcWzNwGlXClko5h25M/vhnnR0dXkpIT5aQy157KZ7G19D23oZP8SLnUDchyvkjp2tIckBVKC+Y+34Ibo1q2q7cedJ5i/vE4UiCX4ocvsLDuAGJ8iToSeZuCAqodzuK3dVRZPc2krKl7kRuE49jHPKhGb1TUMi8yNEpgL9MTaXW9vXzpadpa+meHcF0z/g1KQkCh/rF43Z9griLfVDwK565G+C j3bmETzr rjz2kS2OuJnNAlSH4HYTg3BO1GjxZrEfrNaF8f6Po1a8Hea8+oak7RejFNG02xHsY3Q+Pp2bpl75aYiTob9w4L6qPTLaB9AlxkVo62frrlhOBz698J1x4yyBZCIig7qcu8A/9oi85RCr5DPbRXTQ+ob2QpMCclUvJiah0lgaW8rLNuC6UczuwQKIvV2ei5Q6jaYtvdEiKY/CP80q2csfz+zzMWhnxI4Wz7Nn+6poH5vBFx6nchPk5YrNOec9URWRg2hQ+BsETeerboEKV+dZkePTA7kGjBtgm8riBnHrUz00lxL7rlF4Zlcg36ND5kz3JdElFptld+MAwsuNLOvrQ9MoP1xu6C7ZICFZegjb2IV5vV4yfVVI2L/PkGtGgH1GG0DwH3nkVjI1emM5ZcqysZDwwPsgwG67n6mAv/f5rEGB7A52/XFFj7yu0u8EEdqxR9wgygNGpmBweo3X73s7vrW0MiGeXKTqIy2EuOovpPHR14UcJ4xFeCBIwV9/gCW4PihppAPrWniIDjHXN0Kxw5BWLgPGsr3+L3S/SCUAzcLPlFL+0DQDplE2ld0lHNySruyx/+Jv368rBjkmD1pzOl9dPx/KX+U3y/6Ntv1jkARFkTmwoZ666upfAZB+QWnc0TnlyTcr/ye5K1+HNLAIvhgKFbW/BtJNUEb1E+UW+9lvdx8jHECfLFsaI5Dc8wpbQ6KM9Bnpr2Wv82pKGWdfN6UqLjfq6r+0C+JBfTmTV/D+WXeM+CDpaxYiyJw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: vma_start_write() is used in many places and will grow in size very soon. It is not used in performance critical paths and uninlining it should limit the future code size growth. No functional changes. Signed-off-by: Suren Baghdasaryan Reviewed-by: Vlastimil Babka --- include/linux/mm.h | 12 +++--------- mm/memory.c | 14 ++++++++++++++ 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index cbb4e3dbbaed..3432756d95e6 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -787,6 +787,8 @@ static bool __is_vma_write_locked(struct vm_area_struct *vma, unsigned int *mm_l return (vma->vm_lock_seq == *mm_lock_seq); } +void __vma_start_write(struct vm_area_struct *vma, unsigned int mm_lock_seq); + /* * Begin writing to a VMA. * Exclude concurrent readers under the per-VMA lock until the currently @@ -799,15 +801,7 @@ static inline void vma_start_write(struct vm_area_struct *vma) if (__is_vma_write_locked(vma, &mm_lock_seq)) return; - down_write(&vma->vm_lock.lock); - /* - * We should use WRITE_ONCE() here because we can have concurrent reads - * from the early lockless pessimistic check in vma_start_read(). - * We don't really care about the correctness of that early check, but - * we should use WRITE_ONCE() for cleanliness and to keep KCSAN happy. - */ - WRITE_ONCE(vma->vm_lock_seq, mm_lock_seq); - up_write(&vma->vm_lock.lock); + __vma_start_write(vma, mm_lock_seq); } static inline void vma_assert_write_locked(struct vm_area_struct *vma) diff --git a/mm/memory.c b/mm/memory.c index d0dee2282325..236fdecd44d6 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -6328,6 +6328,20 @@ struct vm_area_struct *lock_mm_and_find_vma(struct mm_struct *mm, #endif #ifdef CONFIG_PER_VMA_LOCK +void __vma_start_write(struct vm_area_struct *vma, unsigned int mm_lock_seq) +{ + down_write(&vma->vm_lock.lock); + /* + * We should use WRITE_ONCE() here because we can have concurrent reads + * from the early lockless pessimistic check in vma_start_read(). + * We don't really care about the correctness of that early check, but + * we should use WRITE_ONCE() for cleanliness and to keep KCSAN happy. + */ + WRITE_ONCE(vma->vm_lock_seq, mm_lock_seq); + up_write(&vma->vm_lock.lock); +} +EXPORT_SYMBOL_GPL(__vma_start_write); + /* * Lookup and lock a VMA under RCU protection. Returned VMA is guaranteed to be * stable and not isolated. If the VMA is not found or is being modified the