From patchwork Thu Jan 9 02:30:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13931816 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51D0FE77188 for ; Thu, 9 Jan 2025 02:30:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 33A576B00A1; Wed, 8 Jan 2025 21:30:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C3E26B00A2; Wed, 8 Jan 2025 21:30:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0CB4B6B00A3; Wed, 8 Jan 2025 21:30:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id D2DA96B00A1 for ; Wed, 8 Jan 2025 21:30:51 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 856FE12174B for ; Thu, 9 Jan 2025 02:30:51 +0000 (UTC) X-FDA: 82986335502.22.BE7B14A Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf19.hostedemail.com (Postfix) with ESMTP id B2AFD1A000E for ; Thu, 9 Jan 2025 02:30:49 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=UEg4sBNs; spf=pass (imf19.hostedemail.com: domain of 32DR_ZwYKCIExzwjsglttlqj.htrqnsz2-rrp0fhp.twl@flex--surenb.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=32DR_ZwYKCIExzwjsglttlqj.htrqnsz2-rrp0fhp.twl@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736389849; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IjUQkFQTPO2uWvItBGSJ4zf4/1EWZkyRsCZwETlqeC8=; b=CA8lCk2bhbk52Mixw+b7jew58rXmyq6Qz72t5RI0Mr987o7cVQehCqyn11f/RLf85K9o0y 0QTTRNEG0Z3Z9k6Qf83En12/WhmiKnV0PywPFWd4Id2YhIwKSIA8AW/7FgmJXJDLmKLdEW wiPIs4wUSyIMKgVWlck6Q/dnv2M9Ab4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736389849; a=rsa-sha256; cv=none; b=Jjib3qrFGLzUYbflOiyc2ImqRixxOU4zI8hffRfA81lBDkHFz44c1M7440s2xc6CYhTj3e 5Cmtq5bF54zcCT9+prkggGSgjwgf32vqGO7OGDUB6cfObm0Rzmeh1nS83zdshyB2hJoy5X FSnWIfG7jOhnMliDBu+/L/Ih7GvETec= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=UEg4sBNs; spf=pass (imf19.hostedemail.com: domain of 32DR_ZwYKCIExzwjsglttlqj.htrqnsz2-rrp0fhp.twl@flex--surenb.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=32DR_ZwYKCIExzwjsglttlqj.htrqnsz2-rrp0fhp.twl@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2efa74481fdso836377a91.1 for ; Wed, 08 Jan 2025 18:30:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736389848; x=1736994648; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=IjUQkFQTPO2uWvItBGSJ4zf4/1EWZkyRsCZwETlqeC8=; b=UEg4sBNsEm6ZBGno1Pd+62B0aNLErFhAkE03GQn/p2j2kScTC9vwAy7A7IKDSBDB7M 10Mr7+d8T2h5AXfyPcTpR1HwJ8Isx9otq5qFXQZFEFVmGQlWDNkpzbB2LIdmkgm1e79X G2/9YKFFkNxmpNNIMABrHjhUoTztMEAD8LDKOeIkGPO51G/5jhFcQ81xLS/l27EJVQpW R4j85gOPD8UU1sM7w+wV6J0uNRHhjpfKM29pvcfUJYb+PCynw8YadbXJrySGhm5FCmcB +H3mNtScOWfP4EH5dcDee+TIYya+fPLs/0/fwt4LZrcCM7TwAmOBQMfR2eZIVkRY3Alf NYeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736389848; x=1736994648; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=IjUQkFQTPO2uWvItBGSJ4zf4/1EWZkyRsCZwETlqeC8=; b=vpVGNcFc60YjyCDRHxO1w9bjr9huyOql9BEyV/5dnHyiN5Q+E4DuoHaSodNKqo+rom 41zlj+KLadOiAddhkdOx46ZohtQs1+GyjbTe7V2Ixhf6WlmlMQOpPQoznvvy79XxdXqH 5pVR+Y/6UsbV+cGiUoR5YizRdMk74Wj4MSBGND1AS3gL+8967TZcCect85aXFsTglluK hfVxo15z8YgvxoaaTOw8nWKkNjzUYCr74ztMBII1MrrXFSRI5PVueoDWrbqO3P/BOHdt Q9DMhcTZj8r77Rb/h6iBhwasTqs3MdMVKHVUXtqFTi1MIoTU+rZTZvEaBd9H5QEhgJka LUTA== X-Forwarded-Encrypted: i=1; AJvYcCVOGaAvaYsEWIJ2ibNoaO4pAk374/BapXSkKpUs2dvxBTgWNI2VYbs4HNZGgpB9ijf0TTZWj6UCbQ==@kvack.org X-Gm-Message-State: AOJu0YwfNzzTau4viB0NkO7yDxfEWn+wAyt+1Dkp77ny5SAvDxadzKGO mb4yLWpk8Rp7WMchp38jnFNreli3IoxtvPmouPA8UZkkDY0sK/NtmzJSpD4wSXPCcWc3Zoi0CWd +1A== X-Google-Smtp-Source: AGHT+IFjjwr5ru1uWVNvi15QsRRv5yn9aK9DytE2ceD8f/6OYw8sNNdl9g7eT88QVCpaGBCf3pVchkAXDj4= X-Received: from pjyd4.prod.google.com ([2002:a17:90a:dfc4:b0:2ea:5fc2:b503]) (user=surenb job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:c2c7:b0:2ee:e518:c1d8 with SMTP id 98e67ed59e1d1-2f548f1c3f0mr7653251a91.30.1736389848454; Wed, 08 Jan 2025 18:30:48 -0800 (PST) Date: Wed, 8 Jan 2025 18:30:18 -0800 In-Reply-To: <20250109023025.2242447-1-surenb@google.com> Mime-Version: 1.0 References: <20250109023025.2242447-1-surenb@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20250109023025.2242447-10-surenb@google.com> Subject: [PATCH v8 09/16] mm: uninline the main body of vma_start_write() From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: peterz@infradead.org, willy@infradead.org, liam.howlett@oracle.com, lorenzo.stoakes@oracle.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mjguzik@gmail.com, oliver.sang@intel.com, mgorman@techsingularity.net, david@redhat.com, peterx@redhat.com, oleg@redhat.com, dave@stgolabs.net, paulmck@kernel.org, brauner@kernel.org, dhowells@redhat.com, hdanton@sina.com, hughd@google.com, lokeshgidra@google.com, minchan@google.com, jannh@google.com, shakeel.butt@linux.dev, souravpanda@google.com, pasha.tatashin@soleen.com, klarasmodin@gmail.com, richard.weiyang@gmail.com, corbet@lwn.net, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com, surenb@google.com X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: B2AFD1A000E X-Stat-Signature: sze9wor6acaog5b1sbizxnxy1mze976z X-Rspam-User: X-HE-Tag: 1736389849-498014 X-HE-Meta: U2FsdGVkX1+u8puWikZgU5L++A8K8yOCUEKRyYErNAA3TW0tfCIq62i+ZzXsdlB44kMRyEmVNPVlUE2bNAY9QOEsNVXiqAyy50weUYwx6bQZP9XyTWNgRw0vsYiykSRsJ+OuYoXUhsFoN6q2Kvl7VGfyg9VvNE+HD8sxEBn4C9nFJyjtFh2OGVxlncR1DJ1Bwy7xnO6ihVNhKGpEfUlp1gipsdopSMNaIGUzIrU/oTrbUeKyTJey2hpetkzQu3aMASyjmarDNtBAMO39TTFXrk6/OWdT6CovAzfPEH4WzcbrtiB8eNY6m0QZYlfGd1x9fGdh20Pq8+CgDC2yNmgYSL304SS0LxWpM6INlJ96NDTFNiJhdDcXyQIQiCi3K9CsY0xNcd/x9MpqDvmmhQwLHhh0wDuicxsTF+n0hlyvPh+JLoQFAIzXz03ZhtyapznxhNFzkASYo0s6ts9Lm7qpWsaVHIzObNFM+8NBwhQ5z+Lh0G0Q6laIXu8I46s7z7gS0JgIyMoJjFNVIwl1IdrSa9PyuzkGrHvHVRJQO8XvdGieYnai4w1SBUGHLjIhqQVJT+F3uunNZO8ZUatzKpByTY4rfvbB1+fsdtstuaPDk1IZx6hEllm5JmvthUbUXfJ17/wyxJnKYElTItjh5yl0eN5Flh5i5qOhkILIjUHGNjWP2rDwJcyDwpfBPj198NPB7HzLkmzOepH+wS3V8NbpWUDzpcc+b1gU0Yn/6Dh6Tq3ljtqVD0mqYICWRV7DzJIF7wvFGOhgNdHhg8DvghMus6m65RVk+iC8YczrPqePpOP1Zfy/u5j5PHPmWPgConAMoxW4Q5m7F5TdWFBKzNUXGjx1QRIu+XNBFa7lo+O45HAtMpYhAtmRPue7FA3qlYdyNh92+Suye7S0eOb+81QsWktSe68hYbeNzpCbsNdXLnxSONij9D+uqJ4doVuxiNXBH0eX6XEjvUXcxEivfqX +rYgtYle tseuzq9DX7qYXLcAP2iBMKFGab3PcdC7fsbsRK0M1bhHc70JA3iXcbO+1jjNhwPAfMQPKFk+eQNJ2w2pVD9LdFGN8/2tr4ntFxJ3q4eCTZwuS3k35QA7dtNlBhEAvwSWBVrGTbnEsNO2sMug8IEKy3NhC1bhEgBr/JAL7DVRNsXGR5t5D82KZ7Rems24ELVf3DbHjcy8PK+U24cYkNTcyWEBEqlg8INecy9G9DN1DyoSpHLUVianF7ac6eWVMk/pjW3+ARRUAHT3HbKAquPyKW+GPGUqzz/ZHHmYbId7k1Fxhj3pwM7QY3Z5KQVs1TC3uW3nOHFF3rYBzx4SxZfRz+pNAZj8eFrHPyNtboVPeAo/uME3vgU4a6QVjRjvHz5Ehvz01qbnEViPkKHQshcdvyLEr+ZGfZ4SFbkolSjCwbK/5Iv25YyInt6Ri0anM4A8y1fQjg/9adGu6wRp2sOPUmqGESPv+NqD2OEMYhdvWuA2KdObj5iSefHRO8kZb7vCHuwKIlfvDtUMVlRlC74qqPVCIWF4lfhLAv39gBVCBgeeoLxrqkdXWZQSfjqjAmLvV4uXjqzECd7phK4YTdWJoN1An/3Yv9JXlWYaEC68pwEJUXCXlzCzZNfl4OYx/jja/MdXyT+K8Zs0lSRpszoAkp8uhzMC9b1McTtgN X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: vma_start_write() is used in many places and will grow in size very soon. It is not used in performance critical paths and uninlining it should limit the future code size growth. No functional changes. Signed-off-by: Suren Baghdasaryan Reviewed-by: Vlastimil Babka --- include/linux/mm.h | 12 +++--------- mm/memory.c | 14 ++++++++++++++ 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 6e6edfd4f3d9..bc8067de41c5 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -787,6 +787,8 @@ static bool __is_vma_write_locked(struct vm_area_struct *vma, unsigned int *mm_l return (vma->vm_lock_seq == *mm_lock_seq); } +void __vma_start_write(struct vm_area_struct *vma, unsigned int mm_lock_seq); + /* * Begin writing to a VMA. * Exclude concurrent readers under the per-VMA lock until the currently @@ -799,15 +801,7 @@ static inline void vma_start_write(struct vm_area_struct *vma) if (__is_vma_write_locked(vma, &mm_lock_seq)) return; - down_write(&vma->vm_lock.lock); - /* - * We should use WRITE_ONCE() here because we can have concurrent reads - * from the early lockless pessimistic check in vma_start_read(). - * We don't really care about the correctness of that early check, but - * we should use WRITE_ONCE() for cleanliness and to keep KCSAN happy. - */ - WRITE_ONCE(vma->vm_lock_seq, mm_lock_seq); - up_write(&vma->vm_lock.lock); + __vma_start_write(vma, mm_lock_seq); } static inline void vma_assert_write_locked(struct vm_area_struct *vma) diff --git a/mm/memory.c b/mm/memory.c index 105b99064ce5..26569a44fb5c 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -6370,6 +6370,20 @@ struct vm_area_struct *lock_mm_and_find_vma(struct mm_struct *mm, #endif #ifdef CONFIG_PER_VMA_LOCK +void __vma_start_write(struct vm_area_struct *vma, unsigned int mm_lock_seq) +{ + down_write(&vma->vm_lock.lock); + /* + * We should use WRITE_ONCE() here because we can have concurrent reads + * from the early lockless pessimistic check in vma_start_read(). + * We don't really care about the correctness of that early check, but + * we should use WRITE_ONCE() for cleanliness and to keep KCSAN happy. + */ + WRITE_ONCE(vma->vm_lock_seq, mm_lock_seq); + up_write(&vma->vm_lock.lock); +} +EXPORT_SYMBOL_GPL(__vma_start_write); + /* * Lookup and lock a VMA under RCU protection. Returned VMA is guaranteed to be * stable and not isolated. If the VMA is not found or is being modified the