From patchwork Mon Jan 9 20:53:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13094293 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 629A0C6379F for ; Mon, 9 Jan 2023 20:54:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0711C8E0016; Mon, 9 Jan 2023 15:54:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F3B448E0001; Mon, 9 Jan 2023 15:54:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E03F28E0016; Mon, 9 Jan 2023 15:54:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D13638E0001 for ; Mon, 9 Jan 2023 15:54:51 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 9829580F38 for ; Mon, 9 Jan 2023 20:54:51 +0000 (UTC) X-FDA: 80336464782.27.BFE29C0 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf21.hostedemail.com (Postfix) with ESMTP id 042211C0009 for ; Mon, 9 Jan 2023 20:54:49 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=aWg1ukrh; spf=pass (imf21.hostedemail.com: domain of 3GX-8YwYKCC4cebOXLQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3GX-8YwYKCC4cebOXLQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1673297690; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0pkQpuFsTWto6vflu1WV/u0ieoG+9lWN639GtltEg4c=; b=FfXrgCWiRhGx5oHr5yTcaglm4nKjcV28L6mR6BHmW6oSxzGNoE/GDTfklWBYszzHmEDSpX 2UUCvT6AIlJAaujx59ayHC/7m/iaACMJv7FxQXjNUhuylRpxVK4M+8FgMfCSiDSwo5of/E goSIXfGOVS1RhmKT8L6CfTXqErBbKyg= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=aWg1ukrh; spf=pass (imf21.hostedemail.com: domain of 3GX-8YwYKCC4cebOXLQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3GX-8YwYKCC4cebOXLQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1673297690; a=rsa-sha256; cv=none; b=0rr2a6y0Go9aqeKpGxrW+G6199HhH5p2nIz+ws7H96kT95NZ+QStZmH7OCSJskCuYBpYjc cBK0N1OHNO032qWqWv2o33wBFd/VggT/A2RDT8vRrAP/O5NcH/Si6PVzvuUEQjpU66Pp4J D8Qw6OqgMDj06uOVrf6V6twbEi12Izg= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-4c11ae6ab25so104150127b3.8 for ; Mon, 09 Jan 2023 12:54:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=0pkQpuFsTWto6vflu1WV/u0ieoG+9lWN639GtltEg4c=; b=aWg1ukrhEhZM2G7oua95ritICYN4v22wUCcY0gEG4tVylqBmv829owF41k3CxkyAaw SuqbgaxpvQ3jo7YPBWRSVqYiX7AqAmWKt0XHBKUG5SuIsHysEKVzXAcfr0KHgTcUUWfl b8z+Kg1PDae7rSd6mmxN8sCBWlq1F+xCCZRkBLC5iLo/l2X3T6+e58rjDwLcesFby2PU TDwKsOLIZU7rmZo8L2TvumA953Nmin0x2oWW50UYSkBp0m/8PkZaL+q137UQ5Jg9l0HG VtNsJ9z3uJiIVmCVloCQiu4j8+XoGxPmrLqPQDbiOLf/gZJpQPJ4VggKimmoURCteWUt n6qA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=0pkQpuFsTWto6vflu1WV/u0ieoG+9lWN639GtltEg4c=; b=ZHd9TJ6OQFORXTfRy5RQQAun65cWr3eM6WctxBI77h71INFdSoExV4eMsQJsgSDbo3 c+1TzQ2gRaNyUv823aOPKQp0V1n9R6iR7qcVSTQvJrs0aMDI/LGyDTiZkfHxIZRUFvW7 wY20P9lFG6lx5ieNnZvDCZSTTStr/NdyZB3fiMpzWYXt+v7ATkrc9EZjABKDDxGSJubz v+Z7qyDF9h/ZRcxf4CKeiO5n+xUGWAhz93O28sFnkrB130hQ1Y2CJNfDuBpifrgrAuW8 4f6M2tQtrDyPnDm7Hde0YcdNyi1xyu87iBoM+zBGD3F/cw3APj5oR0WoRGHXblJHmSPd 3QxA== X-Gm-Message-State: AFqh2kqMCKjNx1Z40cqgkKuDZpxUsT9S9yB1h7ysCZGlPdEY5tU0Ab+P I9n6HKBnhCeiGSADP/4oMtGV40/VLQU= X-Google-Smtp-Source: AMrXdXvMFgW5AuRJO7w2AGyv4a3g+CMQq4rAqpoqS9oQdpJUn0mmZFftW7BbnOBV6/f8EJVvUFaCfl4rQG0= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:200:9393:6f7a:d410:55ca]) (user=surenb job=sendgmr) by 2002:a81:e93:0:b0:3d6:233b:7292 with SMTP id 141-20020a810e93000000b003d6233b7292mr1998513ywo.476.1673297689205; Mon, 09 Jan 2023 12:54:49 -0800 (PST) Date: Mon, 9 Jan 2023 12:53:22 -0800 In-Reply-To: <20230109205336.3665937-1-surenb@google.com> Mime-Version: 1.0 References: <20230109205336.3665937-1-surenb@google.com> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog Message-ID: <20230109205336.3665937-28-surenb@google.com> Subject: [PATCH 27/41] mm/mmap: prevent pagefault handler from racing with mmu_notifier registration From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: michel@lespinasse.org, jglisse@google.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mgorman@techsingularity.net, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, peterz@infradead.org, ldufour@linux.ibm.com, laurent.dufour@fr.ibm.com, paulmck@kernel.org, luto@kernel.org, songliubraving@fb.com, peterx@redhat.com, david@redhat.com, dhowells@redhat.com, hughd@google.com, bigeasy@linutronix.de, kent.overstreet@linux.dev, punit.agrawal@bytedance.com, lstoakes@gmail.com, peterjung1337@gmail.com, rientjes@google.com, axelrasmussen@google.com, joelaf@google.com, minchan@google.com, jannh@google.com, shakeelb@google.com, tatashin@google.com, edumazet@google.com, gthelen@google.com, gurua@google.com, arjunroy@google.com, soheil@google.com, hughlynch@google.com, leewalsh@google.com, posk@google.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-team@android.com, surenb@google.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 042211C0009 X-Stat-Signature: jrfj6uopetfkbsj1fnj1dgsfrnji7pgw X-Rspam-User: X-HE-Tag: 1673297689-6229 X-HE-Meta: U2FsdGVkX1+pYtSDMD/K9m/7F82prhkloXYoVMizZ4KAjeHb3izZhNTvwcQEOpUayutuePwcnf8T9b1Ngc3VlrkyIG3O6YeaPy6A1S1ayldwbgHaJPLvXpyGTwjd+BsJsf4IibvOTx3d0dIjeFO+F7nUSDrpBiiddtlaLLf/YQmeSgsBFOu9WcUcwll60meVBBQk6uz8c5ZoaqOSf3Y4KWM2t9j/mI5SfSd5EjX9Cg90Oo/t/eSldnqrEXj5nT0KOiWqroAaZjyTHIJcyNceubw9Zrq6nX8MvvAgQ+U142QTFdCVhIDoX3egVFsH28hCqrccWiUg4Z/RCGUnc1MERvr0MMoMVBUiuLanCf5nyMcX1/4qkY1dzoQ1Uemk07OGFxg+rXez2PgH7+70mTdkBvmIqFSW9df5quioo7/84F4n5tDVGLqikBvirvgVeLtyY/9WQssh6RMLAl6TOrz25kN7lmIA/E28bjaIDtLFING+ydSUtZsa7hMmyBKj1zapHS8Hty4n80jo0mSvEBEgTfAdxg4ZoVhXmwrJvqre+CKZwa5AAGISAn73HYltMgzFVmzgbEp4/OWtFCwXxvB+I+6yI0ZWNf0gs+gFWTGKJWekZUNyKkCbzuwBpncPRuc4tDj3FDW1MiTClSNBkSGciR7cIaYeLikY7IZ1IjePrlPt7qYpYYsm8VhhkezMORsO2cdHKQzByDKCwl7ymjhEVcgIikLwTiFhxG0mMuEhnBAV8llCroHFsSElfqrzrUvmFUZnrUK1VNg31aO999fslIOTW0jcphlb30uK3+X7qz/OxSuOWc+Yny1FBcvXr9EQCq4qZf2SYKsrlL9gRmEwu05A2/i5SQFAUHtW/BfC45G12NOCIAHXgjOktyr2um9tjGlUyVYiotpAHrModjMspOlXAVC+h5UfCkpoI0qP3Nw1+V9aNc65ypsHflZNfcM5EnggexfeE+/i6d6OBXl rhxcNTJJ lSO+NdmcndjgmKj+xnv9EeMga1C9mzsi3MrylEy9G3NPgUiPGfDsqVO9oJOjYpHqOpS9YFO1g4qoB2v8wN61vF3kyDLfoX1wFZaEHrC7b2FpLNWg1Qxl6A48ruX7UVdJa/tE9RvC78xlI328dQeuVSCo5hWkPiEz4Go/k8GFabSXD+bePPIQGLlTCsnUlHP/fKottNGPnRTxiKdDceqjURzGY1CrngNm7JMv4DQFc0iGqfdnHrObrRcXpIsgBFAZ4+VZ7lrXhZreRKVZd04Q6poCHChllSO4LNq6EnqWDXdBF5G3m+yz0az6Lx0XFq6U4N81VwWEPOY39n6KoReKB/7L9YVGnoovpyjLSFiPpluRxFXskd13tKFDM43sqPUqXAtH9dCz7p+b6W53qs1ZEHvnbmnbFtNEDSVGf X-Bogosity: Ham, tests=bogofilter, spamicity=0.000007, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Page fault handlers might need to fire MMU notifications while a new notifier is being registered. Modify mm_take_all_locks to write-lock all VMAs and prevent this race with fault handlers that would hold VMA locks. VMAs are locked before i_mmap_rwsem and anon_vma to keep the same locking order as in page fault handlers. Signed-off-by: Suren Baghdasaryan --- mm/mmap.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/mmap.c b/mm/mmap.c index 30c7d1c5206e..a256deca0bc0 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -3566,6 +3566,7 @@ static void vm_lock_mapping(struct mm_struct *mm, struct address_space *mapping) * of mm/rmap.c: * - all hugetlbfs_i_mmap_rwsem_key locks (aka mapping->i_mmap_rwsem for * hugetlb mapping); + * - all vmas marked locked * - all i_mmap_rwsem locks; * - all anon_vma->rwseml * @@ -3591,6 +3592,7 @@ int mm_take_all_locks(struct mm_struct *mm) mas_for_each(&mas, vma, ULONG_MAX) { if (signal_pending(current)) goto out_unlock; + vma_write_lock(vma); if (vma->vm_file && vma->vm_file->f_mapping && is_vm_hugetlb_page(vma)) vm_lock_mapping(mm, vma->vm_file->f_mapping); @@ -3677,6 +3679,7 @@ void mm_drop_all_locks(struct mm_struct *mm) if (vma->vm_file && vma->vm_file->f_mapping) vm_unlock_mapping(vma->vm_file->f_mapping); } + vma_write_unlock_mm(mm); mutex_unlock(&mm_all_locks_mutex); }