From patchwork Fri Oct 6 19:53:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13411942 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 969EFE94104 for ; Fri, 6 Oct 2023 19:53:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2EC3280016; Fri, 6 Oct 2023 15:53:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 274E580008; Fri, 6 Oct 2023 15:53:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13C1A80016; Fri, 6 Oct 2023 15:53:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 0332C80008 for ; Fri, 6 Oct 2023 15:53:43 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id D16141602A4 for ; Fri, 6 Oct 2023 19:53:42 +0000 (UTC) X-FDA: 81316086684.13.815994E Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf28.hostedemail.com (Postfix) with ESMTP id 3916BC0011 for ; Fri, 6 Oct 2023 19:53:40 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=up0u2uMR; dmarc=none; spf=none (imf28.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696622021; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ydoOqfQy5A7MAdCeQYAn0mIGPLOFD23QKtcFfuFHsNs=; b=ZqUu7/wH2Oo/Zm5PYUHWX7HMHVpAVv87LZKZjAJqRbrnMk9O35Nk83b4417zU/M1n30qeB U41kHnW+m2DuDJ2g4L4kD+9kJu836YpXINi38xypW/oD9gQLxzVShGlDxPKiR9C4FyPhCo W8yRUjAHj9SmGYzZWq6Qmrm4jE5FUow= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=up0u2uMR; dmarc=none; spf=none (imf28.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696622021; a=rsa-sha256; cv=none; b=4nfxpt2rQnR7q9Pbil7ZOEn9ugn2UnywB82hBK326Q0BZkOQ+K0W9PoJVrRItJPGQTUiPo 9AcceoJ8YeBR35qVXnmtf02Gz7B6OBC2FQ/1X45ttV2NnPFZz5rCM65eggX9CRSWeRlOFF ieBC3t7P0fGUpOoAp8L8EbbtHKx3b2U= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=ydoOqfQy5A7MAdCeQYAn0mIGPLOFD23QKtcFfuFHsNs=; b=up0u2uMR3as+tpovtVU2cjcNoI mcKSM3xaxoUzO6/k+YPPB+JEKIKGX/r0Laevwp8oOslgXCOByoOluj0Ihg3B/O2A+QuAiNq8S6Qm7 8nBk4i1sWoRIptn6J15/aNCpIMM4sADUvaZ5vNQExzroC6RpzHZ2Rk6UCsY6JYwPm4yKKFSqRLa1w IlLTfHmxFwUF2Ps1suUQ+tLZ/2dcAUe/1DOThSJE8Pe00lsLfYzvOR4FAC4MbmrnTzzrYYkaEpWxT T09LJEH7CgnjhYX4hLj5cBufAwAsch3s7HSSDBXkAzRPQ9dAIgkfT12ezxl1Mc+N13IxpMhj/m4qX 3AW0QbPg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qoqsy-00H9GX-MT; Fri, 06 Oct 2023 19:53:20 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, Suren Baghdasaryan Subject: [PATCH v2 3/6] mm: Handle shared faults under the VMA lock Date: Fri, 6 Oct 2023 20:53:15 +0100 Message-Id: <20231006195318.4087158-4-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231006195318.4087158-1-willy@infradead.org> References: <20231006195318.4087158-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 3916BC0011 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: x5tkguh1sphqqirzixgidiiod3str51i X-HE-Tag: 1696622020-634666 X-HE-Meta: U2FsdGVkX19G8scMF4uNiKk9LNxhtvxZu7cNIREdcwNzQkB5QcFiqmIP/QYOYHKkqLXWD+WjhlTDn60DKs/HDgRJ+rywko3OiyrWmlnnE5T0vUIsJZG+RealfLZkj/YTse2fgFKHm5dWJ6uOZp874vI4/PdcNoeUFHHA5W+WmHzwmky33japu3tKShAWyiGsuUve/++ukzUMq2+cIroJjB0daqf2eDS6+nR9MkVs3t0x8WFINLEbrSXJQOTuGG1jgWa4O4huvW+In2eR5NOaXFx7PhX7JIgzwDTNXQmZ+nk4oR9bL1tr7JpdZ0EvVZKBbReqknXphQjV/mAsENcPDL8WRXA1BZ8wvruSkwK3qqkwuCjqBKy4tQPSHsPzoR9FK2B5CzN5ODnEac7tdNX9sO7IpvFcFVOY6ak9zwP3KVY+FIP6nzq8XYo4bN8Zip3D2pTFp34z3IfFmVRzanPaY5oCEvpKqN6raUcDTmTA43ctcOvhaOEjvE7boCRNpfHoBQSWYYLf2smgk5jC1EVYk1+RES/ZysnCXUeSpIx6fhyEOcgs/RIwUS2KF5OIgL/UEyv0sWzLhjX0EacJP0QnlfCaPc6dQqcOX+hwWnKU1FD2SBcDWjIBBYBy7CulfPTJsmXllTSV/4zL4rW9ehYcuz67R7aRIL1/kVGqAK01+MBoa4CrtIQs0T7LVO8TygjIboO2dtQbszmGydw4jwpHkAw5d14Gs9buXs3YpWQixVo5jCcR2iZgMg5MZAnILUoTlZOZF1QyNRwf85ThcjKdPoZXOY+hG8xG/Yd32UkejGJrz958f0lpMaINvGSBHi+jGYNPuoxb3Y1Jwb/I6z2LX9g466WrLVCN6KZLMDykdkU4sEjhOuo1vgW13VonrxBFm6cQiHU5h7UnI0WhxUDopjJxq0djX5pwUayjl+NJHtqiP08i/XizmyAqyYfKOR5G0ZYoAWiKNgrOgo+/TGm IyAJgW8f jBhItDaZxokVVfMl5wO5LsTjKntERPIy8/o1XUx8oK7pqHEHV5fE2xNp2LAiKqFTTBx2E8uoX0gYbK8MGjKNsXpe9LwUSiTWDdbvRuv3UhyItk3md4W0sc6WNT0SrnThsG9hpjeUhbLVvPY2+el3sGb/4tkyJgOcP6VC9F6ttcB/Q9kei5KNehm1g6w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are many implementations of ->fault and some of them depend on mmap_lock being held. All vm_ops that implement ->map_pages() end up calling filemap_fault(), which I have audited to be sure it does not rely on mmap_lock. So (for now) key off ->map_pages existing as a flag to indicate that it's safe to call ->fault while only holding the vma lock. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Suren Baghdasaryan --- mm/memory.c | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index cff78c496728..a9b0c135209a 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3042,6 +3042,21 @@ static inline void wp_page_reuse(struct vm_fault *vmf) count_vm_event(PGREUSE); } +/* + * We could add a bitflag somewhere, but for now, we know that all + * vm_ops that have a ->map_pages have been audited and don't need + * the mmap_lock to be held. + */ +static inline vm_fault_t vmf_can_call_fault(const struct vm_fault *vmf) +{ + struct vm_area_struct *vma = vmf->vma; + + if (vma->vm_ops->map_pages || !(vmf->flags & FAULT_FLAG_VMA_LOCK)) + return 0; + vma_end_read(vma); + return VM_FAULT_RETRY; +} + static vm_fault_t vmf_anon_prepare(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; @@ -4669,10 +4684,9 @@ static vm_fault_t do_shared_fault(struct vm_fault *vmf) vm_fault_t ret, tmp; struct folio *folio; - if (vmf->flags & FAULT_FLAG_VMA_LOCK) { - vma_end_read(vma); - return VM_FAULT_RETRY; - } + ret = vmf_can_call_fault(vmf); + if (ret) + return ret; ret = __do_fault(vmf); if (unlikely(ret & (VM_FAULT_ERROR | VM_FAULT_NOPAGE | VM_FAULT_RETRY)))