From patchwork Tue Jan 2 23:32:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13509579 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AA77C46CD2 for ; Tue, 2 Jan 2024 23:33:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A9A058D002D; Tue, 2 Jan 2024 18:33:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A48308D0006; Tue, 2 Jan 2024 18:33:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8E8B88D002D; Tue, 2 Jan 2024 18:33:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 7C83C8D0006 for ; Tue, 2 Jan 2024 18:33:02 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 090FB160184 for ; Tue, 2 Jan 2024 23:33:02 +0000 (UTC) X-FDA: 81635973804.28.1EAFB9A Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf01.hostedemail.com (Postfix) with ESMTP id 5903A4000C for ; Tue, 2 Jan 2024 23:33:00 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=ZOIheBIQ; spf=pass (imf01.hostedemail.com: domain of 3K52UZQYKCOoegdQZNSaaSXQ.OaYXUZgj-YYWhMOW.adS@flex--surenb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3K52UZQYKCOoegdQZNSaaSXQ.OaYXUZgj-YYWhMOW.adS@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704238380; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=OrZsxOCrBJcDQG2F63wPu/oWesILjPUyL8XFF214j9w=; b=7Z/221QC22VKrD0M5YmukYykoldFWovDNEHVuyYd/Kmo/kpUvH2Oq5B78/zJbN/1ZLjgmj tsYRiPS7F/bY8EwafOP0ADrZ0Km99LV88Es0sIK5mFBnPTqrcVUibZcjpwKEg0FdR/HQLj R01PlGXJLrWfqRtL89i0es6pwp0F50A= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704238380; a=rsa-sha256; cv=none; b=HggllbAQZT/zNU8NUGpE/RQI0vrAQL2WZiAZohUJb7eb5CEtO6aTsIaHs4KLQWzLDT6mK3 98JBQNPnbKH2Ei3U8gkN80r/y0jXz3O481ef5Uc18vrKMi9j5HtZccPF3yOvCNTM4bQyM/ IkXjUp7iveeXlx5oxX2moVA4UpzdJ6I= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=ZOIheBIQ; spf=pass (imf01.hostedemail.com: domain of 3K52UZQYKCOoegdQZNSaaSXQ.OaYXUZgj-YYWhMOW.adS@flex--surenb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3K52UZQYKCOoegdQZNSaaSXQ.OaYXUZgj-YYWhMOW.adS@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dbdb759e73bso5632940276.2 for ; Tue, 02 Jan 2024 15:33:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1704238379; x=1704843179; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=OrZsxOCrBJcDQG2F63wPu/oWesILjPUyL8XFF214j9w=; b=ZOIheBIQpTF4YUmttdjTutM2586hPdpjU1TT1qHI/A67TlpvIn5AHJq5KmRstoJ0+T ZM53ZVNKeOFCk+bhh/WR8Za6TvYK0xQDlG6++WFFOXuyyQS8+s5beZA4D+tsmF0oFNgN LZEOqOyJLDJ8tdDcA5oUV6aWcmQnfXLLahyx+4T3eb2zT0lD8nwCtW8dFtEeI9SKJSNQ ikvvx/hRTr9481HuW16cq4P37jTQM52InMmXtey0YZF/Sdmp3hNG2YaUGlIb0Sn/hsoU d+/wiDYjt9rc6JtOcQ//HyzJvzq+oR6bu5PWmz8x+3pqL2lpYo8pnNpnzX3ecLETiJEX jK3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704238379; x=1704843179; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=OrZsxOCrBJcDQG2F63wPu/oWesILjPUyL8XFF214j9w=; b=MWFKpg6DF8YdAvH3eQnI/JAm3wfcOKgln8DuKP4cTvaSIrUVeIJjz2x+fS2X4FndaX +gvxeGihiwy2qogA1drE2DbyJpxFxtOyEXfMzxV0AaYKGXAr7SERNGA3jDFdmeLb3zFn OJGS4urltxesgQsX5Q3d03ji+jhYZ4domls2hadZPB4gLWnbZ+GtNG9QQJ6TqMNv+DJG Jd10fvcb08knWGxxzHWLOFr3pcc17QIY84zV+cV0ig3ZPZTw4QWcbQ01Byz/tA1LlyRs CYQrXTvIMd7T8cddcdllMVBsgoIY7I0IcJoEYiIQyhfEh+mc7x0IqWaNIRgPtJlxA0yq k9IA== X-Gm-Message-State: AOJu0Yzga3slrRbnfabJ7na7LrL/7Mnnx62aWmAOiDBupDCyc0M6f7AZ zgwpvSfvWChbuTAsaHsN/9Qt8Iq3vedwV2dfLQ== X-Google-Smtp-Source: AGHT+IEuwPzcUwn85Vv4VXikLgRwT5vcLx1QXyFQo+oOhEISU5egcPovU+KpM8prSGIuu4/6FVFM3vh5OGc= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:7c15:8de:bd73:b2f6]) (user=surenb job=sendgmr) by 2002:a25:949:0:b0:dbc:1b8a:ea84 with SMTP id u9-20020a250949000000b00dbc1b8aea84mr417162ybm.7.1704238379350; Tue, 02 Jan 2024 15:32:59 -0800 (PST) Date: Tue, 2 Jan 2024 15:32:56 -0800 Mime-Version: 1.0 X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20240102233256.1077959-1-surenb@google.com> Subject: [PATCH v2 1/1] userfaultfd: fix move_pages_pte() splitting folio under RCU read lock From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: viro@zeniv.linux.org.uk, brauner@kernel.org, shuah@kernel.org, aarcange@redhat.com, lokeshgidra@google.com, peterx@redhat.com, david@redhat.com, ryan.roberts@arm.com, hughd@google.com, mhocko@suse.com, axelrasmussen@google.com, rppt@kernel.org, willy@infradead.org, Liam.Howlett@oracle.com, jannh@google.com, zhangpeng362@huawei.com, bgeffon@google.com, kaleshsingh@google.com, ngeoffray@google.com, jdduke@google.com, surenb@google.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-team@android.com X-Stat-Signature: dprm4h464ie86xcyxfzjcaxxxoecoofz X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 5903A4000C X-Rspam-User: X-HE-Tag: 1704238380-314176 X-HE-Meta: U2FsdGVkX1+h98heXOGZaq5G4EMTK40Ek5j90S/2LSfLfNXSqbteb2sW/M4rwbySryIAQ3mX3xYWnkuRm/dpE3kQKWKMOIaf5LZwng3p9FoxrsQNl64e3pabKcgjvlyt7flM63FbZ48Mq9JlIOlvG+QUQikGhsy6Q0skoikExG+j5wJvLo8RtnyNChtaiQT76R+9hSKEyFcDb+6IvC7eSvIN+4lw2/b5saMtd/W9UCxhf1X87GjPDAm4CE/ako6VDccYgu1S0yw0/eBo0pmLzsKNH8ln26M036FY1hErdBVwwNZM1nlUueQSNkfjbvWrB7NzCUE9GaKcXce1HERhS1nVvUWcMasLJGAmW2HwjmZb8Q0G3QzpW/t2ndyXg9VJ/uuXFQn49bj4GXtkUZQ18lHMqOnebRLC0IMtietKyBJiDQhoWN4Zn/xC/J1WXi4O6cUg/NbaVoX7AlTBDY5ZagGlORrCEMtcYUpW3zxbW92nPQBcQv5Td7jEH4vWj3UGpJs7XMkjc2dXN+R0qZDyTiHbR61cgO7s0dCQ8D6+otxgP4O0Do0y1yBfc24HmO/WrCdm/camlTRp+14TC1+uuWWwzo/LGVaHYCjO3dMnDJzZLJXtUiCHsd5QxuUt+rBPPR6vQ2XUB0YOTqQiSWPyfGuEsxdFJ32g1DZpcEEOPnEZKKlp2UJ17XaWY+Du4aGB911C60Kz35bHL4uCB4i5Eb3dAtIH9a4yHaT9ijNdvguxQ+aUj9Xd3BBcyhQIWLkdDUyr1HNmD4fM99UerrK6aWCIqOB1UYTJf1PTfaGcnkyRWMi0STjwDFtzDF9eJJUVJfF2BdvAtYMOTHIxOTR0EpTnjVhychSttSpvEScCLgbPzyvAvSC+9OhxNGPF8ESLGfsbgVmLkdRRRBXzHw0niLNI1RXU4nMGfdChMnMZpIxX6a2g+fjw1604dbfE7yBVwE6s9p6dulwR8CvkNR7 XJIYxnGo r+ipDMuN+3/kGayRJkHSkQJ2M0HZc06qUQHcm7Nv3jeuKJoQhRsO3FyJfVmeHaJekEnxyxpNTmXmkykW0aEhhMRQykDqH26TzTqdutF0T9hBzMz8+aM4hxHUaK8wpMJL3lvDw6gHgR3JFXIwHBfu2nLUJqNsYwI4zlrO1lfomQpl7OnAYbYpJOw0TJ7nS9NV0syaKD0rDReUYdV7Hdj0deCI7lbYSZLvN0mpq6coJ1n/p2GuQdZTNVb2ECqVc3j5g4fDF3YYFi6Sw5qT4Enkr3azWRhUqBrVWf5QBWmf9Q/272XHsweWpL2+iwG1xBpZ+V+On/rwuiq6Oy/xiR4NNkp51uOxJmGmYH+olwREW2/XebHEa/6Maiv43+CNhTOdrWJOUryyr8/i3ipGdRgOSyuYFIK204SPhqABro7/ieUFtifmO4Zdpkgd5+oj4XZvOT8lhkWXUDWH6h0q50yHDRHmXL9OehiF4ClGFgB8lvns184GmtiiqONMvt4zPWYQA74fCeXkJ1W1kXhfsgjL5QUzUTBMLwT74fcibtpYgSaN8K0S1qI16Jh7srrjziikt8NdW8OsXebcs4rIZGU9cM84MZA/O/Aw/lC5yD/ljmkrTkQGD+rHlv71Ml9L/2/ns9UjIpJjK9/9/WpWqfUnbtTRonIc7KdwrOMVO5fTqZpL8aQq0kUvLo/pglpRiyDiUcCRLCL/Oj8Tu17nR76frn1RAJA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: While testing the split PMD path with lockdep enabled I've got an "Invalid wait context" error caused by split_huge_page_to_list() trying to lock anon_vma->rwsem while inside RCU read section. The issues is due to move_pages_pte() calling split_folio() under RCU read lock. Fix this by unmapping the PTEs and exiting RCU read section before splitting the folio and then retrying. The same retry pattern is used when locking the folio or anon_vma in this function. After splitting the large folio we unlock and release it because after the split the old folio might not be the one that contains the src_addr. Fixes: 94b01c885131 ("userfaultfd: UFFDIO_MOVE uABI") Signed-off-by: Suren Baghdasaryan Reviewed-by: Peter Xu --- Changes from v1 [1]: 1. Reset src_folio and src_folio_pte after folio is split, per Peter Xu [1] https://lore.kernel.org/all/20231230025607.2476912-1-surenb@google.com/ mm/userfaultfd.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 5e718014e671..216ab4c8621f 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -1078,9 +1078,18 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, /* at this point we have src_folio locked */ if (folio_test_large(src_folio)) { + /* split_folio() can block */ + pte_unmap(&orig_src_pte); + pte_unmap(&orig_dst_pte); + src_pte = dst_pte = NULL; err = split_folio(src_folio); if (err) goto out; + /* have to reacquire the folio after it got split */ + folio_unlock(src_folio); + folio_put(src_folio); + src_folio = NULL; + goto retry; } if (!src_anon_vma) {