From patchwork Mon May 22 05:06:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 13249764 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3BEDC77B73 for ; Mon, 22 May 2023 05:06:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 51C3090000B; Mon, 22 May 2023 01:06:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4C963900002; Mon, 22 May 2023 01:06:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3B86490000B; Mon, 22 May 2023 01:06:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 2C2B7900002 for ; Mon, 22 May 2023 01:06:39 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id E59328044A for ; Mon, 22 May 2023 05:06:38 +0000 (UTC) X-FDA: 80816705676.12.61280AE Received: from mail-yw1-f175.google.com (mail-yw1-f175.google.com [209.85.128.175]) by imf05.hostedemail.com (Postfix) with ESMTP id 149F2100011 for ; Mon, 22 May 2023 05:06:36 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="xUO60/u0"; spf=pass (imf05.hostedemail.com: domain of hughd@google.com designates 209.85.128.175 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684731997; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=itm4lMYCv8hAr7oWbErSKKsiGUM8mghvSpbiblkFfbk=; b=W8qvnCrUXigBnKT3u7lRmtTQd/jtUU2DsHk4Ku7xDDwFlNOK40ykxE+WleNd3gw0yUDuc3 3Gz3gBDEyb5MgxkWth+p7gbG+sQm0W1jbCIxYFTEjJzgLpj8TRKLfLKYu+L4js6mbD5R+I QC8dqL5GLnHPY92vAMe1NvsKOVXbGaM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684731997; a=rsa-sha256; cv=none; b=fgOwQSFL//6WsjSih/cYJ6G6ccjux5T+j7KGCSA8K0JcGaHz8xt+RXdt/3LzfmodAGaynx 6eB44lQWcQgTUEQE9y45WmmZq9W0KxLYlMC/0dMS+IErCzRes3KSbI5AGV7Uc83mBNFAbd eEwhl+Oxt7seJaFyKy2sBYHyCqgXalA= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="xUO60/u0"; spf=pass (imf05.hostedemail.com: domain of hughd@google.com designates 209.85.128.175 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yw1-f175.google.com with SMTP id 00721157ae682-565014fc2faso7230947b3.1 for ; Sun, 21 May 2023 22:06:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684731996; x=1687323996; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=itm4lMYCv8hAr7oWbErSKKsiGUM8mghvSpbiblkFfbk=; b=xUO60/u0hCL36TvKXHyURnnxqO5vn7ZybjY7ONPqMVFmXcRaGi5WJ1fjIK85dRtVJa +whV5tpXJtxlVEd0xUGkv7xuykXYLwzIAtD/A83Pd0ByM1utIbe7HqhKdkTngJ8uTZye nXdCtHMDNgLl+HSpG2o6gWxk89SbFvj51bDmDs5EUuxNtQO799f4KEwsfMDYpb1aA0JV G1qpsPEAFFQ60/jP3AFkOagxrcttoK3mZdLQdFsbhRCY1FyAsYOWEizdcI6h50IGdllG JcD8IVUHEVvPfu5cztMQwnxIE3n4xO4E81T6/znW0rbNwJzdKCpC1q5F2RM6c3EnqDt5 BYMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684731996; x=1687323996; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=itm4lMYCv8hAr7oWbErSKKsiGUM8mghvSpbiblkFfbk=; b=i2j3ciCFbcWXCB9g0ZlEredtfiva9qhzocP/Ri+jIeIK5C9/zuB4I1Y951oOc0mAhB 6ddkJrHOe3yYNmV1wmZXGcZjUYDVqaqPgFdjiABJvQNG3fiHDysjtElYYFxh1kvsX6op jEXh1QFewBZe2+ehK+n5SF6vJqupUlxv8VESWB4tQ1uFrFdQxfSez4woW9hksnQ1UBou LUzdMnlZYHoicqescB9akn1oqYeKvRsrJfRI6DN81Zb16tjPWAC1Bujwz6yAKVuXmMVY VEv11NNPf47krf57EhBUjGY7uGgEcyeTxoG9QjwkOGMUQQYmsY8zKiAzkJOTQDFa4ig8 LYGg== X-Gm-Message-State: AC+VfDyVvUJapA6jFcMLLOMOpTPgnuK5IZWJYCvQNbBFYOlfK+RbCUl0 BwsW7iVkw5ADThH1Nh6agRIL/A== X-Google-Smtp-Source: ACHHUZ78QIP2SjKpIXnCqaoasaQ8h+cR3FlbUtT2N1GVyk/3dI0nRSCd0E3wxVyacPDSOVPhhCpzYw== X-Received: by 2002:a0d:ed43:0:b0:561:9d6e:6f45 with SMTP id w64-20020a0ded43000000b005619d6e6f45mr10989561ywe.26.1684731996077; Sun, 21 May 2023 22:06:36 -0700 (PDT) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id v184-20020a8148c1000000b0054f50f71834sm1805106ywa.124.2023.05.21.22.06.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 May 2023 22:06:35 -0700 (PDT) Date: Sun, 21 May 2023 22:06:32 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: Andrew Morton cc: Mike Kravetz , Mike Rapoport , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Suren Baghdasaryan , Qi Zheng , Yang Shi , Mel Gorman , Peter Xu , Peter Zijlstra , Will Deacon , Yu Zhao , Alistair Popple , Ralph Campbell , Ira Weiny , Steven Price , SeongJae Park , Naoya Horiguchi , Christophe Leroy , Zack Rusin , Jason Gunthorpe , Axel Rasmussen , Anshuman Khandual , Pasha Tatashin , Miaohe Lin , Minchan Kim , Christoph Hellwig , Song Liu , Thomas Hellstrom , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 14/31] fs/userfaultfd: retry if pte_offset_map() fails In-Reply-To: <68a97fbe-5c1e-7ac6-72c-7b9c6290b370@google.com> Message-ID: <424274a4-7c13-e14-b380-428fc69a45c5@google.com> References: <68a97fbe-5c1e-7ac6-72c-7b9c6290b370@google.com> MIME-Version: 1.0 X-Stat-Signature: heumjrqjgjwwf9h67zc6u18se5q1f8of X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 149F2100011 X-Rspam-User: X-HE-Tag: 1684731996-307835 X-HE-Meta: U2FsdGVkX1/GaVYkSiJi1gFyJAeAJnmx4HE66NOTF2Q9fw1gm0Ly04QSQ5rN2zJvb8D3VTl0IxS72vQVIYApYdy5zm8ujEuFEvYRnxlvA4q0E9vqmBNH/++yuHI5Z6CS1hwV5SNdRJZbFarmfF7QHgK4ybDSdi3M9l1AmHQtJ9rwx9/mCS93SENny/A8/u4vUnmBCOxn8/X9AQLi7yM/DAe6ZqNY7AVteIAwCJ7nBcW/ZjC+sJ63jDGkT1ChmmQKuNZB9BgGsSwoKopH03kmTbLoDFSRjansD+ymPk9Bx74DIuLNxrebvQUWZN3PY54SHG7FKnTGBz33FqRIC96ozPAxgGWjqkLhuO0zGns+4CtZ0VDXLObdiuwkvfNAODYQxeYkexBMaMXQ/ExtQgyWHM7+v2uGAYbisM17MjNmAYVi23RpAkMNVTRMzvgkVjonX6JNezF+e6rj7VPNptUgABEbuqCxpbm9bCZmuIISkXnX2ujFrtIpcq1QhAVwunNYkr/t1qHP2WEyCdQfighleLaCSxML+DcZvoDFKxU3Zq1EwwWxGNWjCMdT0aoelgPMxzVps+vDOTItuHL8C4z8WJ8kPpVBVtYshFkhfEEq1hu++L3XYijVw2GlS99i+GoTmb3HWw25CpVpGXmVR4/OYe4L0gdpU+ihl0sdyCnOlOAEiCVND2h6ICmcaz2r/lV+TtDOti/RIRBKphulBb3Lmhay2LsPFEbxcH+9uS2mSgCYMtSC1WGBokrKBIV8z1iN4fEcEMVDLs8kvH9N/0k5Y27aF8cYvY3OdCHKeb0abzk85R+syYk3Vjpo/DA5jXhCX/qmGm3B96EcRspzcJ1u/ddzGuvFjpvcI9IucSzraOp7HkESFxv7KZNQtL3JXNraALmbntnVOueenoQt9LboZjeZ1Clf+J+CIUWekgpzvkz+dn4GoMcKdFI771v443WqO1m5ku/ozGPrJ63BHsx goibLda6 QFzLmFkAD+nGxDkBcOYuEEiNEUGGSXi/SKxZGONdZ/fD92JE5TrhmXYS1Ufn6g0XVkEi7dQ0Eag/NIDs91IK+rzycEndI+FSFmr9/Qdqd6wJQ66tv1Ws23J5R1O/DDk9MOksVnKGduDkP/J5PrpcUxLOW9rpA6T5b51dtKhNykmHn7kwQE1Fwxp+cA3rRoYF3N0bzmm0OzCtFRIs3X0CcZLID8znH9MEJ5Rmpbr61p9jstg6DCiw7Qj7uKUDlL8HoIc13AEz98F7RUSNAnrk8fBSLC87mIGbF4YMcyMnKNi04nQkftT0IxjGs42FhlCgRDpoeZ36IyMWYKXZbmrTzzzqZ7GdzobdN/7KfYSEbNdFyCiwBXtQ+J8YtTmQqrv9baVI7bqpPCdVgmIyfBsYAn9/hLj8jst49gWYjnO2RGENc+ojMyFbAzXupqw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Instead of worrying whether the pmd is stable, userfaultfd_must_wait() call pte_offset_map() as before, but go back to try again if that fails. Risk of endless loop? It already broke out if pmd_none(), !pmd_present() or pmd_trans_huge(), and pte_offset_map() would have cleared pmd_bad(): which leaves pmd_devmap(). Presumably pmd_devmap() is inappropriate in a vma subject to userfaultfd (it would have been mistreated before), but add a check just to avoid all possibility of endless loop there. Signed-off-by: Hugh Dickins Acked-by: Peter Xu --- fs/userfaultfd.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index f7a0817b1ec0..ca83423f8d54 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -349,12 +349,13 @@ static inline bool userfaultfd_must_wait(struct userfaultfd_ctx *ctx, if (!pud_present(*pud)) goto out; pmd = pmd_offset(pud, address); +again: _pmd = pmdp_get_lockless(pmd); if (pmd_none(_pmd)) goto out; ret = false; - if (!pmd_present(_pmd)) + if (!pmd_present(_pmd) || pmd_devmap(_pmd)) goto out; if (pmd_trans_huge(_pmd)) { @@ -363,11 +364,11 @@ static inline bool userfaultfd_must_wait(struct userfaultfd_ctx *ctx, goto out; } - /* - * the pmd is stable (as in !pmd_trans_unstable) so we can re-read it - * and use the standard pte_offset_map() instead of parsing _pmd. - */ pte = pte_offset_map(pmd, address); + if (!pte) { + ret = true; + goto again; + } /* * Lockless access: we're in a wait_event so it's ok if it * changes under us. PTE markers should be handled the same as none