From patchwork Mon May 29 06:16:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 13258151 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9936C77B7A for ; Mon, 29 May 2023 06:16:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 57F6F900003; Mon, 29 May 2023 02:16:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 50848900002; Mon, 29 May 2023 02:16:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 38207900003; Mon, 29 May 2023 02:16:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 27643900002 for ; Mon, 29 May 2023 02:16:24 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E1DBB1201FF for ; Mon, 29 May 2023 06:16:23 +0000 (UTC) X-FDA: 80842283046.23.183FA4A Received: from mail-yb1-f178.google.com (mail-yb1-f178.google.com [209.85.219.178]) by imf30.hostedemail.com (Postfix) with ESMTP id 12A3880010 for ; Mon, 29 May 2023 06:16:21 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=TyhMHkB8; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf30.hostedemail.com: domain of hughd@google.com designates 209.85.219.178 as permitted sender) smtp.mailfrom=hughd@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685340982; a=rsa-sha256; cv=none; b=UIZFScnd5LKRgT72cW7CavusW4FwIufP9IjnzMuwLYsZKElA8SxZ/mKjKnZrFfBNSckxxB xSyvLk/IEvUpNJo+lA/bWk/779M/RbkUjhNWYZlbXn9P9cl1lgaq+k2liuNtjE0Cns5Ccn vZ1lz3fMAiuoANrv/cxAoInxkiVtimQ= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=TyhMHkB8; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf30.hostedemail.com: domain of hughd@google.com designates 209.85.219.178 as permitted sender) smtp.mailfrom=hughd@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685340982; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xPq2eB/hW7ShnFytnKcTY7aukisfg/jLLLPKQ4lP3rM=; b=IckPXs4wYseHHwlq822VbVI41WnHilUnB5H6uPFmB3WKycKjB+N2qUopT6AgC1c6BDsxRQ XMa57OOdgD3/t9b89SQNd8SA2SUUtqDFntbZ8wfcT4xebTbaiqv3/iVglvqKjBR0nt2Je2 Sa8QcEyVHRLgnNadulnZgm8vcaTgVs4= Received: by mail-yb1-f178.google.com with SMTP id 3f1490d57ef6-ba8afcc82c0so5784317276.2 for ; Sun, 28 May 2023 23:16:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1685340981; x=1687932981; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=xPq2eB/hW7ShnFytnKcTY7aukisfg/jLLLPKQ4lP3rM=; b=TyhMHkB8EAAdy+hUHa48C2vLWt0oIXv3HPWA7oGBtoKurezV9dmnaHp/Ug08+YXBMJ q7/doDqEaB13J8DWMDOsmxCRPRcN6V2SXOTsYe+dqJjrxA7f0LVq1fNiXr9IMn2dnMMI ESckYbJKgJgrbJ8T+uYkbBEglbSci8JLRkQ5xTjOVaaAsg2STz0pP4v2Ide+0MWDvyHe y1hUI8uDpAzlLTOdYFAa9G5upyPvxazDxgjqrZ/FkHni8ZrS3oyiM+Gr8t/oORl7+XNi bEMYjRBzoNKzLiTXOPBKXj7pF37UgxQaCI1yMdVBb74YQf5DcZFdirdn0N62WO3feJu6 qWEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685340981; x=1687932981; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xPq2eB/hW7ShnFytnKcTY7aukisfg/jLLLPKQ4lP3rM=; b=CrYk/nKBtSSOCSvbicVhay2q/zh9wuFdbjO8azBQ4aERa8KsK1zaVgeSXY+Uj1er4+ u9+Ulnc3zTmOMzVo0WAEbA2+LI/gDrowxT2Jak5FKXHK72xcOUuWdvaPBiQdrp1hnWOq XLvnj5gpTY57KmTSctbvndp034xd6A1vkQPO395m4yAG9WZxUYz2iVSKOWrF7vPbvo4s baxIV1BBNJv8N92CVcUKCEfLpq/SuJT4e+9OvMIG3MymaUNzA1GCimBzthrMMpNJ3+Hj zBWsC5ubedL2eJFxHhTlFSyU0vyU3/uSxN6yt7vDvByEfeBaYTyaEJnJzuxJ+tzR5GS1 L0KA== X-Gm-Message-State: AC+VfDyjfXQuA7yW+cmSsK4pDISnIRWX+h7K4WT+zeuxNQLmY7c5tmdU oSqtUcf3Kp1TltgaEE9hoTdZ5w== X-Google-Smtp-Source: ACHHUZ5ZacpUNHnx35XFkkfr+0IABaEA7Bz0HX1XL9bgqfNu7gBbTpGfMWgF9E5JOg8lgzwGl1APAA== X-Received: by 2002:a0d:f003:0:b0:565:a0c8:7e66 with SMTP id z3-20020a0df003000000b00565a0c87e66mr11630388ywe.0.1685340981021; Sun, 28 May 2023 23:16:21 -0700 (PDT) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id n5-20020a819c45000000b00545a081847fsm3407533ywa.15.2023.05.28.23.16.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 28 May 2023 23:16:20 -0700 (PDT) Date: Sun, 28 May 2023 23:16:16 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: Andrew Morton cc: Mike Kravetz , Mike Rapoport , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Suren Baghdasaryan , Qi Zheng , Yang Shi , Mel Gorman , Peter Xu , Peter Zijlstra , Will Deacon , Yu Zhao , Alistair Popple , Ralph Campbell , Ira Weiny , Steven Price , SeongJae Park , Naoya Horiguchi , Christophe Leroy , Zack Rusin , Jason Gunthorpe , Axel Rasmussen , Anshuman Khandual , Pasha Tatashin , Miaohe Lin , Minchan Kim , Christoph Hellwig , Song Liu , Thomas Hellstrom , Russell King , "David S. Miller" , Michael Ellerman , "Aneesh Kumar K.V" , Heiko Carstens , Christian Borntraeger , Claudio Imbrenda , Alexander Gordeev , Jann Horn , linux-arm-kernel@lists.infradead.org, sparclinux@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 02/12] mm/pgtable: add PAE safety to __pte_offset_map() In-Reply-To: <35e983f5-7ed3-b310-d949-9ae8b130cdab@google.com> Message-ID: <923480d5-35ab-7cac-79d0-343d16e29318@google.com> References: <35e983f5-7ed3-b310-d949-9ae8b130cdab@google.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 12A3880010 X-Stat-Signature: i51xoy87tmqbpy6bzjk1cerihhpjg5y8 X-HE-Tag: 1685340981-2456 X-HE-Meta: U2FsdGVkX18vQ+cRIc7o8ivf6155b2NsLLzGKjHl3k/28z1bvz84lNuC9EoJ+CZ33vGM6PJEIDtjx5jSmndispyha8Irw8kSo2ISo0ox+JguOVxH9wQNHdigVf458QEUfTXjpztqgvsYcMMLZgFIeye1kLuX6J0tHPiEZMMEvNw5mOtmM3Rr1FSnmUZbdUJoScYCskgJJIzJUHoZYgHOJhlnyF73QjIHlt5b8NyMYvBxPemZhjidhKgogwBScowUMPsjV3xkEif0mdNOcwmgZP6Nq3YQMXdqKKqcgL+X0Pm23txBN3o1lzPH8uu3mf/GWNp1I/vE9eUPAVmdeFgAwkJb6CntGTV4W0mo+fsKNcoauu9i8J8RY2THByTbDOW/QXoSlqqWEIELUGxyo0WtosUWE8RaN24WK18lcToT2te+S+efLXRgVLo/yQ7LHiqOE3/w0ig3WcEolRc15zs1gEs4zaqkfdWWj3L2uv1O/PN9/r2OJB5BlFsnv9IxgsQ6CDM/t2sm8lCan1oYfNm8mHwHqJyKOvJbFIII1M4Sqz78ie2oo8OOVEBJHzB0W4na2UUcZOYC5ibhfJA9LWLOa5wZp//+1UyaUWPPxbkgrPr999745bPmemt3mHG9eM7nr0yeBJx4RyQyUjHfuI9xDpOz8J1h5oigElqcMo6wh5CXzLHziKRpb+0UHtOwRVAo9ZsIXol/i0Ny79koaqInpyVoiXzBrQA6mkcBi8EtMOsNxwzWL8Pkc2r4wjEyLZQ6D4pSWwLRa5vN0zPbwU8iYuX4a4YuyJOntGkggI/Q19uPPWfQd0IZoQAkNv8GiOtVbpBfJ0n6q5uPcZPXOcKXufSVo7bmN7ftmaJKqkMiHEY1zROdunH9+1VXQGRcT8q7RdRRt9bOxfyi7ySG7UhzZF5pjeJd9lxzsGLJ9GT4noY9+CEzGKJN4hnLK0LbcqieWNwZHXq9p1nfL25/I7O HWtJmCrr 0ABgkwnLfbw3gLj55LxTOydSdDikOFjldcZ4oSUiqkQwK4uom0fHsWtfuXBaYl2WAZojIidaxhcjx7qnQFvgi1o3JEoUMxSXzkg/OEyMCkrcgjLqJJXzv55jySfZ6ei9ZWjvbhDhApyBsw7ok9hbK9k0eBBu9Z6Gc6FQ2GxtKOHRsbQ+sY5xaFU3YXX0gFiVp77i+DSRUUPcG7VpP9BccfhqgwTWINaZGsfQRq0Eok8PkizcrAvuhys0bieSVUfqvtpslpAztWgIB7/aMdTbhNEENE1Tg9yb0roFpD9QvUeD6BIarTuNByZMI36g+3ZmHiVtoQo45m0tqr98uQs3c9Nxr8MSdnc571TQylSwSu1pffQHes1IfElYUiqMTy6Aqd577zv82evbm4jZP/8WCobvGv1nlHx51fdC28AnLlZ55X09xc0Im90anVA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There is a faint risk that __pte_offset_map(), on a 32-bit architecture with a 64-bit pmd_t e.g. x86-32 with CONFIG_X86_PAE=y, would succeed on a pmdval assembled from a pmd_low and a pmd_high which never belonged together: their combination not pointing to a page table at all, perhaps not even a valid pfn. pmdp_get_lockless() is not enough to prevent that. Guard against that (on such configs) by local_irq_save() blocking TLB flush between present updates, as linux/pgtable.h suggests. It's only needed around the pmdp_get_lockless() in __pte_offset_map(): a race when __pte_offset_map_lock() repeats the pmdp_get_lockless() after getting the lock, would just send it back to __pte_offset_map() again. CONFIG_GUP_GET_PXX_LOW_HIGH is enabled when required by mips, sh and x86. It is not enabled by arm-32 CONFIG_ARM_LPAE: my understanding is that Will Deacon's 2020 enhancements to READ_ONCE() are sufficient for arm. It is not enabled by arc, but its pmd_t is 32-bit even when pte_t 64-bit. Limit the IRQ disablement to CONFIG_HIGHPTE? Perhaps, but would need a little more work, to retry if pmd_low good for page table, but pmd_high non-zero from THP (and that might be making x86-specific assumptions). Signed-off-by: Hugh Dickins --- mm/pgtable-generic.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index 674671835631..d28b63386cef 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -232,12 +232,32 @@ pmd_t pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long address, #endif #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#if defined(CONFIG_GUP_GET_PXX_LOW_HIGH) && \ + (defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_RCU)) +/* + * See the comment above ptep_get_lockless() in include/linux/pgtable.h: + * the barriers in pmdp_get_lockless() cannot guarantee that the value in + * pmd_high actually belongs with the value in pmd_low; but holding interrupts + * off blocks the TLB flush between present updates, which guarantees that a + * successful __pte_offset_map() points to a page from matched halves. + */ +#define config_might_irq_save(flags) local_irq_save(flags) +#define config_might_irq_restore(flags) local_irq_restore(flags) +#else +#define config_might_irq_save(flags) +#define config_might_irq_restore(flags) +#endif + pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp) { + unsigned long __maybe_unused flags; pmd_t pmdval; rcu_read_lock(); + config_might_irq_save(flags); pmdval = pmdp_get_lockless(pmd); + config_might_irq_restore(flags); + if (pmdvalp) *pmdvalp = pmdval; if (unlikely(pmd_none(pmdval) || is_pmd_migration_entry(pmdval)))