From patchwork Fri Jan 28 13:09:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michel Lespinasse X-Patchwork-Id: 12728511 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F406C433EF for ; Fri, 28 Jan 2022 13:10:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7CE656B0089; Fri, 28 Jan 2022 08:10:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 77A7F6B0092; Fri, 28 Jan 2022 08:10:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 55DC26B0089; Fri, 28 Jan 2022 08:10:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0032.hostedemail.com [216.40.44.32]) by kanga.kvack.org (Postfix) with ESMTP id 1CAF76B0080 for ; Fri, 28 Jan 2022 08:10:12 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id DB12C8909D for ; Fri, 28 Jan 2022 13:10:11 +0000 (UTC) X-FDA: 79079729022.26.1640D60 Received: from server.lespinasse.org (server.lespinasse.org [63.205.204.226]) by imf01.hostedemail.com (Postfix) with ESMTP id 7FC9140014 for ; Fri, 28 Jan 2022 13:10:11 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=lespinasse.org; i=@lespinasse.org; q=dns/txt; s=srv-52-ed; t=1643375407; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : from; bh=2AfpzyewhmOlo0fUb0SXI6JxtNUdQC+AYomvKlpL5Ds=; b=Ib2Yh44H5XDzNjpRtIRI02jAIe/KU1DMungZIVzI88t8j0wqwdIWvvyZGtOBMiXiwJPBl 1pvNr0iYL9/q4z0Ag== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lespinasse.org; i=@lespinasse.org; q=dns/txt; s=srv-52-rsa; t=1643375407; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : from; bh=2AfpzyewhmOlo0fUb0SXI6JxtNUdQC+AYomvKlpL5Ds=; b=5vh96h1Q6g4N+Rf+N+L3AUPDrGbYhjVO2jJ3zEgFGhjHvXxcXZ2UelKDr0orcD9V/RPx0 p5Q0nD4RoGulcGvYr4T2j2KS6Y8Nmo1yyVjTSxyPV5ukSEa6Z0WqgfRx/fGV5YLjT2YHiGY MYn4cHvOeaN43jwZLJOzr1/VUkCiNfYsbcIkX2U4U2RnXIiR+uiILVQ5NlydE7BZN1lKDiu Vk1qSP3Q4CcqDst4aKcWBqHyIecdT9aM67g6clkLZv7FImiBwcJSEwCiAr2We5zJHeLqlY3 s/efKuCQOuQ9YI1KuFZR0xC9y+bkr2mnw2hIltaEe6+eCUsrWrmb9JXQqRbg== Received: from zeus.lespinasse.org (zeus.lespinasse.org [IPv6:fd00::150:0]) by server.lespinasse.org (Postfix) with ESMTPS id 0D973160977; Fri, 28 Jan 2022 05:10:07 -0800 (PST) Received: by zeus.lespinasse.org (Postfix, from userid 1000) id E6DC020328; Fri, 28 Jan 2022 05:10:06 -0800 (PST) From: Michel Lespinasse To: Linux-MM , linux-kernel@vger.kernel.org, Andrew Morton Cc: kernel-team@fb.com, Laurent Dufour , Jerome Glisse , Peter Zijlstra , Michal Hocko , Vlastimil Babka , Davidlohr Bueso , Matthew Wilcox , Liam Howlett , Rik van Riel , Paul McKenney , Song Liu , Suren Baghdasaryan , Minchan Kim , Joel Fernandes , David Rientjes , Axel Rasmussen , Andy Lutomirski , Michel Lespinasse Subject: [PATCH v2 14/35] mm: add speculative_page_walk_begin() and speculative_page_walk_end() Date: Fri, 28 Jan 2022 05:09:45 -0800 Message-Id: <20220128131006.67712-15-michel@lespinasse.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20220128131006.67712-1-michel@lespinasse.org> References: <20220128131006.67712-1-michel@lespinasse.org> MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 7FC9140014 X-Stat-Signature: hafwk3kjg8hn45ub4369afof7gak4qy1 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=lespinasse.org header.s=srv-52-ed header.b=Ib2Yh44H; dkim=pass header.d=lespinasse.org header.s=srv-52-rsa header.b=5vh96h1Q; dmarc=pass (policy=none) header.from=lespinasse.org; spf=pass (imf01.hostedemail.com: domain of walken@lespinasse.org designates 63.205.204.226 as permitted sender) smtp.mailfrom=walken@lespinasse.org X-Rspam-User: nil X-HE-Tag: 1643375411-933103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Speculative page faults will use these to protect against races with page table reclamation. This could always be handled by disabling local IRQs as the fast GUP code does; however speculative page faults do not need to protect against races with THP page splitting, so a weaker rcu read lock is sufficient in the MMU_GATHER_RCU_TABLE_FREE case. Signed-off-by: Michel Lespinasse --- mm/memory.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/mm/memory.c b/mm/memory.c index aa24cd8c06e9..663952d14bad 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2725,6 +2725,28 @@ int apply_to_existing_page_range(struct mm_struct *mm, unsigned long addr, } EXPORT_SYMBOL_GPL(apply_to_existing_page_range); +#ifdef CONFIG_SPECULATIVE_PAGE_FAULT + +/* + * speculative_page_walk_begin() ... speculative_page_walk_end() protects + * against races with page table reclamation. + * + * This is similar to what fast GUP does, but fast GUP also needs to + * protect against races with THP page splitting, so it always needs + * to disable interrupts. + * Speculative page faults only need to protect against page table reclamation, + * so rcu_read_lock() is sufficient in the MMU_GATHER_RCU_TABLE_FREE case. + */ +#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE +#define speculative_page_walk_begin() rcu_read_lock() +#define speculative_page_walk_end() rcu_read_unlock() +#else +#define speculative_page_walk_begin() local_irq_disable() +#define speculative_page_walk_end() local_irq_enable() +#endif + +#endif /* CONFIG_SPECULATIVE_PAGE_FAULT */ + /* * handle_pte_fault chooses page fault handler according to an entry which was * read non-atomically. Before making any commitment, on those architectures