From patchwork Thu Jul 13 09:53:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13311667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26F0BC04A6A for ; Thu, 13 Jul 2023 09:40:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3A6C68D000E; Thu, 13 Jul 2023 05:40:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3566A8D0008; Thu, 13 Jul 2023 05:40:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F7118D000E; Thu, 13 Jul 2023 05:40:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 0B06B8D0008 for ; Thu, 13 Jul 2023 05:40:41 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id BE2CB1401B6 for ; Thu, 13 Jul 2023 09:40:40 +0000 (UTC) X-FDA: 81006093840.25.BF178D8 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf03.hostedemail.com (Postfix) with ESMTP id 0CDEF20014 for ; Thu, 13 Jul 2023 09:40:37 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689241238; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kw+TYNhPpY/abAWXtmhBQbSP/ks5Qd9nSeDRwWdiv9o=; b=ez79Sab+tyOikervhcZY0YSJ1qSmoZw6JroL2NHYnf/FWRSpb0T3ymH048F2GtqWFFOg+c HS+Pyq/DOvQT9osfZ/UXb4WAEAC4fxe8io2xRnAEEB5+7/bNPYhJVgwP9es4Is4xvHHqIy MmfZZRj134AEZ1vwpM225omAcCwaN+M= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689241238; a=rsa-sha256; cv=none; b=2YWJ9CpjbdzTsYXlOotL8TSn4dKH01iOq4mgl80+VNzZHhEW6Wb+b8KCEE+xFRW0mi+ljk g81WSc/+FcnOd/lFoaDPU/TFnOdOBByWNR5WrFwo/xBQ/dO2gk6Fsu2khiluH7IWptVyTM P6dFDkEaJ3j/zoopevsIpCW7S1IJDvg= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm500001.china.huawei.com (unknown [172.30.72.53]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4R1qMg1LkQz18Lnh; Thu, 13 Jul 2023 17:39:59 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 13 Jul 2023 17:40:33 +0800 From: Kefeng Wang To: , Andrew Morton , CC: Russell King , Catalin Marinas , Will Deacon , Huacai Chen , WANG Xuerui , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , , , , , , , , Kefeng Wang Subject: [PATCH rfc -next 10/10] loongarch: mm: try VMA lock-based page fault handling first Date: Thu, 13 Jul 2023 17:53:38 +0800 Message-ID: <20230713095339.189715-11-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230713095339.189715-1-wangkefeng.wang@huawei.com> References: <20230713095339.189715-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 0CDEF20014 X-Rspam-User: X-Stat-Signature: 3pgpst815xw9bxirhaia5tu1apahya9i X-Rspamd-Server: rspam03 X-HE-Tag: 1689241237-637535 X-HE-Meta: U2FsdGVkX1/2ykratl6neh6znNkJ3jbQ0umQC1gKDHhlCu+UDmF1zInIGnaXHkXjMXoCjzFVH1Ao6UN4F4zC6wT/tqN/U9bi5BgR9YU7souD/c9IeKBIDBepYLlG9Dz6clq+J5H11wbas47L/V8HqdGul7rIO3CHh8Nfva+BH/fa9mryzQxYw6HJ/YeFr4+7mkskwajnphOMXXkMg5AtgSxzQa/KdkMXa9E6DGuaMIWm5IQFcHtFhHEGUfFVPiKoJxQEsPGPZ0r8Q7UxY5TRsWJGUob7TKNLUAaAMCOnXCPRPdhwnIVhVc+lcEy/jxFalEQleO45A9X0l4sdEIuPGTnCjjGlBwhI0e/ss3ApivH/My0axVBDSLltsF2h+x4o+Y2+rdr9WWzFiIG4TIm6t01uJZmvzoymJQ2LeGMNYSEaz1W4UuMrf76w6Rvm1N8YT6h4CIIv+OxiLAwWlEWB6xMhgDWPK0MBYY6vvmaDICEMl0W/Y6IPvIN8fUEek/fo/xF/nzkRHd+Inq2Cvx4wc9hDCFqksOFDwZQITqr3KJVoQlp+8huK7/b2YUqaceLmG+tfv4w7SxDiTpc7BT0jb47nfL1mnK/Zl3TF3hkSp6pfRlsYDxq7hOSp3eZ+1rbNSWuUq3irJNwhrVPoyzhpYenSlfhyd84PbXMLFiq8GsuvJCeJAxgDw7Q2VZnErtZXTPV4Vd6gYgFZPP9+bf6KUkX61SAU/MtmTnfez8qTXAEDog7MTICCqBBtTgn6BgxJUVNahYzhQt8bIymKbKt2HHMnOG+B0HzfMo74CF5ajy0LaBbHYGKgGxMjOo+QV6hCsxFILvc0wOhMFaPXnzvjd2wJedckIvF6LpR13RzGyt22wDIHIc3yvH9AEEO7/8ssE4wgTLsXEWAjcFD/Bq+B1yuUt20KKkJf4M73NXZItyfMCW6vlDKoBCMKEnk720YK3SEBFySL6byb8oQvKvx TTgLUlxt /yJvrSumbV+P35vN8As7zsNMNTSgNVjgoARTbS3Lyw7VdUpy40y50zodeQKViJOWYtDBE8parTZZecfKmB+Jj8RMDKx5Q61NfvAc0J8pR2t5Fom/EsrpdJE6nHlcozf4Y8SNtOI7hIhenvXY1RKGcBREjTQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Attempt VMA lock-based page fault handling first, and fall back to the existing mmap_lock-based handling if that fails. Signed-off-by: Kefeng Wang --- arch/loongarch/Kconfig | 1 + arch/loongarch/mm/fault.c | 26 ++++++++++++++++++++++++++ 2 files changed, 27 insertions(+) diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index 397203e18800..afb0ccabab97 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -53,6 +53,7 @@ config LOONGARCH select ARCH_SUPPORTS_LTO_CLANG select ARCH_SUPPORTS_LTO_CLANG_THIN select ARCH_SUPPORTS_NUMA_BALANCING + select ARCH_SUPPORTS_PER_VMA_LOCK select ARCH_USE_BUILTIN_BSWAP select ARCH_USE_CMPXCHG_LOCKREF select ARCH_USE_QUEUED_RWLOCKS diff --git a/arch/loongarch/mm/fault.c b/arch/loongarch/mm/fault.c index cde2ea0119fa..7e54bc48813e 100644 --- a/arch/loongarch/mm/fault.c +++ b/arch/loongarch/mm/fault.c @@ -136,6 +136,17 @@ static inline bool access_error(unsigned int flags, struct pt_regs *regs, return false; } +#ifdef CONFIG_PER_VMA_LOCK +int arch_vma_check_access(struct vm_area_struct *vma, + struct vm_locked_fault *vmlf) +{ + if (unlikely(access_error(vmlf->fault_flags, vmlf->regs, vmlf->address, + vma))) + return -EINVAL; + return 0; +} +#endif + /* * This routine handles page faults. It determines the address, * and the problem, and then passes it off to one of the appropriate @@ -149,6 +160,7 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, struct task_struct *tsk = current; struct mm_struct *mm = tsk->mm; struct vm_area_struct *vma = NULL; + struct vm_locked_fault vmlf; vm_fault_t fault; if (kprobe_page_fault(regs, current->thread.trap_nr)) @@ -183,6 +195,19 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, flags |= FAULT_FLAG_WRITE; perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, address); + + VM_LOCKED_FAULT_INIT(vmlf, mm, address, flags, 0, regs, 0); + if (try_vma_locked_page_fault(&vmlf, &fault)) + goto retry; + else if (!(fault | VM_FAULT_RETRY)) + goto done; + + if (fault_signal_pending(fault, regs)) { + if (!user_mode(regs)) + no_context(regs, address); + return; + } + retry: vma = lock_mm_and_find_vma(mm, address, regs); if (unlikely(!vma)) @@ -223,6 +248,7 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, mmap_read_unlock(mm); +done: if (unlikely(fault & VM_FAULT_ERROR)) { if (fault & VM_FAULT_OOM) { do_out_of_memory(regs, address);