From patchwork Mon Aug 21 12:30:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13359369 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2627EEE49AA for ; Mon, 21 Aug 2023 12:31:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A862D8D000A; Mon, 21 Aug 2023 08:31:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A36D18D0002; Mon, 21 Aug 2023 08:31:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8FDAD8D000A; Mon, 21 Aug 2023 08:31:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 821C68D0002 for ; Mon, 21 Aug 2023 08:31:22 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 5636440C0C for ; Mon, 21 Aug 2023 12:31:22 +0000 (UTC) X-FDA: 81148047204.01.2C10D52 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf03.hostedemail.com (Postfix) with ESMTP id A630020014 for ; Mon, 21 Aug 2023 12:31:19 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692621080; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8Za++qJq53kzQeU94pIeGjitsbWDGJeX52Vmc0x0Gzo=; b=Q/Mou9dFyjRhKC71ltWC6DMKIgZgXdUNIXs1U00VK3fgdYfhyj2VNw7JQ5AV58R4pUBJtA 4gEqyIQdtEpAGVJPluFKpBnz/9SBQgsSEw3Yc0pNEFHscgsigRiXZXl6OZ712O9gFQjbci gtWkvk0tNJvtSpzJMdtsVYuXEAxFuOA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692621080; a=rsa-sha256; cv=none; b=YYi9yeay6F3Gytvi0Cm84EsUJDUPg1HrAUi1r0QQ+TXb9ImnG29TSQ6T3ZHTHpbv917b+q OLHJ/wZRYIL6wP32aJS/ME64MPJVIsuu1wLonI97TdhYL7B5Nq2VNPHuSOBEYSBBvT2aIs Lb2PuThK6BM70TijLc0XsX+xOfACIBk= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.56]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4RTsF23D6sztShV; Mon, 21 Aug 2023 20:27:34 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 21 Aug 2023 20:31:15 +0800 From: Kefeng Wang To: Andrew Morton , CC: , , Russell King , Catalin Marinas , Will Deacon , Huacai Chen , WANG Xuerui , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , , "H . Peter Anvin" , , , , , , , Kefeng Wang Subject: [PATCH rfc v2 03/10] x86: mm: use try_vma_locked_page_fault() Date: Mon, 21 Aug 2023 20:30:49 +0800 Message-ID: <20230821123056.2109942-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230821123056.2109942-1-wangkefeng.wang@huawei.com> References: <20230821123056.2109942-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: A630020014 X-Rspam-User: X-Stat-Signature: 5hpmiued73jnny3o5a8zia958upag4oy X-Rspamd-Server: rspam03 X-HE-Tag: 1692621079-691666 X-HE-Meta: U2FsdGVkX19jPPi3RgPbeq+AsG+DGrttrct6jCApZRqtX4bTEecc20adImMSfatDgBNfUUs+5vb93FssRovqzWaQ6+c/ZdFaIvBxr7l/0XYz87RsERNtcQFkUSigFsJtJBSSUhlmCnQ3qO05oWv6FWiy4DdWb/EfAxFQtaMhw7T2Nevs1aTscnD+hcFOjSovaWaFmMJiO6h06EmI0GA5QiF/7EwZ+lWyGJRq0vVQqcncqWn4O1DMjvkyX4QJEu+nOKcOsAwGJYmCMfMCqC19LswstcQVyDHN4wjT/wN5af5Tn6P2ms2TxVlbqa4evcuzOKSHSHecGXzVzIzjt6ZUlbPTiqa0BYV4CdiGkmipS6//Bg7XS8/Vr0TGmPuWaDt+FmD/VC0qe4hBdo2oqg32gwKp37p7A7HkQkVvBag3j/O72X7RI/22WLUQbwZ1W7+qD/eaZ/+IHY+gUiO0B5sfORgeZToJes0w2fdGxJ9zi27n8g/Yinsnpzvz76+4bbb//Ut9aWkBQwTEs1giJbsysK0D8JLRF3gKXT9/dFXANXinf6JB7sPXJ10QcaRHYiYOVfWeqEAD5jCEtswa9wyZbwKquxgK7fUHYe5SHWzMX16D0Tb7DLNL673uj8qV4hTbEPAqBS6WidJL1DO0IwSgPxRGSJNN1wcSiuYloE+EijmXnt+Ot7ozp2tbNJ/ykzTRXZ+tjzMe4RKRN+x2lU/R6eWDwglgZwg0Z0vEwfkYpVKSGsUKtpSG7TtAK3dVGGWs15XNqo7JMKJncZ5HUSpfdkBaHYdZXRQsHkuMi82QIOG6Wi2bpFWO+CxWk5IJzwpoEWd2WUopfzyOKpWafCX2gSPk7kRrWYDEObPpy8obbI05J1MGaeJSVydRsETHNcbQ7EOgGS9ZfuJiSvU6gJLc8/wUHVSwS9TEIqc5wvaaeaFf+K1hr+liMwEF20MgiI2LLDtDby8MVPc5Rc1IvFs lJZ3SxaI iGmDtHY3zcX39h5l20X9cvhWI3su2f5lkXB/lhm0b5C462Z6Cz77O4Kz055bgS9fNZaULnmMuJ/uIYvzHt0gKYrG5EhdHOnWYNZyElODViQTPTKRGB5BNVWYw4RER/ZL901r0GZ/KWda2KU3MtrdJZtJ8uR050y2Ntrbw2rrR4fnCUQxlEPuLKOyiHmAq9wKn8N+j X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use new try_vma_locked_page_fault() helper to simplify code. No functional change intended. Signed-off-by: Kefeng Wang --- arch/x86/mm/fault.c | 55 +++++++++++++++++++-------------------------- 1 file changed, 23 insertions(+), 32 deletions(-) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index ab778eac1952..3edc9edc0b28 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -1227,6 +1227,13 @@ do_kern_addr_fault(struct pt_regs *regs, unsigned long hw_error_code, } NOKPROBE_SYMBOL(do_kern_addr_fault); +#ifdef CONFIG_PER_VMA_LOCK +bool arch_vma_access_error(struct vm_area_struct *vma, struct vm_fault *vmf) +{ + return access_error(vmf->fault_code, vma); +} +#endif + /* * Handle faults in the user portion of the address space. Nothing in here * should check X86_PF_USER without a specific justification: for almost @@ -1241,13 +1248,13 @@ void do_user_addr_fault(struct pt_regs *regs, unsigned long address) { struct vm_area_struct *vma; - struct task_struct *tsk; - struct mm_struct *mm; + struct mm_struct *mm = current->mm; vm_fault_t fault; - unsigned int flags = FAULT_FLAG_DEFAULT; - - tsk = current; - mm = tsk->mm; + struct vm_fault vmf = { + .real_address = address, + .fault_code = error_code, + .flags = FAULT_FLAG_DEFAULT + }; if (unlikely((error_code & (X86_PF_USER | X86_PF_INSTR)) == X86_PF_INSTR)) { /* @@ -1311,7 +1318,7 @@ void do_user_addr_fault(struct pt_regs *regs, */ if (user_mode(regs)) { local_irq_enable(); - flags |= FAULT_FLAG_USER; + vmf.flags |= FAULT_FLAG_USER; } else { if (regs->flags & X86_EFLAGS_IF) local_irq_enable(); @@ -1326,11 +1333,11 @@ void do_user_addr_fault(struct pt_regs *regs, * maybe_mkwrite() can create a proper shadow stack PTE. */ if (error_code & X86_PF_SHSTK) - flags |= FAULT_FLAG_WRITE; + vmf.flags |= FAULT_FLAG_WRITE; if (error_code & X86_PF_WRITE) - flags |= FAULT_FLAG_WRITE; + vmf.flags |= FAULT_FLAG_WRITE; if (error_code & X86_PF_INSTR) - flags |= FAULT_FLAG_INSTRUCTION; + vmf.flags |= FAULT_FLAG_INSTRUCTION; #ifdef CONFIG_X86_64 /* @@ -1350,26 +1357,11 @@ void do_user_addr_fault(struct pt_regs *regs, } #endif - if (!(flags & FAULT_FLAG_USER)) - goto lock_mmap; - - vma = lock_vma_under_rcu(mm, address); - if (!vma) - goto lock_mmap; - - if (unlikely(access_error(error_code, vma))) { - vma_end_read(vma); - goto lock_mmap; - } - fault = handle_mm_fault(vma, address, flags | FAULT_FLAG_VMA_LOCK, regs); - if (!(fault & (VM_FAULT_RETRY | VM_FAULT_COMPLETED))) - vma_end_read(vma); - - if (!(fault & VM_FAULT_RETRY)) { - count_vm_vma_lock_event(VMA_LOCK_SUCCESS); + fault = try_vma_locked_page_fault(&vmf); + if (fault == VM_FAULT_NONE) + goto retry; + if (!(fault & VM_FAULT_RETRY)) goto done; - } - count_vm_vma_lock_event(VMA_LOCK_RETRY); /* Quick path to respond to signals */ if (fault_signal_pending(fault, regs)) { @@ -1379,7 +1371,6 @@ void do_user_addr_fault(struct pt_regs *regs, ARCH_DEFAULT_PKEY); return; } -lock_mmap: retry: vma = lock_mm_and_find_vma(mm, address, regs); @@ -1410,7 +1401,7 @@ void do_user_addr_fault(struct pt_regs *regs, * userland). The return to userland is identified whenever * FAULT_FLAG_USER|FAULT_FLAG_KILLABLE are both set in flags. */ - fault = handle_mm_fault(vma, address, flags, regs); + fault = handle_mm_fault(vma, address, vmf.flags, regs); if (fault_signal_pending(fault, regs)) { /* @@ -1434,7 +1425,7 @@ void do_user_addr_fault(struct pt_regs *regs, * that we made any progress. Handle this case first. */ if (unlikely(fault & VM_FAULT_RETRY)) { - flags |= FAULT_FLAG_TRIED; + vmf.flags |= FAULT_FLAG_TRIED; goto retry; }