From patchwork Mon Aug 21 12:30:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13359372 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3020EE49A6 for ; Mon, 21 Aug 2023 12:31:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DBAD88D000E; Mon, 21 Aug 2023 08:31:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D91D88D0002; Mon, 21 Aug 2023 08:31:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C0CFB8D000E; Mon, 21 Aug 2023 08:31:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id AF2778D0002 for ; Mon, 21 Aug 2023 08:31:26 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 6ADEE40460 for ; Mon, 21 Aug 2023 12:31:26 +0000 (UTC) X-FDA: 81148047372.10.AEF9B5B Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf20.hostedemail.com (Postfix) with ESMTP id 1A5541C002B for ; Mon, 21 Aug 2023 12:31:21 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf20.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692621084; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KQnug/dkwoV99tMtZsLDbSwgWmkgFvFxpomh3L97DHU=; b=a7xbDwKj7ClwRU8h554gzDlup7gOnTHn8ulriAvcqKPWUJVUj5KBou8x7isLYlsMOME2cZ ztxloppibU9578uMDK5fy+9HLc3SW0IiU9bSwapUIKh2DmWupm2t8SoikUflQN5/M0hqbA PRSdNZsR8AmXCW9+gx23ludRaYcYskQ= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf20.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692621084; a=rsa-sha256; cv=none; b=BAbaEL9StRzYK7MzU12fDFfaipthZr3xcT8API3DtBFXet9hcp1It+F4qbYWqHwOGh/nqJ +O2pfRWwsmxoqS8xzZI7q05/SkZAOaJAzao8CbGXuplaWql+8RNFsTg+D8j37jAORHT0/v 6fAcfO6HDZKGT98azaqG2R3psFHj8j8= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4RTsF110wXztSVv; Mon, 21 Aug 2023 20:27:33 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 21 Aug 2023 20:31:14 +0800 From: Kefeng Wang To: Andrew Morton , CC: , , Russell King , Catalin Marinas , Will Deacon , Huacai Chen , WANG Xuerui , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , , "H . Peter Anvin" , , , , , , , Kefeng Wang Subject: [PATCH rfc v2 02/10] arm64: mm: use try_vma_locked_page_fault() Date: Mon, 21 Aug 2023 20:30:48 +0800 Message-ID: <20230821123056.2109942-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230821123056.2109942-1-wangkefeng.wang@huawei.com> References: <20230821123056.2109942-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 1A5541C002B X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: h11qttpynsdpadj5a3fsoa3mwjojypyu X-HE-Tag: 1692621081-416674 X-HE-Meta: U2FsdGVkX18rLe6u7rBPI+XconMIoCnDdakEANCGrH5pPPJn0YPSSlJew+RzPpYZjv1CvtQVO7+UcyhfZTgJxsUNTwtD3gUhrO+CgWsPym0KRsghDYvxNWUofAlVDMVIQeizAa3688RUd8ZKjLscQbWT0WUOEzLM+lkzN5R2fnLSjlq1uKWlw+2vBIpAeBoFRx9JLBRJtxFkaZVMrSEJWm0rEMf6ilI0g9EhrY8tRYtVfHXOu+pofaNo5R/fVq6KgCcNn2kw9MhvY1GLhHV+gfD5Le20/YYX034+ksklArFzOv8/5Zj9jxghykax3mj4Cfz4ADk/HqvltrVaDWjWamF7fg3y04p6Uh8f5iQ+W257ZhKx/UL2vmm4r6/PbyWB77/+oPe+nE6nivRBkK9pPTxJAJUCH8J4oGbQtBMi5xvs8Rkia/kyoYfHh9O3S9rG8VAEQo8I8KJuN36FSg+Vy+WWkmp/RL8eG9L9ejdt7+gzSRKykDzTgbH+Ezp/gLbUYOwABAm+pCYAAxE1t181PgucAU4hHd47CTYzTt6iiB4r+RcyZvE6kgUzN1Az86ESy0ys8LXaNfP7DOxCk/GqlMz5PhRNFZYDzVeedID//zkr/4IKiLbebAeShZhgVVZ5V1rPVBxUqQd6W5MU1WVz26IhR0SGYvI66Od8pjx7gSkjzIbdvv1jeUH5f0Ibv1ia54PJye400+sy6zCcaKUQjiNt3nnJYhwl+gGxm2/vnttdeCfQjV2kHMiO+RB2GKGtnZ0lAGER06pC+32YTEqPqeSPZNbbgtLOrCupkKPxhFwz+f3uJZLfvmTRiN/524m7WW6efuH2BjFUsFBjAg/q001K1p98GoEh6Q/PC/R4lWXXxQMHCOhrZGJSKoIH//5gXVXgRwFLbkeMjF3vNbbWFBuELeB7ZBMFeXUvdil7p+KBZzXcb242ibbhsuxg+EYXmUPWYgfFvPvobLykm8z hW9/SeAc FB9o4e+ZdxKNhDIMIgmNe7WHRVyEKHRemH2rXtMC2BlUwgRyCIXHlC3nsd4Nc5ey62N4dU1f60vRhaRziddysFp05xnIYwVJD81MJXAFXV7yySNSPRb9vPj7RiAE7TemDLHxt3btQcwMTr2RAZVx6hLRtscn2VLA7Jawlnvg0Pbm9swBWpIxkddyx+odYQu2/5aZt X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use new try_vma_locked_page_fault() helper to simplify code, also pass struct vmf to __do_page_fault() directly instead of each independent variable. No functional change intended. Signed-off-by: Kefeng Wang --- arch/arm64/mm/fault.c | 60 ++++++++++++++++--------------------------- 1 file changed, 22 insertions(+), 38 deletions(-) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 2e5d1e238af9..2b7a1e610b3e 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -498,9 +498,8 @@ static void do_bad_area(unsigned long far, unsigned long esr, #define VM_FAULT_BADACCESS ((__force vm_fault_t)0x020000) static vm_fault_t __do_page_fault(struct mm_struct *mm, - struct vm_area_struct *vma, unsigned long addr, - unsigned int mm_flags, unsigned long vm_flags, - struct pt_regs *regs) + struct vm_area_struct *vma, + struct vm_fault *vmf) { /* * Ok, we have a good vm_area for this memory access, so we can handle @@ -508,9 +507,9 @@ static vm_fault_t __do_page_fault(struct mm_struct *mm, * Check that the permissions on the VMA allow for the fault which * occurred. */ - if (!(vma->vm_flags & vm_flags)) + if (!(vma->vm_flags & vmf->vm_flags)) return VM_FAULT_BADACCESS; - return handle_mm_fault(vma, addr, mm_flags, regs); + return handle_mm_fault(vma, vmf->real_address, vmf->flags, vmf->regs); } static bool is_el0_instruction_abort(unsigned long esr) @@ -533,10 +532,12 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, const struct fault_info *inf; struct mm_struct *mm = current->mm; vm_fault_t fault; - unsigned long vm_flags; - unsigned int mm_flags = FAULT_FLAG_DEFAULT; unsigned long addr = untagged_addr(far); struct vm_area_struct *vma; + struct vm_fault vmf = { + .real_address = addr, + .flags = FAULT_FLAG_DEFAULT, + }; if (kprobe_page_fault(regs, esr)) return 0; @@ -549,7 +550,7 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, goto no_context; if (user_mode(regs)) - mm_flags |= FAULT_FLAG_USER; + vmf.flags |= FAULT_FLAG_USER; /* * vm_flags tells us what bits we must have in vma->vm_flags @@ -559,20 +560,20 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, */ if (is_el0_instruction_abort(esr)) { /* It was exec fault */ - vm_flags = VM_EXEC; - mm_flags |= FAULT_FLAG_INSTRUCTION; + vmf.vm_flags = VM_EXEC; + vmf.flags |= FAULT_FLAG_INSTRUCTION; } else if (is_write_abort(esr)) { /* It was write fault */ - vm_flags = VM_WRITE; - mm_flags |= FAULT_FLAG_WRITE; + vmf.vm_flags = VM_WRITE; + vmf.flags |= FAULT_FLAG_WRITE; } else { /* It was read fault */ - vm_flags = VM_READ; + vmf.vm_flags = VM_READ; /* Write implies read */ - vm_flags |= VM_WRITE; + vmf.vm_flags |= VM_WRITE; /* If EPAN is absent then exec implies read */ if (!cpus_have_const_cap(ARM64_HAS_EPAN)) - vm_flags |= VM_EXEC; + vmf.vm_flags |= VM_EXEC; } if (is_ttbr0_addr(addr) && is_el1_permission_fault(addr, esr, regs)) { @@ -587,26 +588,11 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, addr); - if (!(mm_flags & FAULT_FLAG_USER)) - goto lock_mmap; - - vma = lock_vma_under_rcu(mm, addr); - if (!vma) - goto lock_mmap; - - if (!(vma->vm_flags & vm_flags)) { - vma_end_read(vma); - goto lock_mmap; - } - fault = handle_mm_fault(vma, addr, mm_flags | FAULT_FLAG_VMA_LOCK, regs); - if (!(fault & (VM_FAULT_RETRY | VM_FAULT_COMPLETED))) - vma_end_read(vma); - - if (!(fault & VM_FAULT_RETRY)) { - count_vm_vma_lock_event(VMA_LOCK_SUCCESS); + fault = try_vma_locked_page_fault(&vmf); + if (fault == VM_FAULT_NONE) + goto retry; + if (!(fault & VM_FAULT_RETRY)) goto done; - } - count_vm_vma_lock_event(VMA_LOCK_RETRY); /* Quick path to respond to signals */ if (fault_signal_pending(fault, regs)) { @@ -614,8 +600,6 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, goto no_context; return 0; } -lock_mmap: - retry: vma = lock_mm_and_find_vma(mm, addr, regs); if (unlikely(!vma)) { @@ -623,7 +607,7 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, goto done; } - fault = __do_page_fault(mm, vma, addr, mm_flags, vm_flags, regs); + fault = __do_page_fault(mm, vma, &vmf); /* Quick path to respond to signals */ if (fault_signal_pending(fault, regs)) { @@ -637,7 +621,7 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, return 0; if (fault & VM_FAULT_RETRY) { - mm_flags |= FAULT_FLAG_TRIED; + vmf.flags |= FAULT_FLAG_TRIED; goto retry; } mmap_read_unlock(mm);