From patchwork Wed Feb 13 08:06:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 10809307 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 27CCC1399 for ; Wed, 13 Feb 2019 08:06:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 149682C8B9 for ; Wed, 13 Feb 2019 08:06:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 079152C8BC; Wed, 13 Feb 2019 08:06:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4A8AB2C8B9 for ; Wed, 13 Feb 2019 08:06:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C064A8E0003; Wed, 13 Feb 2019 03:06:44 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B8DBE8E0001; Wed, 13 Feb 2019 03:06:44 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A2EC98E0003; Wed, 13 Feb 2019 03:06:44 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id 48FBB8E0001 for ; Wed, 13 Feb 2019 03:06:44 -0500 (EST) Received: by mail-ed1-f69.google.com with SMTP id w51so652878edw.7 for ; Wed, 13 Feb 2019 00:06:44 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=+ic/ZUhdIlQiYtEt1g3sTWSG5TMvSGuJzs/LGycdU/o=; b=W88tjtgNSu28PN3a1BDxuBl1tUCJ+MVAfw1xKNpR8HGSBMG1p9tOn/gDRr6bsNVwEy 7+RG+d9NeXtLJdNVzSWtjaXOc4gAysVUZTcMAsipYC0u5RpGJDrCoJAgAvITiv+lbJyA CEGWet2Pn0TKEKIcmcdZrt9jE2ijGIllSiAQzMpFtm8NkTzkf55rCopSHIivo3Xo42h1 awTcjgDLwdfDI3t26J2ABa+MbguiofYz0Y3Z7w0Uf0MpU63Wz2r56yva4u4TJHLVlAjf qpX1DNBk2jO6u4PMnJaNnh+7C1by5jHQ4+b790v2tECKbUMDookZSvo3Phw2gI4Y5zfs e4Gw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com X-Gm-Message-State: AHQUAuYK29so0s4r+PLgeNxlddyZWiy7Zwis1+aeDy0UdnVQTB2/rHaW YwE/bSutp7pons1o1HNJsGdjJMRUuFq4hYMJcFTmDVzwGKXgM01VN956n33Qq5Qowtlseyn9D37 dOQtbReJMKooRtVDvaeTwwv70LXiJXZDOVP1IBbc4OlPvEVFpL1myKtQ9ys6e9ugkoA== X-Received: by 2002:a05:6402:1295:: with SMTP id w21mr6350776edv.293.1550045203834; Wed, 13 Feb 2019 00:06:43 -0800 (PST) X-Google-Smtp-Source: AHgI3IZqxpRldpWQtOalTxV0whRX0mK17ieXENDv3iMTUprws/0PaaVrtw3hy/Uw9vj0fHg3mx39 X-Received: by 2002:a05:6402:1295:: with SMTP id w21mr6350718edv.293.1550045202836; Wed, 13 Feb 2019 00:06:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550045202; cv=none; d=google.com; s=arc-20160816; b=YKe6oFuKaZ+hKaPCM8AUUnrgr6mbTaDufcwASnGS/Z1fh36TvTobujit+1MGjyxJLS v0ppqKUlJ2tTtYQGJC78pzci3EFLnqcTSgma+W4plXAv42aV40qWwHGixnXN5mDX5uoK 36dI6CtBPIMj3VITL35YTLdiVOlnE1qY5t7cg/z3cREtZv6JkcFao25CtloPlp/fPWEJ KnNsT2ykhUM99wP0atO5Sub0Yd4mdNJu3jHxJUmQCPl44nITuHm4+MDMybbss2DyC36z T81+fY9+AIDYRBetQ8lBRN1iRV02vVtzRr4yU9HOYR9yY6kiPYa6ReDg3irrhg5kmJrE tErw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=+ic/ZUhdIlQiYtEt1g3sTWSG5TMvSGuJzs/LGycdU/o=; b=PBOyURsAdnjnmoZxL44f8o6TkMf4LM8uAkNkdf0zA5X8a1KxaMHk6p5NxXbRL/Pmia 95jbbp0NpY1wmi8FK3RKGU9sZ/E8qd7tuX+DrN4v1/MuY5uH6j7sJ3OiZ/1TqlR74WJH S3xtI0ZlqEEWjJiEwWGbDg/HwoPP/sM5OOPSQ/SoTn0SUFhcFo4LEm0YGx1ge0EOkhwk KeQXDULUDwzYkOGQ0Erm/EyuEd2w8URHuM5gf1R94WN2+PE2niyInW61FTVg6YotY99I v3baFUiEZdsu7cgdVXdr/airEZm+mhqa/P5GYyNgwK80k/lPjLpdTtVkY0BE6TXhjuPT 9CSw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id o21si799544edc.54.2019.02.13.00.06.42 for ; Wed, 13 Feb 2019 00:06:42 -0800 (PST) Received-SPF: pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9AE501596; Wed, 13 Feb 2019 00:06:41 -0800 (PST) Received: from p8cg001049571a15.blr.arm.com (p8cg001049571a15.blr.arm.com [10.162.43.147]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 2FBFA3F575; Wed, 13 Feb 2019 00:06:37 -0800 (PST) From: Anshuman Khandual To: linux-mm@kvack.org, akpm@linux-foundation.org Cc: mhocko@kernel.org, kirill@shutemov.name, kirill.shutemov@linux.intel.com, vbabka@suse.cz, will.deacon@arm.com, catalin.marinas@arm.com, dave.hansen@intel.com Subject: [RFC 1/4] mm: Introduce lazy exec permission setting on a page Date: Wed, 13 Feb 2019 13:36:28 +0530 Message-Id: <1550045191-27483-2-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1550045191-27483-1-git-send-email-anshuman.khandual@arm.com> References: <1550045191-27483-1-git-send-email-anshuman.khandual@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Setting an exec permission on a page normally triggers I-cache invalidation which might be expensive. I-cache invalidation is not mandatory on a given page if there is no immediate exec access on it. Non-fault modification of user page table from generic memory paths like migration can be improved if setting of the exec permission on the page can be deferred till actual use. This introduces [pte|pmd]_mklazyexec() which clears the exec permission on a page during migration. This exec permission deferral must be enabled back with maybe_[pmd]_mkexec() during exec page fault (FAULT_FLAG_INSTRUCTION) if the corresponding VMA contains exec flag (VM_EXEC). This framework is encapsulated under CONFIG_ARCH_SUPPORTS_LAZY_EXEC so that non-subscribing architectures don't take any performance hit. For now only generic memory migration path will be using this framework but later it can be extended to other generic memory paths as well. Signed-off-by: Anshuman Khandual --- include/asm-generic/pgtable.h | 12 ++++++++++++ include/linux/mm.h | 26 ++++++++++++++++++++++++++ mm/Kconfig | 9 +++++++++ mm/huge_memory.c | 5 +++++ mm/hugetlb.c | 2 ++ mm/memory.c | 4 ++++ mm/migrate.c | 2 ++ 7 files changed, 60 insertions(+) diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h index 05e61e6..d35d129 100644 --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -26,6 +26,18 @@ #define USER_PGTABLES_CEILING 0UL #endif +#ifndef CONFIG_ARCH_SUPPORTS_LAZY_EXEC +static inline pte_t pte_mklazyexec(pte_t entry) +{ + return entry; +} + +static inline pmd_t pmd_mklazyexec(pmd_t entry) +{ + return entry; +} +#endif + #ifndef __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS extern int ptep_set_access_flags(struct vm_area_struct *vma, unsigned long address, pte_t *ptep, diff --git a/include/linux/mm.h b/include/linux/mm.h index 80bb640..04d7a0a 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -755,6 +755,32 @@ static inline pte_t maybe_mkwrite(pte_t pte, struct vm_area_struct *vma) return pte; } +#ifdef CONFIG_ARCH_SUPPORTS_LAZY_EXEC +static inline pte_t maybe_mkexec(pte_t entry, struct vm_area_struct *vma) +{ + if (unlikely(vma->vm_flags & VM_EXEC)) + return pte_mkexec(entry); + return entry; +} + +static inline pmd_t maybe_pmd_mkexec(pmd_t entry, struct vm_area_struct *vma) +{ + if (unlikely(vma->vm_flags & VM_EXEC)) + return pmd_mkexec(entry); + return entry; +} +#else +static inline pte_t maybe_mkexec(pte_t entry, struct vm_area_struct *vma) +{ + return entry; +} + +static inline pmd_t maybe_pmd_mkexec(pmd_t entry, struct vm_area_struct *vma) +{ + return entry; +} +#endif + vm_fault_t alloc_set_pte(struct vm_fault *vmf, struct mem_cgroup *memcg, struct page *page); vm_fault_t finish_fault(struct vm_fault *vmf); diff --git a/mm/Kconfig b/mm/Kconfig index 25c71eb..5c046cb 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -322,6 +322,15 @@ config DEFAULT_MMAP_MIN_ADDR This value can be changed after boot using the /proc/sys/vm/mmap_min_addr tunable. +config ARCH_SUPPORTS_LAZY_EXEC + bool "Architecture supports deferred exec permission setting" + help + Some architectures can improve performance during non-fault page + table modifications paths with deferred exec permission setting + which helps in avoiding expensive I-cache invalidations. This + requires arch implementation of ptep_set_access_flags() to allow + non-exec to exec transition. + config ARCH_SUPPORTS_MEMORY_FAILURE bool diff --git a/mm/huge_memory.c b/mm/huge_memory.c index faf357e..9ef7662 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1126,6 +1126,8 @@ void huge_pmd_set_accessed(struct vm_fault *vmf, pmd_t orig_pmd) if (write) entry = pmd_mkdirty(entry); haddr = vmf->address & HPAGE_PMD_MASK; + if (vmf->flags & FAULT_FLAG_INSTRUCTION) + entry = maybe_pmd_mkexec(entry, vmf->vma); if (pmdp_set_access_flags(vmf->vma, haddr, vmf->pmd, entry, write)) update_mmu_cache_pmd(vmf->vma, vmf->address, vmf->pmd); @@ -1290,6 +1292,8 @@ vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t orig_pmd) pmd_t entry; entry = pmd_mkyoung(orig_pmd); entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma); + if (vmf->flags & FAULT_FLAG_INSTRUCTION) + entry = maybe_pmd_mkexec(entry, vma); if (pmdp_set_access_flags(vma, haddr, vmf->pmd, entry, 1)) update_mmu_cache_pmd(vma, vmf->address, vmf->pmd); ret |= VM_FAULT_WRITE; @@ -2944,6 +2948,7 @@ void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) pmde = pmd_mksoft_dirty(pmde); if (is_write_migration_entry(entry)) pmde = maybe_pmd_mkwrite(pmde, vma); + pmde = pmd_mklazyexec(pmde); flush_cache_range(vma, mmun_start, mmun_start + HPAGE_PMD_SIZE); if (PageAnon(new)) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index afef616..ea41832 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -4018,6 +4018,8 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, entry = huge_pte_mkdirty(entry); } entry = pte_mkyoung(entry); + if (flags & FAULT_FLAG_INSTRUCTION) + entry = maybe_mkexec(entry, vma); if (huge_ptep_set_access_flags(vma, haddr, ptep, entry, flags & FAULT_FLAG_WRITE)) update_mmu_cache(vma, haddr, ptep); diff --git a/mm/memory.c b/mm/memory.c index e11ca9d..74c406b 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2218,6 +2218,8 @@ static inline void wp_page_reuse(struct vm_fault *vmf) flush_cache_page(vma, vmf->address, pte_pfn(vmf->orig_pte)); entry = pte_mkyoung(vmf->orig_pte); entry = maybe_mkwrite(pte_mkdirty(entry), vma); + if (vmf->flags & FAULT_FLAG_INSTRUCTION) + entry = maybe_mkexec(entry, vma); if (ptep_set_access_flags(vma, vmf->address, vmf->pte, entry, 1)) update_mmu_cache(vma, vmf->address, vmf->pte); pte_unmap_unlock(vmf->pte, vmf->ptl); @@ -3804,6 +3806,8 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) entry = pte_mkdirty(entry); } entry = pte_mkyoung(entry); + if (vmf->flags & FAULT_FLAG_INSTRUCTION) + entry = maybe_mkexec(entry, vmf->vma); if (ptep_set_access_flags(vmf->vma, vmf->address, vmf->pte, entry, vmf->flags & FAULT_FLAG_WRITE)) { update_mmu_cache(vmf->vma, vmf->address, vmf->pte); diff --git a/mm/migrate.c b/mm/migrate.c index d4fd680..7587717 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -257,6 +257,7 @@ static bool remove_migration_pte(struct page *page, struct vm_area_struct *vma, if (PageHuge(new)) { pte = pte_mkhuge(pte); pte = arch_make_huge_pte(pte, vma, new, 0); + pte = pte_mklazyexec(pte); set_huge_pte_at(vma->vm_mm, pvmw.address, pvmw.pte, pte); if (PageAnon(new)) hugepage_add_anon_rmap(new, vma, pvmw.address); @@ -265,6 +266,7 @@ static bool remove_migration_pte(struct page *page, struct vm_area_struct *vma, } else #endif { + pte = pte_mklazyexec(pte); set_pte_at(vma->vm_mm, pvmw.address, pvmw.pte, pte); if (PageAnon(new)) From patchwork Wed Feb 13 08:06:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 10809309 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B15031399 for ; Wed, 13 Feb 2019 08:06:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A06DC2BEA5 for ; Wed, 13 Feb 2019 08:06:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 943C32C8BB; Wed, 13 Feb 2019 08:06:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3C20D2BEA5 for ; Wed, 13 Feb 2019 08:06:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8825C8E0004; Wed, 13 Feb 2019 03:06:48 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 832978E0001; Wed, 13 Feb 2019 03:06:48 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6AF0B8E0004; Wed, 13 Feb 2019 03:06:48 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id 234BE8E0001 for ; Wed, 13 Feb 2019 03:06:48 -0500 (EST) Received: by mail-ed1-f69.google.com with SMTP id x15so665757edd.2 for ; Wed, 13 Feb 2019 00:06:48 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=urH6OH0gut0tDgnHnlkLf46J8KjXnTjzYyRqRVAC1C0=; b=an+GpR3hitDBj+4FGvvlBKFM3/fs6a2JcaVSp98BvJfTJJAM7d+kS1aNp3u1kq7o+Z ooGvEM5E3ff7o5x6yROcp28bvowai7zy/Ks+6ve4NMPqgVA2T3fedjkjzeCKIdU3BRQx kG3kuk76VvGKB5G7qV5y+bWhbSJx1sYc4OBVMHNYCLW+GLcfFDg3rVVlbDErwiojX4NU YcD6hi4pAUmgea86FVwAK0HnWk6O6k+DUPZST/D7WXR4airmf3FHukUiXKOTekU/iLBb i1p5WTatCVlRySzScCo7MjsKoKsjRmkjwOVmT3vCe3Ei42A1hM9WcE3uf7gbdBMlFTsa 11QA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com X-Gm-Message-State: AHQUAuaX/G7yb6l+7tJfFOhrjBMs5iNH13k0JEzMtqgtE99kSqm4ksbl tgfRd4l8TJmzOc79E8twXQuE5NFX/HSLa3WFDHnbnlpO6AWJNbEbMpcfSk97vJ21m3eL74Ee2vo DonNwPO5X/iDYrANWh1DiuOW+7LlIfECb4e8BpLyFPeiRHqLdUNpRNGDRtIT7c3GbDg== X-Received: by 2002:a50:8b42:: with SMTP id l60mr6365089edl.61.1550045207634; Wed, 13 Feb 2019 00:06:47 -0800 (PST) X-Google-Smtp-Source: AHgI3IaVEfBorSaR0O7jLJtoj1r2iB4ISGtshFBPJChWA4zopYNF/TpLabh3Y7x3KaBrmluSCmma X-Received: by 2002:a50:8b42:: with SMTP id l60mr6365028edl.61.1550045206554; Wed, 13 Feb 2019 00:06:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550045206; cv=none; d=google.com; s=arc-20160816; b=b9u0+jjV+1iFQZiEz8KNMY4z56PIHhKbJuIfAo0Hy8R8/iNVxrm+FM7W/1/OyNKKjb bvf/OCHDKUfMb+jVvo+V1SFvvDuUMn1GDYVN0MZ56d3cwjy2f5/UZLMHKWJZ3e7zdWUN +51wlLGz+cENIX3Tq6KRRbugWSqVr3a2kGg4HEB+Ucwqd7T4KUl73z9zYUf9GqjJZQ8S cbmy2TJdR6o7EAYvGzN6VNnsfpEiGI1wkekI724GljpU+gypYWiPINnp9dFkajZLUfjg W255+BnAUaozxpirN5AFsIXSpGojUb7Ix9tLb9Q4nYagoyBMtP0jeYXJ4nt0bPGjngha ZFWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=urH6OH0gut0tDgnHnlkLf46J8KjXnTjzYyRqRVAC1C0=; b=TgjthD35x7TYinJllYE+VchIUc3sI2L3tKxOeuCaiEZCL+xDE0NLJr0jP09Tp7ES4A hFB+te8KDgGrEIVmW8h58h/YZF/UZeWWnAL0OU31h9ek5YHIBuPi08oxRAT/q3mxjZZI SX8QL1F0leiDHLVCJEwXFB5SpoHsjGovgABS6foQDjmjvK3t9UewL0JuD7C8CPqaSFog +YPHRFU08ee1vv2eMURxmyQWUix9yF7K2IRdKbgqOFXvZyndY54m9gK91F9SzV0IBAZk KA5y6y0r3xi1gjJe7pPV73r1oMxqWel/XS61NGRK8cvlLkKLPGjEfo06GXvw6UW0J6kf u64g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id t23si3044404ejf.99.2019.02.13.00.06.46 for ; Wed, 13 Feb 2019 00:06:46 -0800 (PST) Received-SPF: pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8337115BF; Wed, 13 Feb 2019 00:06:45 -0800 (PST) Received: from p8cg001049571a15.blr.arm.com (p8cg001049571a15.blr.arm.com [10.162.43.147]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 22D493F575; Wed, 13 Feb 2019 00:06:41 -0800 (PST) From: Anshuman Khandual To: linux-mm@kvack.org, akpm@linux-foundation.org Cc: mhocko@kernel.org, kirill@shutemov.name, kirill.shutemov@linux.intel.com, vbabka@suse.cz, will.deacon@arm.com, catalin.marinas@arm.com, dave.hansen@intel.com Subject: [RFC 2/4] arm64/mm: Identify user level instruction faults Date: Wed, 13 Feb 2019 13:36:29 +0530 Message-Id: <1550045191-27483-3-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1550045191-27483-1-git-send-email-anshuman.khandual@arm.com> References: <1550045191-27483-1-git-send-email-anshuman.khandual@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Page fault flags (FAULT_FLAG_XXX) need to be passed down fault handling path for appropriate action and reporting. Identify user instruction fetch faults and mark them with FAULT_FLAG_INSTRUCTION. Signed-off-by: Anshuman Khandual --- arch/arm64/mm/fault.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index efb7b2c..591670d 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -468,6 +468,9 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, mm_flags |= FAULT_FLAG_WRITE; } + if (is_el0_instruction_abort(esr)) + mm_flags |= FAULT_FLAG_INSTRUCTION; + if (is_ttbr0_addr(addr) && is_el1_permission_fault(addr, esr, regs)) { /* regs->orig_addr_limit may be 0 if we entered from EL0 */ if (regs->orig_addr_limit == KERNEL_DS) From patchwork Wed Feb 13 08:06:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 10809311 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CBA401399 for ; Wed, 13 Feb 2019 08:06:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BB41C2BEA5 for ; Wed, 13 Feb 2019 08:06:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AF61C2C8BB; Wed, 13 Feb 2019 08:06:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9CE7A2C8B9 for ; Wed, 13 Feb 2019 08:06:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E286E8E0005; Wed, 13 Feb 2019 03:06:51 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D89BF8E0001; Wed, 13 Feb 2019 03:06:51 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C51438E0005; Wed, 13 Feb 2019 03:06:51 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id 6E8438E0001 for ; Wed, 13 Feb 2019 03:06:51 -0500 (EST) Received: by mail-ed1-f71.google.com with SMTP id f17so646683edt.20 for ; Wed, 13 Feb 2019 00:06:51 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=podC4yBE/C4EalrFbyfqigYUTjh8/0NvFB8BYgeqwzI=; b=WQOglXebwMBGN93B/UGp8Day8CBGAw/k6HPCSOrtY632OFRUS4d75jGtFqvXHKTQvG fFeDrJqGPDDvGQ6PG3Penz+6EJpgo+dKaR8VwcvBNa5YZMQKJZmvS3lWLnwwf+1Vwbaz QIkoPMTt2w0J7gvGnksfE74kr/F/LbWxOF6Nmk7N1uVP/O/Qe+ePFjXvnNAo+hItj+Xm KTF67Lug036FRGw42Lh1lbQ8p8CgdqKq2seJSQ78cuSHhDxFRPAmgp7Z6zW2XmouwLhp dbpQIZLBIzn6g5gimalesGrjNGNtFyFtHv20JZfeqF0p6yujHzkaeZLE1ze1M9Tcz9FJ V8Eg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com X-Gm-Message-State: AHQUAuZaagh6FV8KS/XjciyqCXtBAXVcjbg8aPVQWPg3XjgJ28++hA7M wswQhqn+SFdtxY9HOBdRl+/zxotdOINL+1UiePwChgSEsLxIsyrATpE9pgtV3QYSclhIhixHQ27 m1eIj+lpIdqWHAmAeH9v5VMvRD43jwnsrorjN8B8/15YSMwA7+bnYK+HJW5hSyJ5/5Q== X-Received: by 2002:a05:6402:1588:: with SMTP id c8mr2205569edv.176.1550045210904; Wed, 13 Feb 2019 00:06:50 -0800 (PST) X-Google-Smtp-Source: AHgI3Ia2B1lP/e5gVO4YsRjrK3zlJG2Ui4t3Ae6gmL6MDmJOQ7+x2xgX0SIP+2L+ADHpCG9dEFT5 X-Received: by 2002:a05:6402:1588:: with SMTP id c8mr2205527edv.176.1550045209965; Wed, 13 Feb 2019 00:06:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550045209; cv=none; d=google.com; s=arc-20160816; b=ASu/5BglPOI81MJkOBX4XcWWC9rx9oOPgyIwEoPIP8YendpclIF28l3/DDYnd1AU/F ONCCo677gzduN4J6s1iYJ/DI1eJjiM/Rnspcl86R161VlEZIHKqH5PBZ/+zZvzuuA4eY mksyTlVkqVZuvYyG6SGqqa0htyYG3qoLGzPZn6lwV+s5H491626OgXTl8Oz2onpLH5/K OdztdR67CiL93CX5HSw3ErkVQ7Rf/F7b0g2gPq8vS9BrvWckw4FpVsellRJCFYl3ukns 9CPCotk+SvA101JY/usq533fXWJIHXvu9P4tVSAE/N3iEquCBLgGOTH5Crqkhld0U3Dl N4zA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=podC4yBE/C4EalrFbyfqigYUTjh8/0NvFB8BYgeqwzI=; b=tw5HyGhkJit/yXJAxdJYgx1pLJ29FxclLy7s4dXP1zOrkAr+/vb7UMBAOcXxqyVmdc I3YZsrvHwgvwZk/yOX/YjMrT5PlNYIRWd8qFM59BnjMrDSShaeA53RGgBQxJT4PDkR2S OuAb4AqSsEkAyb8YnHMlFprJRTHRFHCTT6R+idCkXuE5UdWv4c6ZWLA5vu+6bmZ50Knd EpxTlrJym/G4EoaT5xonjwLRqlWomCIfskNVoxSqT7R76VQhh47Vsc4UiuWCpnKgWCa1 yP6G8IGBabKUnifB5bNlavgdL/yeH4BYaKQYlHT9sVAc7UyyxfnnXRPCfdCtdWT6/T1K 1Hgg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id 12si2621876edx.436.2019.02.13.00.06.49 for ; Wed, 13 Feb 2019 00:06:49 -0800 (PST) Received-SPF: pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E152A1596; Wed, 13 Feb 2019 00:06:48 -0800 (PST) Received: from p8cg001049571a15.blr.arm.com (p8cg001049571a15.blr.arm.com [10.162.43.147]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 146E13F575; Wed, 13 Feb 2019 00:06:45 -0800 (PST) From: Anshuman Khandual To: linux-mm@kvack.org, akpm@linux-foundation.org Cc: mhocko@kernel.org, kirill@shutemov.name, kirill.shutemov@linux.intel.com, vbabka@suse.cz, will.deacon@arm.com, catalin.marinas@arm.com, dave.hansen@intel.com Subject: [RFC 3/4] arm64/mm: Allow non-exec to exec transition in ptep_set_access_flags() Date: Wed, 13 Feb 2019 13:36:30 +0530 Message-Id: <1550045191-27483-4-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1550045191-27483-1-git-send-email-anshuman.khandual@arm.com> References: <1550045191-27483-1-git-send-email-anshuman.khandual@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP ptep_set_access_flags() updates page table for a mapped page entry which still got a fault probably because of a different permission than what it is mapped with. Previously an exec enabled page always gets required permission in the page table entry. Hence ptep_set_access_flags() never had to move an entry from non-exec to exec. This is going to change with deferred exec permission setting with later patches. Hence allow non-exec to exec transition here and do the required I-cache invalidation. Signed-off-by: Anshuman Khandual --- arch/arm64/mm/fault.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 591670d..1540fc1 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -227,22 +227,25 @@ int ptep_set_access_flags(struct vm_area_struct *vma, if (pte_same(pte, entry)) return 0; - /* only preserve the access flags and write permission */ - pte_val(entry) &= PTE_RDONLY | PTE_AF | PTE_WRITE | PTE_DIRTY; + /* only preserve the access flags, write and exec permission */ + pte_val(entry) &= PTE_RDONLY | PTE_AF | PTE_WRITE | PTE_DIRTY | PTE_UXN; + + if (pte_user_exec(entry)) + __sync_icache_dcache(pte); /* * Setting the flags must be done atomically to avoid racing with the - * hardware update of the access/dirty state. The PTE_RDONLY bit must - * be set to the most permissive (lowest value) of *ptep and entry - * (calculated as: a & b == ~(~a | ~b)). + * hardware update of the access/dirty state. The PTE_RDONLY bit and + * PTE_UXN must be set to the most permissive (lowest value) of *ptep + * and entry (calculated as: a & b == ~(~a | ~b)). */ - pte_val(entry) ^= PTE_RDONLY; + pte_val(entry) ^= PTE_RDONLY | PTE_UXN; pteval = pte_val(pte); do { old_pteval = pteval; - pteval ^= PTE_RDONLY; + pteval ^= PTE_RDONLY | PTE_UXN; pteval |= pte_val(entry); - pteval ^= PTE_RDONLY; + pteval ^= PTE_RDONLY | PTE_UXN; pteval = cmpxchg_relaxed(&pte_val(*ptep), old_pteval, pteval); } while (pteval != old_pteval); From patchwork Wed Feb 13 08:06:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 10809313 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E3B341399 for ; Wed, 13 Feb 2019 08:06:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D424E2C8B9 for ; Wed, 13 Feb 2019 08:06:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C44DE2BEA5; Wed, 13 Feb 2019 08:06:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 525762BEA5 for ; Wed, 13 Feb 2019 08:06:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6892D8E0006; Wed, 13 Feb 2019 03:06:55 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 60ED68E0001; Wed, 13 Feb 2019 03:06:55 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 48DD48E0006; Wed, 13 Feb 2019 03:06:55 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id DE3138E0001 for ; Wed, 13 Feb 2019 03:06:54 -0500 (EST) Received: by mail-ed1-f72.google.com with SMTP id x15so665915edd.2 for ; Wed, 13 Feb 2019 00:06:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=lLU9a6ZQu0wuumQ0c/KrMdC8hoqxGAN16ZlPM6u6TQ0=; b=ugLHYkkdeMi6KdU4Utjx4WBJU8KixKbCtqiFkJDbimh5vkhhcJjcEdvqVyxbpbxKfA kw1tF+5RlEFrvJXs0WTUvx1Z3II/eDM9ja6QuXaXcpg9S1yrbZPlYsy2YxaKqCxxbxJk Ah/scg3BEbcKTFiSxIM5ybcVRAW9KCCaJdj1eYggnCXR2ziG4k8oecDSP8jTNhEfh9UY ml6YPo9NeRRiOpcMGvWlVGbe8tkrtZGKiOmr55WovrZxLlRa+o7CYMwfm8Dkjnu9fc1w xgWiNJDE64ei5RyKu9CspSVl7wXz+Ab/OFMHcOQE8ocNr27DyUaiV/IANVOp2PRPEY4s wO5g== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com X-Gm-Message-State: AHQUAuaKi+TH/oJ3tZO2zbLA4uYCY3pPdbeGYmoVrE/+84/PJlp/EV7M 5cExH1EQM0/THRpK/NlADkwpUVIy8hINd32T9OIGpLE0HBW7PI8RbpAQS/4hYvEbiW8fCGRjuQl 5bCCdVSMeOtfOWmY2/rqM7+d83vofkJ057aFQPzQko+Qm8ppqACEoReq5je7hEqSlBg== X-Received: by 2002:aa7:c0c4:: with SMTP id j4mr6332646edp.182.1550045214418; Wed, 13 Feb 2019 00:06:54 -0800 (PST) X-Google-Smtp-Source: AHgI3IYV7GKfOCGWNKpOzc87kKvs/n5mGYEDsbcaW6H+1Vt/W+w1xzZrwE8fTstLKE4K1IaOephj X-Received: by 2002:aa7:c0c4:: with SMTP id j4mr6332590edp.182.1550045213148; Wed, 13 Feb 2019 00:06:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550045213; cv=none; d=google.com; s=arc-20160816; b=Ybei5Cm1MR5eIvwQH9I29JaQlUCZKsFYew2GLXPOPK8cptpoLGtErklx9CQ5nJXKFV V43CWOE3I5VyqLji6GtVTUKvO5gE+UF5k+rqw/KrmUKmSiwn58DkndnUTNzLLHTpHg4X 4EB961o8WQ8rIHOIekMHpY30JGZhqeWVS3YDgxwbXYA54rrmsfDO0uDQHWPbk1x97Q8F xbs7Tz+knS5seBvaNgMLmgExzkYPLp9RjCFy5pbBV8lpQOOBamYy2KvK0IZM/SxKwgl1 PET4S5aKt351RBkaUQ5HFqCVSBpkCe2J/XasljMvv+vsWncP5O0FA1bXkCOusALL1tiY JsJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=lLU9a6ZQu0wuumQ0c/KrMdC8hoqxGAN16ZlPM6u6TQ0=; b=jTtBkkkk+kmVbai0OaultWH1afIhRnnbqwU0U+MzJseziQIwCFt2xxSUsUkdXUHp+l e/qYKKPWpRJqVqmpEFbFa2hB6Qmhu3GAxV2+wGalhzyDY+s5WDlF8OYMsy8bzCF9IRGF MiUedkZ1hBeOQeB45lnxKmj+INMmVMWpvxHWTbaN3Aiu0NTeFJ8/LHbwM4Xm3JGB9Is5 ydB+FpUvJVWLf0yMhhCYNxY2y1R9hDiM+9ircsjrKVeDRgf5PvR3nWzXzHMrqriBLWMU Fhy7Q9OIr7CRCjFjwYwURxussHcM+G4lY5Z+ahLd3uShdHDJ6jH2XRHBQAWKAdJwIm/a c6LA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id t17si1525918eds.207.2019.02.13.00.06.52 for ; Wed, 13 Feb 2019 00:06:53 -0800 (PST) Received-SPF: pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2A4561596; Wed, 13 Feb 2019 00:06:52 -0800 (PST) Received: from p8cg001049571a15.blr.arm.com (p8cg001049571a15.blr.arm.com [10.162.43.147]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 5D1023F575; Wed, 13 Feb 2019 00:06:49 -0800 (PST) From: Anshuman Khandual To: linux-mm@kvack.org, akpm@linux-foundation.org Cc: mhocko@kernel.org, kirill@shutemov.name, kirill.shutemov@linux.intel.com, vbabka@suse.cz, will.deacon@arm.com, catalin.marinas@arm.com, dave.hansen@intel.com Subject: [RFC 4/4] arm64/mm: Enable ARCH_SUPPORTS_LAZY_EXEC Date: Wed, 13 Feb 2019 13:36:31 +0530 Message-Id: <1550045191-27483-5-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1550045191-27483-1-git-send-email-anshuman.khandual@arm.com> References: <1550045191-27483-1-git-send-email-anshuman.khandual@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Make arm64 subscribe to ARCH_SUPPORTS_LAZY_EXEC framework and provided all required helpers for this purpose. This moves away execution cost from the migration path to exec fault path as expected. Signed-off-by: Anshuman Khandual --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/pgtable.h | 17 +++++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a4168d3..3cdb3e4 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -59,6 +59,7 @@ config ARM64 select ARCH_USE_CMPXCHG_LOCKREF select ARCH_USE_QUEUED_RWLOCKS select ARCH_USE_QUEUED_SPINLOCKS + select ARCH_SUPPORTS_LAZY_EXEC select ARCH_SUPPORTS_MEMORY_FAILURE select ARCH_SUPPORTS_ATOMIC_RMW select ARCH_SUPPORTS_INT128 if GCC_VERSION >= 50000 || CC_IS_CLANG diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index de70c1e..f2a5716 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -217,6 +217,18 @@ static inline pmd_t pmd_mkcont(pmd_t pmd) return __pmd(pmd_val(pmd) | PMD_SECT_CONT); } +#ifdef CONFIG_ARCH_SUPPORTS_LAZY_EXEC +static inline pte_t pte_mkexec(pte_t pte) +{ + return clear_pte_bit(pte, __pgprot(PTE_UXN)); +} + +static inline pte_t pte_mklazyexec(pte_t pte) +{ + return set_pte_bit(pte, __pgprot(PTE_UXN)); +} +#endif + static inline void set_pte(pte_t *ptep, pte_t pte) { WRITE_ONCE(*ptep, pte); @@ -355,6 +367,11 @@ static inline int pmd_protnone(pmd_t pmd) } #endif +#ifdef CONFIG_ARCH_SUPPORTS_LAZY_EXEC +#define pmd_mkexec(pmd) pte_pmd(pte_mkexec(pmd_pte(pmd))) +#define pmd_mklazyexec(pmd) pte_pmd(pte_mklazyexec(pmd_pte(pmd))) +#endif + /* * THP definitions. */