From patchwork Mon Sep 30 01:57:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jia He X-Patchwork-Id: 11165937 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7E49B13B1 for ; Mon, 30 Sep 2019 01:58:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 486A5218AC for ; Mon, 30 Sep 2019 01:58:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 486A5218AC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 782E36B0005; Sun, 29 Sep 2019 21:58:01 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 70BCF6B0006; Sun, 29 Sep 2019 21:58:01 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5AC7E6B0007; Sun, 29 Sep 2019 21:58:01 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0082.hostedemail.com [216.40.44.82]) by kanga.kvack.org (Postfix) with ESMTP id 32EE36B0005 for ; Sun, 29 Sep 2019 21:58:01 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id BF8AC181AC9BA for ; Mon, 30 Sep 2019 01:58:00 +0000 (UTC) X-FDA: 75989926320.12.loaf99_62ba841198e35 X-Spam-Summary: 10,1,0,1f3fcb9d487a22a6,d41d8cd98f00b204,justin.he@arm.com,:catalin.marinas@arm.com:will@kernel.org:mark.rutland@arm.com:james.morse@arm.com:maz@kernel.org:willy@infradead.org:kirill.shutemov@linux.intel.com:linux-arm-kernel@lists.infradead.org:linux-kernel@vger.kernel.org::punitagrawal@gmail.com:tglx@linutronix.de:akpm@linux-foundation.org:hejianet@gmail.com:kaly.xin@arm.com:justin.he@arm.com,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1345:1359:1437:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:2895:2901:3138:3139:3140:3141:3142:3352:3865:3866:3867:3868:3871:3872:4250:4321:5007:6119:6261:7903:8634:10004:11026:11473:11657:11658:11914:12043:12048:12296:12297:12438:12555:12895:12986:13069:13311:13357:13523:13524:14096:14181:14384:14394:14721:21080:21451:21627:30054:30070,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutra l,Custom X-HE-Tag: loaf99_62ba841198e35 X-Filterd-Recvd-Size: 2647 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Mon, 30 Sep 2019 01:57:58 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7902C1570; Sun, 29 Sep 2019 18:57:57 -0700 (PDT) Received: from localhost.localdomain (entos-thunderx2-02.shanghai.arm.com [10.169.40.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A23343F706; Sun, 29 Sep 2019 18:57:53 -0700 (PDT) From: Jia He To: Catalin Marinas , Will Deacon , Mark Rutland , James Morse , Marc Zyngier , Matthew Wilcox , "Kirill A. Shutemov" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Punit Agrawal , Thomas Gleixner , Andrew Morton , hejianet@gmail.com, Kaly Xin , Jia He Subject: [PATCH v10 1/3] arm64: cpufeature: introduce helper cpu_has_hw_af() Date: Mon, 30 Sep 2019 09:57:38 +0800 Message-Id: <20190930015740.84362-2-justin.he@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190930015740.84362-1-justin.he@arm.com> References: <20190930015740.84362-1-justin.he@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We unconditionally set the HW_AFDBM capability and only enable it on CPUs which really have the feature. But sometimes we need to know whether this cpu has the capability of HW AF. So decouple AF from DBM by new helper cpu_has_hw_af(). Signed-off-by: Jia He Suggested-by: Suzuki Poulose Reviewed-by: Catalin Marinas --- arch/arm64/include/asm/cpufeature.h | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index 9cde5d2e768f..949bc7c85030 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -659,6 +659,16 @@ static inline u32 id_aa64mmfr0_parange_to_phys_shift(int parange) default: return CONFIG_ARM64_PA_BITS; } } + +/* Check whether hardware update of the Access flag is supported */ +static inline bool cpu_has_hw_af(void) +{ + if (IS_ENABLED(CONFIG_ARM64_HW_AFDBM)) + return read_cpuid(ID_AA64MMFR1_EL1) & 0xf; + + return false; +} + #endif /* __ASSEMBLY__ */ #endif From patchwork Mon Sep 30 01:57:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jia He X-Patchwork-Id: 11165939 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E1A491599 for ; Mon, 30 Sep 2019 01:58:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AE89B2190F for ; Mon, 30 Sep 2019 01:58:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AE89B2190F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CB26D6B0006; Sun, 29 Sep 2019 21:58:05 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C3CB76B0007; Sun, 29 Sep 2019 21:58:05 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B2AAB6B0008; Sun, 29 Sep 2019 21:58:05 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0106.hostedemail.com [216.40.44.106]) by kanga.kvack.org (Postfix) with ESMTP id 8C03E6B0006 for ; Sun, 29 Sep 2019 21:58:05 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id F13ED180AD803 for ; Mon, 30 Sep 2019 01:58:04 +0000 (UTC) X-FDA: 75989926488.09.page43_63582ba73db52 X-Spam-Summary: 2,0,0,52295d28532fefb0,d41d8cd98f00b204,justin.he@arm.com,:catalin.marinas@arm.com:will@kernel.org:mark.rutland@arm.com:james.morse@arm.com:maz@kernel.org:willy@infradead.org:kirill.shutemov@linux.intel.com:linux-arm-kernel@lists.infradead.org:linux-kernel@vger.kernel.org::punitagrawal@gmail.com:tglx@linutronix.de:akpm@linux-foundation.org:hejianet@gmail.com:kaly.xin@arm.com:justin.he@arm.com,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1345:1359:1437:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:2693:2901:3138:3139:3140:3141:3142:3352:3865:3867:3868:3871:3872:3874:5007:6119:6261:7903:8634:10004:11026:11657:11658:11914:12043:12048:12114:12297:12438:12555:12895:13069:13161:13229:13311:13357:14096:14181:14384:14394:14721:21080:21451:21627:21795:30051:30054,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0: 0:0,LFti X-HE-Tag: page43_63582ba73db52 X-Filterd-Recvd-Size: 2965 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Mon, 30 Sep 2019 01:58:02 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AAC311597; Sun, 29 Sep 2019 18:58:01 -0700 (PDT) Received: from localhost.localdomain (entos-thunderx2-02.shanghai.arm.com [10.169.40.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id DEE5B3F706; Sun, 29 Sep 2019 18:57:57 -0700 (PDT) From: Jia He To: Catalin Marinas , Will Deacon , Mark Rutland , James Morse , Marc Zyngier , Matthew Wilcox , "Kirill A. Shutemov" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Punit Agrawal , Thomas Gleixner , Andrew Morton , hejianet@gmail.com, Kaly Xin , Jia He Subject: [PATCH v10 2/3] arm64: mm: implement arch_faults_on_old_pte() on arm64 Date: Mon, 30 Sep 2019 09:57:39 +0800 Message-Id: <20190930015740.84362-3-justin.he@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190930015740.84362-1-justin.he@arm.com> References: <20190930015740.84362-1-justin.he@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On arm64 without hardware Access Flag, copying fromuser will fail because the pte is old and cannot be marked young. So we always end up with zeroed page after fork() + CoW for pfn mappings. we don't always have a hardware-managed access flag on arm64. Hence implement arch_faults_on_old_pte on arm64 to indicate that it might cause page fault when accessing old pte. Signed-off-by: Jia He Reviewed-by: Catalin Marinas --- arch/arm64/include/asm/pgtable.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 7576df00eb50..e96fb82f62de 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -885,6 +885,20 @@ static inline void update_mmu_cache(struct vm_area_struct *vma, #define phys_to_ttbr(addr) (addr) #endif +/* + * On arm64 without hardware Access Flag, copying from user will fail because + * the pte is old and cannot be marked young. So we always end up with zeroed + * page after fork() + CoW for pfn mappings. We don't always have a + * hardware-managed access flag on arm64. + */ +static inline bool arch_faults_on_old_pte(void) +{ + WARN_ON(preemptible()); + + return !cpu_has_hw_af(); +} +#define arch_faults_on_old_pte arch_faults_on_old_pte + #endif /* !__ASSEMBLY__ */ #endif /* __ASM_PGTABLE_H */ From patchwork Mon Sep 30 01:57:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jia He X-Patchwork-Id: 11165941 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E0EF313B1 for ; Mon, 30 Sep 2019 01:58:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A10AD21835 for ; Mon, 30 Sep 2019 01:58:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A10AD21835 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B045D6B0007; Sun, 29 Sep 2019 21:58:09 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A8EC06B0008; Sun, 29 Sep 2019 21:58:09 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 958966B000A; Sun, 29 Sep 2019 21:58:09 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0187.hostedemail.com [216.40.44.187]) by kanga.kvack.org (Postfix) with ESMTP id 6D67A6B0007 for ; Sun, 29 Sep 2019 21:58:09 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 22405181AC9B6 for ; Mon, 30 Sep 2019 01:58:09 +0000 (UTC) X-FDA: 75989926698.12.mom35_63f1090f31604 X-Spam-Summary: 2,0,0,157f0899718cc761,d41d8cd98f00b204,justin.he@arm.com,:catalin.marinas@arm.com:will@kernel.org:mark.rutland@arm.com:james.morse@arm.com:maz@kernel.org:willy@infradead.org:kirill.shutemov@linux.intel.com:linux-arm-kernel@lists.infradead.org:linux-kernel@vger.kernel.org::punitagrawal@gmail.com:tglx@linutronix.de:akpm@linux-foundation.org:hejianet@gmail.com:kaly.xin@arm.com:justin.he@arm.com,RULES_HIT:2:41:355:379:541:800:960:967:973:988:989:1042:1260:1261:1345:1359:1437:1535:1605:1606:1730:1747:1777:1792:2194:2199:2393:2525:2559:2563:2682:2685:2693:2859:2901:2918:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4118:4250:4321:5007:6117:6119:6261:6671:7208:7903:8634:8660:9025:9108:10004:11026:11473:11658:11914:12043:12048:12291:12295:12296:12297:12438:12555:12683:12895:12986:13007:13148:13161:13221:13229:13230:14096:14394:21080:21451:21627:21772:2179 5:30003: X-HE-Tag: mom35_63f1090f31604 X-Filterd-Recvd-Size: 7250 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Mon, 30 Sep 2019 01:58:06 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DF4C115A2; Sun, 29 Sep 2019 18:58:05 -0700 (PDT) Received: from localhost.localdomain (entos-thunderx2-02.shanghai.arm.com [10.169.40.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 1C5563F706; Sun, 29 Sep 2019 18:58:01 -0700 (PDT) From: Jia He To: Catalin Marinas , Will Deacon , Mark Rutland , James Morse , Marc Zyngier , Matthew Wilcox , "Kirill A. Shutemov" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Punit Agrawal , Thomas Gleixner , Andrew Morton , hejianet@gmail.com, Kaly Xin , Jia He Subject: [PATCH v10 3/3] mm: fix double page fault on arm64 if PTE_AF is cleared Date: Mon, 30 Sep 2019 09:57:40 +0800 Message-Id: <20190930015740.84362-4-justin.he@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190930015740.84362-1-justin.he@arm.com> References: <20190930015740.84362-1-justin.he@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When we tested pmdk unit test [1] vmmalloc_fork TEST1 in arm64 guest, there will be a double page fault in __copy_from_user_inatomic of cow_user_page. Below call trace is from arm64 do_page_fault for debugging purpose [ 110.016195] Call trace: [ 110.016826] do_page_fault+0x5a4/0x690 [ 110.017812] do_mem_abort+0x50/0xb0 [ 110.018726] el1_da+0x20/0xc4 [ 110.019492] __arch_copy_from_user+0x180/0x280 [ 110.020646] do_wp_page+0xb0/0x860 [ 110.021517] __handle_mm_fault+0x994/0x1338 [ 110.022606] handle_mm_fault+0xe8/0x180 [ 110.023584] do_page_fault+0x240/0x690 [ 110.024535] do_mem_abort+0x50/0xb0 [ 110.025423] el0_da+0x20/0x24 The pte info before __copy_from_user_inatomic is (PTE_AF is cleared): [ffff9b007000] pgd=000000023d4f8003, pud=000000023da9b003, pmd=000000023d4b3003, pte=360000298607bd3 As told by Catalin: "On arm64 without hardware Access Flag, copying from user will fail because the pte is old and cannot be marked young. So we always end up with zeroed page after fork() + CoW for pfn mappings. we don't always have a hardware-managed access flag on arm64." This patch fix it by calling pte_mkyoung. Also, the parameter is changed because vmf should be passed to cow_user_page() Add a WARN_ON_ONCE when __copy_from_user_inatomic() returns error in case there can be some obscure use-case.(by Kirill) [1] https://github.com/pmem/pmdk/tree/master/src/test/vmmalloc_fork Signed-off-by: Jia He Reported-by: Yibo Cai Reviewed-by: Catalin Marinas Acked-by: Kirill A. Shutemov --- mm/memory.c | 99 +++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 84 insertions(+), 15 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index b1ca51a079f2..1f56b0118ef5 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -118,6 +118,13 @@ int randomize_va_space __read_mostly = 2; #endif +#ifndef arch_faults_on_old_pte +static inline bool arch_faults_on_old_pte(void) +{ + return false; +} +#endif + static int __init disable_randmaps(char *s) { randomize_va_space = 0; @@ -2145,32 +2152,82 @@ static inline int pte_unmap_same(struct mm_struct *mm, pmd_t *pmd, return same; } -static inline void cow_user_page(struct page *dst, struct page *src, unsigned long va, struct vm_area_struct *vma) +static inline bool cow_user_page(struct page *dst, struct page *src, + struct vm_fault *vmf) { + bool ret; + void *kaddr; + void __user *uaddr; + bool force_mkyoung; + struct vm_area_struct *vma = vmf->vma; + struct mm_struct *mm = vma->vm_mm; + unsigned long addr = vmf->address; + debug_dma_assert_idle(src); + if (likely(src)) { + copy_user_highpage(dst, src, addr, vma); + return true; + } + /* * If the source page was a PFN mapping, we don't have * a "struct page" for it. We do a best-effort copy by * just copying from the original user address. If that * fails, we just zero-fill it. Live with it. */ - if (unlikely(!src)) { - void *kaddr = kmap_atomic(dst); - void __user *uaddr = (void __user *)(va & PAGE_MASK); + kaddr = kmap_atomic(dst); + uaddr = (void __user *)(addr & PAGE_MASK); + + /* + * On architectures with software "accessed" bits, we would + * take a double page fault, so mark it accessed here. + */ + force_mkyoung = arch_faults_on_old_pte() && !pte_young(vmf->orig_pte); + if (force_mkyoung) { + pte_t entry; + + vmf->pte = pte_offset_map_lock(mm, vmf->pmd, addr, &vmf->ptl); + if (!likely(pte_same(*vmf->pte, vmf->orig_pte))) { + /* + * Other thread has already handled the fault + * and we don't need to do anything. If it's + * not the case, the fault will be triggered + * again on the same address. + */ + ret = false; + goto pte_unlock; + } + + entry = pte_mkyoung(vmf->orig_pte); + if (ptep_set_access_flags(vma, addr, vmf->pte, entry, 0)) + update_mmu_cache(vma, addr, vmf->pte); + } + /* + * This really shouldn't fail, because the page is there + * in the page tables. But it might just be unreadable, + * in which case we just give up and fill the result with + * zeroes. + */ + if (__copy_from_user_inatomic(kaddr, uaddr, PAGE_SIZE)) { /* - * This really shouldn't fail, because the page is there - * in the page tables. But it might just be unreadable, - * in which case we just give up and fill the result with - * zeroes. + * Give a warn in case there can be some obscure + * use-case */ - if (__copy_from_user_inatomic(kaddr, uaddr, PAGE_SIZE)) - clear_page(kaddr); - kunmap_atomic(kaddr); - flush_dcache_page(dst); - } else - copy_user_highpage(dst, src, va, vma); + WARN_ON_ONCE(1); + clear_page(kaddr); + } + + ret = true; + +pte_unlock: + if (force_mkyoung) + pte_unmap_unlock(vmf->pte, vmf->ptl); + kunmap_atomic(kaddr); + flush_dcache_page(dst); + + return ret; } static gfp_t __get_fault_gfp_mask(struct vm_area_struct *vma) @@ -2327,7 +2384,19 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) vmf->address); if (!new_page) goto oom; - cow_user_page(new_page, old_page, vmf->address, vma); + + if (!cow_user_page(new_page, old_page, vmf)) { + /* + * COW failed, if the fault was solved by other, + * it's fine. If not, userspace would re-fault on + * the same address and we will handle the fault + * from the second attempt. + */ + put_page(new_page); + if (old_page) + put_page(old_page); + return 0; + } } if (mem_cgroup_try_charge_delay(new_page, mm, GFP_KERNEL, &memcg, false))