From patchwork Wed Feb 5 12:53:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Thomas_Hellstr=C3=B6m_=28Intel=29?= X-Patchwork-Id: 11366289 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF692138D for ; Wed, 5 Feb 2020 12:54:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9F7C020702 for ; Wed, 5 Feb 2020 12:54:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=shipmail.org header.i=@shipmail.org header.b="iq0wNGkO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9F7C020702 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=shipmail.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A7B4C6B0007; Wed, 5 Feb 2020 07:54:30 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A06EA6B000E; Wed, 5 Feb 2020 07:54:30 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E2D76B0036; Wed, 5 Feb 2020 07:54:30 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0091.hostedemail.com [216.40.44.91]) by kanga.kvack.org (Postfix) with ESMTP id 3B7436B000E for ; Wed, 5 Feb 2020 07:54:30 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 0A55D2DFC for ; Wed, 5 Feb 2020 12:54:30 +0000 (UTC) X-FDA: 76456067100.06.men33_27308cdd13450 X-Spam-Summary: 2,0,0,cc2be5926e1e8d3c,d41d8cd98f00b204,thomas_os@shipmail.org,::dri-devel@lists.freedesktop.org:linux-kernel@vger.kernel.org:pv-drivers@vmware.com:linux-graphics-maintainer@vmware.com:thellstrom@vmware.com:akpm@linux-foundation.org:mhocko@suse.com:willy@infradead.org:kirill.shutemov@linux.intel.com:rcampbell@nvidia.com:jglisse@redhat.com:christian.koenig@amd.com:dan.j.williams@intel.com,RULES_HIT:41:152:355:379:541:800:960:968:973:988:989:1260:1261:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1543:1593:1594:1676:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3354:3865:3867:3868:3870:3871:3872:3874:4250:4419:5007:6119:6261:6653:6742:7576:7903:10004:10226:10400:11026:11232:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12679:12895:13180:13229:13894:14181:14394:14659:14721:21080:21451:21627:21939:21990:30003:30054:30064,0,RBL:213.80.101.70:@shipmail.org:.lbl8.mailshell.net-62.2.203.100 64.100.201.201,CacheIP:none,Bay esian:0. X-HE-Tag: men33_27308cdd13450 X-Filterd-Recvd-Size: 5961 Received: from ste-pvt-msa1.bahnhof.se (ste-pvt-msa1.bahnhof.se [213.80.101.70]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Wed, 5 Feb 2020 12:54:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by ste-pvt-msa1.bahnhof.se (Postfix) with ESMTP id CD4E03F695; Wed, 5 Feb 2020 13:54:27 +0100 (CET) Authentication-Results: ste-pvt-msa1.bahnhof.se; dkim=pass (1024-bit key; unprotected) header.d=shipmail.org header.i=@shipmail.org header.b=iq0wNGkO; dkim-atps=neutral X-Virus-Scanned: Debian amavisd-new at bahnhof.se X-Spam-Flag: NO X-Spam-Score: -2.099 X-Spam-Level: X-Spam-Status: No, score=-2.099 tagged_above=-999 required=6.31 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from ste-pvt-msa1.bahnhof.se ([127.0.0.1]) by localhost (ste-pvt-msa1.bahnhof.se [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IdEYKwXtpuB6; Wed, 5 Feb 2020 13:54:23 +0100 (CET) Received: from mail1.shipmail.org (h-205-35.A357.priv.bahnhof.se [155.4.205.35]) (Authenticated sender: mb878879) by ste-pvt-msa1.bahnhof.se (Postfix) with ESMTPA id A930B3F669; Wed, 5 Feb 2020 13:54:18 +0100 (CET) Received: from localhost.localdomain.localdomain (h-205-35.A357.priv.bahnhof.se [155.4.205.35]) by mail1.shipmail.org (Postfix) with ESMTPSA id 32FC1360413; Wed, 5 Feb 2020 13:54:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=shipmail.org; s=mail; t=1580907258; bh=b14LVZl/+itpO85NpGNOLmY5imFMUhPrKjnSv3EgZ4c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iq0wNGkOta0XVDOb4TsmUnEau5JJ6UnHcTcKtTptRJf9qO7ebH2AyH4imdZyDk5pA cDvaeW7pk2xtLSWFFCiGOAZgbJH4su10Ij45+oCozqPfgS7PhIUl0ndlqaQ3ywps3E kkw7Ip76PhyQnFUv/o06hpT5I45oPMNK6i5B9LcA= From: =?utf-8?q?Thomas_Hellstr=C3=B6m_=28VMware=29?= To: linux-mm@kvack.org, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org Cc: pv-drivers@vmware.com, linux-graphics-maintainer@vmware.com, Thomas Hellstrom , Andrew Morton , Michal Hocko , "Matthew Wilcox (Oracle)" , "Kirill A. Shutemov" , Ralph Campbell , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Dan Williams Subject: [PATCH v3 3/9] mm: Split huge pages on write-notify or COW Date: Wed, 5 Feb 2020 13:53:47 +0100 Message-Id: <20200205125353.2760-4-thomas_os@shipmail.org> X-Mailer: git-send-email 2.21.1 In-Reply-To: <20200205125353.2760-1-thomas_os@shipmail.org> References: <20200205125353.2760-1-thomas_os@shipmail.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Thomas Hellstrom We currently only do COW and write-notify on the PTE level, so if the huge_fault() handler returns VM_FAULT_FALLBACK on wp faults, split the huge pages and page-table entries. Also do this for huge PUDs if there is no huge_fault() handler and the vma is not anonymous, similar to how it's done for PMDs. Note that fs/dax.c does the splitting in the huge_fault() handler, but as huge_fault() is implemented by modules we need to consider whether to export the splitting functions for use in the modules or whether to try to keep calls in the core. Opt for the latter. A follow-up patch can remove the dax.c split_huge_pmd() if needed. Cc: Andrew Morton Cc: Michal Hocko Cc: "Matthew Wilcox (Oracle)" Cc: "Kirill A. Shutemov" Cc: Ralph Campbell Cc: "Jérôme Glisse" Cc: "Christian König" Cc: Dan Williams Signed-off-by: Thomas Hellstrom --- mm/memory.c | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 17aadc751e5c..4c49fe963e5c 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3880,11 +3880,14 @@ static inline vm_fault_t wp_huge_pmd(struct vm_fault *vmf, pmd_t orig_pmd) { if (vma_is_anonymous(vmf->vma)) return do_huge_pmd_wp_page(vmf, orig_pmd); - if (vmf->vma->vm_ops->huge_fault) - return vmf->vma->vm_ops->huge_fault(vmf, PE_SIZE_PMD); + if (vmf->vma->vm_ops->huge_fault) { + vm_fault_t ret = vmf->vma->vm_ops->huge_fault(vmf, PE_SIZE_PMD); - /* COW handled on pte level: split pmd */ - VM_BUG_ON_VMA(vmf->vma->vm_flags & VM_SHARED, vmf->vma); + if (!(ret & VM_FAULT_FALLBACK)) + return ret; + } + + /* COW or write-notify handled on pte level: split pmd. */ __split_huge_pmd(vmf->vma, vmf->pmd, vmf->address, false, NULL); return VM_FAULT_FALLBACK; @@ -3897,12 +3900,20 @@ static inline bool vma_is_accessible(struct vm_area_struct *vma) static vm_fault_t create_huge_pud(struct vm_fault *vmf) { -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && \ + defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) /* No support for anonymous transparent PUD pages yet */ if (vma_is_anonymous(vmf->vma)) - return VM_FAULT_FALLBACK; - if (vmf->vma->vm_ops->huge_fault) - return vmf->vma->vm_ops->huge_fault(vmf, PE_SIZE_PUD); + goto split; + if (vmf->vma->vm_ops->huge_fault) { + vm_fault_t ret = vmf->vma->vm_ops->huge_fault(vmf, PE_SIZE_PUD); + + if (!(ret & VM_FAULT_FALLBACK)) + return ret; + } +split: + /* COW or write-notify not handled on PUD level: split pud.*/ + __split_huge_pud(vmf->vma, vmf->pud, vmf->address); #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ return VM_FAULT_FALLBACK; }