From patchwork Fri Jan 24 09:09:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Thomas_Hellstr=C3=B6m_=28Intel=29?= X-Patchwork-Id: 11349927 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C7517924 for ; Fri, 24 Jan 2020 09:09:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 855C720838 for ; Fri, 24 Jan 2020 09:09:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=shipmail.org header.i=@shipmail.org header.b="lmfX8Jm+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 855C720838 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=shipmail.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id ADA0A6B02BF; Fri, 24 Jan 2020 04:09:55 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A554D6B02BA; Fri, 24 Jan 2020 04:09:55 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8CEBB6B02BF; Fri, 24 Jan 2020 04:09:55 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0177.hostedemail.com [216.40.44.177]) by kanga.kvack.org (Postfix) with ESMTP id 6D3EC6B02BA for ; Fri, 24 Jan 2020 04:09:55 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 35E4683FD for ; Fri, 24 Jan 2020 09:09:55 +0000 (UTC) X-FDA: 76411955550.16.level50_9091d9f64f412 X-Spam-Summary: 2,0,0,476bd3be6f7f508c,d41d8cd98f00b204,thomas_os@shipmail.org,::dri-devel@lists.freedesktop.org:linux-kernel@vger.kernel.org:pv-drivers@vmware.com:linux-graphics-maintainer@vmware.com:thellstrom@vmware.com:akpm@linux-foundation.org:mhocko@suse.com:willy@infradead.org:kirill.shutemov@linux.intel.com:rcampbell@nvidia.com:jglisse@redhat.com:christian.koenig@amd.com:dan.j.williams@intel.com,RULES_HIT:41:152:355:379:541:800:960:968:973:988:989:1260:1261:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1543:1593:1594:1676:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3354:3865:3867:3868:3870:3871:3872:3874:4117:4250:4419:5007:6119:6261:6653:6742:7576:7903:10004:10226:10400:11026:11232:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12679:12895:13180:13229:13894:14181:14394:14659:14721:21080:21451:21627:21939:21990:30003:30054:30064,0,RBL:213.80.101.71:@shipmail.org:.lbl8.mailshell.net-62.2.203.100 64.100.201.201,CacheIP:non e,Bayesi X-HE-Tag: level50_9091d9f64f412 X-Filterd-Recvd-Size: 6074 Received: from ste-pvt-msa2.bahnhof.se (ste-pvt-msa2.bahnhof.se [213.80.101.71]) by imf29.hostedemail.com (Postfix) with ESMTP for ; Fri, 24 Jan 2020 09:09:53 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by ste-pvt-msa2.bahnhof.se (Postfix) with ESMTP id 681DF3F6EE; Fri, 24 Jan 2020 10:09:51 +0100 (CET) Authentication-Results: ste-pvt-msa2.bahnhof.se; dkim=pass (1024-bit key; unprotected) header.d=shipmail.org header.i=@shipmail.org header.b=lmfX8Jm+; dkim-atps=neutral X-Virus-Scanned: Debian amavisd-new at bahnhof.se X-Spam-Flag: NO X-Spam-Score: -2.099 X-Spam-Level: X-Spam-Status: No, score=-2.099 tagged_above=-999 required=6.31 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ste-ftg-msa2.bahnhof.se (amavisd-new); dkim=pass (1024-bit key) header.d=shipmail.org Received: from ste-pvt-msa2.bahnhof.se ([127.0.0.1]) by localhost (ste-ftg-msa2.bahnhof.se [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ja-EDtmmEIsb; Fri, 24 Jan 2020 10:09:50 +0100 (CET) Received: from mail1.shipmail.org (h-205-35.A357.priv.bahnhof.se [155.4.205.35]) (Authenticated sender: mb878879) by ste-pvt-msa2.bahnhof.se (Postfix) with ESMTPA id 524263F5D8; Fri, 24 Jan 2020 10:09:48 +0100 (CET) Received: from localhost.localdomain.localdomain (h-205-35.A357.priv.bahnhof.se [155.4.205.35]) by mail1.shipmail.org (Postfix) with ESMTPSA id E914F360653; Fri, 24 Jan 2020 10:09:47 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=shipmail.org; s=mail; t=1579856988; bh=3/LLclKt7+aCOVoEUoZ2rgY+/+paTMjeNbg1mLjGtXI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lmfX8Jm+eY3Ry7JJWrVaHib0VAmvysHOsKTd9EAgG/hvEs5eV4BZxhxGhZkhc3KUb qjticevxCPxJ7uQTZL5bBAT/UuQbgXOtIDKWZyMumrDy3cVT4UDntUaK2AQro++6tB l+ZCGTshLQUO4igVhjWCh145hJkc3nYcxQvdNfJY= From: =?utf-8?q?Thomas_Hellstr=C3=B6m_=28VMware=29?= To: linux-mm@kvack.org, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org Cc: pv-drivers@vmware.com, linux-graphics-maintainer@vmware.com, Thomas Hellstrom , Andrew Morton , Michal Hocko , "Matthew Wilcox (Oracle)" , "Kirill A. Shutemov" , Ralph Campbell , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Dan Williams Subject: [PATCH 3/9] mm: Split huge pages on write-notify or COW Date: Fri, 24 Jan 2020 10:09:34 +0100 Message-Id: <20200124090940.26571-4-thomas_os@shipmail.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200124090940.26571-1-thomas_os@shipmail.org> References: <20200124090940.26571-1-thomas_os@shipmail.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Thomas Hellstrom We currently only do COW and write-notify on the PTE level, so if the huge_fault() handler returns VM_FAULT_FALLBACK on wp faults, split the huge pages and page-table entries. Also do this for huge PUDs if there is no huge_fault() handler and the vma is not anonymous, similar to how it's done for PMDs. Note that fs/dax.c does the splitting in the huge_fault() handler, but as huge_fault() is implemented by modules we need to consider whether to export the splitting functions for use in the modules or whether to try to keep calls in the core. Opt for the latter. A follow-up patch can remove the dax.c split_huge_pmd() if needed. Cc: Andrew Morton Cc: Michal Hocko Cc: "Matthew Wilcox (Oracle)" Cc: "Kirill A. Shutemov" Cc: Ralph Campbell Cc: "Jérôme Glisse" Cc: "Christian König" Cc: Dan Williams Signed-off-by: Thomas Hellstrom --- mm/memory.c | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 17aadc751e5c..4c49fe963e5c 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3880,11 +3880,14 @@ static inline vm_fault_t wp_huge_pmd(struct vm_fault *vmf, pmd_t orig_pmd) { if (vma_is_anonymous(vmf->vma)) return do_huge_pmd_wp_page(vmf, orig_pmd); - if (vmf->vma->vm_ops->huge_fault) - return vmf->vma->vm_ops->huge_fault(vmf, PE_SIZE_PMD); + if (vmf->vma->vm_ops->huge_fault) { + vm_fault_t ret = vmf->vma->vm_ops->huge_fault(vmf, PE_SIZE_PMD); - /* COW handled on pte level: split pmd */ - VM_BUG_ON_VMA(vmf->vma->vm_flags & VM_SHARED, vmf->vma); + if (!(ret & VM_FAULT_FALLBACK)) + return ret; + } + + /* COW or write-notify handled on pte level: split pmd. */ __split_huge_pmd(vmf->vma, vmf->pmd, vmf->address, false, NULL); return VM_FAULT_FALLBACK; @@ -3897,12 +3900,20 @@ static inline bool vma_is_accessible(struct vm_area_struct *vma) static vm_fault_t create_huge_pud(struct vm_fault *vmf) { -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && \ + defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) /* No support for anonymous transparent PUD pages yet */ if (vma_is_anonymous(vmf->vma)) - return VM_FAULT_FALLBACK; - if (vmf->vma->vm_ops->huge_fault) - return vmf->vma->vm_ops->huge_fault(vmf, PE_SIZE_PUD); + goto split; + if (vmf->vma->vm_ops->huge_fault) { + vm_fault_t ret = vmf->vma->vm_ops->huge_fault(vmf, PE_SIZE_PUD); + + if (!(ret & VM_FAULT_FALLBACK)) + return ret; + } +split: + /* COW or write-notify not handled on PUD level: split pud.*/ + __split_huge_pud(vmf->vma, vmf->pud, vmf->address); #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ return VM_FAULT_FALLBACK; }