From patchwork Mon Feb 3 15:43:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Thomas_Hellstr=C3=B6m_=28Intel=29?= X-Patchwork-Id: 11363027 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0E47B924 for ; Mon, 3 Feb 2020 15:43:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CE48821582 for ; Mon, 3 Feb 2020 15:43:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=shipmail.org header.i=@shipmail.org header.b="lDw4bzhC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CE48821582 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=shipmail.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 08E7A6B0682; Mon, 3 Feb 2020 10:43:19 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 03EB46B0683; Mon, 3 Feb 2020 10:43:18 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E96E96B0684; Mon, 3 Feb 2020 10:43:18 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0146.hostedemail.com [216.40.44.146]) by kanga.kvack.org (Postfix) with ESMTP id D1BC16B0682 for ; Mon, 3 Feb 2020 10:43:18 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 9F900824805A for ; Mon, 3 Feb 2020 15:43:18 +0000 (UTC) X-FDA: 76449234876.22.wash00_56e6728343446 X-Spam-Summary: 2,0,0,ddcea75cb050b9fa,d41d8cd98f00b204,thomas_os@shipmail.org,::linux-kernel@vger.kernel.org:thellstrom@vmware.com:akpm@linux-foundation.org:steven.price@arm.com,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1431:1437:1515:1535:1542:1676:1711:1730:1747:1777:1792:2393:2559:2562:2730:2895:3138:3139:3140:3141:3142:3354:3865:3866:3867:3868:3871:3874:4250:4321:4605:5007:6119:6261:6653:7576:7903:10004:11026:11658:11914:12043:12114:12291:12296:12297:12438:12517:12519:12555:12679:12683:12895:12986:13894:14096:14181:14394:14721:21063:21080:21212:21451:21627:21990:30012:30054:30070:30075,0,RBL:79.136.2.41:@shipmail.org:.lbl8.mailshell.net-62.14.203.100 64.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:30,LUA_SUMMARY:none X-HE-Tag: wash00_56e6728343446 X-Filterd-Recvd-Size: 5109 Received: from pio-pvt-msa2.bahnhof.se (pio-pvt-msa2.bahnhof.se [79.136.2.41]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Mon, 3 Feb 2020 15:43:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by pio-pvt-msa2.bahnhof.se (Postfix) with ESMTP id 70BB03F386; Mon, 3 Feb 2020 16:43:15 +0100 (CET) Authentication-Results: pio-pvt-msa2.bahnhof.se; dkim=pass (1024-bit key; unprotected) header.d=shipmail.org header.i=@shipmail.org header.b=lDw4bzhC; dkim-atps=neutral X-Virus-Scanned: Debian amavisd-new at bahnhof.se X-Spam-Flag: NO X-Spam-Score: -2.099 X-Spam-Level: X-Spam-Status: No, score=-2.099 tagged_above=-999 required=6.31 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from pio-pvt-msa2.bahnhof.se ([127.0.0.1]) by localhost (pio-pvt-msa2.bahnhof.se [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nV0aPQ4GvCBo; Mon, 3 Feb 2020 16:43:14 +0100 (CET) Received: from mail1.shipmail.org (h-205-35.A357.priv.bahnhof.se [155.4.205.35]) (Authenticated sender: mb878879) by pio-pvt-msa2.bahnhof.se (Postfix) with ESMTPA id C0FCD3F374; Mon, 3 Feb 2020 16:43:13 +0100 (CET) Received: from localhost.localdomain.localdomain (h-205-35.A357.priv.bahnhof.se [155.4.205.35]) by mail1.shipmail.org (Postfix) with ESMTPSA id 202013600E5; Mon, 3 Feb 2020 16:43:13 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=shipmail.org; s=mail; t=1580744593; bh=XlNdoAIJhpLSyjWqttR1EHRGL0pqRlRHRNorOyAJqJ8=; h=From:To:Cc:Subject:Date:From; b=lDw4bzhCl2B//23TdxB/WCEoTB7BQ0+kdara1T10lvEvnwRZRyjBZKtocQrzdtSB6 a4uJ9XBMY4T1+rNKOTOqk+H2glaGG7hKVqcn1eksu0CtrOkrd5aKO/dZ3KfdL2RcwP P3Noxm3r0uayKt/Pvb0keFvZZWgeCIXbYVPynHyk= From: =?utf-8?q?Thomas_Hellstr=C3=B6m_=28VMware=29?= To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Thomas Hellstrom , Andrew Morton , Steven Price Subject: [PATCH] mm/mapping_dirty_helpers: Update huge page-table entry callbacks Date: Mon, 3 Feb 2020 16:43:05 +0100 Message-Id: <20200203154305.15045-1-thomas_os@shipmail.org> X-Mailer: git-send-email 2.21.1 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Thomas Hellstrom Following the update of pagewalk code commit a07984d48146 ("mm: pagewalk: add p4d_entry() and pgd_entry()") we can modify the mapping_dirty_helpers' huge page-table entry callbacks to avoid splitting when a huge pud or -pmd is encountered. Cc: Andrew Morton Signed-off-by: Thomas Hellstrom Reviewed-by: Steven Price --- v2: Added R-B from Steven. --- mm/mapping_dirty_helpers.c | 42 ++++++++++++++++++++++++++++++++++---- 1 file changed, 38 insertions(+), 4 deletions(-) diff --git a/mm/mapping_dirty_helpers.c b/mm/mapping_dirty_helpers.c index 71070dda9643..2c7d03675903 100644 --- a/mm/mapping_dirty_helpers.c +++ b/mm/mapping_dirty_helpers.c @@ -111,26 +111,60 @@ static int clean_record_pte(pte_t *pte, unsigned long addr, return 0; } -/* wp_clean_pmd_entry - The pagewalk pmd callback. */ +/* + * wp_clean_pmd_entry - The pagewalk pmd callback. + * + * Dirty-tracking should take place on the PTE level, so + * WARN() if encountering a dirty huge pmd. + * Furthermore, never split huge pmds, since that currently + * causes dirty info loss. The pagefault handler should do + * that if needed. + */ static int wp_clean_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long end, struct mm_walk *walk) { - /* Dirty-tracking should be handled on the pte level */ pmd_t pmdval = pmd_read_atomic(pmd); + if (!pmd_trans_unstable(&pmdval)) + return 0; + + if (pmd_none(pmdval)) { + walk->action = ACTION_AGAIN; + return 0; + } + + /* Huge pmd, present or migrated */ + walk->action = ACTION_CONTINUE; if (pmd_trans_huge(pmdval) || pmd_devmap(pmdval)) WARN_ON(pmd_write(pmdval) || pmd_dirty(pmdval)); return 0; } -/* wp_clean_pud_entry - The pagewalk pud callback. */ +/* + * wp_clean_pud_entry - The pagewalk pud callback. + * + * Dirty-tracking should take place on the PTE level, so + * WARN() if encountering a dirty huge puds. + * Furthermore, never split huge puds, since that currently + * causes dirty info loss. The pagefault handler should do + * that if needed. + */ static int wp_clean_pud_entry(pud_t *pud, unsigned long addr, unsigned long end, struct mm_walk *walk) { - /* Dirty-tracking should be handled on the pte level */ pud_t pudval = READ_ONCE(*pud); + if (!pud_trans_unstable(&pudval)) + return 0; + + if (pud_none(pudval)) { + walk->action = ACTION_AGAIN; + return 0; + } + + /* Huge pud */ + walk->action = ACTION_CONTINUE; if (pud_trans_huge(pudval) || pud_devmap(pudval)) WARN_ON(pud_write(pudval) || pud_dirty(pudval));