From patchwork Tue Aug 28 11:20:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 10578283 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AB9F7174C for ; Tue, 28 Aug 2018 11:21:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9B80A299CC for ; Tue, 28 Aug 2018 11:21:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8DD29297DB; Tue, 28 Aug 2018 11:21:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0A801297DB for ; Tue, 28 Aug 2018 11:21:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 010156B45EE; Tue, 28 Aug 2018 07:20:59 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F01066B45F0; Tue, 28 Aug 2018 07:20:58 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA2DE6B45F1; Tue, 28 Aug 2018 07:20:58 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) by kanga.kvack.org (Postfix) with ESMTP id 90B2E6B45EE for ; Tue, 28 Aug 2018 07:20:58 -0400 (EDT) Received: by mail-pl1-f200.google.com with SMTP id 2-v6so584008plc.11 for ; Tue, 28 Aug 2018 04:20:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=u5LLVwzqsd0W3khip77eczbmhBS8m0M7MliDfum5fbo=; b=lpJE/cCxp+swSmhNUjRHWD7Gw30dIpiwG1IDLi+yqcCdVHHaqDwaJjy2EXfsoT2jF6 GjznfnMX59yNuG2VfX51mu3P1jTWWj1aDsaokfM5xK3/cW6KtyJVvOX+/L+ZF+ZBIsi6 2iqdYmDkL9Q0JmN+0uNSdgylYEhbpFMJYRXw2TWMnK6nXMaOrPxBhj7a7/NTJrB10TtW v1hRRnXssyA+DvWrIWBMEz+okKOZNJiZmnTlvssz7g9DOOS8C2cs1+3KoiLgHeLN6Jiu JJUdJr7114IY5TOdxQPtOL9p2Piousuja3LjoWK4hRYkoqCi9VT1GmuKRNV9LDl0Hyzo ++dg== X-Gm-Message-State: APzg51DKUCvFJhCqOO8juPwyNu4wiE0bddHGJm7SOjcI7RoLZqN4a5Uz SrvTc7XAv8xEsvRCSnu5+H8pAjDPfhZKPdhOmmmIoHJqxjWjFD7KCy6KRun5TW8q4qtLqiJf3Z+ MEmOlB+cfZVRGSOhXs0fyeX7vxOMQNHwArAFKfMvwcyNIe/bhtXIW1Hnt90vpLCP39f6IXLvTd8 embUtSDovc7AvTf4HeqYqFMcF8UwgqOKMrkfKixWFLzTYNJVH3Z5cgb4rTBc/O/OHQgepPi/4cr +cONzku2fWLYrT/w/ZLlw/+l60Ho3phhB//pPZkwWgPv3UB0ebStlyEYlYaMjjxEEHtHefsE/Ic foH+djOg7AC0FL0Kwu2xo1ohGKc0mAdr3XuBtZaNDHEOQIBP7A3lj9oqLZRPb0Egf6YwRHi0/zk R X-Received: by 2002:a17:902:48c8:: with SMTP id u8-v6mr1143130plh.152.1535455258266; Tue, 28 Aug 2018 04:20:58 -0700 (PDT) X-Received: by 2002:a17:902:48c8:: with SMTP id u8-v6mr1143078plh.152.1535455257491; Tue, 28 Aug 2018 04:20:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535455257; cv=none; d=google.com; s=arc-20160816; b=UcIgFewQBUdPV3qNqVKLdfGUcFmSDEagS7698QAWhgMH+f1c3Cv0HGOYGxUyVWVw3r lcyyFnZ+CW1uRnr7EbOgIXuwYddIEmXB7tOXUMyM5EmcXJUujuYrI0YEAo19BSsn3Knn OKGJL/vI081rPWaByqRmvKAHg4KkxiAYj3e1r42AEHZ71n41sT9qkt7S8tsOyr8xbasp QcZZZx9XIQdRXnX1y1z7T3jyds04xJ7ZkKUEP4VJqza6VCCAXUdRRjfwYC0VnyeYFCkQ g0hSvCYSqCSoQyaxoYIZ/Cr4YS2qx7oFEPHwhTtoowg59u55TR5U1MQcz+npTRiQUduB 0m8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=u5LLVwzqsd0W3khip77eczbmhBS8m0M7MliDfum5fbo=; b=k3XRGxdpMvNf9gTvuxFRExcIbVntpogAni1QJ0kn6ZrpSIovr3YIaBwTEUpyHjrWTl rY6navx5dCy/aJLVbJlzxjJjd57pqAevW2satIxKYySLrwdsKejzONME0B2s0mZHee9j rN5eyDZSLU76rSHfJBzJCBrld6GX53Be/QwWUeYig6P3Y05bVR/Xff0MED+tC/3aO/11 ZfncvGPgDOTvejgrn9v2lNtP65lkpyNyyVvq8KTO6Lh3m5j+so61x46zRxFwhZo3ZcdQ BmKleMQhBoLgS70Ghy2AHQVJaA0/LLrzYTYj3WdKQt+YA/Q7hfA0mm962jPYN0fRN84i 1aNg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=JwRKkOpx; spf=pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=npiggin@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id e2-v6sor263779pls.43.2018.08.28.04.20.57 for (Google Transport Security); Tue, 28 Aug 2018 04:20:57 -0700 (PDT) Received-SPF: pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=JwRKkOpx; spf=pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=npiggin@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=u5LLVwzqsd0W3khip77eczbmhBS8m0M7MliDfum5fbo=; b=JwRKkOpxwjV2ovQ7J2Ho5+UlDBygiVUre+hfgsY9ifuXf7lMftdoyq0EjG5Y4wV9jI edNRMwartMVSqXKLDu9KmlMzCUzsjm1SEnGMe2GKjqwu5rEkfsiRHxxqwBpQMsqp84uM 61qFr8Q4SUNZ52mt4/gmik+I7401vIvRBgcknLsZKC4trEXjn7k+zwZSc1V1G9+jFkyv BoW03rpPR4WSm6aaQuOxSQqpT2H0MJDPANu/Bs7ctr23p5f5jUTAAiVEgXGHIM0Ktpv1 1j6HNaUmy1Rnx0kpiaEMO7Ut3drfbl2yP3OpVKcWN5uUo30RDYFEChjPVH3nS0A+Qxzg 7TOw== X-Google-Smtp-Source: ANB0VdYR8Iq6a5/JlwAWF9iIuyrZ3FY50H+M9B7BfYKPAMTl0ScSRzxrBp5pb47wFpOFK+WGxxiTKg== X-Received: by 2002:a17:902:6845:: with SMTP id f5-v6mr1121540pln.17.1535455257066; Tue, 28 Aug 2018 04:20:57 -0700 (PDT) Received: from roar.au.ibm.com (59-102-81-67.tpgi.com.au. [59.102.81.67]) by smtp.gmail.com with ESMTPSA id s3-v6sm3287917pgj.84.2018.08.28.04.20.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 28 Aug 2018 04:20:56 -0700 (PDT) From: Nicholas Piggin To: linux-mm@kvack.org Cc: Nicholas Piggin , linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, Andrew Morton , Linus Torvalds Subject: [PATCH 3/3] mm: optimise pte dirty/accessed bit setting by demand based pte insertion Date: Tue, 28 Aug 2018 21:20:34 +1000 Message-Id: <20180828112034.30875-4-npiggin@gmail.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180828112034.30875-1-npiggin@gmail.com> References: <20180828112034.30875-1-npiggin@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Similarly to the previous patch, this tries to optimise dirty/accessed bits in ptes to avoid access costs of hardware setting them. Signed-off-by: Nicholas Piggin --- mm/huge_memory.c | 12 +++++++----- mm/memory.c | 8 +++++--- 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 5fb1a43e12e0..2c169041317f 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1197,6 +1197,7 @@ static vm_fault_t do_huge_pmd_wp_page_fallback(struct vm_fault *vmf, for (i = 0; i < HPAGE_PMD_NR; i++, haddr += PAGE_SIZE) { pte_t entry; entry = mk_pte(pages[i], vma->vm_page_prot); + entry = pte_mkyoung(entry); entry = maybe_mkwrite(pte_mkdirty(entry), vma); memcg = (void *)page_private(pages[i]); set_page_private(pages[i], 0); @@ -2067,7 +2068,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, struct page *page; pgtable_t pgtable; pmd_t old_pmd, _pmd; - bool young, write, soft_dirty, pmd_migration = false; + bool young, write, dirty, soft_dirty, pmd_migration = false; unsigned long addr; int i; @@ -2145,8 +2146,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, page = pmd_page(old_pmd); VM_BUG_ON_PAGE(!page_count(page), page); page_ref_add(page, HPAGE_PMD_NR - 1); - if (pmd_dirty(old_pmd)) - SetPageDirty(page); + dirty = pmd_dirty(old_pmd); write = pmd_write(old_pmd); young = pmd_young(old_pmd); soft_dirty = pmd_soft_dirty(old_pmd); @@ -2176,8 +2176,10 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, entry = maybe_mkwrite(entry, vma); if (!write) entry = pte_wrprotect(entry); - if (!young) - entry = pte_mkold(entry); + if (young) + entry = pte_mkyoung(entry); + if (dirty) + entry = pte_mkdirty(entry); if (soft_dirty) entry = pte_mksoft_dirty(entry); } diff --git a/mm/memory.c b/mm/memory.c index 3d8bf8220bd0..d205ba69918c 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1830,10 +1830,9 @@ static int insert_pfn(struct vm_area_struct *vma, unsigned long addr, entry = pte_mkspecial(pfn_t_pte(pfn, prot)); out_mkwrite: - if (mkwrite) { - entry = pte_mkyoung(entry); + entry = pte_mkyoung(entry); + if (mkwrite) entry = maybe_mkwrite(pte_mkdirty(entry), vma); - } set_pte_at(mm, addr, pte, entry); update_mmu_cache(vma, addr, pte); /* XXX: why not for insert_page? */ @@ -2560,6 +2559,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) } flush_cache_page(vma, vmf->address, pte_pfn(vmf->orig_pte)); entry = mk_pte(new_page, vma->vm_page_prot); + entry = pte_mkyoung(entry); entry = maybe_mkwrite(pte_mkdirty(entry), vma); /* * Clear the pte entry and flush it first, before updating the @@ -3069,6 +3069,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) inc_mm_counter_fast(vma->vm_mm, MM_ANONPAGES); dec_mm_counter_fast(vma->vm_mm, MM_SWAPENTS); pte = mk_pte(page, vma->vm_page_prot); + pte = pte_mkyoung(pte); if ((vmf->flags & FAULT_FLAG_WRITE) && reuse_swap_page(page, NULL)) { pte = maybe_mkwrite(pte_mkdirty(pte), vma); vmf->flags &= ~FAULT_FLAG_WRITE; @@ -3479,6 +3480,7 @@ vm_fault_t alloc_set_pte(struct vm_fault *vmf, struct mem_cgroup *memcg, flush_icache_page(vma, page); entry = mk_pte(page, vma->vm_page_prot); + entry = pte_mkyoung(entry); if (write) entry = maybe_mkwrite(pte_mkdirty(entry), vma); /* copy-on-write page */