From patchwork Thu Jul 12 07:36:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 10521137 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 285926028E for ; Thu, 12 Jul 2018 07:36:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DB10F294A7 for ; Thu, 12 Jul 2018 07:36:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CEBA2294AB; Thu, 12 Jul 2018 07:36:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5FF53294A7 for ; Thu, 12 Jul 2018 07:36:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 956236B000D; Thu, 12 Jul 2018 03:36:43 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8DD546B000E; Thu, 12 Jul 2018 03:36:43 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 781C46B0010; Thu, 12 Jul 2018 03:36:43 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf0-f199.google.com (mail-pf0-f199.google.com [209.85.192.199]) by kanga.kvack.org (Postfix) with ESMTP id 3052D6B000D for ; Thu, 12 Jul 2018 03:36:43 -0400 (EDT) Received: by mail-pf0-f199.google.com with SMTP id f9-v6so16865436pfn.22 for ; Thu, 12 Jul 2018 00:36:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id; bh=ghUSxcLT8+56FiqKCN6Yqqgm6Xx8XeRTIoYlQ+pa4mY=; b=DknySxQ3VH2Ww6Cq9VVhcPUI3Y7iebHQ4LEwrlxBDpy+nZYu2GcULTHtBlXaGdUr+F J+qOkDAniSYfqxVFRZGPgxvRvs9rk/h1nwmiceIO/UHjJAACOQ8Jb/PVkHft65SR9TOI yNCsQUSdyNRYR5Yes76KiI0NXl5oiKnDMm2BeMuBswjDopNvRHPmZUHpllVI5A6Z1FKb lQs12c+YEOovlayazUUt8bZAPME22P67XIVf6o6g3aSQMdfRXC7sWPvC7X3Fv+SQy7kx c5qHJWX9yvs5tBBao3XjnmJibltpdDgbfroN9M13NqFMRCNbzY5PFU4c3pLLOyBO/mLR Mxrw== X-Gm-Message-State: AOUpUlGf/QFKH2CCiSRjX7oTFlXTFvWMOeDbl1GKszb/vG95mqCKihsw OV+aWewpagmV3Te5FeDDb2zW9dJ2DhJ/0bqGZOB8/cDcBO1teF7ERubrqWEJEsidi8GnAM5391x 90tLYUIILXMDsOgrNrrtoFD28lDiHq3uwVL5sNtl3VW+dCKXL5iz+OF7ad5cqVwgUkLJJgvSsIh pwvPeQ9t5NbCigSe50OZUAle06xzvvm0Y/ikma8tf8NYNq5QTNF2XATpikFhfB8Xz4mX4TN5y8X N5UoAcssPN+dGgz+MJW5+u5LcgS3xu3VNfc46dYRNlP11S2Hv0u0/dzbUxomQy8Z74fTiiLJmFm YogVMvELwwEd8gLLs5n1Aszw1dSL7tvhmSoY3q6h5g7FkpROC/Cr13W1eteHvei8dkCFKZkjnL4 H X-Received: by 2002:a17:902:3381:: with SMTP id b1-v6mr1120121plc.248.1531381002870; Thu, 12 Jul 2018 00:36:42 -0700 (PDT) X-Received: by 2002:a17:902:3381:: with SMTP id b1-v6mr1120087plc.248.1531381002083; Thu, 12 Jul 2018 00:36:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1531381002; cv=none; d=google.com; s=arc-20160816; b=XZpyKJoaNb5QVpw1o4JUgyJQAjRQjUa3zkvK65y2sNKCk+CN49TDrXsndB7ZWTOIc9 ssSk7g8ZAPTbaiJqOOeEkjk7E1128um9kBLMkDOMPeKbJ1njCNgOXZmUBwVUNvwZNuwd vOW2wtOgFJ8kzzYPT7Muv46bFHf8M5AUXvkNYZ2iKphQs1j2UjjJJiB4qgVy3mU78tdY xvvY1jv8cgfgtAMFiK5ulXc1upOOOEIy0qk12W0OPSmE8Ilv0zPxtTsQ4d5Z+kT8yOjg zYxYDcqYh2JFrmUv4yNePb2c2DbnQsLU88X+sUNc/dxjBItmQg0eKBq+wbOeRQljDCyA r2wQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=ghUSxcLT8+56FiqKCN6Yqqgm6Xx8XeRTIoYlQ+pa4mY=; b=s8pbNfyWPartITpyxs98iHO2poA6bMNgC8Zz7B0yn3YPTwI0To/uo5CaCJQg9seYXO LLqe9hAlAaaIstMg5IL/g6MCPKnPkZB/sHsAjcXofRrf1xw4W0R2AhE6eeaQ0IVk565T lfxNauu3nCUW9c7Mo+Q0MvNMFvf8QSBP7YZV3vDGD+TzIWjTn6bl/OlaT0LrbZ9Mfy/d VBO1TXay7QEY0KhY0HGIjUwEsf+o42PlNGgTMg5v05mzLQfe3TET2KO2b5sTrLjoVMhY IL2pjDHV/dEIQ8hfrLzQbSVEgkTi1xLjoozgu/0fB30Wa7aNcSsnD2tXFdK2KKOP6QuQ qS3g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=H2pgq8oF; spf=pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=npiggin@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id a8-v6sor6954857plp.49.2018.07.12.00.36.41 for (Google Transport Security); Thu, 12 Jul 2018 00:36:42 -0700 (PDT) Received-SPF: pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=H2pgq8oF; spf=pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=npiggin@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=ghUSxcLT8+56FiqKCN6Yqqgm6Xx8XeRTIoYlQ+pa4mY=; b=H2pgq8oFJfDQyh0uyKgOp4I661UDXI62fc9eeVUpHr9NqcOgYq7SQRjDwg8GcVSqyD 0ce8GHNUNvon1ZByLzbcfUsRIV/Qj+WIpvRPtgOqdE7DCwEBqPFmJB7cHbjDWQQJs9ff bof3MRBftnR0ynLBPa51wbqU8lSARhXyc0Z7IglrgAU50bEVZCklAxbnDmVDLf/gu+f4 ZIxGVoPN5Sgm3isjVeXS6GqK6VqiJ2uzUmvvTvyDwdBHKZcuEyJZL5xg8GK3Jv51HcNQ Ph0HC1+Ok92egp4pNkQ50SRHqWLYuSECsqfeUdgL/ZXuax0G/3Y8EM0G0maZNykTJD7J ABkA== X-Google-Smtp-Source: AAOMgpcNXAW3oIY9o0hvER5dVOGSsZUqpalYMgZEmlX/zAAcdHirMicnjA2Z0vPfYvKcCEkhYvYyHQ== X-Received: by 2002:a17:902:6845:: with SMTP id f5-v6mr1079611pln.173.1531381001534; Thu, 12 Jul 2018 00:36:41 -0700 (PDT) Received: from roar.ozlabs.ibm.com ([122.99.82.10]) by smtp.gmail.com with ESMTPSA id z8-v6sm36143283pfg.24.2018.07.12.00.36.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Jul 2018 00:36:40 -0700 (PDT) From: Nicholas Piggin To: linux-mm@kvack.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org, linux-arch@vger.kernel.org Subject: [RFC PATCH] mm: optimise pte dirty/accessed bits handling in fork Date: Thu, 12 Jul 2018 17:36:33 +1000 Message-Id: <20180712073633.1702-1-npiggin@gmail.com> X-Mailer: git-send-email 2.17.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP fork clears dirty/accessed bits from new ptes in the child, even though the mapping allows such accesses. This logic has existed for ~ever, and certainly well before physical page reclaim and cleaning was not strongly tied to pte access state as it is today. Now that is the case, this access bit clearing logic does not do much. Other than this case, Linux is "eager" to set dirty/accessed bits when setting up mappings, which avoids micro-faults (and page faults on CPUs that implement these bits in software). With this patch, there are no cases I could instrument where dirty/accessed bits do not match the access permissions without memory pressure (and without more exotic things like migration). This speeds up a fork/exit microbenchmark by about 5% on POWER9 (which uses a software fault fallback mechanism to set these bits). I expect x86 CPUs will barely be noticable, but would be interesting to see. Other archs might care more, and anyway it's always good if we can remove code and make things a bit faster. I don't *think* I'm missing anything fundamental, but would be good to be sure. Comments? Thanks, Nick --- mm/huge_memory.c | 4 ++-- mm/memory.c | 10 +++++----- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1cd7c1a57a14..c1d41cad9aad 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -974,7 +974,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable); pmdp_set_wrprotect(src_mm, addr, src_pmd); - pmd = pmd_mkold(pmd_wrprotect(pmd)); + pmd = pmd_wrprotect(pmd); set_pmd_at(dst_mm, addr, dst_pmd, pmd); ret = 0; @@ -1065,7 +1065,7 @@ int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm, } pudp_set_wrprotect(src_mm, addr, src_pud); - pud = pud_mkold(pud_wrprotect(pud)); + pud = pud_wrprotect(pud); set_pud_at(dst_mm, addr, dst_pud, pud); ret = 0; diff --git a/mm/memory.c b/mm/memory.c index 7206a634270b..3fea40da3a58 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1023,12 +1023,12 @@ copy_one_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, } /* - * If it's a shared mapping, mark it clean in - * the child + * Child inherits dirty and young bits from parent. There is no + * point clearing them because any cleaning or aging has to walk + * all ptes anyway, and it will notice the bits set in the parent. + * Leaving them set avoids stalls and even page faults on CPUs that + * handle these bits in software. */ - if (vm_flags & VM_SHARED) - pte = pte_mkclean(pte); - pte = pte_mkold(pte); page = vm_normal_page(vma, addr, pte); if (page) {