From patchwork Wed Jul 25 15:52:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 10544243 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AF964139A for ; Wed, 25 Jul 2018 15:53:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9E5202A5C1 for ; Wed, 25 Jul 2018 15:53:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 91D5D2A5FF; Wed, 25 Jul 2018 15:53:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 30D3E2A5C1 for ; Wed, 25 Jul 2018 15:53:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C1BFB6B028C; Wed, 25 Jul 2018 11:53:02 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BA3C76B028D; Wed, 25 Jul 2018 11:53:02 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 95C406B0291; Wed, 25 Jul 2018 11:53:02 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id 4F2866B028C for ; Wed, 25 Jul 2018 11:53:02 -0400 (EDT) Received: by mail-pf1-f198.google.com with SMTP id j15-v6so989717pff.12 for ; Wed, 25 Jul 2018 08:53:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=gLcXLHSvMjLl1a5SqU+HvKEkz+b8ngxKOrFFxTPno9c=; b=L9n/ATa0af6oTRmCeEJtIDXk00hHhyogCRssPFFvxOe1o79kSw83kV+HpNLgbibDuR 2P4t/k6ijtip8Qi3ypZDuWG1djYUulZXkgpYBlzxCyFOcADtyT9+YJat5+mHRmLk8ISN BwLE9Jga2WNp2ezZlGzg1GOKL119saYrCGJMH2SdU4A+x8SrCWBA3YwRNwk2kGMq5SpF d8p1y/TEx220eAPkJZQAHZ7WDfT0gGtGdUllWf/SClN5Rvyx7PImdxB2g8bKoytvKdJQ 3/0KJD36xXFto4aQoU2UwjAuX8O9l3KWPCvcydFXVYvN7io/FaexyNHVnmQOQ36PfmRY w7rg== X-Gm-Message-State: AOUpUlGXGKgtHEltS2oc9mnu2IDFYC5CCqaV9Ow72U0RStlaW+NBQ8AR oU+/u16vZ0GHnJjog3JnYSECtid1BOKMP8Yeu814EAQYqD+6covvBqHLAADcD8H+UcYkhslnq5H 52yJ3nGPoXsGe+Ip6+pIZ3WiJcMZHxwvMFt9zCuv9hNtqBG3zfvDGkJfdd1S+VU/9CXAEVTf/x9 TdJzvrdBoNf/Y9cJ3PubbVrjg/T5NYypQE+eMMGCqm/8DXc5qNBNJ7grxoC90TqaEkZ4CvWybqA SLtG6p0v/dNmvs7TcFQ7GUT8Kqv9djbVmMA0HziZ68O0VpHruefHNGL/8WeIpOt+jU6ZgcuPBM/ BdxpSOfEzokaxxwK+ZagH6R71Dw6xWc2fGcvOJkcPwxfOFuoee1rHJLjCITlGGLyzphFwcMdv8o N X-Received: by 2002:a63:c902:: with SMTP id o2-v6mr21100152pgg.118.1532533982030; Wed, 25 Jul 2018 08:53:02 -0700 (PDT) X-Received: by 2002:a63:c902:: with SMTP id o2-v6mr21100122pgg.118.1532533981421; Wed, 25 Jul 2018 08:53:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1532533981; cv=none; d=google.com; s=arc-20160816; b=l6keGQO7oJiU7P4OJlHnYlvZZ7EtOOUYyVAccH5goJ590B/O8zvzNfkK/pwK0J675j l8vmA1taQOInY2RSceh4uj3rlsp2nCeRDqqc4W/mJxefCgKNWKr0IQgQ0ve+J7chSCuj q7tXaYxRN91IVCY/PJmz4fnVH1bAtK6GWhStqvDU54Y6sJphS2DxjneoYTcL5Zn5Rw1t fSDtbUXhrJXHIIRn1W+e1necLTcT2qTpTrNFaf3hdWT7PEcLSsdeaVTNwprmAi4CNOhn l77eo9ajsimLEAixCJkInm0oRfIu9tY/imRxds/ft9I0byInUA10cfUt2BdrTPv/Lk5U xfiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=gLcXLHSvMjLl1a5SqU+HvKEkz+b8ngxKOrFFxTPno9c=; b=xJAVTZ0xIhp9qWgDqZYbsiYZr6xnBVWxlXdCBp3R7uK3geti/cExU0yhuBdkInSbdE DevJqVfB3sAbNUuyMskpF4teJ3TZFgXjciGMahFTwyH0LYRof9n/JopD6s4G5+5zIJMW Grs+r9cRCmb2NQuzfZiF0XukFMtc1NvxdtB26Za/bKwszJuPvNPrbY8/auu8Cvf9XSk4 VPHq3GsL8juMuKeeekhBSXZyuD0qyPh9WK945mZhrMoqeyaqFdncMegVqfRaRJiI+HR/ +cGMHS5DgB7anWNC1s927ojFBUoayx20Qq+ovb4Zqr22JpHlPT9vVLLtgP4+zQgkCQfu 7m8Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=dp250E5Z; spf=pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=npiggin@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id h3-v6sor4121712pld.86.2018.07.25.08.53.01 for (Google Transport Security); Wed, 25 Jul 2018 08:53:01 -0700 (PDT) Received-SPF: pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=dp250E5Z; spf=pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=npiggin@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=gLcXLHSvMjLl1a5SqU+HvKEkz+b8ngxKOrFFxTPno9c=; b=dp250E5Z4av6TXwK2FiGOeTPADAjItKut5H7zFcF/d/fRqS0s7hknOVTU5+VMXfkaX 7g6kTdbtmk/9F1vjL0RBc0x4ii5PRtKgiqpLol4KYSCCMf34MNfVuHnVoRCmwW4BcCwI S7jqxfOjmvOEbfKDt700G6izbgGXRzHoMRvCws4TQ3Ep8jLrrpEggfkwcGJ5D/Vswtok QWz/ISpB5YCqt6DgFlm+wNse/+Uafm2BuIh5Fw5zSjVYfWo0zGjtcdmAQ2snYolKEEVt 4Uh7ww5zAgPc8impdYT5CsrefOveyK50E+Sfg2bQKE47FLZo1lmOOkKZMpqFAmzJSS17 nWRw== X-Google-Smtp-Source: AAOMgpd85CNFa7uhnNYwyQIBOy3EvOvHYch1sIdeIaHCb3oR5VPQYm8BX8WmRkAFxDf6mENjqT64gw== X-Received: by 2002:a17:902:758c:: with SMTP id j12-v6mr21455822pll.195.1532533981052; Wed, 25 Jul 2018 08:53:01 -0700 (PDT) Received: from roar.au.ibm.com ([61.69.188.107]) by smtp.gmail.com with ESMTPSA id j72-v6sm23438315pge.19.2018.07.25.08.52.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Jul 2018 08:53:00 -0700 (PDT) From: Nicholas Piggin To: linux-mm@kvack.org Cc: Nicholas Piggin , linux-arch@vger.kernel.org Subject: [RFC PATCH 2/4] mm: zap_pte_range only flush under ptl if a dirty shared page was unmapped Date: Thu, 26 Jul 2018 01:52:44 +1000 Message-Id: <20180725155246.1085-3-npiggin@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180725155246.1085-1-npiggin@gmail.com> References: <20180725155246.1085-1-npiggin@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The force_flush is used for two cases, a tlb batch full, and a shared dirty page unmapped. Only the latter is required to flush the TLB under the page table lock, because the problem is page_mkclean returning when there are still writable TLB entries the page can be modified with. We are encountering cases of soft lockups due to high TLB flush latency with very large guests. There is probably some contetion in hypervisor and interconnect tuning to be done, and it's actually a hash MMU guest which has a whole other set of issues, but I'm looking for general ways to reduce TLB fushing under locks. --- mm/memory.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 773d588b371d..1161ed3f1d0b 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1281,6 +1281,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, { struct mm_struct *mm = tlb->mm; int force_flush = 0; + int locked_flush = 0; int rss[NR_MM_COUNTERS]; spinlock_t *ptl; pte_t *start_pte; @@ -1322,6 +1323,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, if (!PageAnon(page)) { if (pte_dirty(ptent)) { force_flush = 1; + locked_flush = 1; set_page_dirty(page); } if (pte_young(ptent) && @@ -1384,7 +1386,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, arch_leave_lazy_mmu_mode(); /* Do the actual TLB flush before dropping ptl */ - if (force_flush) + if (locked_flush) tlb_flush_mmu_tlbonly(tlb); pte_unmap_unlock(start_pte, ptl); @@ -1395,8 +1397,12 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, * memory too. Restart if we didn't do everything. */ if (force_flush) { - force_flush = 0; + if (!locked_flush) + tlb_flush_mmu_tlbonly(tlb); tlb_flush_mmu_free(tlb); + + force_flush = 0; + locked_flush = 0; if (addr != end) goto again; }