From patchwork Thu Aug 23 08:47:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 10573569 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6537B112E for ; Thu, 23 Aug 2018 08:47:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 50EA22BA27 for ; Thu, 23 Aug 2018 08:47:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 44F7D2BA41; Thu, 23 Aug 2018 08:47:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 24FC22BA27 for ; Thu, 23 Aug 2018 08:47:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C2D3D6B2926; Thu, 23 Aug 2018 04:47:35 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BD7936B2927; Thu, 23 Aug 2018 04:47:35 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A2DAE6B2928; Thu, 23 Aug 2018 04:47:35 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by kanga.kvack.org (Postfix) with ESMTP id 51C786B2926 for ; Thu, 23 Aug 2018 04:47:35 -0400 (EDT) Received: by mail-pf1-f197.google.com with SMTP id c8-v6so2847425pfn.2 for ; Thu, 23 Aug 2018 01:47:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=vm5WJYEScMr7UWDWXPbr88R9HWjvqSW7wxj8DCyVjYs=; b=hMs0UYqmAYuOLrmGOfcNsydLo4EVbB5NzHfLiiDEH3CfKTTZAoBSvyN9RIwu9Jc35U hNBg4jnZkj/3iO/+mW2/8nU9b9UI0HsBgeyoRoOutiB1hfOkmTzMo3azka7Ue52nEdh/ Ax5MZwoua/IZvjd9hCFHvHbzMeVR2CPtk36BnufFEcw+we2Qa3hpHlLBonu9dm4KN+wV WUhPvK5rpCUn1/SB9bDggn7912+KJmODHlGpcPVEjIEEme+NnCos3HdzaKN8hnhLOUUk iZ9q5+JLQG6pwS21HfsjM/EaIdqd058nf2/wN2WZXNxkU7D24v/ccLot4fLGl2romTf6 tYig== X-Gm-Message-State: AOUpUlGO/W6LDDTunuVfQPKEMIet/n+J9O4ShYz50V6GEID8avzfwupU SYnkcEbD0TTJxynvBe60EkDuy7pAEaeY94XR+z1hgMGxhBHONLNMxxVqCroiOvmz8Uhs8Jhq5Ta B8tnyB/TIbvVU1fcL5EyJfaM/ofshSI6koZQGFX4fE0gZW0wuuY7exZ6RaLYt6sVkeiSPTUGG1o ES1rTTkSlYgUvxsF5VnLIixRXpMdNNd2AhisfN7Tz1UymMSmTzzkhtYJjhl2sUHD3qGV7D3k+GR XoR6ln5lbr6piHOQUy24ZsbRV1H/eOQo+LwBjuDIbmq21gRL5rpbw3aKms+V8mqW+6bD8bRgf/L sQVjrP24exRa0sjYGFZLSqHaLsT15MVvSns4hYPFjoX5kSWzUZbtlsBAK0kYEzvYzdnZxyqtR1M H X-Received: by 2002:a62:8559:: with SMTP id u86-v6mr62233762pfd.32.1535014054964; Thu, 23 Aug 2018 01:47:34 -0700 (PDT) X-Received: by 2002:a62:8559:: with SMTP id u86-v6mr62233723pfd.32.1535014054145; Thu, 23 Aug 2018 01:47:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535014054; cv=none; d=google.com; s=arc-20160816; b=SVG9459EtuClUImj1r+U5b+GqyWOqbluVS59DWFbrPxVnww0DYx2CotQNi5aBnprqC x1AotvFj6LMI0678ThioSPRWGTBBH05jdRtFvcxboCQoyo0yUnPniLKItz1VgMi78Veh /Z1gG7ydhUq8ffCZQYycSKAbEnPLiy0CZISTwrFreoho/TwMZwYaeblbQGCqZpc4tJvQ her9hjY/IW5XOEPv5oAcNr7DsHdJCPo1K/mrlXz1T8NjoQ9UTyCrH60v34dXJUe2y8zq zobH72VbSmg1VdpQ/IB+rRhfWvK7elPiGcKwo1F8l9cbc7lXBaxm/LSeVWryQ1h0UIQh /xYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=vm5WJYEScMr7UWDWXPbr88R9HWjvqSW7wxj8DCyVjYs=; b=EreiySvDT3pTflUw+Rnry9uc42Ayr5rlpbuTyn4YwFEuszYJX34FGj4V7gCG9bcm5d +M1M7Pcfi5Q5duBezlHemEv5EHap+tG8HkpCJB7ZW5xasmmwZ494p50yJwi5gi58aA1Q 2hYxGm8S1L9L5cJ+aoOV7kOylYb9loS0Nrkde14WkYedU3iZ3MB6wAnm9s/jbouvlTTP fwK+d45b6rMLNnc1hRCZ2GOLCBV0jYwgxf2L6lVAF9wmoMIgpCw6SQv6j3qK44YXgzpa 1uBPkEcz6JEOW4aqCXSb6MkKMesVdAka0qlGDbMMVNEl2acmtFCiMRdObIOOcSIi12EA N9Cg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=gEYBvPCj; spf=pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=npiggin@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id r37-v6sor735631pgb.330.2018.08.23.01.47.34 for (Google Transport Security); Thu, 23 Aug 2018 01:47:34 -0700 (PDT) Received-SPF: pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=gEYBvPCj; spf=pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=npiggin@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vm5WJYEScMr7UWDWXPbr88R9HWjvqSW7wxj8DCyVjYs=; b=gEYBvPCjdTml9yIgaT/W/ou3ACAqcWJwpC5ZL0E4EgrZoLqbUra2Rwg1DAYe8iKJhf zq3c95YV4k46qxBQyAKkdclcDfYUib1+cgop+zBiCUT1aekJgyWIbamjOYVe7O/+iXAv dOLLpwCgvXwZA2E8OixpCseW2sZuiHSRJBY6/fuOO0tBZ5/afFYhbAhwVul45WraIpvV HD75sXtJTgxrQ+i72/GsCGEUmKr5+KTPURlVS6aIMNqgVfAcIpT4Fzyt3IO0qOKyIFsa 4AXjJ3xm8omnlemZtY23Zhj3RYm1ae2mTuUPQszHgEJ1W6R9eM/3gSxzLjUE4K08Od5c 4yKg== X-Google-Smtp-Source: AA+uWPyGlyWnj3nMPrxY4SPnBTjWZjcKNxEdhGijGKj4PrMsrgME/v8IMpsaIZLBPJdZiyrMNgxmmA== X-Received: by 2002:a63:2e09:: with SMTP id u9-v6mr52325298pgu.294.1535014053904; Thu, 23 Aug 2018 01:47:33 -0700 (PDT) Received: from roar.ozlabs.ibm.com. ([122.99.82.10]) by smtp.gmail.com with ESMTPSA id w5-v6sm5170503pfn.44.2018.08.23.01.47.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 23 Aug 2018 01:47:33 -0700 (PDT) From: Nicholas Piggin To: Peter Zijlstra Cc: Nicholas Piggin , torvalds@linux-foundation.org, luto@kernel.org, x86@kernel.org, bp@alien8.de, will.deacon@arm.com, riel@surriel.com, jannh@google.com, ascannell@google.com, dave.hansen@intel.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, David Miller , Martin Schwidefsky , Michael Ellerman , linux-arch@vger.kernel.org Subject: [RFC PATCH 2/2] mm: mmu_notifier fix for tlb_end_vma Date: Thu, 23 Aug 2018 18:47:09 +1000 Message-Id: <20180823084709.19717-3-npiggin@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180823084709.19717-1-npiggin@gmail.com> References: <20180823084709.19717-1-npiggin@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The generic tlb_end_vma does not call invalidate_range mmu notifier, and it resets resets the mmu_gather range, which means the notifier won't be called on part of the range in case of an unmap that spans multiple vmas. ARM64 seems to be the only arch I could see that has notifiers and uses the generic tlb_end_vma. I have not actually tested it. Signed-off-by: Nicholas Piggin Acked-by: Will Deacon Reported-by: Guenter Roeck Signed-off-by: Will Deacon --- include/asm-generic/tlb.h | 17 +++++++++++++---- mm/memory.c | 10 ---------- 2 files changed, 13 insertions(+), 14 deletions(-) diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h index e811ef7b8350..79cb76b89c26 100644 --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -15,6 +15,7 @@ #ifndef _ASM_GENERIC__TLB_H #define _ASM_GENERIC__TLB_H +#include #include #include #include @@ -138,6 +139,16 @@ static inline void __tlb_reset_range(struct mmu_gather *tlb) } } +static inline void tlb_flush_mmu_tlbonly(struct mmu_gather *tlb) +{ + if (!tlb->end) + return; + + tlb_flush(tlb); + mmu_notifier_invalidate_range(tlb->mm, tlb->start, tlb->end); + __tlb_reset_range(tlb); +} + static inline void tlb_remove_page_size(struct mmu_gather *tlb, struct page *page, int page_size) { @@ -186,10 +197,8 @@ static inline void tlb_remove_check_page_size_change(struct mmu_gather *tlb, #define __tlb_end_vma(tlb, vma) \ do { \ - if (!tlb->fullmm && tlb->end) { \ - tlb_flush(tlb); \ - __tlb_reset_range(tlb); \ - } \ + if (!tlb->fullmm) \ + tlb_flush_mmu_tlbonly(tlb); \ } while (0) #ifndef tlb_end_vma diff --git a/mm/memory.c b/mm/memory.c index 7c58310734eb..c4a7b6cb17c8 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -238,16 +238,6 @@ void arch_tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm, __tlb_reset_range(tlb); } -static void tlb_flush_mmu_tlbonly(struct mmu_gather *tlb) -{ - if (!tlb->end) - return; - - tlb_flush(tlb); - mmu_notifier_invalidate_range(tlb->mm, tlb->start, tlb->end); - __tlb_reset_range(tlb); -} - static void tlb_flush_mmu_free(struct mmu_gather *tlb) { struct mmu_gather_batch *batch;