From patchwork Sat Nov 28 16:01:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 11937583 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.5 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F764C63777 for ; Sat, 28 Nov 2020 16:01:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0EFE124673 for ; Sat, 28 Nov 2020 16:01:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CgE6KOyT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0EFE124673 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4145D6B006E; Sat, 28 Nov 2020 11:01:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3C4E26B0070; Sat, 28 Nov 2020 11:01:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2B3D56B0071; Sat, 28 Nov 2020 11:01:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0018.hostedemail.com [216.40.44.18]) by kanga.kvack.org (Postfix) with ESMTP id 13AF96B006E for ; Sat, 28 Nov 2020 11:01:54 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id C9345362A for ; Sat, 28 Nov 2020 16:01:53 +0000 (UTC) X-FDA: 77534292906.19.offer89_4900a6c27391 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id 068691ACC39 for ; Sat, 28 Nov 2020 16:01:51 +0000 (UTC) X-HE-Tag: offer89_4900a6c27391 X-Filterd-Recvd-Size: 5434 Received: from mail-pf1-f195.google.com (mail-pf1-f195.google.com [209.85.210.195]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Sat, 28 Nov 2020 16:01:51 +0000 (UTC) Received: by mail-pf1-f195.google.com with SMTP id 131so7072181pfb.9 for ; Sat, 28 Nov 2020 08:01:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=2O6qcQHJ1HIADKlavbzosnF3a3wMtszkYQ/PuPl66qI=; b=CgE6KOyTfg3utDcRpZdx3wNg5OyhBXbYyc7PH8oa6j60BTtpQLlHDpvZodUSgtskAv kqzv+kzRC8gJVu60u38BrPXBYCz+1pEo0KvxCwXjcwsZBMYy4UyOWOY3pEIZ4CakN9AD IlOOeUveQZn2f5Ib6HdtyeE6Xd8O/3Ajz61BxBQaI2kgQLf9g57ysabjeDv/7CTlsJSV 6GIrzobXL4pzBVG6qscprgZn16B+G8oGEPy3hSkuzq1R6I3T2JlwKWSi3WfN7MY57x/P zPH8/jK+/t1BSRextQVoRrJHkVB9PMJuGQ9K8ajrDnT+LwE9dGk9paNd5x5WY4BqSCif XnDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=2O6qcQHJ1HIADKlavbzosnF3a3wMtszkYQ/PuPl66qI=; b=CqOTCEs6t9Dr4BGeLCbLdzofmxyOCIFclNek+x3GEcc36cg8j+i78wk++4I4OpoECV dYlXcls62MZ+ijeQz9IuXM9JHJDTwnPgurQ0GN3u0Ul58N/07+Qyf1fwrQ4NjQ6x/TUo eRdWMMbhsCrItOVN1hQqh+pdsDwEo1vIfyv/0z3hB6v9f41VJKBUi+rS/4IuWiTCvC7n QIWzunsvtJzO3Xlm1T8n9o0OLsxUBUFoWDelS++1THWYIvEtFslDYeNskrVmGvuXm1pz noHnsyCFl5bqaXc0H2Yh0mq+gCirTv9N5GtUqxIhuMfzEQe9Yv/bflJ9TJ5ZsV+K/1tk WV9Q== X-Gm-Message-State: AOAM533QL+LQOzqMdEs2qMEnfD0jh+yRb4YFCidVZxvcF7A3erGs4rf6 /dK63vV0QawQhLNw3c9ZLdo= X-Google-Smtp-Source: ABdhPJwuiEos39Kd+yWnNTMmDC8yimmI7zJPon0FpMljcy6a+V40CGpf8EDqJAwgKioFXcJ84fR1pw== X-Received: by 2002:a17:90a:f695:: with SMTP id cl21mr16872124pjb.137.1606579310741; Sat, 28 Nov 2020 08:01:50 -0800 (PST) Received: from bobo.ibm.com (193-116-103-132.tpgi.com.au. [193.116.103.132]) by smtp.gmail.com with ESMTPSA id d4sm9762607pjz.28.2020.11.28.08.01.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 28 Nov 2020 08:01:50 -0800 (PST) From: Nicholas Piggin To: linux-kernel@vger.kernel.org Cc: Nicholas Piggin , x86@kernel.org, Mathieu Desnoyers , Arnd Bergmann , Peter Zijlstra , linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-mm@kvack.org, Anton Blanchard Subject: [PATCH 0/8] shoot lazy tlbs Date: Sun, 29 Nov 2020 02:01:33 +1000 Message-Id: <20201128160141.1003903-1-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000003, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is a rebase now on top of Arnd's asm-generic tree, which has reduced most of the fluff from this patch series. The x86 refactoring is still in the way a bit, I hope to get some movement on that rather than rebase the main patches off it, because I think it's a good cleanup. I think it could go in a generic mm/scheduler series if we get arch acks because it's really just refactoring wrappers. The main result is reduced contention on lazy tlb mm refcount that helps very big systems. Thanks, Nick Nicholas Piggin (8): lazy tlb: introduce exit_lazy_tlb x86: use exit_lazy_tlb rather than membarrier_mm_sync_core_before_usermode x86: remove ARCH_HAS_SYNC_CORE_BEFORE_USERMODE lazy tlb: introduce lazy mm refcount helper functions lazy tlb: allow lazy tlb mm switching to be configurable lazy tlb: shoot lazies, a non-refcounting lazy tlb option powerpc: use lazy mm refcount helper functions powerpc/64s: enable MMU_LAZY_TLB_SHOOTDOWN .../membarrier-sync-core/arch-support.txt | 6 +- arch/Kconfig | 24 +++++ arch/arm/mach-rpc/ecard.c | 3 +- arch/powerpc/Kconfig | 1 + arch/powerpc/kernel/smp.c | 2 +- arch/powerpc/mm/book3s64/radix_tlb.c | 5 +- arch/x86/Kconfig | 1 - arch/x86/include/asm/mmu_context.h | 27 ++++++ arch/x86/kernel/alternative.c | 2 +- arch/x86/kernel/cpu/mce/core.c | 2 +- drivers/misc/sgi-gru/grufault.c | 2 +- drivers/misc/sgi-gru/gruhandles.c | 2 +- drivers/misc/sgi-gru/grukservices.c | 2 +- fs/exec.c | 6 +- include/asm-generic/mmu_context.h | 21 ++++ include/linux/sched/mm.h | 34 ++++--- include/linux/sync_core.h | 21 ---- init/Kconfig | 3 - kernel/cpu.c | 6 +- kernel/exit.c | 2 +- kernel/fork.c | 53 ++++++++++ kernel/kthread.c | 12 ++- kernel/sched/core.c | 97 +++++++++++++------ kernel/sched/sched.h | 4 +- 24 files changed, 247 insertions(+), 91 deletions(-) delete mode 100644 include/linux/sync_core.h