From patchwork Mon Aug 22 02:15:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12950120 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E78E6C00140 for ; Mon, 22 Aug 2022 02:17:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=upJkHbF9k4lw7sYMPOcSlNjG2YfEUw4nFnYHb2rz7Z8=; b=r96XRp0Uv4SdtV FGbytBypUbGq0xEODaMl/hqQj+pOerfQcmScmgnZJXVDELt/Itcr/lvINH0U2mIwWOhE4gHDQ8WL8 I5MZJA11OLbDCPD+j+3AKldovB0sHKp7xIG57yGlevCd27Sp2HaDqyGUYgPVJlURIUpEefUAdr+PW YUitHdHis1TKPDxA+He52eFfMBcjIeHe1B7jRp+ThWc/VrVdPWyyLexOUa2Hy0VgMsT6H2gNujzZt 3WcLShDwQnl6N9+AhRAoiHQG1CDaiMHJ2j7voSdKlFWKC+PUyzdV0PC9S4kbAvIpR52894YlXsE93 gsje2KpPMb4qCKEiS0RA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oPwyp-003l33-Mr; Mon, 22 Aug 2022 02:15:55 +0000 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oPwyj-003kyB-Vv; Mon, 22 Aug 2022 02:15:53 +0000 Received: by mail-pf1-x434.google.com with SMTP id g129so5704204pfb.8; Sun, 21 Aug 2022 19:15:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=1iHPy2MJm6s7J5XdJO/siHoKhzaKKWD/EXtuZwe5/AE=; b=mKi9In1cguwoEOIvsqd6r243Uwmb97pem6sMmVbQ/lX7DKmHGQ4Cz8u7NaUAMolab3 HmdgTa5cqnb+GfgxLAPvRC2VcgnhWhU/izaXxri5eH+EgDoV+6M0er9uK9apm/9k6Wa+ O6vWMuPgbVirYyTOwBORRtmkCs2pdpI1E3AlnfMY1g6INcFNsiY5OxCc+eK21YnW41W/ 9gNulkHUPIkdHnMs5921OdcrPwHgoXozCfdiUiIdwYospQ2cvraEremg4xnjuFFKYkFu gDjp9ye/aSfYYNx/jRfdI/eCYmQuHZst3at0UYKdKX8Agaty19wI93E6qRscBDPMtYU4 QtCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=1iHPy2MJm6s7J5XdJO/siHoKhzaKKWD/EXtuZwe5/AE=; b=Ja8bB+KQCCYjXE7jRa6zXBbozm4Hqybi3eNMKGNAcyVeBDNJA2xI96RKzKbb3F26Z6 MXtt4OgYajAhtP9ABGE42ciec4CxHuK/h4OOqezXpg5USSOuJlNO3x36KdBtwsxZ8lpl JOlp/RMsK7oo2mej6wtGk4+BEpWHxiczZi9/TXQWTvqseEOVxcM3JUD8J6IE7V4nJhoq XCjGz6OLIGF+ynuBqAYHIYD0bfXyng5I5npf3XxAsBNLkbshlcnfnaPMYV8r9x7pMV0y qrZhZ2UPAe+JmdZiI+d7R589l7r78WfLrXnr65uZbPzKwNV4GvD3tUBxIP37sBi7VvdR jxog== X-Gm-Message-State: ACgBeo1Ov5y8fKiZDt/r0gy0tMHhBq8uFsvxr3gEGXgftemZ0m/blJDa 2rUsVlShWTUvlTZS/NbRlpV2znyIYg== X-Google-Smtp-Source: AA6agR5RhOvnr+n8C+QckGvAPfLQPD+H4Z9Kyq6FS5FDBt09E1rsRMXpjGgfl8V7Hzvo18Hl1aKTcg== X-Received: by 2002:a63:ce06:0:b0:41d:dcc3:aa6e with SMTP id y6-20020a63ce06000000b0041ddcc3aa6emr15448270pgf.251.1661134548425; Sun, 21 Aug 2022 19:15:48 -0700 (PDT) Received: from piliu.users.ipa.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id k3-20020aa79723000000b005321340753fsm7312139pfg.103.2022.08.21.19.15.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Aug 2022 19:15:47 -0700 (PDT) From: Pingfan Liu To: linux-arm-kernel@lists.infradead.org, linux-ia64@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Pingfan Liu , Russell King , Catalin Marinas , Will Deacon , Paul Walmsley , Palmer Dabbelt , Albert Ou , Peter Zijlstra , "Eric W. Biederman" , Mark Rutland , Marco Elver , Masami Hiramatsu , Dan Li , Song Liu , Sami Tolvanen , Arnd Bergmann , Linus Walleij , Ard Biesheuvel , Tony Lindgren , Nick Hawkins , John Crispin , Geert Uytterhoeven , Andrew Morton , Bjorn Andersson , Anshuman Khandual , Thomas Gleixner , Steven Price Subject: [RFC 02/10] cpu/hotplug: Compile smp_shutdown_nonboot_cpus() conditioned on CONFIG_SHUTDOWN_NONBOOT_CPUS Date: Mon, 22 Aug 2022 10:15:12 +0800 Message-Id: <20220822021520.6996-3-kernelfans@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220822021520.6996-1-kernelfans@gmail.com> References: <20220822021520.6996-1-kernelfans@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220821_191551_208472_6A5C01EA X-CRM114-Status: GOOD ( 15.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Only arm/arm64/ia64/riscv share the smp_shutdown_nonboot_cpus(). So compiling this code conditioned on the macro CONFIG_SHUTDOWN_NONBOOT_CPUS. Later this macro will brace the quick kexec reboot code. Signed-off-by: Pingfan Liu Cc: Russell King Cc: Catalin Marinas Cc: Will Deacon Cc: Paul Walmsley Cc: Palmer Dabbelt Cc: Albert Ou Cc: Peter Zijlstra Cc: "Eric W. Biederman" Cc: Mark Rutland Cc: Marco Elver Cc: Masami Hiramatsu Cc: Dan Li Cc: Song Liu Cc: Sami Tolvanen Cc: Arnd Bergmann Cc: Linus Walleij Cc: Ard Biesheuvel Cc: Tony Lindgren Cc: Nick Hawkins Cc: John Crispin Cc: Geert Uytterhoeven Cc: Andrew Morton Cc: Bjorn Andersson Cc: Anshuman Khandual Cc: Thomas Gleixner Cc: Steven Price To: linux-arm-kernel@lists.infradead.org To: linux-ia64@vger.kernel.org To: linux-riscv@lists.infradead.org To: linux-kernel@vger.kernel.org --- arch/Kconfig | 4 ++++ arch/arm/Kconfig | 1 + arch/arm64/Kconfig | 1 + arch/ia64/Kconfig | 1 + arch/riscv/Kconfig | 1 + kernel/cpu.c | 3 +++ 6 files changed, 11 insertions(+) diff --git a/arch/Kconfig b/arch/Kconfig index f330410da63a..be447537d0f6 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -14,6 +14,10 @@ menu "General architecture-dependent options" config CRASH_CORE bool +config SHUTDOWN_NONBOOT_CPUS + select KEXEC_CORE + bool + config KEXEC_CORE select CRASH_CORE bool diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 87badeae3181..711cfdb4f9f4 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -129,6 +129,7 @@ config ARM select PCI_SYSCALL if PCI select PERF_USE_VMALLOC select RTC_LIB + select SHUTDOWN_NONBOOT_CPUS select SYS_SUPPORTS_APM_EMULATION select THREAD_INFO_IN_TASK select HAVE_ARCH_VMAP_STACK if MMU && ARM_HAS_GROUP_RELOCS diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 571cc234d0b3..8c481a0b1829 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -223,6 +223,7 @@ config ARM64 select PCI_SYSCALL if PCI select POWER_RESET select POWER_SUPPLY + select SHUTDOWN_NONBOOT_CPUS select SPARSE_IRQ select SWIOTLB select SYSCTL_EXCEPTION_TRACE diff --git a/arch/ia64/Kconfig b/arch/ia64/Kconfig index 26ac8ea15a9e..8a3ddea97d1b 100644 --- a/arch/ia64/Kconfig +++ b/arch/ia64/Kconfig @@ -52,6 +52,7 @@ config IA64 select ARCH_CLOCKSOURCE_DATA select GENERIC_TIME_VSYSCALL select LEGACY_TIMER_TICK + select SHUTDOWN_NONBOOT_CPUS select SWIOTLB select SYSCTL_ARCH_UNALIGN_NO_WARN select HAVE_MOD_ARCH_SPECIFIC diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index ed66c31e4655..02606a48c5ea 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -120,6 +120,7 @@ config RISCV select PCI_MSI if PCI select RISCV_INTC select RISCV_TIMER if RISCV_SBI + select SHUTDOWN_NONBOOT_CPUS select SPARSE_IRQ select SYSCTL_EXCEPTION_TRACE select THREAD_INFO_IN_TASK diff --git a/kernel/cpu.c b/kernel/cpu.c index 338e1d426c7e..2be6ba811a01 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1258,6 +1258,8 @@ int remove_cpu(unsigned int cpu) } EXPORT_SYMBOL_GPL(remove_cpu); +#ifdef CONFIG_SHUTDOWN_NONBOOT_CPUS + void smp_shutdown_nonboot_cpus(unsigned int primary_cpu) { unsigned int cpu; @@ -1299,6 +1301,7 @@ void smp_shutdown_nonboot_cpus(unsigned int primary_cpu) cpu_maps_update_done(); } +#endif #else #define takedown_cpu NULL From patchwork Mon Aug 22 02:15:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12950122 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D554DC00140 for ; Mon, 22 Aug 2022 02:17:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=k7okh9vcaYMXWw4ABpSJKARZj7sfgjBf0AKxAeqpo78=; b=s/79ZdgJK4Sb+2 Z0VpMO7TH2jFK73T9GavNat7FqOQ+mwd7a4I/I4Djk7kHQmcoKGlumw6O1RtH70uV0LaY5t2kt6DJ 90URc6aa5ugFgwe5r7ISN3yZohLOyoduHYJENW8ScCGPj9se+ExwoJ4a3nSG3I1CFOeYuY411QVLZ budr7Q763U9vDnyyy0cLjgHmkdCU2D/sLODRLnFE63F6paQsngdU7SoL7bGKSRvNC3r8j3eqqCsPD pdFPlgARDNfinGXFQF3f8lG04mKULi6G4Y7bZH9OcPgBVklNnjn/HEGos+8owfY5RbrzuxiN8joY6 ATMbj1mLFw3Doiqur9mQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oPwz8-003lHu-E0; Mon, 22 Aug 2022 02:16:14 +0000 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oPwyr-003kyB-0B; Mon, 22 Aug 2022 02:15:59 +0000 Received: by mail-pf1-x434.google.com with SMTP id g129so5704368pfb.8; Sun, 21 Aug 2022 19:15:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=WLszNFH6rAx833ludRXMY401Pm+H+RTV6I0FYUgHTg0=; b=SqqVBMOnC7CzBii1m0ocTxbCUywgAQHCwLfgqx83GK4H1dtDbCF9O6MIHnUWaRicTM 26QmKxmEBTyB+zRhgsS02GmtOBoSUG3n9vNLTx/oVFe/iLv2YXG5DBfeJVvp3N98Nqw3 QrOZY8/GZMChwXBy246ZcaxGqyefNTj4lrsFOg7n9KSwMyGWA/PJun+O3rN7k6IKejoS W0WGG2sDMSITPd9l5SFj5FDjiB8VxiEy3h01jTkhf4lwHS2n4jaEtlm95zcq47ioODO/ +xzMNd/XLGxseTgLin18K/K9szEkCq6SJSsPsuypa2ZcYXvVOjiUTEtup+0S+TuAjbog PD6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=WLszNFH6rAx833ludRXMY401Pm+H+RTV6I0FYUgHTg0=; b=PNUfZZ0eKvKrt4bKFblF0yH6ak5G7DUDpVVhVR32qRdWaxALUCFwpLkjqPf1f6Vd1n iOal/26t+lhBQmX0r/MCnZk7WyVAmq9S0yYqzhSGhbnDh99TGL8NfbZv36Ct4TCTPkt+ YYDKvWy+U4b1UjrGMCPQqRIYr1cRDD+OcmVPZJApKcBAHaoCHH0+yNrYy71w0XYeD8Fa CBzAX+GLSpV7Hb2/ZZVw1FuZAA1veaFN4ALMTOK2k/ciY4dmIMrknhgbBWhOWq4Agx+2 sQCPQZjTpkUa1v0ZrvUEGCGB5zb9WTtatyQMKLXoi9GRMEU1bZYMFmNhkXGFZG6f/B1w e/mw== X-Gm-Message-State: ACgBeo34Vd51k/qTedY0+up6X6tiPHIMOrTo4k6lJZYL4plI8/vXdyBE TzR6xGeVK8fpK1LNYMQJp0BYSUVgDw== X-Google-Smtp-Source: AA6agR6OqW8j4jhaYEyPSXqhTTqweo/eD8PzrvQ6+13CJ2uOIZ0noYxOQmPKYGKHiep+6to1lt7J9Q== X-Received: by 2002:a05:6a00:23c1:b0:536:463e:e53b with SMTP id g1-20020a056a0023c100b00536463ee53bmr9714834pfc.43.1661134556164; Sun, 21 Aug 2022 19:15:56 -0700 (PDT) Received: from piliu.users.ipa.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id k3-20020aa79723000000b005321340753fsm7312139pfg.103.2022.08.21.19.15.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Aug 2022 19:15:55 -0700 (PDT) From: Pingfan Liu To: linux-arm-kernel@lists.infradead.org, linux-ia64@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Pingfan Liu , Thomas Gleixner , Steven Price , Kuppuswamy Sathyanarayanan , "Jason A. Donenfeld" , Frederic Weisbecker , Russell King , Catalin Marinas , Will Deacon , Paul Walmsley , Palmer Dabbelt , Albert Ou , Peter Zijlstra , "Eric W. Biederman" Subject: [RFC 03/10] cpu/hotplug: Introduce fast kexec reboot Date: Mon, 22 Aug 2022 10:15:13 +0800 Message-Id: <20220822021520.6996-4-kernelfans@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220822021520.6996-1-kernelfans@gmail.com> References: <20220822021520.6996-1-kernelfans@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220821_191557_166431_EB12EB48 X-CRM114-Status: GOOD ( 34.54 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org *** Current situation 'slow kexec reboot' *** At present, some architectures rely on smp_shutdown_nonboot_cpus() to implement "kexec -e". Since smp_shutdown_nonboot_cpus() tears down the cpus serially, it is very slow. Take a close look, a cpu_down() processing on a single cpu can approximately be divided into two stages: -1. from CPUHP_ONLINE to CPUHP_TEARDOWN_CPU -2. from CPUHP_TEARDOWN_CPU to CPUHP_AP_IDLE_DEAD which is by stop_machine_cpuslocked(take_cpu_down, NULL, cpumask_of(cpu)); and runs on the teardown cpu. If these processes can run in parallel, then, the reboot can be speeded up. That is the aim of this patch. *** Contrast to other implements *** X86 and PowerPC have their own machine_shutdown(), which does not reply on the cpu hot-removing mechanism. They just discriminate some critical component and tears them down in per cpu NMI handler during the kexec reboot. But for some architectures, let's say arm64, it is not easy to define these critical component due to various chipmakers' implements. As a result, sticking to the cpu hot-removing mechanism is the simplest way to re-implement the parallel. It also renders an opportunity to implement the cpu_down() in parallel in future (not done by this series). *** Things worthy of consideration *** 1. The definition of a clean boundary between the first kernel and the new kernel -1.1 firmware The firmware's internal state should enter into a proper state. And this is achieved by the firmware's cpuhp_step's teardown interface if any. -1.2 CPU internal Whether the cache or PMU needs a clean shutdown before rebooting. 2. The dependency of each cpuhp_step The boundary of a clean cut involves only few cpuhp_step, but they may propagate to other cpuhp_step by the way of the dependency. This series does not bother to judge the dependency, instead, just iterate downside each cpuhp_step. And this stragegy demands that each cpuhp_step's teardown interface supports parallel. *** Solution *** Ideally, if the interface _cpu_down() can be enhanced to enable parallel, then the fast reboot can be achieved. But revisiting the two parts of the current cpu_down() process, the second part 'stop_machine_cpuslocked()' is a blockade. Packed inside the _cpu_down(), stop_machine_cpuslocked() only allow one cpu to execute the teardown. So this patch breaks down the process of _cpu_down(), and divides the teardown into three steps. And the exposed stop_machine_cpuslocked() can be used to support parallel. 1. Bring each AP from CPUHP_ONLINE to CPUHP_TEARDOWN_CPU in parallel. 2. Sync on BP to wait all APs to enter CPUHP_TEARDOWN_CPU state 3. Bring each AP from CPUHP_TEARDOWN_CPU to CPUHP_AP_IDLE_DEAD by the interface of stop_machine_cpuslocked() in parallel. Apparently, the step 2 is introduced in order to satisfy the condition on which stop_machine_cpuslocked() can start on each cpu. Then the rest issue is about how to support parallel in step 1&3. Furtunately, each subsystem has its own carefully designed lock mechanism. In each cpuhp_step teardown interface, adopting to the subsystem's lock rule will make things work. *** No rollback if failure *** During kexec reboot, the devices have already been shutdown, there is no way for system to roll back to a workable state. So this series also does not consider the rollback issue. Signed-off-by: Pingfan Liu Cc: Thomas Gleixner Cc: Steven Price Cc: Kuppuswamy Sathyanarayanan Cc: "Jason A. Donenfeld" Cc: Frederic Weisbecker Cc: Russell King Cc: Catalin Marinas Cc: Will Deacon Cc: Paul Walmsley Cc: Palmer Dabbelt Cc: Albert Ou Cc: Peter Zijlstra Cc: "Eric W. Biederman" To: linux-arm-kernel@lists.infradead.org To: linux-ia64@vger.kernel.org To: linux-riscv@lists.infradead.org To: linux-kernel@vger.kernel.org --- kernel/cpu.c | 139 +++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 129 insertions(+), 10 deletions(-) diff --git a/kernel/cpu.c b/kernel/cpu.c index 2be6ba811a01..94ab2727d6bb 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1260,10 +1260,125 @@ EXPORT_SYMBOL_GPL(remove_cpu); #ifdef CONFIG_SHUTDOWN_NONBOOT_CPUS -void smp_shutdown_nonboot_cpus(unsigned int primary_cpu) +/* + * Push all of cpus to the state CPUHP_AP_ONLINE_IDLE. + * Since kexec-reboot has already shut down all devices, there is no way to + * roll back, the cpus' teardown also requires no rollback, instead, just throw + * warning. + */ +static void cpus_down_no_rollback(struct cpumask *cpus) { + struct cpuhp_cpu_state *st; unsigned int cpu; + + /* launch ap work one by one, but not wait for completion */ + for_each_cpu(cpu, cpus) { + st = per_cpu_ptr(&cpuhp_state, cpu); + /* + * If the current CPU state is in the range of the AP hotplug thread, + * then we need to kick the thread. + */ + if (st->state > CPUHP_TEARDOWN_CPU) { + cpuhp_set_state(cpu, st, CPUHP_TEARDOWN_CPU); + /* In order to parallel, async. And there is no way to rollback */ + cpuhp_kick_ap_work_async(cpu); + } + } + + /* wait for all ap work completion */ + for_each_cpu(cpu, cpus) { + st = per_cpu_ptr(&cpuhp_state, cpu); + wait_for_ap_thread(st, st->bringup); + if (st->result) + pr_warn("cpu %u refuses to offline due to %d\n", cpu, st->result); + else if (st->state > CPUHP_TEARDOWN_CPU) + pr_warn("cpu %u refuses to offline, state: %d\n", cpu, st->state); + } +} + +static int __takedown_cpu_cleanup(unsigned int cpu) +{ + struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu); + + /* + * The teardown callback for CPUHP_AP_SCHED_STARTING will have removed + * all runnable tasks from the CPU, there's only the idle task left now + * that the migration thread is done doing the stop_machine thing. + * + * Wait for the stop thread to go away. + */ + wait_for_ap_thread(st, false); + BUG_ON(st->state != CPUHP_AP_IDLE_DEAD); + + hotplug_cpu__broadcast_tick_pull(cpu); + /* This actually kills the CPU. */ + __cpu_die(cpu); + + tick_cleanup_dead_cpu(cpu); + rcutree_migrate_callbacks(cpu); + return 0; +} + +/* + * There is a sync that all ap threads are done before calling this func. + */ +static void takedown_cpus_no_rollback(struct cpumask *cpus) +{ + struct cpuhp_cpu_state *st; + unsigned int cpu; + + for_each_cpu(cpu, cpus) { + st = per_cpu_ptr(&cpuhp_state, cpu); + WARN_ON(st->state != CPUHP_TEARDOWN_CPU); + /* No invoke to takedown_cpu(), so set the state by manual */ + st->state = CPUHP_AP_ONLINE; + cpuhp_set_state(cpu, st, CPUHP_AP_OFFLINE); + } + + irq_lock_sparse(); + /* ask stopper kthreads to execute take_cpu_down() in parallel */ + stop_machine_cpuslocked(take_cpu_down, NULL, cpus); + + /* Finally wait for completion and clean up */ + for_each_cpu(cpu, cpus) + __takedown_cpu_cleanup(cpu); + irq_unlock_sparse(); +} + +static bool check_quick_reboot(void) +{ + return false; +} + +static struct cpumask kexec_ap_map; + +void smp_shutdown_nonboot_cpus_quick_path(unsigned int primary_cpu) +{ + struct cpumask *cpus = &kexec_ap_map; + /* + * To prevent other subsystem from access to __cpu_online_mask, but internally, + * __cpu_disable() accesses the bitmap in parral and needs its own local lock. + */ + cpus_write_lock(); + + cpumask_copy(cpus, cpu_online_mask); + cpumask_clear_cpu(primary_cpu, cpus); + cpus_down_no_rollback(cpus); + takedown_cpus_no_rollback(cpus); + /* + * For some subsystems, there are still remains for offline cpus from + * CPUHP_BRINGUP_CPU to CPUHP_OFFLINE. But since none of them interact + * with hardwares or firmware, they have no effect on the new kernel. + * So skipping the cpuhp callbacks in that range + */ + + cpus_write_unlock(); +} + +void smp_shutdown_nonboot_cpus(unsigned int primary_cpu) +{ int error; + unsigned int cpu; cpu_maps_update_begin(); @@ -1275,15 +1390,19 @@ void smp_shutdown_nonboot_cpus(unsigned int primary_cpu) if (!cpu_online(primary_cpu)) primary_cpu = cpumask_first(cpu_online_mask); - for_each_online_cpu(cpu) { - if (cpu == primary_cpu) - continue; - - error = cpu_down_maps_locked(cpu, CPUHP_OFFLINE); - if (error) { - pr_err("Failed to offline CPU%d - error=%d", - cpu, error); - break; + if (check_quick_reboot()) { + smp_shutdown_nonboot_cpus_quick_path(primary_cpu); + } else { + for_each_online_cpu(cpu) { + if (cpu == primary_cpu) + continue; + + error = cpu_down_maps_locked(cpu, CPUHP_OFFLINE); + if (error) { + pr_err("Failed to offline CPU%d - error=%d", + cpu, error); + break; + } } } From patchwork Mon Aug 22 02:15:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12950123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 30870C00140 for ; Mon, 22 Aug 2022 02:18:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=TQAuQCdkUZbYlNNEmBLe2+ocBAt3gWIvlieib8RT3Vg=; b=WeB7Bz2qzC8Dxg 1IZWi4fAMxo126bSuBRt1kZ7gVIMcMG2mccqm6g9XvdHDNmLDMyxA7jvO5IJkY0PgwmHwRYf2nKnp iHDM5Z8BxC8uyXZlCovFtvYjrH2zGvQL5e60QhS+V1N4ALPiqkZr/HTVrq0y8G0aXiIruPUxwr+01 q3EQxt6lgYAQBoEsDpty+IQR0Q2eZ0WWL8WqcCBc9WsE30PbThArwpVtqF8dIn8qRxEGFWoMWaau0 EUUUthhyBgTiXZNABxM3jVT1HqLEUQE7DFLY5feCxvWyQPDICfizIDmuhqX+P+F2trIpNgjE5Xomr yJjXjG4z8zi1lIOeehcA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oPwzR-003lWL-Pe; Mon, 22 Aug 2022 02:16:33 +0000 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oPwz1-003l8x-Js; Mon, 22 Aug 2022 02:16:10 +0000 Received: by mail-pg1-x535.google.com with SMTP id v4so8164299pgi.10; Sun, 21 Aug 2022 19:16:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=xSWHQT0Gh+ASwqDbuftZ9hOwt+7ZpIdbcTfHZLJUjew=; b=drdEGnn6bSG1bQXrr6SCO9vL8sEJJnSyNh0s4SAbgA50zj3osmvGAr+RilNPSGW7tp KAfFuc5fnb/IN3BmLw8Lx1KUVaemwzWMqg+7fGeE+qpb2RuR+Kib91bE6zeP6pdAU+Iv rHPp+ANtA230uW3x89/bKkqo7TORCuWAHvqz4rYuwfwa0RHsGIPHziXZEfrA5sbdna54 tQSSr4N/VaS9RF7zANbJqRDu+IaNTN4xu3gRYTJ/+K+ZA/pY1IFky9Y0+JvCObZ18NYD vzacLfpll9SGbrBMx+pDubr0qCbwovinLpHj7a49H5uO4QvUDuMrSrN+nvwfBoxvGTK5 Cm8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=xSWHQT0Gh+ASwqDbuftZ9hOwt+7ZpIdbcTfHZLJUjew=; b=FXghNO9xmt4HbVo97uNW8okrVNi4qRWz5Z/u6YeiPyAS7cdcmphJYtg0s1oc8aLj3e al8QVK/zMGufaj5qgUrBozJmM3NY40hVKuZZw/POcZmnz//8Ug1avALF8QcePU5Wu71u Xg9gvYDxSt1nvaVFACMFSSlUxkVw3i2IB4FwDy5DJaRlyLVV3yJ71H6VlkCU5inVDecP gmIbPArTeOoHN3h1znQ/ZEcqLT3oRvJuiBMpBaT7Wihpws7dAehhkNvo8yP1mevxTPSL eEAnsSiDKn6OOduFSDsmaSnPSu/qVdbTQy20fio54gjZt9+HfFfMwHPyYcfGesUBFd57 DwWQ== X-Gm-Message-State: ACgBeo2TMLBXneFMQxUuN1jVH2vwZYnN9O25RaL1MparAucmhKuoZS4f m7uYmAimvA48aILK6A8y6+Jr3k7YQQ== X-Google-Smtp-Source: AA6agR7KljxZ9kbPwullzRTYyS9R74MZRULUzSQZWPtq0VQwcgMNVf0XgsbzpqwnU5ffstdl5YcXvw== X-Received: by 2002:a63:c1f:0:b0:41a:9b73:a89e with SMTP id b31-20020a630c1f000000b0041a9b73a89emr14967609pgl.342.1661134566121; Sun, 21 Aug 2022 19:16:06 -0700 (PDT) Received: from piliu.users.ipa.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id k3-20020aa79723000000b005321340753fsm7312139pfg.103.2022.08.21.19.15.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Aug 2022 19:16:05 -0700 (PDT) From: Pingfan Liu To: linux-arm-kernel@lists.infradead.org, linux-ia64@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Pingfan Liu , Thomas Gleixner , Steven Price , Kuppuswamy Sathyanarayanan , "Jason A. Donenfeld" , Frederic Weisbecker , Russell King , Catalin Marinas , Will Deacon , Paul Walmsley , Palmer Dabbelt , Albert Ou , Peter Zijlstra , "Eric W. Biederman" Subject: [RFC 04/10] cpu/hotplug: Check the capability of kexec quick reboot Date: Mon, 22 Aug 2022 10:15:14 +0800 Message-Id: <20220822021520.6996-5-kernelfans@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220822021520.6996-1-kernelfans@gmail.com> References: <20220822021520.6996-1-kernelfans@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220821_191607_821173_4099B6E9 X-CRM114-Status: GOOD ( 17.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The kexec quick reboot needs each involved cpuhp_step to run in parallel. There are lots of teardown cpuhp_step, but not all of them belong to arm/arm64/riscv kexec reboot path. So introducing a member 'support_kexec_parallel' in the struct cpuhp_step to signal whether the teardown supports parallel or not. If a cpuhp_step is used in kexec reboot, then it needs to support parallel to enable the quick reboot. The function check_quick_reboot() checks all teardown cpuhp_steps and report those unsupported if any. Signed-off-by: Pingfan Liu Cc: Thomas Gleixner Cc: Steven Price Cc: Kuppuswamy Sathyanarayanan Cc: "Jason A. Donenfeld" Cc: Frederic Weisbecker Cc: Russell King Cc: Catalin Marinas Cc: Will Deacon Cc: Paul Walmsley Cc: Palmer Dabbelt Cc: Albert Ou Cc: Peter Zijlstra Cc: "Eric W. Biederman" To: linux-arm-kernel@lists.infradead.org To: linux-ia64@vger.kernel.org To: linux-riscv@lists.infradead.org To: linux-kernel@vger.kernel.org --- include/linux/cpuhotplug.h | 2 ++ kernel/cpu.c | 28 +++++++++++++++++++++++++++- 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index f61447913db9..73093fc15aec 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -374,6 +374,8 @@ static inline int cpuhp_setup_state_multi(enum cpuhp_state state, (void *) teardown, true); } +void cpuhp_set_step_parallel(enum cpuhp_state state); + int __cpuhp_state_add_instance(enum cpuhp_state state, struct hlist_node *node, bool invoke); int __cpuhp_state_add_instance_cpuslocked(enum cpuhp_state state, diff --git a/kernel/cpu.c b/kernel/cpu.c index 94ab2727d6bb..1261c3f3be51 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -137,6 +137,9 @@ struct cpuhp_step { /* public: */ bool cant_stop; bool multi_instance; +#ifdef CONFIG_SHUTDOWN_NONBOOT_CPUS + bool support_kexec_parallel; +#endif }; static DEFINE_MUTEX(cpuhp_state_mutex); @@ -147,6 +150,14 @@ static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state) return cpuhp_hp_states + state; } +#ifdef CONFIG_SHUTDOWN_NONBOOT_CPUS +void cpuhp_set_step_parallel(enum cpuhp_state state) +{ + cpuhp_hp_states[state].support_kexec_parallel = true; +} +EXPORT_SYMBOL(cpuhp_set_step_parallel); +#endif + static bool cpuhp_step_empty(bool bringup, struct cpuhp_step *step) { return bringup ? !step->startup.single : !step->teardown.single; @@ -1347,7 +1358,22 @@ static void takedown_cpus_no_rollback(struct cpumask *cpus) static bool check_quick_reboot(void) { - return false; + struct cpuhp_step *step; + enum cpuhp_state state; + bool ret = true; + + for (state = CPUHP_ONLINE; state >= CPUHP_AP_OFFLINE; state--) { + step = cpuhp_get_step(state); + if (step->teardown.single == NULL) + continue; + if (step->support_kexec_parallel == false) { + pr_info("cpuhp state:%d, %s, does not support cpudown in parallel\n", + state, step->name); + ret = false; + } + } + + return ret; } static struct cpumask kexec_ap_map; From patchwork Mon Aug 22 02:15:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12950124 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2D2A6C00140 for ; Mon, 22 Aug 2022 02:18:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2qJ9alaaJx84hFVnr5+fa+byGSX3LaQUp4IE7SAB5Mg=; b=OYrehm046qrLZn jZRzQBEHBKglHOPRLt5abau/CEQVbchEw7Zv0E+2Si1yzNuTxei+zdic8ANys9pDcQL+sboWembLG Uq/ak610BurgC6uM8Rh4ss5hQQAzJrdlwEYJQOSH9oHaX5Xm6mhiVsbyzryT37Yzd56SkgmFX0Qcu Uw5aRT4G5+f4QnG/voB/4T5OO8gtFDdysBBF9YrBc6ReiWsN+o6KES/4GmWBv2TaZVNJlYNARGsRZ RjOKz5NEiDRO+KAV1QRGLF4TDXrpRTQMiVY4l/LqcRCtLkAaqmFZ4SsRJ3+IAgXt1DnTeK6kL9zOJ vTUsiID9W0FhfhsgtfsQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oPwzt-003lvv-74; Mon, 22 Aug 2022 02:17:01 +0000 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oPwz4-003lCB-Lc for linux-arm-kernel@lists.infradead.org; Mon, 22 Aug 2022 02:16:12 +0000 Received: by mail-pf1-x432.google.com with SMTP id p9so7942924pfq.13 for ; Sun, 21 Aug 2022 19:16:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=gsR63uXiD4J4dRjeMZsAaLt2fDvFcRZsnDNlGseIAjE=; b=Pk0rIhqwd8AcUkq5KmJe9mjrUF6uNVfJWfjWUkbfwuQIGmV+U3/UFgPVoh1gYGq+bd lbCSRPMb+1x0M86+ymqwzFqN8nb+l68wSITrsmEixSWhlHgDZLB/R0svIyy5MO54R6Ym PwVfSVkrB8SoRCQ7vzL1LQZzM9iHBwApzh6vrmeiZsltT6ynaL3sdrb1ZhJGPhmFyOD3 4s2db1YqPkPeaWJ6qcSkXxuYfgRl3Eokk297cD1GGHz/Q5l5MVSGxbVpp3c4meit/hP+ bSsrrIKUfzH3yc9hotWq9Z7duppQSn0jH3GGkXZbsN3yxozQbvoh5tqw/X+vXNhleLsl cX7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=gsR63uXiD4J4dRjeMZsAaLt2fDvFcRZsnDNlGseIAjE=; b=6IdXOaP48SoYfOCJuIsVEe+0W/8Wy2zUw/6Cy+RDHEfmboo9IPloAYrd2sgAoawwtS qjOEFPklNxlPbO2wds/EeFTUyORBwH3g+TvSY34xEuPtu7gqpo4OlC8i7SFkF39TPaun eSkBJ/Pi9VYWUkLSnCuRKqL8mfTHKzyfTJE/wTsWqc2sBR22ej67uxI+srNqqB2xS1wm hle/auSQuw6uNBoBnIVkf230+fIK2UYuPh3gZ/8YNe34pmI8Rf+8xIi3JlB1D+bjBjoD YCLZyr111ZsgiJTkEOsPylWZcLxmPUl1HgUEIwtHDF00zoZCVxteS8vs/AfBsDBbHMnp 9wWw== X-Gm-Message-State: ACgBeo2XucSFS7A908zwNIDMFlqTQHvq018Vi0ZtdCrMycALmjVzbwFh KoOAUREnR/tSbK7O66HNIU/KZFZ1Eg== X-Google-Smtp-Source: AA6agR796+xdB/mm2oEQUhLS61HVQaOmFzmcD/yNrFj1veXL2aZ9RxDMlPamwMMKooh9so1wCzMtIA== X-Received: by 2002:a63:4b62:0:b0:421:8c8b:163c with SMTP id k34-20020a634b62000000b004218c8b163cmr14624442pgl.182.1661134569229; Sun, 21 Aug 2022 19:16:09 -0700 (PDT) Received: from piliu.users.ipa.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id k3-20020aa79723000000b005321340753fsm7312139pfg.103.2022.08.21.19.16.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Aug 2022 19:16:08 -0700 (PDT) From: Pingfan Liu To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Pingfan Liu , Will Deacon , Mark Rutland Subject: [RFC 05/10] perf/arm-dsu: Make dsu_pmu_cpu_teardown() parallel Date: Mon, 22 Aug 2022 10:15:15 +0800 Message-Id: <20220822021520.6996-6-kernelfans@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220822021520.6996-1-kernelfans@gmail.com> References: <20220822021520.6996-1-kernelfans@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220821_191610_810584_6EDF4277 X-CRM114-Status: GOOD ( 13.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In the case of kexec quick reboot, dsu_pmu_cpu_teardown() confronts parallel and lock are needed to protect the contest on a dsu_pmu. Signed-off-by: Pingfan Liu Cc: Will Deacon Cc: Mark Rutland To: linux-arm-kernel@lists.infradead.org To: linux-kernel@vger.kernel.org --- drivers/perf/arm_dsu_pmu.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/drivers/perf/arm_dsu_pmu.c b/drivers/perf/arm_dsu_pmu.c index a36698a90d2f..aa9f4393ff0c 100644 --- a/drivers/perf/arm_dsu_pmu.c +++ b/drivers/perf/arm_dsu_pmu.c @@ -833,16 +833,23 @@ static int dsu_pmu_cpu_teardown(unsigned int cpu, struct hlist_node *node) struct dsu_pmu *dsu_pmu = hlist_entry_safe(node, struct dsu_pmu, cpuhp_node); - if (!cpumask_test_and_clear_cpu(cpu, &dsu_pmu->active_cpu)) + raw_spin_lock(&dsu_pmu->pmu_lock); + if (!cpumask_test_and_clear_cpu(cpu, &dsu_pmu->active_cpu)) { + raw_spin_unlock(&dsu_pmu->pmu_lock); return 0; + } dst = dsu_pmu_get_online_cpu_any_but(dsu_pmu, cpu); /* If there are no active CPUs in the DSU, leave IRQ disabled */ - if (dst >= nr_cpu_ids) + if (dst >= nr_cpu_ids) { + raw_spin_unlock(&dsu_pmu->pmu_lock); return 0; + } - perf_pmu_migrate_context(&dsu_pmu->pmu, cpu, dst); + /* dst should not be in dying mask. So after setting, blocking parallel */ dsu_pmu_set_active_cpu(dst, dsu_pmu); + raw_spin_unlock(&dsu_pmu->pmu_lock); + perf_pmu_migrate_context(&dsu_pmu->pmu, cpu, dst); return 0; } @@ -858,6 +865,7 @@ static int __init dsu_pmu_init(void) if (ret < 0) return ret; dsu_pmu_cpuhp_state = ret; + cpuhp_set_step_parallel(ret); return platform_driver_register(&dsu_pmu_driver); } From patchwork Mon Aug 22 02:15:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12950125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CEA4BC00140 for ; Mon, 22 Aug 2022 02:19:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kZlNbV7EDLTy6763GvSSB0INfZbHOguucqjqxgWVmZc=; b=xCS/yGPeq/yW0g q0h7HkUgbRA2AZoagc2jrZGzkCZq40Q+HPERVjMEunfC89t2vGZTzMePArPqS/F50CEinsDPZFYdH +nyTQtBWhWzGYbUdPNsn1Nls0AjiGzyNBSmS0Vw/QKrS9lQ6kcrLOMbJ63uQcH06SVEv81X47tixh 5qdmBBpoRh5piJR3xVibvLWyFGNxwJEhaw0LiGtv9ok6a1XyKK15nOCctWvAPkQk8/4v6JyPcKZmK 4GZzdhQe21Dt865l2Y7cJrg/09H1I28ab9K7JXxcf7vckeKya6BmnKCE2XlT37emIuwDHPv906ESV HMzYSpVUNrPun+hCbhmw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oPx0p-003mY3-Sk; Mon, 22 Aug 2022 02:18:01 +0000 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oPwzU-003lYa-4T for linux-arm-kernel@lists.infradead.org; Mon, 22 Aug 2022 02:16:40 +0000 Received: by mail-pj1-x102a.google.com with SMTP id m15so1522643pjj.3 for ; Sun, 21 Aug 2022 19:16:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=xB76u9yqtIZ+mSa30ADibl4N4bymJ9Dh7UlLFPT7o98=; b=HRSBxEVCt3rvrI/7lmMMzAo0rRbfGOgqEC67o825HvHiARvDrjB8+tDVoRpWMePXYp 9oQI+id67tHzlJ/A1v+NZiooV2Ufdbed57m4RPzeWT4HTWQ3q2I+/Pe+oeuuLSkGr3sH OFGxMUhrOnPgyl+8PJjFb2VlRGBH65CMs+CTIcK33IedsTj91WJy8qIYPkCyA3YhPpbN xDCCgOZKQ5Y1aN3JjfMjYvKZvr6vDO7+ZcRdt9uyjXkR87MKlV9+WnblT5SVy92eqfSx w3yN/jWsdSyvHft8Zf3G5N2NxPQml5tWr62cry/lKZIDETPbgI06U3OybrR+Eaet25zB MNXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=xB76u9yqtIZ+mSa30ADibl4N4bymJ9Dh7UlLFPT7o98=; b=ttvFlBCnMjSEA77sONU0SsjOC/IpPUQ4GuBFHe9f8Dcixw2mWAKZa1JuHOHDRtOwUq wXqxysN1xBGJYSQWJCWOqxrCQKUlUxhIgsEjSQe/Imk5kiF3AWq2//xmxQspk4hHzm6t SYxNW947wupIvgu737F3hTodB91NeBYOTWtEVVWmUfIqSCpc12ute8n9lNTjsST4JUIW ub//m04g43x2YTZyF7D629f7UiHEAmS8F8JyVsqpHJIr4G8s5uhU/AVLPgDhLFyJoSlu 8RZtD/Xo7iSmMpnO4IKlP+qTTcV37q7DLRmqLIVv3+2JPxQammeB9/m2GE4VYnE9Wroq vDzg== X-Gm-Message-State: ACgBeo2Rmgpbor0+rvqWtPPU+JBmBMeL6yHOcOmRZLc8N7/vw+NnL9Ig nyL/iV7KYi96Ex0nlOeu7WIMv6B9sA== X-Google-Smtp-Source: AA6agR522vRbRjKGj/9trWnx6g9g6lhC9pFTs134FCjBnfDght6iTV00F7UnaCUQri0jee4v1rjxPQ== X-Received: by 2002:a17:90b:33ce:b0:1fb:1aec:ffac with SMTP id lk14-20020a17090b33ce00b001fb1aecffacmr5850314pjb.137.1661134594615; Sun, 21 Aug 2022 19:16:34 -0700 (PDT) Received: from piliu.users.ipa.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id k3-20020aa79723000000b005321340753fsm7312139pfg.103.2022.08.21.19.16.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Aug 2022 19:16:34 -0700 (PDT) From: Pingfan Liu To: linux-arm-kernel@lists.infradead.org, dmaengine@vger.kernel.org, linux-fpga@vger.kernel.org, intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Cc: Pingfan Liu , Russell King , Shawn Guo , Sascha Hauer , Pengutronix Kernel Team , Fabio Estevam , NXP Linux Team , Fenghua Yu , Dave Jiang , Vinod Koul , Wu Hao , Tom Rix , Moritz Fischer , Xu Yilun , Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Tvrtko Ursulin , David Airlie , Daniel Vetter , Will Deacon , Mark Rutland , Frank Li , Shaokun Zhang , Qi Liu , Andy Gross , Bjorn Andersson , Konrad Dybcio , Khuong Dinh , Li Yang , Yury Norov Subject: [RFC 08/10] cpuhp: Replace cpumask_any_but(cpu_online_mask, cpu) Date: Mon, 22 Aug 2022 10:15:18 +0800 Message-Id: <20220822021520.6996-9-kernelfans@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220822021520.6996-1-kernelfans@gmail.com> References: <20220822021520.6996-1-kernelfans@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220821_191638_139038_13AF3FB8 X-CRM114-Status: GOOD ( 17.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In a kexec quick reboot path, the dying cpus are still on cpu_online_mask. During the teardown of cpu, a subsystem needs to migrate its broker to a real online cpu. This patch replaces cpumask_any_but(cpu_online_mask, cpu) in a teardown procedure with cpumask_not_dying_but(cpu_online_mask, cpu). Signed-off-by: Pingfan Liu Cc: Russell King Cc: Shawn Guo Cc: Sascha Hauer Cc: Pengutronix Kernel Team Cc: Fabio Estevam Cc: NXP Linux Team Cc: Fenghua Yu Cc: Dave Jiang Cc: Vinod Koul Cc: Wu Hao Cc: Tom Rix Cc: Moritz Fischer Cc: Xu Yilun Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Rodrigo Vivi Cc: Tvrtko Ursulin Cc: David Airlie Cc: Daniel Vetter Cc: Will Deacon Cc: Mark Rutland Cc: Frank Li Cc: Shaokun Zhang Cc: Qi Liu Cc: Andy Gross Cc: Bjorn Andersson Cc: Konrad Dybcio Cc: Khuong Dinh Cc: Li Yang Cc: Yury Norov To: linux-arm-kernel@lists.infradead.org To: dmaengine@vger.kernel.org To: linux-fpga@vger.kernel.org To: intel-gfx@lists.freedesktop.org To: dri-devel@lists.freedesktop.org To: linux-arm-msm@vger.kernel.org To: linuxppc-dev@lists.ozlabs.org To: linux-kernel@vger.kernel.org --- arch/arm/mach-imx/mmdc.c | 2 +- arch/arm/mm/cache-l2x0-pmu.c | 2 +- drivers/dma/idxd/perfmon.c | 2 +- drivers/fpga/dfl-fme-perf.c | 2 +- drivers/gpu/drm/i915/i915_pmu.c | 2 +- drivers/perf/arm-cci.c | 2 +- drivers/perf/arm-ccn.c | 2 +- drivers/perf/arm-cmn.c | 4 ++-- drivers/perf/arm_dmc620_pmu.c | 2 +- drivers/perf/arm_dsu_pmu.c | 2 +- drivers/perf/arm_smmuv3_pmu.c | 2 +- drivers/perf/fsl_imx8_ddr_perf.c | 2 +- drivers/perf/hisilicon/hisi_uncore_pmu.c | 2 +- drivers/perf/marvell_cn10k_tad_pmu.c | 2 +- drivers/perf/qcom_l2_pmu.c | 2 +- drivers/perf/qcom_l3_pmu.c | 2 +- drivers/perf/xgene_pmu.c | 2 +- drivers/soc/fsl/qbman/bman_portal.c | 2 +- drivers/soc/fsl/qbman/qman_portal.c | 2 +- 19 files changed, 20 insertions(+), 20 deletions(-) diff --git a/arch/arm/mach-imx/mmdc.c b/arch/arm/mach-imx/mmdc.c index af12668d0bf5..a109a7ea8613 100644 --- a/arch/arm/mach-imx/mmdc.c +++ b/arch/arm/mach-imx/mmdc.c @@ -220,7 +220,7 @@ static int mmdc_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) if (!cpumask_test_and_clear_cpu(cpu, &pmu_mmdc->cpu)) return 0; - target = cpumask_any_but(cpu_online_mask, cpu); + target = cpumask_not_dying_but(cpu_online_mask, cpu); if (target >= nr_cpu_ids) return 0; diff --git a/arch/arm/mm/cache-l2x0-pmu.c b/arch/arm/mm/cache-l2x0-pmu.c index 993fefdc167a..1b0037ef7fa5 100644 --- a/arch/arm/mm/cache-l2x0-pmu.c +++ b/arch/arm/mm/cache-l2x0-pmu.c @@ -428,7 +428,7 @@ static int l2x0_pmu_offline_cpu(unsigned int cpu) if (!cpumask_test_and_clear_cpu(cpu, &pmu_cpu)) return 0; - target = cpumask_any_but(cpu_online_mask, cpu); + target = cpumask_not_dying_but(cpu_online_mask, cpu); if (target >= nr_cpu_ids) return 0; diff --git a/drivers/dma/idxd/perfmon.c b/drivers/dma/idxd/perfmon.c index d73004f47cf4..f3f1ccb55f73 100644 --- a/drivers/dma/idxd/perfmon.c +++ b/drivers/dma/idxd/perfmon.c @@ -528,7 +528,7 @@ static int perf_event_cpu_offline(unsigned int cpu, struct hlist_node *node) if (!cpumask_test_and_clear_cpu(cpu, &perfmon_dsa_cpu_mask)) return 0; - target = cpumask_any_but(cpu_online_mask, cpu); + target = cpumask_not_dying_but(cpu_online_mask, cpu); /* migrate events if there is a valid target */ if (target < nr_cpu_ids) diff --git a/drivers/fpga/dfl-fme-perf.c b/drivers/fpga/dfl-fme-perf.c index 587c82be12f7..57804f28357e 100644 --- a/drivers/fpga/dfl-fme-perf.c +++ b/drivers/fpga/dfl-fme-perf.c @@ -948,7 +948,7 @@ static int fme_perf_offline_cpu(unsigned int cpu, struct hlist_node *node) if (cpu != priv->cpu) return 0; - target = cpumask_any_but(cpu_online_mask, cpu); + target = cpumask_not_dying_but(cpu_online_mask, cpu); if (target >= nr_cpu_ids) return 0; diff --git a/drivers/gpu/drm/i915/i915_pmu.c b/drivers/gpu/drm/i915/i915_pmu.c index 958b37123bf1..f866f9223492 100644 --- a/drivers/gpu/drm/i915/i915_pmu.c +++ b/drivers/gpu/drm/i915/i915_pmu.c @@ -1068,7 +1068,7 @@ static int i915_pmu_cpu_offline(unsigned int cpu, struct hlist_node *node) return 0; if (cpumask_test_and_clear_cpu(cpu, &i915_pmu_cpumask)) { - target = cpumask_any_but(topology_sibling_cpumask(cpu), cpu); + target = cpumask_not_dying_but(topology_sibling_cpumask(cpu), cpu); /* Migrate events if there is a valid target */ if (target < nr_cpu_ids) { diff --git a/drivers/perf/arm-cci.c b/drivers/perf/arm-cci.c index 03b1309875ae..481da937fb9d 100644 --- a/drivers/perf/arm-cci.c +++ b/drivers/perf/arm-cci.c @@ -1447,7 +1447,7 @@ static int cci_pmu_offline_cpu(unsigned int cpu) if (!g_cci_pmu || cpu != g_cci_pmu->cpu) return 0; - target = cpumask_any_but(cpu_online_mask, cpu); + target = cpumask_not_dying_but(cpu_online_mask, cpu); if (target >= nr_cpu_ids) return 0; diff --git a/drivers/perf/arm-ccn.c b/drivers/perf/arm-ccn.c index 728d13d8e98a..573d6906ec9b 100644 --- a/drivers/perf/arm-ccn.c +++ b/drivers/perf/arm-ccn.c @@ -1205,7 +1205,7 @@ static int arm_ccn_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) if (cpu != dt->cpu) return 0; - target = cpumask_any_but(cpu_online_mask, cpu); + target = cpumask_not_dying_but(cpu_online_mask, cpu); if (target >= nr_cpu_ids) return 0; perf_pmu_migrate_context(&dt->pmu, cpu, target); diff --git a/drivers/perf/arm-cmn.c b/drivers/perf/arm-cmn.c index 80d8309652a4..1847182a1ed3 100644 --- a/drivers/perf/arm-cmn.c +++ b/drivers/perf/arm-cmn.c @@ -1787,9 +1787,9 @@ static int arm_cmn_pmu_offline_cpu(unsigned int cpu, struct hlist_node *cpuhp_no node = dev_to_node(cmn->dev); if (cpumask_and(&mask, cpumask_of_node(node), cpu_online_mask) && cpumask_andnot(&mask, &mask, cpumask_of(cpu))) - target = cpumask_any(&mask); + target = cpumask_not_dying_but(&mask, cpu); else - target = cpumask_any_but(cpu_online_mask, cpu); + target = cpumask_not_dying_but(cpu_online_mask, cpu); if (target < nr_cpu_ids) arm_cmn_migrate(cmn, target); return 0; diff --git a/drivers/perf/arm_dmc620_pmu.c b/drivers/perf/arm_dmc620_pmu.c index 280a6ae3e27c..3a0a2bb92e12 100644 --- a/drivers/perf/arm_dmc620_pmu.c +++ b/drivers/perf/arm_dmc620_pmu.c @@ -611,7 +611,7 @@ static int dmc620_pmu_cpu_teardown(unsigned int cpu, if (cpu != irq->cpu) return 0; - target = cpumask_any_but(cpu_online_mask, cpu); + target = cpumask_not_dying_but(cpu_online_mask, cpu); if (target >= nr_cpu_ids) return 0; diff --git a/drivers/perf/arm_dsu_pmu.c b/drivers/perf/arm_dsu_pmu.c index aa9f4393ff0c..e19ce0406b02 100644 --- a/drivers/perf/arm_dsu_pmu.c +++ b/drivers/perf/arm_dsu_pmu.c @@ -236,7 +236,7 @@ static int dsu_pmu_get_online_cpu_any_but(struct dsu_pmu *dsu_pmu, int cpu) cpumask_and(&online_supported, &dsu_pmu->associated_cpus, cpu_online_mask); - return cpumask_any_but(&online_supported, cpu); + return cpumask_not_dying_but(&online_supported, cpu); } static inline bool dsu_pmu_counter_valid(struct dsu_pmu *dsu_pmu, u32 idx) diff --git a/drivers/perf/arm_smmuv3_pmu.c b/drivers/perf/arm_smmuv3_pmu.c index 00d4c45a8017..827315d31056 100644 --- a/drivers/perf/arm_smmuv3_pmu.c +++ b/drivers/perf/arm_smmuv3_pmu.c @@ -640,7 +640,7 @@ static int smmu_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) if (cpu != smmu_pmu->on_cpu) return 0; - target = cpumask_any_but(cpu_online_mask, cpu); + target = cpumask_not_dying_but(cpu_online_mask, cpu); if (target >= nr_cpu_ids) return 0; diff --git a/drivers/perf/fsl_imx8_ddr_perf.c b/drivers/perf/fsl_imx8_ddr_perf.c index 8e058e08fe81..4e0276fc1548 100644 --- a/drivers/perf/fsl_imx8_ddr_perf.c +++ b/drivers/perf/fsl_imx8_ddr_perf.c @@ -664,7 +664,7 @@ static int ddr_perf_offline_cpu(unsigned int cpu, struct hlist_node *node) if (cpu != pmu->cpu) return 0; - target = cpumask_any_but(cpu_online_mask, cpu); + target = cpumask_not_dying_but(cpu_online_mask, cpu); if (target >= nr_cpu_ids) return 0; diff --git a/drivers/perf/hisilicon/hisi_uncore_pmu.c b/drivers/perf/hisilicon/hisi_uncore_pmu.c index fbc8a93d5eac..8c39da8f4b3c 100644 --- a/drivers/perf/hisilicon/hisi_uncore_pmu.c +++ b/drivers/perf/hisilicon/hisi_uncore_pmu.c @@ -518,7 +518,7 @@ int hisi_uncore_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) /* Choose a new CPU to migrate ownership of the PMU to */ cpumask_and(&pmu_online_cpus, &hisi_pmu->associated_cpus, cpu_online_mask); - target = cpumask_any_but(&pmu_online_cpus, cpu); + target = cpumask_not_dying_but(&pmu_online_cpus, cpu); if (target >= nr_cpu_ids) return 0; diff --git a/drivers/perf/marvell_cn10k_tad_pmu.c b/drivers/perf/marvell_cn10k_tad_pmu.c index 69c3050a4348..268e3288893d 100644 --- a/drivers/perf/marvell_cn10k_tad_pmu.c +++ b/drivers/perf/marvell_cn10k_tad_pmu.c @@ -387,7 +387,7 @@ static int tad_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) if (cpu != pmu->cpu) return 0; - target = cpumask_any_but(cpu_online_mask, cpu); + target = cpumask_not_dying_but(cpu_online_mask, cpu); if (target >= nr_cpu_ids) return 0; diff --git a/drivers/perf/qcom_l2_pmu.c b/drivers/perf/qcom_l2_pmu.c index 30234c261b05..8823d0bb6476 100644 --- a/drivers/perf/qcom_l2_pmu.c +++ b/drivers/perf/qcom_l2_pmu.c @@ -822,7 +822,7 @@ static int l2cache_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) /* Any other CPU for this cluster which is still online */ cpumask_and(&cluster_online_cpus, &cluster->cluster_cpus, cpu_online_mask); - target = cpumask_any_but(&cluster_online_cpus, cpu); + target = cpumask_not_dying_but(&cluster_online_cpus, cpu); if (target >= nr_cpu_ids) { disable_irq(cluster->irq); return 0; diff --git a/drivers/perf/qcom_l3_pmu.c b/drivers/perf/qcom_l3_pmu.c index 1ff2ff6582bf..ba26b2fa0736 100644 --- a/drivers/perf/qcom_l3_pmu.c +++ b/drivers/perf/qcom_l3_pmu.c @@ -718,7 +718,7 @@ static int qcom_l3_cache_pmu_offline_cpu(unsigned int cpu, struct hlist_node *no if (!cpumask_test_and_clear_cpu(cpu, &l3pmu->cpumask)) return 0; - target = cpumask_any_but(cpu_online_mask, cpu); + target = cpumask_not_dying_but(cpu_online_mask, cpu); if (target >= nr_cpu_ids) return 0; perf_pmu_migrate_context(&l3pmu->pmu, cpu, target); diff --git a/drivers/perf/xgene_pmu.c b/drivers/perf/xgene_pmu.c index 0c32dffc7ede..069eb0a0d3ba 100644 --- a/drivers/perf/xgene_pmu.c +++ b/drivers/perf/xgene_pmu.c @@ -1804,7 +1804,7 @@ static int xgene_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) if (!cpumask_test_and_clear_cpu(cpu, &xgene_pmu->cpu)) return 0; - target = cpumask_any_but(cpu_online_mask, cpu); + target = cpumask_not_dying_but(cpu_online_mask, cpu); if (target >= nr_cpu_ids) return 0; diff --git a/drivers/soc/fsl/qbman/bman_portal.c b/drivers/soc/fsl/qbman/bman_portal.c index 4d7b9caee1c4..8ebcf87e7d06 100644 --- a/drivers/soc/fsl/qbman/bman_portal.c +++ b/drivers/soc/fsl/qbman/bman_portal.c @@ -67,7 +67,7 @@ static int bman_offline_cpu(unsigned int cpu) return 0; /* use any other online CPU */ - cpu = cpumask_any_but(cpu_online_mask, cpu); + cpu = cpumask_not_dying_but(cpu_online_mask, cpu); irq_set_affinity(pcfg->irq, cpumask_of(cpu)); return 0; } diff --git a/drivers/soc/fsl/qbman/qman_portal.c b/drivers/soc/fsl/qbman/qman_portal.c index e23b60618c1a..3807a8285ced 100644 --- a/drivers/soc/fsl/qbman/qman_portal.c +++ b/drivers/soc/fsl/qbman/qman_portal.c @@ -148,7 +148,7 @@ static int qman_offline_cpu(unsigned int cpu) pcfg = qman_get_qm_portal_config(p); if (pcfg) { /* select any other online CPU */ - cpu = cpumask_any_but(cpu_online_mask, cpu); + cpu = cpumask_not_dying_but(cpu_online_mask, cpu); irq_set_affinity(pcfg->irq, cpumask_of(cpu)); qman_portal_update_sdest(pcfg, cpu); } From patchwork Mon Aug 22 02:15:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12950126 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6401DC00140 for ; Mon, 22 Aug 2022 02:19:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=hiihpLPve/REZQhUrDB+KXE5M9jiubMXMaKcfLXun9c=; b=bUE+RzKMjqckN3 CMs+EalgWpdcl8gU5pnQHB7/t8zsPF9fU6mCUuJa/NleYPE6WaTEPBwtqFAarGQ0Mo6ofx8AoLI49 k5f9BV2wd0DKa2cOCFnmpCO/PNLGSTlIPXXkRNpxI0UrR6b4dX/yt5aV0CJ0CLLAMTIiOwRqez1Md vppXgngwUx4crf+WDF0ltZrC3qpIr+9lONaP5iSIESkK+2UJr4Cneluj/VMPUeNe9P2Wj/EQv9rPt Arp9V20Rj2+WmvtsBupCRnSiDJQsLlArg8kw6yZzQNd1HuVEJQO+XDkx+4dcM4MgnaiiTr5iWS/eq Hkw1FIYmtbxpf7iGluAA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oPx1C-003mtS-1B; Mon, 22 Aug 2022 02:18:22 +0000 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oPwzc-003leM-LB for linux-arm-kernel@lists.infradead.org; Mon, 22 Aug 2022 02:16:46 +0000 Received: by mail-pl1-x62e.google.com with SMTP id m2so8679226pls.4 for ; Sun, 21 Aug 2022 19:16:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=PHnhWNxyM6E9IxuW9zPMNCsqVfB66utEZnswI2QVnUY=; b=moyaT8LSzbsNueFItnBmXfweqqJW9rd5NwnfMitdVq1qYPLzlWYAklL72LaXB9oDvS JpCgS1YpoIHyPnh0KSuubJSnDsEtvOt1uncB6yxmaK8MQOhSUye5ULeoyFZ8gLHcgLtm CZha7A0JSZxdrdNhCQIMcHj8tXNLhk7KWvT6y/0bqmWktx98ivUpz7ndxNdcnYsvd4Gv djm4Wghi5lRR/MwRT4vwVy70tUYaHEdli5D2yg/fRmtSZ4BS+A33PfPf0bFw2aQWfAY8 ieb8o1pFrhi9TXDdC7A1CU5ozlkzki6nUWCPZilyRhE0maI0dWIklZ1jVX0QH4g3WCqd nXxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=PHnhWNxyM6E9IxuW9zPMNCsqVfB66utEZnswI2QVnUY=; b=3ndb89pHmOOcts49TQhPplytTKJ93aAuGjT6svekScZiFVpO37prpPC3nlv95ySyT4 m+XWg7z17n6pEF1KXVuqr/IK+mY5Hrj1jrAfHqEMOsXKviDAc4qAMfcaRglXX4K/Roxz rMXcj1nHwO/ELxtee7QIawEXI0zcOO6BwwW1d8qUPtjijIYNv1M9wlwwvsffNdbHZbvf Dy8HXKeNnCescFSo2Y5fdbqUSZlikILg8Jy+WK9C5ZlJ7JrFVVhfaHkSJYuokG+e2gzq W2K9hhY+RF82qYYuKh/MB7qZm06XmjRbtnTcBVLfmV0J/ZNCinJ+hYvGFdfh4RLN91MK czIg== X-Gm-Message-State: ACgBeo2FCALlF3jLQhtRhAMhlPH4tqhPSTkyT4tm3vFFxFQ/4QSxg9fx Tly/M35cM4Tlstrpb4x54pqtEEzn+A== X-Google-Smtp-Source: AA6agR57jl9X7wl2LO5S9m+7exo7QRftfzMvBzue936dsrZ1sTIFM7iDKbI/NpJ/4k6h+XhaKlz7WA== X-Received: by 2002:a17:90a:c402:b0:1f8:c335:d4d7 with SMTP id i2-20020a17090ac40200b001f8c335d4d7mr26607609pjt.242.1661134602655; Sun, 21 Aug 2022 19:16:42 -0700 (PDT) Received: from piliu.users.ipa.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id k3-20020aa79723000000b005321340753fsm7312139pfg.103.2022.08.21.19.16.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Aug 2022 19:16:42 -0700 (PDT) From: Pingfan Liu To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Pingfan Liu , Catalin Marinas , Will Deacon , Viresh Kumar , Sudeep Holla , Phil Auld , Rob Herring , Ben Dooks Subject: [RFC 10/10] arm64: smp: Make __cpu_disable() parallel Date: Mon, 22 Aug 2022 10:15:20 +0800 Message-Id: <20220822021520.6996-11-kernelfans@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220822021520.6996-1-kernelfans@gmail.com> References: <20220822021520.6996-1-kernelfans@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220821_191644_768074_2B1471E9 X-CRM114-Status: GOOD ( 15.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On a dying cpu, take_cpu_down()->__cpu_disable(), which means if the teardown path supports parallel, __cpu_disable() confront the parallel, which may ruin cpu_online_mask etc if no extra lock provides the protection. At present, the cpumask is protected by cpu_add_remove_lock, that lock is quite above __cpu_disable(). In order to protect __cpu_disable() from parrallel in kexec quick reboot path, introducing a local lock cpumap_lock. Signed-off-by: Pingfan Liu Cc: Catalin Marinas Cc: Will Deacon Cc: Viresh Kumar Cc: Sudeep Holla Cc: Phil Auld Cc: Rob Herring Cc: Ben Dooks To: linux-arm-kernel@lists.infradead.org To: linux-kernel@vger.kernel.org --- arch/arm64/kernel/smp.c | 31 +++++++++++++++++++++++-------- 1 file changed, 23 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index ffc5d76cf695..fee8879048b0 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -287,6 +287,28 @@ static int op_cpu_disable(unsigned int cpu) return 0; } +static DEFINE_SPINLOCK(cpumap_lock); + +static void __cpu_clear_maps(unsigned int cpu) +{ + /* + * In the case of kexec rebooting, the cpu_add_remove_lock mutex can not protect + */ + if (kexec_in_progress) + spin_lock(&cpumap_lock); + remove_cpu_topology(cpu); + numa_remove_cpu(cpu); + + /* + * Take this CPU offline. Once we clear this, we can't return, + * and we must not schedule until we're ready to give up the cpu. + */ + set_cpu_online(cpu, false); + if (kexec_in_progress) + spin_unlock(&cpumap_lock); + +} + /* * __cpu_disable runs on the processor to be shutdown. */ @@ -299,14 +321,7 @@ int __cpu_disable(void) if (ret) return ret; - remove_cpu_topology(cpu); - numa_remove_cpu(cpu); - - /* - * Take this CPU offline. Once we clear this, we can't return, - * and we must not schedule until we're ready to give up the cpu. - */ - set_cpu_online(cpu, false); + __cpu_clear_maps(cpu); ipi_teardown(cpu); /*