From patchwork Tue Aug 22 21:26:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Anderson X-Patchwork-Id: 13361464 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BC898EE49AF for ; Tue, 22 Aug 2023 21:31:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=m2c2AUIfJ1TUw1Xm7nx/Gwfcp0MuvnkKcrvu0kPDFzE=; b=bPkc+iwZX1ryXZ Oqg/sapQOxI9BBx2S40hvZbdNziK7JiK6IeoRDrPyQBB0zEa3pgE7WvxcHCZZpZt+geeYAS9xG1YT l/NqHPC9dY1Bbk60G00gkZYEN6mJwUFGpaPTNAO3y4X17bn8g7r92gl+WPozm2fPuPFdIxJ9UL1aw Ly7GiO9p6vlHSc/OiZEl+XM+hJZqRk/hnNCWVaY4H59eiqywQrChvDnclFQ6fGfZxfw1E4i0lZscQ lJ3eaB9bsh4yv0LB6/aZZRrHe7zppo3cZ8aEOSbd2YSx4NpQeBtweOSJrhuA3kx1znxYTbV7Zt8R7 1JmXILtj6lQcfr3GXoQw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qYYxk-00Goj7-1M; Tue, 22 Aug 2023 21:30:56 +0000 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qYYxZ-00Goem-2Z for linux-arm-kernel@lists.infradead.org; Tue, 22 Aug 2023 21:30:47 +0000 Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1bf1935f6c2so33576375ad.1 for ; Tue, 22 Aug 2023 14:30:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1692739845; x=1693344645; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=r711O7pLCkc011P6uB8Yxj4dgxTwik66DVJv4kl9k04=; b=W0uZtEg26EQDXtm77b7PMkmXqmK8numJ/QwcYv95/Ts6EKBMM1HlDLpVbRYCbw81V8 Ys7pwl6rqKA9WNOgCz8XJG9/v+w1frEI0o721hNsP19Nw0Spayq6Lm27T/gNm6EMSKw9 WqPt9Ocgsh5MCAecWVIawb+ehMRZn8ysWhur0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692739845; x=1693344645; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=r711O7pLCkc011P6uB8Yxj4dgxTwik66DVJv4kl9k04=; b=UJaCPmlCtGfgycQhgHA1g5TddHMVNxLrdYf5e25E01OnFyjgs1BDWwwhIB7F5ADXyC x35LRgVrMUMG1NaI16CkO6Mlhil55bdMFZ3YNObht4BvTgFD4B1QH32C+NxAG+mn4K/H 64iJakNHuQhoEcDCTwrSkwULkwiN5Z/BNe5GmN49V100r1dorYmeXe6bzpU38S1+lce9 oaNpg0J968p3S38GmWmYk8kLJM6CUiMTxE1UFPgREFO+HqMFJjBfjrp4EZKh7J3ir/aN x1RZgGrRdT+DBURQwj1VQNdwtmCxnjusWcUBROGd1bFYcFgXmI6ALKY4//Bf2NscXUcn 1qCg== X-Gm-Message-State: AOJu0Yxdql79C1yrx9m1yJF4gxKvGwbC+cL9buWDM0FJQAz8LAsZWvd6 VUIVc93AcWOXBUh1xSGzu24oEQ== X-Google-Smtp-Source: AGHT+IGW5Y+S3zA9vhrULldHH4rcxrlq9t2jTqJpEv0NMlrGdVI7baMUBuuwP3C8hHCjW6dG/GGHVg== X-Received: by 2002:a17:902:6504:b0:1bc:2fe1:1821 with SMTP id b4-20020a170902650400b001bc2fe11821mr13113821plk.17.1692739845263; Tue, 22 Aug 2023 14:30:45 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:83f0:2bc5:38c4:a9de]) by smtp.gmail.com with ESMTPSA id t7-20020a170902e84700b001befac3b3cbsm9451475plg.290.2023.08.22.14.30.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Aug 2023 14:30:44 -0700 (PDT) From: Douglas Anderson To: Mark Rutland , Catalin Marinas , Will Deacon , Sumit Garg , Daniel Thompson , Marc Zyngier Cc: Lecopzer Chen , linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, Peter Zijlstra , kgdb-bugreport@lists.sourceforge.net, ito-yuichi@fujitsu.com, Stephen Boyd , Thomas Gleixner , Chen-Yu Tsai , Ard Biesheuvel , "Rafael J . Wysocki" , Masayoshi Mizuma , Douglas Anderson , linux-kernel@vger.kernel.org Subject: [PATCH v10 1/6] irqchip/gic-v3: Enable support for SGIs to act as NMIs Date: Tue, 22 Aug 2023 14:26:56 -0700 Message-ID: <20230822142644.v10.1.I1223c11c88937bd0cbd9b086d4ef216985797302@changeid> X-Mailer: git-send-email 2.42.0.rc1.204.g551eb34607-goog In-Reply-To: <20230822212927.249645-1-dianders@chromium.org> References: <20230822212927.249645-1-dianders@chromium.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230822_143045_835339_968FFCE4 X-CRM114-Status: GOOD ( 23.53 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org As of commit 6abbd6988971 ("irqchip/gic, gic-v3: Make SGIs use handle_percpu_devid_irq()") SGIs are treated the same as PPIs/EPPIs and use handle_percpu_devid_irq() by default. Unfortunately, handle_percpu_devid_irq() isn't NMI safe, and so to run in an NMI context those should use handle_percpu_devid_fasteoi_nmi(). In order to accomplish this, we just have to make room for SGIs in the array of refcounts that keeps track of which interrupts are set as NMI. We also rename the array and create a new indexing scheme that accounts for SGIs. Also, enable NMI support prior to gic_smp_init() as allocation of SGIs as IRQs/NMIs happen as part of this routine. Co-developed-by: Sumit Garg Signed-off-by: Sumit Garg Signed-off-by: Douglas Anderson --- In v10 I removed the previous Reviewed-by and Tested-by tags since the patch contents changed pretty drastically. I'll also note that this change is a little more black magic to me than others in this series. I don't have a massive amounts of familiarity with all the moving parts of gic-v3, so I mostly just followed Mark Rutland's advice [1]. Please pay extra attention to make sure I didn't do anything too terrible. Mark's advice wasn't a full patch and I ended up doing a bit of work to translate it to reality, so I did not add him as "Co-developed-by" here. Mark: if you would like this tag then please provide it and your Signed-off-by. I certainly won't object. [1] https://lore.kernel.org/r/ZNC-YRQopO0PaIIo@FVFF77S0Q05N.cambridge.arm.com Changes in v10: - Rewrite as needed for 5.11+ as per Mark Rutland and Sumit. drivers/irqchip/irq-gic-v3.c | 54 ++++++++++++++++++++++++------------ 1 file changed, 36 insertions(+), 18 deletions(-) diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index eedfa8e9f077..49d18cf3f636 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -78,6 +78,8 @@ static DEFINE_STATIC_KEY_TRUE(supports_deactivate_key); #define GIC_LINE_NR min(GICD_TYPER_SPIS(gic_data.rdists.gicd_typer), 1020U) #define GIC_ESPI_NR GICD_TYPER_ESPIS(gic_data.rdists.gicd_typer) +#define SGI_NR 16 + /* * The behaviours of RPR and PMR registers differ depending on the value of * SCR_EL3.FIQ, and the behaviour of non-secure priority registers of the @@ -125,8 +127,8 @@ EXPORT_SYMBOL(gic_nonsecure_priorities); __priority; \ }) -/* ppi_nmi_refs[n] == number of cpus having ppi[n + 16] set as NMI */ -static refcount_t *ppi_nmi_refs; +/* rdist_nmi_refs[n] == number of cpus having the rdist interrupt n set as NMI */ +static refcount_t *rdist_nmi_refs; static struct gic_kvm_info gic_v3_kvm_info __initdata; static DEFINE_PER_CPU(bool, has_rss); @@ -519,9 +521,22 @@ static u32 __gic_get_ppi_index(irq_hw_number_t hwirq) } } -static u32 gic_get_ppi_index(struct irq_data *d) +static u32 __gic_get_rdist_idx(irq_hw_number_t hwirq) +{ + switch (__get_intid_range(hwirq)) { + case SGI_RANGE: + case PPI_RANGE: + return hwirq; + case EPPI_RANGE: + return hwirq - EPPI_BASE_INTID + 32; + default: + unreachable(); + } +} + +static u32 gic_get_rdist_idx(struct irq_data *d) { - return __gic_get_ppi_index(d->hwirq); + return __gic_get_rdist_idx(d->hwirq); } static int gic_irq_nmi_setup(struct irq_data *d) @@ -545,11 +560,14 @@ static int gic_irq_nmi_setup(struct irq_data *d) /* desc lock should already be held */ if (gic_irq_in_rdist(d)) { - u32 idx = gic_get_ppi_index(d); + u32 idx = gic_get_rdist_idx(d); - /* Setting up PPI as NMI, only switch handler for first NMI */ - if (!refcount_inc_not_zero(&ppi_nmi_refs[idx])) { - refcount_set(&ppi_nmi_refs[idx], 1); + /* + * Setting up a percpu interrupt as NMI, only switch handler + * for first NMI + */ + if (!refcount_inc_not_zero(&rdist_nmi_refs[idx])) { + refcount_set(&rdist_nmi_refs[idx], 1); desc->handle_irq = handle_percpu_devid_fasteoi_nmi; } } else { @@ -582,10 +600,10 @@ static void gic_irq_nmi_teardown(struct irq_data *d) /* desc lock should already be held */ if (gic_irq_in_rdist(d)) { - u32 idx = gic_get_ppi_index(d); + u32 idx = gic_get_rdist_idx(d); /* Tearing down NMI, only switch handler for last NMI */ - if (refcount_dec_and_test(&ppi_nmi_refs[idx])) + if (refcount_dec_and_test(&rdist_nmi_refs[idx])) desc->handle_irq = handle_percpu_devid_irq; } else { desc->handle_irq = handle_fasteoi_irq; @@ -1279,10 +1297,10 @@ static void gic_cpu_init(void) rbase = gic_data_rdist_sgi_base(); /* Configure SGIs/PPIs as non-secure Group-1 */ - for (i = 0; i < gic_data.ppi_nr + 16; i += 32) + for (i = 0; i < gic_data.ppi_nr + SGI_NR; i += 32) writel_relaxed(~0, rbase + GICR_IGROUPR0 + i / 8); - gic_cpu_config(rbase, gic_data.ppi_nr + 16, gic_redist_wait_for_rwp); + gic_cpu_config(rbase, gic_data.ppi_nr + SGI_NR, gic_redist_wait_for_rwp); /* initialise system registers */ gic_cpu_sys_reg_init(); @@ -1939,12 +1957,13 @@ static void gic_enable_nmi_support(void) return; } - ppi_nmi_refs = kcalloc(gic_data.ppi_nr, sizeof(*ppi_nmi_refs), GFP_KERNEL); - if (!ppi_nmi_refs) + rdist_nmi_refs = kcalloc(gic_data.ppi_nr + SGI_NR, + sizeof(*rdist_nmi_refs), GFP_KERNEL); + if (!rdist_nmi_refs) return; - for (i = 0; i < gic_data.ppi_nr; i++) - refcount_set(&ppi_nmi_refs[i], 0); + for (i = 0; i < gic_data.ppi_nr + SGI_NR; i++) + refcount_set(&rdist_nmi_refs[i], 0); pr_info("Pseudo-NMIs enabled using %s ICC_PMR_EL1 synchronisation\n", gic_has_relaxed_pmr_sync() ? "relaxed" : "forced"); @@ -2061,6 +2080,7 @@ static int __init gic_init_bases(phys_addr_t dist_phys_base, gic_dist_init(); gic_cpu_init(); + gic_enable_nmi_support(); gic_smp_init(); gic_cpu_pm_init(); @@ -2073,8 +2093,6 @@ static int __init gic_init_bases(phys_addr_t dist_phys_base, gicv2m_init(handle, gic_data.domain); } - gic_enable_nmi_support(); - return 0; out_free: From patchwork Tue Aug 22 21:26:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Anderson X-Patchwork-Id: 13361463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 32931EE49A5 for ; Tue, 22 Aug 2023 21:31:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vE4I3yLrkoFu9yvJx357Q+QjZe4BDJQDzApYZCTp6yQ=; b=VFbRuv5tPG7dWa /SFNMpfyzLMqRkpi4wOlID0Mo/Lt/mksiTtk3GuWL0yDjqAFqSUMbMEvKBs/xvfYb8nAcZyG/gESb LRbL+InncO37nwwPUe+lF8tSePRc5xYjxSylwO1EHEm3apEVAHabd6v+iUMD5twPvg6ETSKXPUmvz ClU3AHqMoUVehjiutkFxZJRf2u72LQWXmF1aYAdwywRce5qP8MdqOFsXCKvxg7893Dmzj6oOXQyag qcQozJU4hTCNzxC7jj2w0mEXhE3Zej7v0tdoWEkBPHQp/NVWIHI5beketZl2ZZYdJ9ujA+bRr+Nag gsYF3K3udqq2Xg3VrnfQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qYYxk-00GojO-2z; Tue, 22 Aug 2023 21:30:56 +0000 Received: from mail-pl1-x631.google.com ([2607:f8b0:4864:20::631]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qYYxc-00Gofn-2L for linux-arm-kernel@lists.infradead.org; Tue, 22 Aug 2023 21:30:50 +0000 Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1bdc243d62bso32344945ad.3 for ; Tue, 22 Aug 2023 14:30:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1692739848; x=1693344648; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DRXZdprKPZs0xk/TIH25QXtGg9YiAlfKIIpohS5vrjQ=; b=VC9hZKzbdMPhWPoT/CJpgKx+lHz3MuRKi7xqq07FBHgIMAV6jdjAsGxc2oPCq5LdiD FT9TBosfHaLzOnTftw1blVl3kZEL5lLkpIenrJsSd4HqMrukFEHtIYNNkoT19AyNPYqZ XWLrShyvPU7jO3ZGVV82MhHq4fTZ7rwjwD5kk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692739848; x=1693344648; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DRXZdprKPZs0xk/TIH25QXtGg9YiAlfKIIpohS5vrjQ=; b=GYYPO/L+UgrGE167lMvTw5e5INY9JElStGI6XLaQJUl9LKvdERIT7/kqrLPb5RULW8 LmcQqPoWmheEvhyWRsg//AAPIY3FN3KXcJXkBunqKoC5/c94pKLXYul9w3/yARfdgrHm A6VtusnhvdbvGh1p30OY+xXveU1FuJUxV8OPBegEzt0yEWwDCaDhaj8ADy3yrH1ZDwxP No3VolmtKZcp3g3Z4AUCGxGnOd5Wmaeqt46nBqbY3cRs3iZYJcuU54JcZxwGKERIICps HGRcix8XPGlmZ3LKl0lO9x9z9t1POilmfrEzutzRr3F8DBP+MjcDOCRrnSI9x7zqMQbA 6n2A== X-Gm-Message-State: AOJu0YwmG67zVBZJi6ujVqQrUJbLN3H5DnlXTzf7uqBsZUA9G4mpV0mj s6pqbwmPJYn0PW1+X998xaRbwA== X-Google-Smtp-Source: AGHT+IHOe4StZk9p/o2Zvwe4Q/iRgNulteHoQQ85TwT9W8Rl6RYQ6kjOCWG1GGHBmeC9Gt4Hw7/U1A== X-Received: by 2002:a17:903:234a:b0:1bf:4833:9c2b with SMTP id c10-20020a170903234a00b001bf48339c2bmr10564192plh.31.1692739847743; Tue, 22 Aug 2023 14:30:47 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:83f0:2bc5:38c4:a9de]) by smtp.gmail.com with ESMTPSA id t7-20020a170902e84700b001befac3b3cbsm9451475plg.290.2023.08.22.14.30.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Aug 2023 14:30:47 -0700 (PDT) From: Douglas Anderson To: Mark Rutland , Catalin Marinas , Will Deacon , Sumit Garg , Daniel Thompson , Marc Zyngier Cc: Lecopzer Chen , linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, Peter Zijlstra , kgdb-bugreport@lists.sourceforge.net, ito-yuichi@fujitsu.com, Stephen Boyd , Thomas Gleixner , Chen-Yu Tsai , Ard Biesheuvel , "Rafael J . Wysocki" , Masayoshi Mizuma , Douglas Anderson , Frederic Weisbecker , Guo Ren , linux-kernel@vger.kernel.org Subject: [PATCH v10 2/6] arm64: idle: Tag the arm64 idle functions as __cpuidle Date: Tue, 22 Aug 2023 14:26:57 -0700 Message-ID: <20230822142644.v10.2.I4baba13e220bdd24d11400c67f137c35f07f82c7@changeid> X-Mailer: git-send-email 2.42.0.rc1.204.g551eb34607-goog In-Reply-To: <20230822212927.249645-1-dianders@chromium.org> References: <20230822212927.249645-1-dianders@chromium.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230822_143048_769235_603CE78B X-CRM114-Status: GOOD ( 18.24 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org As per the (somewhat recent) comment before the definition of `__cpuidle`, the tag is like `noinstr` but also marks a function so it can be identified by cpu_in_idle(). Let'a add this. After doing this then when we dump stack traces of all processors using nmi_cpu_backtrace() then instead of getting useless backtraces we get things like: NMI backtrace for cpu N skipped: idling at cpu_do_idle+0x94/0x98 NOTE: this patch won't make cpu_in_idle() work perfectly for arm64, but it doesn't hurt and does catch some cases. Specifically an example that wasn't caught in my testing looked like this: gic_cpu_sys_reg_init+0x1f8/0x314 gic_cpu_pm_notifier+0x40/0x78 raw_notifier_call_chain+0x5c/0x134 cpu_pm_notify+0x38/0x64 cpu_pm_exit+0x20/0x2c psci_enter_idle_state+0x48/0x70 cpuidle_enter_state+0xb8/0x260 cpuidle_enter+0x44/0x5c do_idle+0x188/0x30c Acked-by: Mark Rutland Signed-off-by: Douglas Anderson Reviewed-by: Stephen Boyd --- (no changes since v9) Changes in v9: - Added to commit message that this doesn't catch all cases. Changes in v8: - "Tag the arm64 idle functions as __cpuidle" new for v8 arch/arm64/kernel/idle.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/idle.c b/arch/arm64/kernel/idle.c index c1125753fe9b..05cfb347ec26 100644 --- a/arch/arm64/kernel/idle.c +++ b/arch/arm64/kernel/idle.c @@ -20,7 +20,7 @@ * ensure that interrupts are not masked at the PMR (because the core will * not wake up if we block the wake up signal in the interrupt controller). */ -void noinstr cpu_do_idle(void) +void __cpuidle cpu_do_idle(void) { struct arm_cpuidle_irq_context context; @@ -35,7 +35,7 @@ void noinstr cpu_do_idle(void) /* * This is our default idle handler. */ -void noinstr arch_cpu_idle(void) +void __cpuidle arch_cpu_idle(void) { /* * This should do all the clock switching and wait for interrupt From patchwork Tue Aug 22 21:26:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Anderson X-Patchwork-Id: 13361462 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EC452EE49AE for ; Tue, 22 Aug 2023 21:31:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=k9hAGGDpNbE9iP1yJLMJvYS00J20Hut6vvO9KPYao+4=; b=ue5sMmuThfyeYp V/Nj9Toete9OP1Aau0jDeAMH76cuLC1SMCKNhpLoXVkSv32Hi6ccg9QUEDa02Vtfac2SzmWRrRrKG y03ePLZSlSlgVuliemYSWLedUHxktLYFXwql5cmm3/tJgWhioVgPQijHU658CKmdsQGeAd8kUyODb /AD0aWBHnZEl4z2GNIMGEHskkzgziUsnIqrLNC1/moZyp+OiboxPPp87o2KTk2qT5DFN1q7u/ylpk Wzgq0Iu9oSbHz+4nWCfBJIB0VRgTZbaRvufOnLI/hG7hxGxsAm306pOQq8YinJNrCgoPcTdNdbFnP waC/fNmI2ZaMI/VNz9eg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qYYxl-00Gojm-1Z; Tue, 22 Aug 2023 21:30:57 +0000 Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qYYxe-00Gogk-2I for linux-arm-kernel@lists.infradead.org; Tue, 22 Aug 2023 21:30:52 +0000 Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-1bf078d5fb7so33245605ad.0 for ; Tue, 22 Aug 2023 14:30:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1692739850; x=1693344650; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7NTSblsAu8PNy2OgT43FimYv6UYWrsT3fb8iGcpeqoY=; b=SSwVk0/Y6ZNUglAJO0DD6i6/m5NNNngXeG3Y6lw2BIZyrknN1/imUbQMG5WLp0Op+m d9VQYQnoaA6skHN+tGDuiemWjBBac93s/XMTWlw8mW/NLtczFpdMrwj2baINYvFmYBlV +hNDcfecQe2m+uT4ySEkasCQpLR/uSQXdaOkQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692739850; x=1693344650; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7NTSblsAu8PNy2OgT43FimYv6UYWrsT3fb8iGcpeqoY=; b=Cp3tdGru7tyS3ZWvwh76+a5YUgvsUH3a+smPvTrZbxjG76FsLUBh1WxwOAcOnyvgZ/ /vKFdn84dD15TE9MDBCoYql8DsE/RxsZ37WiVF5L3jDRHE6MR8osqRAQpaUs7nn2eFBF OgxN149RZQwic5CNWvKjcRUl+2onX9z8ZGyj2HvBpEoCjC1AMeEvNus004MJxLomM2X9 kFX23G5yszVESwLY62v02bbv2rq726TgHR5aJU0oCR2G/YRuWMgILAWswf8b0N2CtfXL zhjqWLqIQxMmlDqodhp5qmM44EMR1fwhV7Z8bBIaCqde9oy3XE+k44j2SerfNg5ST9s+ p/yA== X-Gm-Message-State: AOJu0Yz2WJ7ZSfM8HyxSIrDlLi9fWhPAJeG/5CEzapkCXDlD/4IMUKDQ 6D3NMc5tnSNL0Qb/Q0IsPtUc6w== X-Google-Smtp-Source: AGHT+IHA2klR/hpR4pC+xNHo1EGJFPmRBiLjPNVhxt8TvBFzkQhWq2zyCjyW7Wf1mZrMjB+TX9rFuw== X-Received: by 2002:a17:902:b403:b0:1bd:bfc0:4627 with SMTP id x3-20020a170902b40300b001bdbfc04627mr7708342plr.40.1692739850223; Tue, 22 Aug 2023 14:30:50 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:83f0:2bc5:38c4:a9de]) by smtp.gmail.com with ESMTPSA id t7-20020a170902e84700b001befac3b3cbsm9451475plg.290.2023.08.22.14.30.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Aug 2023 14:30:49 -0700 (PDT) From: Douglas Anderson To: Mark Rutland , Catalin Marinas , Will Deacon , Sumit Garg , Daniel Thompson , Marc Zyngier Cc: Lecopzer Chen , linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, Peter Zijlstra , kgdb-bugreport@lists.sourceforge.net, ito-yuichi@fujitsu.com, Stephen Boyd , Thomas Gleixner , Chen-Yu Tsai , Ard Biesheuvel , "Rafael J . Wysocki" , Masayoshi Mizuma , Douglas Anderson , D Scott Phillips , Ingo Molnar , Josh Poimboeuf , Kees Cook , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Sami Tolvanen , Valentin Schneider , linux-kernel@vger.kernel.org Subject: [PATCH v10 3/6] arm64: smp: Remove dedicated wakeup IPI Date: Tue, 22 Aug 2023 14:26:58 -0700 Message-ID: <20230822142644.v10.3.I7209db47ef8ec151d3de61f59005bbc59fe8f113@changeid> X-Mailer: git-send-email 2.42.0.rc1.204.g551eb34607-goog In-Reply-To: <20230822212927.249645-1-dianders@chromium.org> References: <20230822212927.249645-1-dianders@chromium.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230822_143050_757873_B4C9458C X-CRM114-Status: GOOD ( 22.10 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Mark Rutland To enable NMI backtrace and KGDB's NMI cpu roundup, we need to free up at least one dedicated IPI. On arm64 the IPI_WAKEUP IPI is only used for the ACPI parking protocol, which itself is only used on some very early ARMv8 systems which couldn't implement PSCI. Remove the IPI_WAKEUP IPI, and rely on the IPI_RESCHEDULE IPI to wake CPUs from the parked state. This will cause a tiny amonut of redundant work to check the thread flags, but this is miniscule in relation to the cost of taking and handling the IPI in the first place. We can safely handle redundant IPI_RESCHEDULE IPIs, so there should be no functional impact as a result of this change. Signed-off-by: Mark Rutland Signed-off-by: Douglas Anderson Cc: Catalin Marinas Cc: Marc Zyngier Cc: Sumit Garg Cc: Will Deacon --- I have no idea how to test this. I just took Mark's patch and jammed it into my series. Logicially the patch seems reasonable to me. Changes in v10: - ("arm64: smp: Remove dedicated wakeup IPI") new for v10. arch/arm64/include/asm/smp.h | 4 ++-- arch/arm64/kernel/acpi_parking_protocol.c | 2 +- arch/arm64/kernel/smp.c | 28 +++++++++-------------- 3 files changed, 14 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h index 9b31e6d0da17..798fd76a883a 100644 --- a/arch/arm64/include/asm/smp.h +++ b/arch/arm64/include/asm/smp.h @@ -89,9 +89,9 @@ extern void arch_send_call_function_single_ipi(int cpu); extern void arch_send_call_function_ipi_mask(const struct cpumask *mask); #ifdef CONFIG_ARM64_ACPI_PARKING_PROTOCOL -extern void arch_send_wakeup_ipi_mask(const struct cpumask *mask); +extern void arch_send_wakeup_ipi(int cpu); #else -static inline void arch_send_wakeup_ipi_mask(const struct cpumask *mask) +static inline void arch_send_wakeup_ipi(int cpu) { BUILD_BUG(); } diff --git a/arch/arm64/kernel/acpi_parking_protocol.c b/arch/arm64/kernel/acpi_parking_protocol.c index b1990e38aed0..e1be29e608b7 100644 --- a/arch/arm64/kernel/acpi_parking_protocol.c +++ b/arch/arm64/kernel/acpi_parking_protocol.c @@ -103,7 +103,7 @@ static int acpi_parking_protocol_cpu_boot(unsigned int cpu) &mailbox->entry_point); writel_relaxed(cpu_entry->gic_cpu_id, &mailbox->cpu_id); - arch_send_wakeup_ipi_mask(cpumask_of(cpu)); + arch_send_wakeup_ipi(cpu); return 0; } diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 960b98b43506..bfe60adbc453 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -72,7 +72,6 @@ enum ipi_msg_type { IPI_CPU_CRASH_STOP, IPI_TIMER, IPI_IRQ_WORK, - IPI_WAKEUP, NR_IPI }; @@ -764,7 +763,6 @@ static const char *ipi_types[NR_IPI] __tracepoint_string = { [IPI_CPU_CRASH_STOP] = "CPU stop (for crash dump) interrupts", [IPI_TIMER] = "Timer broadcast interrupts", [IPI_IRQ_WORK] = "IRQ work interrupts", - [IPI_WAKEUP] = "CPU wake-up interrupts", }; static void smp_cross_call(const struct cpumask *target, unsigned int ipinr); @@ -797,13 +795,6 @@ void arch_send_call_function_single_ipi(int cpu) smp_cross_call(cpumask_of(cpu), IPI_CALL_FUNC); } -#ifdef CONFIG_ARM64_ACPI_PARKING_PROTOCOL -void arch_send_wakeup_ipi_mask(const struct cpumask *mask) -{ - smp_cross_call(mask, IPI_WAKEUP); -} -#endif - #ifdef CONFIG_IRQ_WORK void arch_irq_work_raise(void) { @@ -897,14 +888,6 @@ static void do_handle_IPI(int ipinr) break; #endif -#ifdef CONFIG_ARM64_ACPI_PARKING_PROTOCOL - case IPI_WAKEUP: - WARN_ONCE(!acpi_parking_protocol_valid(cpu), - "CPU%u: Wake-up IPI outside the ACPI parking protocol\n", - cpu); - break; -#endif - default: pr_crit("CPU%u: Unknown IPI message 0x%x\n", cpu, ipinr); break; @@ -979,6 +962,17 @@ void arch_smp_send_reschedule(int cpu) smp_cross_call(cpumask_of(cpu), IPI_RESCHEDULE); } +#ifdef CONFIG_ARM64_ACPI_PARKING_PROTOCOL +void arch_send_wakeup_ipi(int cpu) +{ + /* + * We use a scheduler IPI to wake the CPU as this avoids the need for a + * dedicated IPI and we can safely handle spurious scheduler IPIs. + */ + arch_smp_send_reschedule(cpu); +} +#endif + #ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST void tick_broadcast(const struct cpumask *mask) { From patchwork Tue Aug 22 21:26:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Anderson X-Patchwork-Id: 13361465 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9ECA5EE49B1 for ; Tue, 22 Aug 2023 21:31:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=OmjAHWN+9qwrTuMfA0jwY+kpO/B9HM/zA1ayxEUPHgs=; b=cXQtwNxH7N4DIl tdMYF3bAMDyZlcFZ8hc3FRdx8lnVgFFjLepgxpAXubqv/yXO8rOTcEuszBVdSt/L0dB3XS42DHH5w n5O4NqszIFaehB3QMNJwLRknDFX3skOQsurhKuO1zwfdB6YpbC2B3JEgFnYgt8nHDFfryvqJGo2xB MyKXtTJF44TqJlcFAyhjq+TwXKFPGOKqGFn2pnfE/Bvt/wMCZyUaq6JhYGEW6KFPGKpV77ii0KIbf vXEuisKd82ffORV8FAfXvEJtbM6yHwHiiHVRJ+uxL+eeDrTZOkvLPyl/b1Uvb+Ax4MmC3xEbQAhWu ec/jO0WpK00P05tKocYQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qYYxm-00Gok5-00; Tue, 22 Aug 2023 21:30:58 +0000 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qYYxh-00Goha-2S for linux-arm-kernel@lists.infradead.org; Tue, 22 Aug 2023 21:30:55 +0000 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1bf3a2f4528so36871765ad.2 for ; Tue, 22 Aug 2023 14:30:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1692739852; x=1693344652; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rKZ6vpkmoXshy0WXF69vUhU6JiKC/AMqd7Fq/TEmz2w=; b=e0UE++PmLtN2Wv0Jj/gw8YSmQIuKof+cdGUnV8LtEBcFGaV1iIh3pI90em7t8Wg+Ia Zp++cPMF6vyua9rsgbr+BqpbFkssXLC2YN/xcuinOzPtTi2Fmr0KtHvjcTBKd3zak4aQ 7qDORQFpfMRhhs9NKqGq93HCYinF2lvC9agZw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692739852; x=1693344652; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rKZ6vpkmoXshy0WXF69vUhU6JiKC/AMqd7Fq/TEmz2w=; b=XQpbRwguyL1qXe9FEZneRSTF8ht8KLjSG34ukCm7awwq8q6T/ijc6rqge/RuuqIUCz X7Fsn3yuOp+6D0Wn44FeMnMKmV9eYSSa6bWAAEXpX5iGSCOuGHD3eghTrbX7T6AGZii5 VD96R/ZPkBZW1Mu/aibCIDSL1FAMNSdrq3ZVzUZUYUu/Bu0L4Q0QipXeHSFfnCOvRSpD alPKriXLJ+PnYNharVtBr2sxT8Y/VXLkvRyHmsBIoMkrh/TEHIp2YKy9NtBdTz25Hjed Fn/I05yxsh/Hl3B98XrvKbXZTBvaqxgLFqIoCh9eqLtGavFhx4CXTkPnqkG0wWoGXgiP uN6Q== X-Gm-Message-State: AOJu0YySC7Cy1ne6jnFy1+QZmumeihLlY6h/5LoLsoEDtlV87VmEsfU6 quOLtkZHCjxFhc0f9A2pBQvqzQ== X-Google-Smtp-Source: AGHT+IHv614Zsz+g1uwMiDi74vgcx/0NSFoXfnRW0gsCWP7mBJdr1fbUHmRVXO/8i4UzaPLRDck+7g== X-Received: by 2002:a17:902:efc1:b0:1bc:66a3:9a4b with SMTP id ja1-20020a170902efc100b001bc66a39a4bmr9087350plb.5.1692739852664; Tue, 22 Aug 2023 14:30:52 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:83f0:2bc5:38c4:a9de]) by smtp.gmail.com with ESMTPSA id t7-20020a170902e84700b001befac3b3cbsm9451475plg.290.2023.08.22.14.30.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Aug 2023 14:30:52 -0700 (PDT) From: Douglas Anderson To: Mark Rutland , Catalin Marinas , Will Deacon , Sumit Garg , Daniel Thompson , Marc Zyngier Cc: Lecopzer Chen , linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, Peter Zijlstra , kgdb-bugreport@lists.sourceforge.net, ito-yuichi@fujitsu.com, Stephen Boyd , Thomas Gleixner , Chen-Yu Tsai , Ard Biesheuvel , "Rafael J . Wysocki" , Masayoshi Mizuma , Douglas Anderson , D Scott Phillips , Ingo Molnar , Josh Poimboeuf , Valentin Schneider , linux-kernel@vger.kernel.org Subject: [PATCH v10 4/6] arm64: smp: Add arch support for backtrace using pseudo-NMI Date: Tue, 22 Aug 2023 14:26:59 -0700 Message-ID: <20230822142644.v10.4.Ie6c132b96ebbbcddbf6954b9469ed40a6960343c@changeid> X-Mailer: git-send-email 2.42.0.rc1.204.g551eb34607-goog In-Reply-To: <20230822212927.249645-1-dianders@chromium.org> References: <20230822212927.249645-1-dianders@chromium.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230822_143053_804878_97AE470B X-CRM114-Status: GOOD ( 28.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Enable arch_trigger_cpumask_backtrace() support on arm64. This enables things much like they are enabled on arm32 (including some of the funky logic around NR_IPI, nr_ipi, and MAX_IPI) but with the difference that, unlike arm32, we'll try to enable the backtrace to use pseudo-NMI. NOTE: this patch is a squash of the little bit of code adding the ability to mark an IPI to try to use pseudo-NMI plus the little bit of code to hook things up for kgdb. This approach was decided upon in the discussion of v9 [1]. This patch depends on commit 36759e343ff9 ("nmi_backtrace: allow excluding an arbitrary CPU") since that commit changed the prototype of arch_trigger_cpumask_backtrace(), which this patch implements. [1] https://lore.kernel.org/r/ZORY51mF4alI41G1@FVFF77S0Q05N Co-developed-by: Sumit Garg Signed-off-by: Sumit Garg Co-developed-by: Mark Rutland Signed-off-by: Mark Rutland Signed-off-by: Douglas Anderson --- Changes in v10: - Backtrace now directly supported in smp.c - Squash backtrace into patch adding support for pseudo-NMI IPIs. Changes in v9: - Added comments that we might not be using NMI always. - Fold in v8 patch #10 ("Fallback to a regular IPI if NMI isn't enabled") - Moved header file out of "include" since it didn't need to be there. - Remove arm64_supports_nmi() - Renamed "NMI IPI" to "debug IPI" since it might not be backed by NMI. - arch_trigger_cpumask_backtrace() no longer returns bool Changes in v8: - Removed "#ifdef CONFIG_SMP" since arm64 is always SMP - debug_ipi_setup() and debug_ipi_teardown() no longer take cpu param arch/arm64/include/asm/irq.h | 3 ++ arch/arm64/kernel/smp.c | 86 +++++++++++++++++++++++++++++++----- 2 files changed, 78 insertions(+), 11 deletions(-) diff --git a/arch/arm64/include/asm/irq.h b/arch/arm64/include/asm/irq.h index fac08e18bcd5..50ce8b697ff3 100644 --- a/arch/arm64/include/asm/irq.h +++ b/arch/arm64/include/asm/irq.h @@ -6,6 +6,9 @@ #include +void arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu); +#define arch_trigger_cpumask_backtrace arch_trigger_cpumask_backtrace + struct pt_regs; int set_handle_irq(void (*handle_irq)(struct pt_regs *)); diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index bfe60adbc453..15b66dc1391b 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include @@ -72,12 +73,18 @@ enum ipi_msg_type { IPI_CPU_CRASH_STOP, IPI_TIMER, IPI_IRQ_WORK, - NR_IPI + NR_IPI, + /* + * CPU_BACKTRACE is special and not included in NR_IPI + * or tracable with trace_ipi_* + */ + IPI_CPU_BACKTRACE = NR_IPI, + MAX_IPI }; static int ipi_irq_base __read_mostly; static int nr_ipi __read_mostly = NR_IPI; -static struct irq_desc *ipi_desc[NR_IPI] __read_mostly; +static struct irq_desc *ipi_desc[MAX_IPI] __read_mostly; static void ipi_setup(int cpu); @@ -845,6 +852,22 @@ static void __noreturn ipi_cpu_crash_stop(unsigned int cpu, struct pt_regs *regs #endif } +void arm64_backtrace_ipi(cpumask_t *mask) +{ + __ipi_send_mask(ipi_desc[IPI_CPU_BACKTRACE], mask); +} + +void arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) +{ + /* + * NOTE: though nmi_trigger_cpumask_backtrace has "nmi_" in the name, + * nothing about it truly needs to be backed by an NMI, it's just that + * it's _allowed_ to work with NMIs. If ipi_should_be_nmi() returned + * false our backtrace attempt will just be backed by a regular IPI. + */ + nmi_trigger_cpumask_backtrace(mask, exclude_cpu, arm64_backtrace_ipi); +} + /* * Main handler for inter-processor interrupts */ @@ -888,6 +911,14 @@ static void do_handle_IPI(int ipinr) break; #endif + case IPI_CPU_BACKTRACE: + /* + * NOTE: in some cases this _won't_ be NMI context. See the + * comment in arch_trigger_cpumask_backtrace(). + */ + nmi_cpu_backtrace(get_irq_regs()); + break; + default: pr_crit("CPU%u: Unknown IPI message 0x%x\n", cpu, ipinr); break; @@ -909,6 +940,19 @@ static void smp_cross_call(const struct cpumask *target, unsigned int ipinr) __ipi_send_mask(ipi_desc[ipinr], target); } +static bool ipi_should_be_nmi(enum ipi_msg_type ipi) +{ + if (!system_uses_irq_prio_masking()) + return false; + + switch (ipi) { + case IPI_CPU_BACKTRACE: + return true; + default: + return false; + } +} + static void ipi_setup(int cpu) { int i; @@ -916,8 +960,14 @@ static void ipi_setup(int cpu) if (WARN_ON_ONCE(!ipi_irq_base)) return; - for (i = 0; i < nr_ipi; i++) - enable_percpu_irq(ipi_irq_base + i, 0); + for (i = 0; i < nr_ipi; i++) { + if (ipi_should_be_nmi(i)) { + prepare_percpu_nmi(ipi_irq_base + i); + enable_percpu_nmi(ipi_irq_base + i, 0); + } else { + enable_percpu_irq(ipi_irq_base + i, 0); + } + } } #ifdef CONFIG_HOTPLUG_CPU @@ -928,8 +978,14 @@ static void ipi_teardown(int cpu) if (WARN_ON_ONCE(!ipi_irq_base)) return; - for (i = 0; i < nr_ipi; i++) - disable_percpu_irq(ipi_irq_base + i); + for (i = 0; i < nr_ipi; i++) { + if (ipi_should_be_nmi(i)) { + disable_percpu_nmi(ipi_irq_base + i); + teardown_percpu_nmi(ipi_irq_base + i); + } else { + disable_percpu_irq(ipi_irq_base + i); + } + } } #endif @@ -937,15 +993,23 @@ void __init set_smp_ipi_range(int ipi_base, int n) { int i; - WARN_ON(n < NR_IPI); - nr_ipi = min(n, NR_IPI); + WARN_ON(n < MAX_IPI); + nr_ipi = min(n, MAX_IPI); for (i = 0; i < nr_ipi; i++) { int err; - err = request_percpu_irq(ipi_base + i, ipi_handler, - "IPI", &cpu_number); - WARN_ON(err); + if (ipi_should_be_nmi(i)) { + err = request_percpu_nmi(ipi_base + i, ipi_handler, + "IPI", &cpu_number); + WARN(err, "Could not request IPI %d as NMI, err=%d\n", + i, err); + } else { + err = request_percpu_irq(ipi_base + i, ipi_handler, + "IPI", &cpu_number); + WARN(err, "Could not request IPI %d as IRQ, err=%d\n", + i, err); + } ipi_desc[i] = irq_to_desc(ipi_base + i); irq_set_status_flags(ipi_base + i, IRQ_HIDDEN); From patchwork Tue Aug 22 21:27:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Anderson X-Patchwork-Id: 13361467 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1AEA0EE49AF for ; Tue, 22 Aug 2023 21:31:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=M5U5/3Vz8vMlQXGfHOsLleK6KJSioo7l4KOA1874Ous=; b=gIsS/b6fwzX/wd zK+88nXQilpLPlpMzO1hfg6Bf+/Hrp+t0ub7P3lKs0UPp3twsQXVskBFmD2VVh3Ha5h7ra3Sp/HBb 9PuyG7UXGUOYzcKgbvc1ZHu3dES1PV0kNrcXUdaL4XOYfn/1RfPBxObYtXew5Sks01GJFC9oPet5x OSSND1UHMfEkT/24yI3yOk3CXX4ikLkq3bKfIxR+LlGZxPnemn3eliOVk1NqhNO8ee770ktukR8H8 M/i1F6cN7f9QOjeB6WnSJJEYoq2raIKB4fzid9YsCarf97sPrFXC93V6MaByV73FVa8FoI/GlTdgy mtbViTuRrxE9AqOoBMBA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qYYyC-00Gorb-1f; Tue, 22 Aug 2023 21:31:24 +0000 Received: from mail-pl1-x62f.google.com ([2607:f8b0:4864:20::62f]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qYYxj-00GoiT-1U for linux-arm-kernel@lists.infradead.org; Tue, 22 Aug 2023 21:30:56 +0000 Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1bf0b24d925so32454535ad.3 for ; Tue, 22 Aug 2023 14:30:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1692739855; x=1693344655; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sHNo52jMQHCTTXv1njx0att1y1oc+xWVwUeLIRRfBAg=; b=YC0PScEPND5aZfA24ZoaI8wC1TSlRy2EYibMqFd2iUs8w7ZRXT3RKFlha3o6KDnWYU nghvF99uhQidPrOCTS7kFbZPlAlehSol1ZWc0mEDaepO6A4obV5tBwUEQIFSbbZWPuUB UgSIx4tkrAH1Cw/lnH0Vt27I3n+AIHy+x0WXA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692739855; x=1693344655; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sHNo52jMQHCTTXv1njx0att1y1oc+xWVwUeLIRRfBAg=; b=A9Yfzy3yLB2x8LKoGvrDdp/Xogpk0Ff8MAj2fzk6SQdm3eEO+B0t8cIOKIRtLvK/4A /+xt13tflZK6gwRL8TYAE/7gMGc6z9DbEMqor4vOK9iRl65jIDHVbvuy3ROi6V/NYHyR 3T0LhxxiUMjTp5kZdhLIEGrfP+fWOPYYqYuXZWpyMPsOProilWcMDKFIugwc+JbK7jYz S+kxdodceMa5Fg/3TkdikqwMfgamZWojeBbzFAW7Y9TkqPfwkDdOzjSPHK45W/yBQsPG qdFx06AWGd5TDKfcH7SpHWqh5tG75ByXUf9LIUkCNnZkqfmHKY7cBrvE5ZdyAxDmjEgg T8IA== X-Gm-Message-State: AOJu0YxqgWzrwd4u+l0G3dMutSqmo5mOsddX0ZpolayeIbZCD1lCBI5O 1bkuWeRC7pKpA++5liQCqeWPvA== X-Google-Smtp-Source: AGHT+IHswwKksNgSxyqElMvXEh66yr67LTxIGthRXPjy6R7WvznnHL9O4TkqksLBbcEBtlvUxYpXTg== X-Received: by 2002:a17:902:e842:b0:1b8:63c6:84ab with SMTP id t2-20020a170902e84200b001b863c684abmr9273364plg.61.1692739854952; Tue, 22 Aug 2023 14:30:54 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:83f0:2bc5:38c4:a9de]) by smtp.gmail.com with ESMTPSA id t7-20020a170902e84700b001befac3b3cbsm9451475plg.290.2023.08.22.14.30.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Aug 2023 14:30:54 -0700 (PDT) From: Douglas Anderson To: Mark Rutland , Catalin Marinas , Will Deacon , Sumit Garg , Daniel Thompson , Marc Zyngier Cc: Lecopzer Chen , linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, Peter Zijlstra , kgdb-bugreport@lists.sourceforge.net, ito-yuichi@fujitsu.com, Stephen Boyd , Thomas Gleixner , Chen-Yu Tsai , Ard Biesheuvel , "Rafael J . Wysocki" , Masayoshi Mizuma , Douglas Anderson , D Scott Phillips , Ingo Molnar , Josh Poimboeuf , Valentin Schneider , linux-kernel@vger.kernel.org Subject: [PATCH v10 5/6] arm64: smp: IPI_CPU_STOP and IPI_CPU_CRASH_STOP should try for NMI Date: Tue, 22 Aug 2023 14:27:00 -0700 Message-ID: <20230822142644.v10.5.Ifadbfd45b22c52edcb499034dd4783d096343260@changeid> X-Mailer: git-send-email 2.42.0.rc1.204.g551eb34607-goog In-Reply-To: <20230822212927.249645-1-dianders@chromium.org> References: <20230822212927.249645-1-dianders@chromium.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230822_143055_495736_CA231C26 X-CRM114-Status: GOOD ( 12.39 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org There's no reason why IPI_CPU_STOP and IPI_CPU_CRASH_STOP can't be handled as NMI. They are very simple and everything in them is NMI-safe. Mark them as things to use NMI for if NMI is available. Suggested-by: Mark Rutland Signed-off-by: Douglas Anderson Reviewed-by: Stephen Boyd --- I don't actually have any good way to test/validate this patch. It's added to the series at Mark's request. Changes in v10: - ("IPI_CPU_STOP and IPI_CPU_CRASH_STOP should try for NMI") new for v10. arch/arm64/kernel/smp.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 15b66dc1391b..c51d54019f5f 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -946,6 +946,8 @@ static bool ipi_should_be_nmi(enum ipi_msg_type ipi) return false; switch (ipi) { + case IPI_CPU_STOP: + case IPI_CPU_CRASH_STOP: case IPI_CPU_BACKTRACE: return true; default: From patchwork Tue Aug 22 21:27:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Anderson X-Patchwork-Id: 13361466 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 598B4EE49A5 for ; Tue, 22 Aug 2023 21:31:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=jFHojD6GU4quuJrYLxcRP2ibFATT0n8d5ZRwLjcsOh0=; b=YmpHfSWJ7kkQHy /lSFMoN3dhiQxHOyftNvHQtvWuGOEWUyUa2fzOVcgK5b78tgY7ch+oLbWjv2ss5xejJeEpj9RnBFU xjrvAhDtujKOZpWiSRyV0SxWkacx1AAA0P6g5Ts3S7CnO0agg/QCNtsCkMk9tF4OEtwS3/1Vrmzaj /SXJMgg26THur4l0x14cTE31TEYiZFn3YL5RbfTcxd6Kdvzt5VhDgFEgkhtKbf5tbe1DMgNZps/GG EZaG18WZD34BWc8Icaj+zlWynz7BdDIXLWuYiAJzvcHCJityUyj4EbbmjkwIjBxjdEFb9LaZr6dvn eYXOm6BP2RyFHvJIOieQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qYYyD-00Gorn-02; Tue, 22 Aug 2023 21:31:25 +0000 Received: from mail-pf1-x431.google.com ([2607:f8b0:4864:20::431]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qYYxm-00Gojy-15 for linux-arm-kernel@lists.infradead.org; Tue, 22 Aug 2023 21:30:59 +0000 Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-689f9576babso3353581b3a.0 for ; Tue, 22 Aug 2023 14:30:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1692739857; x=1693344657; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=P5gCGegQW5oGbMQ/BTeF2BnmDhxlWa9PqRY9ItnmdA4=; b=i2j2emfnK5XP7t+5MPqLiRHwEldyIx58mwxs8qZCGbml5ejZ6JH7+3EA7zApwPx7Xp DKzJZt+j4/BQ260X/yfDtcOiRK549Q7XtaY8dwyRKZzevzWXRbEs0+J82fq0ifmhegao Q+N6UM9e5wW7FSXIc832M2QlxQB51Qtz5z/vA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692739857; x=1693344657; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=P5gCGegQW5oGbMQ/BTeF2BnmDhxlWa9PqRY9ItnmdA4=; b=NAqWS8wdcDIv2LGcluyWrf8vG6WCS6YHNMVd/cDbrVFziLELcSsN0AHJI7GHFhMG5p VBeZsrUo/0DJkRwT1VxmI7XwYsvEyo1M46HauLvoMI7ZBifTu1ypAV16gHuk8JiKS5sz SI3QSnxwdocqZrSDmCXZ3puQSc9w94YVbneJnl8w7aV0Tsz/2TD9iePWiTc5NFID+bBo hQgKwmx+Tqc7HvKzlWT0xCPXcpPh7riSuulZeZ98ztYoRoCS0WJIf5vF3IexDPxFBQHe woFzFxCCSwKY60XkJQI7sseCU57g8sdjmkZ34ohNKK1RkHuZ1kfRKaLLJ8/41KnBH87j B2pA== X-Gm-Message-State: AOJu0Yyf50fFmgLr2bytMSSEDwuIMjK7Ol/YNFVfFZrNfvBSlR68XneF O6eqy1HeP9eV4qBcI26PgA5bHg== X-Google-Smtp-Source: AGHT+IHzEMJvwLDxrbrpnP1PL0V1S39JCggLWYRvJ4nQH33U88JSDAsszYF9eNIHgI3tAnb1soYQPw== X-Received: by 2002:a17:903:41cc:b0:1af:aafb:64c8 with SMTP id u12-20020a17090341cc00b001afaafb64c8mr8892237ple.21.1692739857271; Tue, 22 Aug 2023 14:30:57 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:83f0:2bc5:38c4:a9de]) by smtp.gmail.com with ESMTPSA id t7-20020a170902e84700b001befac3b3cbsm9451475plg.290.2023.08.22.14.30.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Aug 2023 14:30:56 -0700 (PDT) From: Douglas Anderson To: Mark Rutland , Catalin Marinas , Will Deacon , Sumit Garg , Daniel Thompson , Marc Zyngier Cc: Lecopzer Chen , linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, Peter Zijlstra , kgdb-bugreport@lists.sourceforge.net, ito-yuichi@fujitsu.com, Stephen Boyd , Thomas Gleixner , Chen-Yu Tsai , Ard Biesheuvel , "Rafael J . Wysocki" , Masayoshi Mizuma , Douglas Anderson , D Scott Phillips , Josh Poimboeuf , Valentin Schneider , linux-kernel@vger.kernel.org Subject: [PATCH v10 6/6] arm64: kgdb: Implement kgdb_roundup_cpus() to enable pseudo-NMI roundup Date: Tue, 22 Aug 2023 14:27:01 -0700 Message-ID: <20230822142644.v10.6.I2ef26d1b3bfbed2d10a281942b0da7d9854de05e@changeid> X-Mailer: git-send-email 2.42.0.rc1.204.g551eb34607-goog In-Reply-To: <20230822212927.249645-1-dianders@chromium.org> References: <20230822212927.249645-1-dianders@chromium.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230822_143058_380040_92A62942 X-CRM114-Status: GOOD ( 25.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Up until now we've been using the generic (weak) implementation for kgdb_roundup_cpus() when using kgdb on arm64. Let's move to a custom one. The advantage here is that, when pseudo-NMI is enabled on a device, we'll be able to round up CPUs using pseudo-NMI. This allows us to debug CPUs that are stuck with interrupts disabled. If pseudo-NMIs are not enabled then we'll fallback to just using an IPI, which is still slightly better than the generic implementation since it avoids the potential situation described in the generic kgdb_call_nmi_hook(). Co-developed-by: Sumit Garg Signed-off-by: Sumit Garg Signed-off-by: Douglas Anderson --- I debated whether this should be in "arch/arm64/kernel/smp.c" or if I should try to find a way for it to go into "arch/arm64/kernel/kgdb.c". In the end this is so little code that it didn't seem worth it to find a way to export the IPI defines or to otherwise come up with some API between kgdb.c and smp.c. If someone has strong feelings and wants this to change, please shout and give details of your preferred solution. FWIW, it seems like ~half the other platforms put this in "smp.c" with an ifdef for KGDB and the other half put it in "kgdb.c" with an ifdef for SMP. :-P Changes in v10: - Don't allocate the cpumask on the stack; just iterate. - Moved kgdb calls to smp.c to avoid needing to export IPI info. - kgdb now has its own IPI. Changes in v9: - Remove fallback for when debug IPI isn't available. - Renamed "NMI IPI" to "debug IPI" since it might not be backed by NMI. arch/arm64/kernel/smp.c | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index c51d54019f5f..5ee6b69b4360 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -32,6 +32,7 @@ #include #include #include +#include #include #include @@ -75,10 +76,11 @@ enum ipi_msg_type { IPI_IRQ_WORK, NR_IPI, /* - * CPU_BACKTRACE is special and not included in NR_IPI + * CPU_BACKTRACE and KGDB_ROUNDUP are special and not included in NR_IPI * or tracable with trace_ipi_* */ IPI_CPU_BACKTRACE = NR_IPI, + IPI_KGDB_ROUNDUP, MAX_IPI }; @@ -868,6 +870,22 @@ void arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) nmi_trigger_cpumask_backtrace(mask, exclude_cpu, arm64_backtrace_ipi); } +#ifdef CONFIG_KGDB +void kgdb_roundup_cpus(void) +{ + int this_cpu = raw_smp_processor_id(); + int cpu; + + for_each_online_cpu(cpu) { + /* No need to roundup ourselves */ + if (cpu == this_cpu) + continue; + + __ipi_send_single(ipi_desc[IPI_KGDB_ROUNDUP], cpu); + } +} +#endif + /* * Main handler for inter-processor interrupts */ @@ -919,6 +937,10 @@ static void do_handle_IPI(int ipinr) nmi_cpu_backtrace(get_irq_regs()); break; + case IPI_KGDB_ROUNDUP: + kgdb_nmicallback(cpu, get_irq_regs()); + break; + default: pr_crit("CPU%u: Unknown IPI message 0x%x\n", cpu, ipinr); break; @@ -949,6 +971,7 @@ static bool ipi_should_be_nmi(enum ipi_msg_type ipi) case IPI_CPU_STOP: case IPI_CPU_CRASH_STOP: case IPI_CPU_BACKTRACE: + case IPI_KGDB_ROUNDUP: return true; default: return false;