From patchwork Sun Feb 9 04:16:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anup Patel X-Patchwork-Id: 13966649 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AE080C02199 for ; Sun, 9 Feb 2025 04:32:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=l67vx6+O+Gzz9kxEY1fAybGouRr4OJWWYxHIz/fmS4Q=; b=diyPgotolpePER g2/zSSTY7bzCilr0SdWIlK84PQMRLWSGyPCPjSlgtYOXGDh6nKLRA0841k2RHLeEzjqf5Gg1vXS/6 xtBFFpOPVkPrIZbPJEWjoYCTp/DlC53nEYaQ8/OUF7iSxcffaIAYGA7BXrmP1yuSX/WT5TrajYY6U Fuob2zjkzn4WWjd0ZG0muStYIuM3pWgag1OgKErxeF8CcKma8Z4TDnjQYtQI1YO1XCyzf66RVSNS8 w43fXIXhY/Dszvogh2YrAGEZhVgH+NKsY6FS3yyp5P8DJEKCHmItnJzvYb8QV/BYKwHr/VPFyM0HF 7bcWZ7HHsNAB3xk6XO6w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tgyzz-0000000E50N-07VM; Sun, 09 Feb 2025 04:32:51 +0000 Received: from mail-pj1-x1030.google.com ([2607:f8b0:4864:20::1030]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tgymP-0000000E2lX-1qF4 for linux-riscv@lists.infradead.org; Sun, 09 Feb 2025 04:18:51 +0000 Received: by mail-pj1-x1030.google.com with SMTP id 98e67ed59e1d1-2f9bac7699aso4844038a91.1 for ; Sat, 08 Feb 2025 20:18:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; t=1739074729; x=1739679529; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KEjndyBD35lV6nr6NSLWRRyly4db2axy7koFOHZTKsQ=; b=CrX4FJud+8PLqilHcLMUJvGFX7l5ypDZVsamK/wbsufkTHijrWP/+DSRasNrGY5zbm geBTq639QkwYl6YLUvOJnqZi0RaZAI8jcGQfCoInMN+awl6ox/kfDiXJXh/jlcxMfW5S 799Jr+cBnYbVffEC1R3ANniIB4qUFpDH0FEV2WEJTqAkFFRFfFbREr/qSFaovQfMmHF1 cPriR6IVo+abpImm1YLLctjhSXqtxmG3TZvAnQeuWxTrw2UYWVMzlDau5+Xg1Hk3cYAx IN0JybrYii/v4L8iu19vngNVrVzUxitkwqY7gBIvHs2iFc/Ur0fdBincBFXluy/Dtb5d VYCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739074729; x=1739679529; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KEjndyBD35lV6nr6NSLWRRyly4db2axy7koFOHZTKsQ=; b=w9KOOaT4eOoHNv9+2Jcm/lGbwSmVD4eBMT7O6NcbKenl1kX6rA5FzE62gjPyn+h7Ri XHUgld7lS7xqdh1djNl7gS9Ve0U2vOjiANJHJeET45isgBkWVCOzZQF7121PiC5jCzlR +NjoDpukN8Goj7rvK3B5Fb+WUc39OeVHET5I4qkU0tjAlkowy0A9p1JvVUJFey/od2BU n3GmYPylTWA9fZVeu5jYsRnXITuB3kX0lQTICws2uHuN91KW/A+C+njegHTzp5WhYJCl gCyd+lMY/ArxUyAQIdqEMBBRw9ZXxJZ720itxUf+yEUajcVHdrGFvlWr4+8gOg5iQMau jkWw== X-Forwarded-Encrypted: i=1; AJvYcCUmcOKraBbGj70gVJ06/KuvAaY21i0vC3govqRebRhJ54JkvHYoYDSkaYyDwk+EtqaoJqOgTESW7KQDkw==@lists.infradead.org X-Gm-Message-State: AOJu0YyR2rCVp3dNXalnf6+5OeBKot6C14xc89eJRZuuwpGlDcCB36ix 107cVQzj0j39TxswkFnvGYMqc9XnJLozhayiQYjrUJMoQxqG4zJpZ4v5iy4UxF4= X-Gm-Gg: ASbGnctB38ww+ddeHz4URoxBx1Fqo9L4w9T6PN/nYHvIk+VxCf90E1I6KmThUTil0+8 gI97OIASqEHrJy+zhLDmsF+Xo6hkd4uPTj+lcRr280FytmVrD9beT+pHQIXyaVmyNC/bihw0R6h 5PAIkA6LXlB3/lZuO3CalQDhYYzVFcs71ydr1FWa+aMO7LI90cquqfDygrjWs3ER+Zi5PUUcrQd BFNRU5gu0pF9yrlRuULSNea+eAdZIv7aKyECr2c+T1PkZwj9Hc29CqAAb5jd4tYhtUylvQGgXM6 49G5z6Bd6W85VgjlpKQi7X5Vkk2GknT8nXwNdRkFScSbIj0WFfj+xvU= X-Google-Smtp-Source: AGHT+IFzlqSdIaYHLSZI599cWVBA7dW7jlgFu/OL/NbLRylvBc/ItMiequCd7pzuj7u4ZD8d08JvJw== X-Received: by 2002:a05:6a00:2342:b0:730:7d3f:8c70 with SMTP id d2e1a72fcca58-7307d3f9339mr5011190b3a.21.1739074728662; Sat, 08 Feb 2025 20:18:48 -0800 (PST) Received: from anup-ubuntu-vm.localdomain ([103.97.166.196]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-73073eba116sm1898410b3a.124.2025.02.08.20.18.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Feb 2025 20:18:48 -0800 (PST) From: Anup Patel To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org Subject: [PATCH v5 11/11] irqchip/riscv-imsic: Special handling for non-atomic device MSI update Date: Sun, 9 Feb 2025 09:46:55 +0530 Message-ID: <20250209041655.331470-12-apatel@ventanamicro.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250209041655.331470-1-apatel@ventanamicro.com> References: <20250209041655.331470-1-apatel@ventanamicro.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250208_201849_483379_45D3E908 X-CRM114-Status: GOOD ( 21.70 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anup Patel , Andrew Lunn , Atish Patra , imx@lists.linux.dev, Marc Zyngier , Sascha Hauer , Paul Walmsley , linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Palmer Dabbelt , Pengutronix Kernel Team , hpa@zytor.com, Anup Patel , Andrew Jones , Shawn Guo , Gregory Clement , linux-arm-kernel@lists.infradead.org, Sebastian Hesselbarth Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Device having non-atomic MSI update might see an intermediate state when changing target IMSIC vector from one CPU to another. To avoid losing interrupt to such intermediate state, do the following (just like x86 APIC): 1) First write a temporary IMSIC vector to the device which has MSI address same as the old IMSIC vector but with MSI data matches the new IMSIC vector. 2) Next write the new IMSIC vector to the device. Based on the above, the __imsic_local_sync() must check pending status of both old MSI data and new MSI data on the old CPU. In addition, the movement of IMSIC vector for non-atomic device MSI update must be done in interrupt context using IRQCHIP_MOVE_DEFERRED. Signed-off-by: Anup Patel --- drivers/irqchip/irq-riscv-imsic-platform.c | 73 +++++++++++++++++++++- drivers/irqchip/irq-riscv-imsic-state.c | 31 +++++++-- 2 files changed, 98 insertions(+), 6 deletions(-) diff --git a/drivers/irqchip/irq-riscv-imsic-platform.c b/drivers/irqchip/irq-riscv-imsic-platform.c index 6bf5d63f614e..828102c46f51 100644 --- a/drivers/irqchip/irq-riscv-imsic-platform.c +++ b/drivers/irqchip/irq-riscv-imsic-platform.c @@ -64,6 +64,11 @@ static int imsic_irq_retrigger(struct irq_data *d) return 0; } +static void imsic_irq_ack(struct irq_data *d) +{ + irq_move_irq(d); +} + static void imsic_irq_compose_vector_msg(struct imsic_vector *vec, struct msi_msg *msg) { phys_addr_t msi_addr; @@ -97,6 +102,21 @@ static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask bool force) { struct imsic_vector *old_vec, *new_vec; + struct imsic_vector tmp_vec; + + /* + * Requirements for the downstream irqdomains (or devices): + * + * 1) Downstream irqdomains (or devices) with atomic MSI update can + * happily do imsic_irq_set_affinity() in the process-context on + * any CPU so the irqchip of such irqdomains must not set the + * IRQCHIP_MOVE_DEFERRED flag. + * + * 2) Downstream irqdomains (or devices) with non-atomic MSI update + * must do imsic_irq_set_affinity() in the interrupt-context upon + * next interrupt so the irqchip of such irqdomains must set the + * IRQCHIP_MOVE_DEFERRED flag. + */ old_vec = irq_data_get_irq_chip_data(d); if (WARN_ON(!old_vec)) @@ -115,6 +135,33 @@ static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask if (!new_vec) return -ENOSPC; + /* + * Device having non-atomic MSI update might see an intermediate + * state when changing target IMSIC vector from one CPU to another. + * + * To avoid losing interrupt to such intermediate state, do the + * following (just like x86 APIC): + * + * 1) First write a temporary IMSIC vector to the device which + * has MSI address same as the old IMSIC vector but MSI data + * matches the new IMSIC vector. + * + * 2) Next write the new IMSIC vector to the device. + * + * Based on the above, the __imsic_local_sync() must check pending + * status of both old MSI data and new MSI data on the old CPU. + */ + + if (!irq_can_move_in_process_context(d) && + new_vec->local_id != old_vec->local_id) { + /* Setup temporary vector */ + tmp_vec.cpu = old_vec->cpu; + tmp_vec.local_id = new_vec->local_id; + + /* Point device to the temporary vector */ + imsic_msi_update_msg(irq_get_irq_data(d->irq), &tmp_vec); + } + /* Point device to the new vector */ imsic_msi_update_msg(irq_get_irq_data(d->irq), new_vec); @@ -171,6 +218,7 @@ static struct irq_chip imsic_irq_base_chip = { .irq_force_complete_move = imsic_irq_force_complete_move, #endif .irq_retrigger = imsic_irq_retrigger, + .irq_ack = imsic_irq_ack, .irq_compose_msi_msg = imsic_irq_compose_msg, .flags = IRQCHIP_SKIP_SET_WAKE | IRQCHIP_MASK_ON_SUSPEND, @@ -190,7 +238,7 @@ static int imsic_irq_domain_alloc(struct irq_domain *domain, unsigned int virq, return -ENOSPC; irq_domain_set_info(domain, virq, virq, &imsic_irq_base_chip, vec, - handle_simple_irq, NULL, NULL); + handle_edge_irq, NULL, NULL); irq_set_noprobe(virq); irq_set_affinity(virq, cpu_online_mask); irq_data_update_effective_affinity(irq_get_irq_data(virq), cpumask_of(vec->cpu)); @@ -229,15 +277,36 @@ static const struct irq_domain_ops imsic_base_domain_ops = { #endif }; +static bool imsic_init_dev_msi_info(struct device *dev, + struct irq_domain *domain, + struct irq_domain *real_parent, + struct msi_domain_info *info) +{ + if (!msi_lib_init_dev_msi_info(dev, domain, real_parent, info)) + return false; + + switch (info->bus_token) { + case DOMAIN_BUS_PCI_DEVICE_MSI: + case DOMAIN_BUS_PCI_DEVICE_MSIX: + info->chip->flags |= IRQCHIP_MOVE_DEFERRED; + break; + default: + break; + } + + return true; +} + static const struct msi_parent_ops imsic_msi_parent_ops = { .supported_flags = MSI_GENERIC_FLAGS_MASK | MSI_FLAG_PCI_MSIX, .required_flags = MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS | MSI_FLAG_PCI_MSI_MASK_PARENT, + .chip_flags = MSI_CHIP_FLAG_SET_ACK, .bus_select_token = DOMAIN_BUS_NEXUS, .bus_select_mask = MATCH_PCI_MSI | MATCH_PLATFORM_MSI, - .init_dev_msi_info = msi_lib_init_dev_msi_info, + .init_dev_msi_info = imsic_init_dev_msi_info, }; int imsic_irqdomain_init(void) diff --git a/drivers/irqchip/irq-riscv-imsic-state.c b/drivers/irqchip/irq-riscv-imsic-state.c index d0148e48ab05..3a2a381e4fa1 100644 --- a/drivers/irqchip/irq-riscv-imsic-state.c +++ b/drivers/irqchip/irq-riscv-imsic-state.c @@ -126,8 +126,8 @@ void __imsic_eix_update(unsigned long base_id, unsigned long num_id, bool pend, static bool __imsic_local_sync(struct imsic_local_priv *lpriv) { - struct imsic_local_config *mlocal; - struct imsic_vector *vec, *mvec; + struct imsic_local_config *tlocal, *mlocal; + struct imsic_vector *vec, *tvec, *mvec; bool ret = true; int i; @@ -169,13 +169,36 @@ static bool __imsic_local_sync(struct imsic_local_priv *lpriv) */ mvec = READ_ONCE(vec->move_next); if (mvec) { - if (__imsic_id_read_clear_pending(i)) { + /* + * Device having non-atomic MSI update might see an + * intermediate state so check both old ID and new ID + * for pending interrupts. + * + * For details, refer imsic_irq_set_affinity(). + */ + + tvec = vec->local_id == mvec->local_id ? + NULL : &lpriv->vectors[mvec->local_id]; + if (tvec && + !irq_can_move_in_process_context(irq_get_irq_data(vec->irq)) && + __imsic_id_read_clear_pending(tvec->local_id)) { + /* Retrigger temporary vector if it was already in-use */ + if (READ_ONCE(tvec->enable)) { + tlocal = per_cpu_ptr(imsic->global.local, tvec->cpu); + writel_relaxed(tvec->local_id, tlocal->msi_va); + } + + mlocal = per_cpu_ptr(imsic->global.local, mvec->cpu); + writel_relaxed(mvec->local_id, mlocal->msi_va); + } + + if (__imsic_id_read_clear_pending(vec->local_id)) { mlocal = per_cpu_ptr(imsic->global.local, mvec->cpu); writel_relaxed(mvec->local_id, mlocal->msi_va); } WRITE_ONCE(vec->move_next, NULL); - imsic_vector_free(&lpriv->vectors[i]); + imsic_vector_free(vec); } skip: