[net-next,3/7] net: mscc: ocelot: optimize ocelot_mm_irq()

Commit Message

Vladimir Oltean April 15, 2023, 5:05 p.m. UTC

The MAC Merge IRQ of all ports is shared with the PTP TX timestamp IRQ
of all ports, which means that currently, when a PTP TX timestamp is
generated, felix_irq_handler() also polls for the MAC Merge layer status
of all ports, looking for changes. This makes the kernel do more work,
and under certain circumstances may make ptp4l require a
tx_timestamp_timeout argument higher than before.

Changes to the MAC Merge layer status are only to be expected under
certain conditions - its TX direction needs to be enabled - so we can
check early if that is the case, and omit register access otherwise.

Make ocelot_mm_update_port_status() skip register access if
mm->tx_enabled is unset, and also call it once more, outside IRQ
context, from ocelot_port_set_mm(), when mm->tx_enabled transitions from
true to false, because an IRQ is also expected in that case.

Also, a port may have its MAC Merge layer enabled but it may not have
generated the interrupt. In that case, there's no point in writing to
DEV_MM_STATUS to acknowledge that IRQ. We can reduce the number of
register writes per port with MM enabled by keeping an "ack" variable
which writes the "write-one-to-clear" bits. Those are 3 in number:
PRMPT_ACTIVE_STICKY, UNEXP_RX_PFRM_STICKY and UNEXP_TX_PFRM_STICKY.
The other fields in DEV_MM_STATUS are read-only and it doesn't matter
what is written to them, so writing zero is just fine.

Signed-off-by: Vladimir Oltean <[e-mail]>
---
Diff: patch is new.

 drivers/net/ethernet/mscc/ocelot_mm.c | 30 +++++++++++++++++++++++++--
 include/soc/mscc/ocelot.h             |  1 +
 2 files changed, 29 insertions(+), 2 deletions(-)

Comments

Simon Horman April 17, 2023, 12:55 p.m. UTC | #1

On Sat, Apr 15, 2023 at 08:05:47PM +0300, Vladimir Oltean wrote:
> The MAC Merge IRQ of all ports is shared with the PTP TX timestamp IRQ
> of all ports, which means that currently, when a PTP TX timestamp is
> generated, felix_irq_handler() also polls for the MAC Merge layer status
> of all ports, looking for changes. This makes the kernel do more work,
> and under certain circumstances may make ptp4l require a
> tx_timestamp_timeout argument higher than before.
> 
> Changes to the MAC Merge layer status are only to be expected under
> certain conditions - its TX direction needs to be enabled - so we can
> check early if that is the case, and omit register access otherwise.
> 
> Make ocelot_mm_update_port_status() skip register access if
> mm->tx_enabled is unset, and also call it once more, outside IRQ
> context, from ocelot_port_set_mm(), when mm->tx_enabled transitions from
> true to false, because an IRQ is also expected in that case.
> 
> Also, a port may have its MAC Merge layer enabled but it may not have
> generated the interrupt. In that case, there's no point in writing to
> DEV_MM_STATUS to acknowledge that IRQ. We can reduce the number of
> register writes per port with MM enabled by keeping an "ack" variable
> which writes the "write-one-to-clear" bits. Those are 3 in number:
> PRMPT_ACTIVE_STICKY, UNEXP_RX_PFRM_STICKY and UNEXP_TX_PFRM_STICKY.
> The other fields in DEV_MM_STATUS are read-only and it doesn't matter
> what is written to them, so writing zero is just fine.
> 
> Signed-off-by: Vladimir Oltean <[e-mail]>

Reviewed-by: Simon Horman <[e-mail]>

No need to respin on my account.
However, I do observe that this patch is doing several things,
and I do wonder if it could have been more than one patch.

Florian Fainelli April 17, 2023, 12:59 p.m. UTC | #2

On 4/15/2023 10:05 AM, Vladimir Oltean wrote:
> The MAC Merge IRQ of all ports is shared with the PTP TX timestamp IRQ
> of all ports, which means that currently, when a PTP TX timestamp is
> generated, felix_irq_handler() also polls for the MAC Merge layer status
> of all ports, looking for changes. This makes the kernel do more work,
> and under certain circumstances may make ptp4l require a
> tx_timestamp_timeout argument higher than before.
> 
> Changes to the MAC Merge layer status are only to be expected under
> certain conditions - its TX direction needs to be enabled - so we can
> check early if that is the case, and omit register access otherwise.
> 
> Make ocelot_mm_update_port_status() skip register access if
> mm->tx_enabled is unset, and also call it once more, outside IRQ
> context, from ocelot_port_set_mm(), when mm->tx_enabled transitions from
> true to false, because an IRQ is also expected in that case.
> 
> Also, a port may have its MAC Merge layer enabled but it may not have
> generated the interrupt. In that case, there's no point in writing to
> DEV_MM_STATUS to acknowledge that IRQ. We can reduce the number of
> register writes per port with MM enabled by keeping an "ack" variable
> which writes the "write-one-to-clear" bits. Those are 3 in number:
> PRMPT_ACTIVE_STICKY, UNEXP_RX_PFRM_STICKY and UNEXP_TX_PFRM_STICKY.
> The other fields in DEV_MM_STATUS are read-only and it doesn't matter
> what is written to them, so writing zero is just fine.
> 
> Signed-off-by: Vladimir Oltean <[e-mail]>

Reviewed-by: Florian Fainelli <[e-mail]>

Patch

diff --git a/drivers/net/ethernet/mscc/ocelot_mm.c b/drivers/net/ethernet/mscc/ocelot_mm.c
index d2df47e6f8f6..ce6429d46814 100644
--- a/drivers/net/ethernet/mscc/ocelot_mm.c
+++ b/drivers/net/ethernet/mscc/ocelot_mm.c
@@ -54,7 +54,10 @@  static void ocelot_mm_update_port_status(struct ocelot *ocelot, int port)
 	struct ocelot_port *ocelot_port = ocelot->ports[port];
 	struct ocelot_mm_state *mm = &ocelot->mm[port];
 	enum ethtool_mm_verify_status verify_status;
-	u32 val;
+	u32 val, ack = 0;
+
+	if (!mm->tx_enabled)
+		return;
 
 	val = ocelot_port_readl(ocelot_port, DEV_MM_STATUS);
 
@@ -71,21 +74,28 @@  static void ocelot_mm_update_port_status(struct ocelot *ocelot, int port)
 
 		dev_dbg(ocelot->dev, "Port %d TX preemption %s\n",
 			port, mm->tx_active ? "active" : "inactive");
+
+		ack |= DEV_MM_STAT_MM_STATUS_PRMPT_ACTIVE_STICKY;
 	}
 
 	if (val & DEV_MM_STAT_MM_STATUS_UNEXP_RX_PFRM_STICKY) {
 		dev_err(ocelot->dev,
 			"Unexpected P-frame received on port %d while verification was unsuccessful or not yet verified\n",
 			port);
+
+		ack |= DEV_MM_STAT_MM_STATUS_UNEXP_RX_PFRM_STICKY;
 	}
 
 	if (val & DEV_MM_STAT_MM_STATUS_UNEXP_TX_PFRM_STICKY) {
 		dev_err(ocelot->dev,
 			"Unexpected P-frame requested to be transmitted on port %d while verification was unsuccessful or not yet verified, or MM_TX_ENA=0\n",
 			port);
+
+		ack |= DEV_MM_STAT_MM_STATUS_UNEXP_TX_PFRM_STICKY;
 	}
 
-	ocelot_port_writel(ocelot_port, val, DEV_MM_STATUS);
+	if (ack)
+		ocelot_port_writel(ocelot_port, ack, DEV_MM_STATUS);
 }
 
 void ocelot_mm_irq(struct ocelot *ocelot)
@@ -107,11 +117,14 @@  int ocelot_port_set_mm(struct ocelot *ocelot, int port,
 {
 	struct ocelot_port *ocelot_port = ocelot->ports[port];
 	u32 mm_enable = 0, verify_disable = 0, add_frag_size;
+	struct ocelot_mm_state *mm;
 	int err;
 
 	if (!ocelot->mm_supported)
 		return -EOPNOTSUPP;
 
+	mm = &ocelot->mm[port];
+
 	err = ethtool_mm_frag_size_min_to_add(cfg->tx_min_frag_size,
 					      &add_frag_size, extack);
 	if (err)
@@ -145,6 +158,19 @@  int ocelot_port_set_mm(struct ocelot *ocelot, int port,
 		       QSYS_PREEMPTION_CFG,
 		       port);
 
+	/* The switch will emit an IRQ when TX is disabled, to notify that it
+	 * has become inactive. We optimize ocelot_mm_update_port_status() to
+	 * not bother processing MM IRQs at all for ports with TX disabled,
+	 * but we need to ACK this IRQ now, while mm->tx_enabled is still set,
+	 * otherwise we get an IRQ storm.
+	 */
+	if (mm->tx_enabled && !cfg->tx_enabled) {
+		ocelot_mm_update_port_status(ocelot, port);
+		WARN_ON(mm->tx_active);
+	}
+
+	mm->tx_enabled = cfg->tx_enabled;
+
 	mutex_unlock(&ocelot->fwd_domain_lock);
 
 	return 0;
diff --git a/include/soc/mscc/ocelot.h b/include/soc/mscc/ocelot.h
index 9599be6a0a39..ee8d43dc5c06 100644
--- a/include/soc/mscc/ocelot.h
+++ b/include/soc/mscc/ocelot.h
@@ -745,6 +745,7 @@  struct ocelot_mirror {
 
 struct ocelot_mm_state {
 	enum ethtool_mm_verify_status verify_status;
+	bool tx_enabled;
 	bool tx_active;
 };