| Message ID | 1459909562-22865-6-git-send-email-troy.kisky@boundarydevices.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
From: Troy Kisky <troy.kisky@boundarydevices.com> Sent: Wednesday, April 06, 2016 10:26 AM > To: netdev@vger.kernel.org; davem@davemloft.net; Fugang Duan > <fugang.duan@nxp.com>; lznuaa@gmail.com > Cc: Fabio Estevam <fabio.estevam@nxp.com>; l.stach@pengutronix.de; > andrew@lunn.ch; tremyfr@gmail.com; gerg@uclinux.org; linux-arm- > kernel@lists.infradead.org; johannes@sipsolutions.net; > stillcompiling@gmail.com; sergei.shtylyov@cogentembedded.com; > arnd@arndb.de; Troy Kisky <troy.kisky@boundarydevices.com> > Subject: [PATCH net-next V3 05/16] net: fec: reduce interrupts > > By clearing the NAPI interrupts in the NAPI routine and not in the interrupt > handler, we can reduce the number of interrupts. We also don't need any status > variables as the registers are still valid. > > Also, notice that if budget pkts are received, the next call to fec_enet_rx_napi > will now continue to receive the previously pending packets. > > To test that this actually reduces interrupts, try this command before/after patch > > cat /proc/interrupts |grep ether; \ > ping -s2800 192.168.0.201 -f -c1000 ; \ > cat /proc/interrupts |grep ether > > For me, before this patch is 2996 interrupts. > After patch is 2010 interrupts. > > Signed-off-by: Troy Kisky <troy.kisky@boundarydevices.com> > As my previous comments on V2, if you want to improve performance, you can try to separate tx and rx napi process like calling netif_tx_napi_add() to initialize tx NAPI context.
From: Troy Kisky <troy.kisky@boundarydevices.com> Date: Tue, 5 Apr 2016 19:25:51 -0700 > By clearing the NAPI interrupts in the NAPI routine > and not in the interrupt handler, we can reduce the > number of interrupts. We also don't need any status > variables as the registers are still valid. > > Also, notice that if budget pkts are received, the > next call to fec_enet_rx_napi will now continue to > receive the previously pending packets. > > To test that this actually reduces interrupts, try > this command before/after patch > > cat /proc/interrupts |grep ether; \ > ping -s2800 192.168.0.201 -f -c1000 ; \ > cat /proc/interrupts |grep ether > > For me, before this patch is 2996 interrupts. > After patch is 2010 interrupts. > > Signed-off-by: Troy Kisky <troy.kisky@boundarydevices.com> I really don't think this is a good idea at all. I would instead really rather see you stash away the status register values into some piece of software state, and then re-read them before you are about to finish a NAPI poll cycle.
On 4/6/2016 2:20 PM, David Miller wrote: > From: Troy Kisky <troy.kisky@boundarydevices.com> > Date: Tue, 5 Apr 2016 19:25:51 -0700 > >> By clearing the NAPI interrupts in the NAPI routine >> and not in the interrupt handler, we can reduce the >> number of interrupts. We also don't need any status >> variables as the registers are still valid. >> >> Also, notice that if budget pkts are received, the >> next call to fec_enet_rx_napi will now continue to >> receive the previously pending packets. >> >> To test that this actually reduces interrupts, try >> this command before/after patch >> >> cat /proc/interrupts |grep ether; \ >> ping -s2800 192.168.0.201 -f -c1000 ; \ >> cat /proc/interrupts |grep ether >> >> For me, before this patch is 2996 interrupts. >> After patch is 2010 interrupts. >> >> Signed-off-by: Troy Kisky <troy.kisky@boundarydevices.com> > > I really don't think this is a good idea at all. > > I would instead really rather see you stash away the > status register values into some piece of software state, > and then re-read them before you are about to finish a > NAPI poll cycle. > > Sure, that's an easy change. But if a TX int is what caused the interrupt and masks them, and then a RX packet happens before napi runs, do you want the TX serviced 1st, or RX ?
From: Troy Kisky <troy.kisky@boundarydevices.com> Date: Wed, 6 Apr 2016 17:42:47 -0700 > Sure, that's an easy change. But if a TX int is what caused the > interrupt and masks them, and then a RX packet happens before napi > runs, do you want the TX serviced 1st, or RX ? If you properly split your driver up into seperate interrupt/poll instances, one for TX one for RX, you wouldn't need to ask me that question now would you? :-)
On 4/6/2016 8:57 PM, David Miller wrote: > From: Troy Kisky <troy.kisky@boundarydevices.com> > Date: Wed, 6 Apr 2016 17:42:47 -0700 > >> Sure, that's an easy change. But if a TX int is what caused the >> interrupt and masks them, and then a RX packet happens before napi >> runs, do you want the TX serviced 1st, or RX ? > > If you properly split your driver up into seperate interrupt/poll > instances, one for TX one for RX, you wouldn't need to ask me > that question now would you? > > :-) > I absolutely claim no ownership :-)
diff --git a/drivers/net/ethernet/freescale/fec.h b/drivers/net/ethernet/freescale/fec.h index 6dd0ba8..9d5bdc6 100644 --- a/drivers/net/ethernet/freescale/fec.h +++ b/drivers/net/ethernet/freescale/fec.h @@ -505,11 +505,7 @@ struct fec_enet_private { unsigned int total_tx_ring_size; unsigned int total_rx_ring_size; - - unsigned long work_tx; - unsigned long work_rx; - unsigned long work_ts; - unsigned long work_mdio; + uint events; struct platform_device *pdev; diff --git a/drivers/net/ethernet/freescale/fec_main.c b/drivers/net/ethernet/freescale/fec_main.c index b4d46f8..918ac82 100644 --- a/drivers/net/ethernet/freescale/fec_main.c +++ b/drivers/net/ethernet/freescale/fec_main.c @@ -70,8 +70,6 @@ static void fec_enet_itr_coal_init(struct net_device *ndev); #define DRIVER_NAME "fec" -#define FEC_ENET_GET_QUQUE(_x) ((_x == 0) ? 1 : ((_x == 1) ? 2 : 0)) - /* Pause frame feild and FIFO threshold */ #define FEC_ENET_FCE (1 << 5) #define FEC_ENET_RSEM_V 0x84 @@ -1257,21 +1255,6 @@ static void fec_txq(struct net_device *ndev, struct fec_enet_priv_tx_q *txq) writel(0, txq->bd.reg_desc_active); } -static void -fec_enet_tx(struct net_device *ndev) -{ - struct fec_enet_private *fep = netdev_priv(ndev); - struct fec_enet_priv_tx_q *txq; - u16 queue_id; - /* First process class A queue, then Class B and Best Effort queue */ - for_each_set_bit(queue_id, &fep->work_tx, FEC_ENET_MAX_TX_QS) { - clear_bit(queue_id, &fep->work_tx); - txq = fep->tx_queue[FEC_ENET_GET_QUQUE(queue_id)]; - fec_txq(ndev, txq); - } - return; -} - static int fec_enet_new_rxbdp(struct net_device *ndev, struct bufdesc *bdp, struct sk_buff *skb) { @@ -1505,70 +1488,34 @@ rx_processing_done: return pkt_received; } -static int -fec_enet_rx(struct net_device *ndev, int budget) -{ - int pkt_received = 0; - u16 queue_id; - struct fec_enet_private *fep = netdev_priv(ndev); - struct fec_enet_priv_rx_q *rxq; - - for_each_set_bit(queue_id, &fep->work_rx, FEC_ENET_MAX_RX_QS) { - clear_bit(queue_id, &fep->work_rx); - rxq = fep->rx_queue[FEC_ENET_GET_QUQUE(queue_id)]; - pkt_received += fec_rxq(ndev, rxq, budget - pkt_received); - } - return pkt_received; -} - -static bool -fec_enet_collect_events(struct fec_enet_private *fep, uint int_events) -{ - if (int_events == 0) - return false; - - if (int_events & FEC_ENET_RXF_0) - fep->work_rx |= (1 << 2); - if (int_events & FEC_ENET_RXF_1) - fep->work_rx |= (1 << 0); - if (int_events & FEC_ENET_RXF_2) - fep->work_rx |= (1 << 1); - - if (int_events & FEC_ENET_TXF_0) - fep->work_tx |= (1 << 2); - if (int_events & FEC_ENET_TXF_1) - fep->work_tx |= (1 << 0); - if (int_events & FEC_ENET_TXF_2) - fep->work_tx |= (1 << 1); - - return true; -} - static irqreturn_t fec_enet_interrupt(int irq, void *dev_id) { struct net_device *ndev = dev_id; struct fec_enet_private *fep = netdev_priv(ndev); - uint int_events; irqreturn_t ret = IRQ_NONE; + uint eir = readl(fep->hwp + FEC_IEVENT); + uint int_events = eir & readl(fep->hwp + FEC_IMASK); - int_events = readl(fep->hwp + FEC_IEVENT); - writel(int_events, fep->hwp + FEC_IEVENT); - fec_enet_collect_events(fep, int_events); - - if ((fep->work_tx || fep->work_rx) && fep->link) { - ret = IRQ_HANDLED; - + if (int_events & (FEC_ENET_RXF | FEC_ENET_TXF)) { if (napi_schedule_prep(&fep->napi)) { /* Disable the NAPI interrupts */ writel(FEC_NAPI_IMASK, fep->hwp + FEC_IMASK); __napi_schedule(&fep->napi); + int_events &= ~(FEC_ENET_RXF | FEC_ENET_TXF); + ret = IRQ_HANDLED; + } else { + fep->events |= int_events; + netdev_info(ndev, "couldn't schedule NAPI\n"); } } - if (int_events & FEC_ENET_MII) { + if (int_events) { ret = IRQ_HANDLED; - complete(&fep->mdio_done); + writel(int_events, fep->hwp + FEC_IEVENT); + if (int_events & FEC_ENET_MII) { + complete(&fep->mdio_done); + } } if (fep->ptp_clock) @@ -1581,16 +1528,39 @@ static int fec_enet_rx_napi(struct napi_struct *napi, int budget) { struct net_device *ndev = napi->dev; struct fec_enet_private *fep = netdev_priv(ndev); - int pkts; - - pkts = fec_enet_rx(ndev, budget); + int pkts = 0; + uint events; - fec_enet_tx(ndev); + do { + events = readl(fep->hwp + FEC_IEVENT); + if (fep->events) { + events |= fep->events; + fep->events = 0; + } + events &= FEC_ENET_RXF | FEC_ENET_TXF; + if (!events) { + if (budget) { + napi_complete(napi); + writel(FEC_DEFAULT_IMASK, fep->hwp + FEC_IMASK); + } + return pkts; + } - if (pkts < budget) { - napi_complete(napi); - writel(FEC_DEFAULT_IMASK, fep->hwp + FEC_IMASK); - } + writel(events, fep->hwp + FEC_IEVENT); + if (events & FEC_ENET_RXF_1) + pkts += fec_rxq(ndev, fep->rx_queue[1], budget - pkts); + if (events & FEC_ENET_RXF_2) + pkts += fec_rxq(ndev, fep->rx_queue[2], budget - pkts); + if (events & FEC_ENET_RXF_0) + pkts += fec_rxq(ndev, fep->rx_queue[0], budget - pkts); + if (events & FEC_ENET_TXF_1) + fec_txq(ndev, fep->tx_queue[1]); + if (events & FEC_ENET_TXF_2) + fec_txq(ndev, fep->tx_queue[2]); + if (events & FEC_ENET_TXF_0) + fec_txq(ndev, fep->tx_queue[0]); + } while (pkts < budget); + fep->events |= events & FEC_ENET_RXF; /* save for next callback */ return pkts; }
By clearing the NAPI interrupts in the NAPI routine and not in the interrupt handler, we can reduce the number of interrupts. We also don't need any status variables as the registers are still valid. Also, notice that if budget pkts are received, the next call to fec_enet_rx_napi will now continue to receive the previously pending packets. To test that this actually reduces interrupts, try this command before/after patch cat /proc/interrupts |grep ether; \ ping -s2800 192.168.0.201 -f -c1000 ; \ cat /proc/interrupts |grep ether For me, before this patch is 2996 interrupts. After patch is 2010 interrupts. Signed-off-by: Troy Kisky <troy.kisky@boundarydevices.com> --- v3: Fix introduced bug of checking for FEC_ENET_TS_TIMER before calling fec_ptp_check_pps_event Changed commit message to show measured changes. Used netdev_info instead of pr_info. Fugang Duan suggested splitting TX and RX into two NAPI contexts, but that should be a separate patch as it is unrelated to what this patch does. --- drivers/net/ethernet/freescale/fec.h | 6 +- drivers/net/ethernet/freescale/fec_main.c | 118 +++++++++++------------------- 2 files changed, 45 insertions(+), 79 deletions(-)