From patchwork Sun Oct 29 18:35:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mirsad Todorovac X-Patchwork-Id: 13439856 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4AC7E10942 for ; Sun, 29 Oct 2023 18:42:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=alu.unizg.hr header.i=@alu.unizg.hr header.b="y877EDNJ"; dkim=pass (2048-bit key) header.d=alu.unizg.hr header.i=@alu.unizg.hr header.b="HsvHeQ7y" Received: from domac.alu.hr (domac.alu.unizg.hr [161.53.235.3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BB2BDBC; Sun, 29 Oct 2023 11:42:19 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by domac.alu.hr (Postfix) with ESMTP id 5843560187; Sun, 29 Oct 2023 19:42:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698604937; bh=aGnuUMcaiZKyu6bxd0gchGWjjPRuLHhYcfTW+nYpD4s=; h=From:To:Cc:Subject:Date:From; b=y877EDNJQzI9JIXiUlWCazdj+b4MMqsdq0bFqrO2rUhZ03UL4zD7xLuJhC1O8qrLs nch/gc7UfcU7LnuAJA6glsXD+vQS/jYEGDFOo6SJxrQ957REI6/NcD3S3F5qhQxcgT fbvclOGxQhpH2+Ut8LfTHIa1HA5dSx6o+AfGWPTFQVO1BWaNlf5P+tFfzhGU3UXN2r Cny24dOrz96lGwnGEgXD+gT2W2WoEugXWiNC4Da9vlVSwv0lrSSOsQrf42vNLlQxkg PzX6/xNjvqp4bL86aISd3ZrY4ymNlQazFQuxOBu6zc3WLUcFC+cYdHpeso+awcVkS6 orZpeMMpeaA5w== X-Virus-Scanned: Debian amavisd-new at domac.alu.hr Received: from domac.alu.hr ([127.0.0.1]) by localhost (domac.alu.hr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b2ku-x8BoIkh; Sun, 29 Oct 2023 19:42:14 +0100 (CET) Received: from defiant.home (78-3-40-247.adsl.net.t-com.hr [78.3.40.247]) by domac.alu.hr (Postfix) with ESMTPSA id 7982E60173; Sun, 29 Oct 2023 19:42:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698604934; bh=aGnuUMcaiZKyu6bxd0gchGWjjPRuLHhYcfTW+nYpD4s=; h=From:To:Cc:Subject:Date:From; b=HsvHeQ7yq9Szwl4hvaxE8a4VLqpAv0B+NaVAGWA11ZuW2w7Bmg0Dy5rZBa3R+jyxn zC2u7Pglbh1aMUD5V5hNWPbIa+4kICgNACg8fKUiVYloqyf/zfkXLn+imEsT4/ZRoH k2c+ZjIR5hUO1AbAuFfZpB2OpM3wJA7bgZNiXEITVGXgFcue3488iO6KrZKK+CLBle 5WVh3ewa0gtEGUAncXgQVip5tU8acc9ZlBliH2TwOKtRSU9DxX2mIdNUFBXZSTVVyd +X3OlgYYSpsRbmmxsed4/+VPgKKYGO5QK1iPF+xR5zEtE+dEt58+l3Yic5o/wGlX04 l3RDk8wdD5Ogw== From: Mirsad Goran Todorovac To: Jason Gunthorpe , Joerg Roedel , Lu Baolu , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Heiner Kallweit , nic_swsd@realtek.com, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirsad Goran Todorovac , Marco Elver Subject: [PATCH v5 1/7] r8169: Add r8169_mac_ocp_(write|modify)_seq helpers to reduce spinlock contention Date: Sun, 29 Oct 2023 19:35:55 +0100 Message-Id: <20231029183600.451694-1-mirsad.todorovac@alu.unizg.hr> X-Mailer: git-send-email 2.34.1 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org A pair of new helpers r8168_mac_ocp_write_seq() and r8168_mac_ocp_modify_seq() are introduced. The motivation for these helpers was the locking overhead of 130 consecutive r8168_mac_ocp_write() calls in the RTL8411b reset after the NIC gets confused if the PHY is powered-down. To quote Heiner: On RTL8411b the RX unit gets confused if the PHY is powered-down. This was reported in [0] and confirmed by Realtek. Realtek provided a sequence to fix the RX unit after PHY wakeup. A series of about 130 r8168_mac_ocp_write() calls is performed to program the RTL registers for recovery, each doing an expensive spin_lock_irqsave() and spin_unlock_irqrestore(). Each mac ocp write is made of: static void __r8168_mac_ocp_write(struct rtl8169_private *tp, u32 reg, u32 data) { if (rtl_ocp_reg_failure(reg)) return; RTL_W32(tp, OCPDR, OCPAR_FLAG | (reg << 15) | data); } static void r8168_mac_ocp_write(struct rtl8169_private *tp, u32 reg, u32 data) { unsigned long flags; raw_spin_lock_irqsave(&tp->mac_ocp_lock, flags); __r8168_mac_ocp_write(tp, reg, data); raw_spin_unlock_irqrestore(&tp->mac_ocp_lock, flags); } Register programming is done through RTL_W32() macro which expands into #define RTL_W32(tp, reg, val32) writel((val32), tp->mmio_addr + (reg)) which is further (on Alpha): extern inline void writel(u32 b, volatile void __iomem *addr) { mb(); __raw_writel(b, addr); } or on i386/x86_64: #define build_mmio_write(name, size, type, reg, barrier) \ static inline void name(type val, volatile void __iomem *addr) \ { asm volatile("mov" size " %0,%1": :reg (val), \ "m" (*(volatile type __force *)addr) barrier); } build_mmio_write(writel, "l", unsigned int, "r", :"memory") This obviously involves iat least a compiler barrier. mb() expands into something like this i.e. on x86_64: #define mb() asm volatile("lock; addl $0,0(%%esp)" ::: "memory") This means a whole lot of memory bus stalls: for spin_lock_irqsave(), memory barrier, writel(), and spin_unlock_irqrestore(). With about 130 of these sequential calls to r8168_mac_ocp_write() this looks like a lock storm that will stall all of the cores and CPUs on the same memory controller for certain time I/O takes to finish. In a sequential case of RTL register programming, the writes to RTL registers can be coalesced under a same raw spinlock. This can dramatically decrease the number of bus stalls in a multicore or multi-CPU system. Macro helpers r8168_mac_ocp_write_seq() and r8168_mac_ocp_modify_seq() are provided to reduce lock contention: static void rtl_hw_start_8411_2(struct rtl8169_private *tp) { ... /* The following Realtek-provided magic fixes an issue with the RX unit * getting confused after the PHY having been powered-down. */ static const struct recover_8411b_info init_zero_seq[] = { { 0xFC28, 0x0000 }, { 0xFC2A, 0x0000 }, { 0xFC2C, 0x0000 }, ... }; ... r8168_mac_ocp_write_seq(tp, init_zero_seq); ... } The hex data is preserved intact through s/r8168_mac_ocp_write[(]tp,/{ / and s/[)];/ },/ functions that only changed the function names and the ending of the line, so the actual hex data is unchanged. To repeat, the reason for the introduction of the original commit was to enable recovery of the RX unit on the RTL8411b which was confused by the powered-down PHY. This sequence of r8168_mac_ocp_write() calls amplifies the problem into a series of about 500+ memory bus locks, most waiting for the main memory read, modify and write under a LOCK. The memory barrier in RTL_W32 should suffice for the programming sequence to reach RTL NIC registers. [0] https://bugzilla.redhat.com/show_bug.cgi?id=1692075 Fixes: fe4e8db0392a6 ("r8169: fix issue with confused RX unit after PHY power-down on RTL8411b") Fixes: 91c8643578a21 ("r8169: use spinlock to protect mac ocp register access") Fixes: d6c36cbc5e533 ("r8169: Use a raw_spinlock_t for the register locks.") Cc: Heiner Kallweit Cc: Marco Elver Cc: nic_swsd@realtek.com Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org Link: https://lore.kernel.org/lkml/20231028005153.2180411-1-mirsad.todorovac@alu.unizg.hr/ Link: https://lore.kernel.org/lkml/20231028110459.2644926-1-mirsad.todorovac@alu.unizg.hr/ Signed-off-by: Mirsad Goran Todorovac --- v5: added unlocked primitives to allow mac ocs modify grouping applied coalescing of mac ocp writes/modifies for 8168ep and 8117 some formatting fixes to please checkpatch.pl v4: fixed complaints as advised by Heiner and checkpatch.pl split the patch into five sections to be more easily manipulated and reviewed introduced r8168_mac_ocp_write_seq() applied coalescing of mac ocp writes/modifies for 8168H, 8125 and 8125B v3: removed register/mask pair array sentinels, so using ARRAY_SIZE(). avoided duplication of RTL_W32() call code as advised by Heiner. drivers/net/ethernet/realtek/r8169_main.c | 71 +++++++++++++++++++++-- 1 file changed, 66 insertions(+), 5 deletions(-) diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index 361b90007148..da1f5d1b4fd5 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -888,7 +888,7 @@ static int r8168_phy_ocp_read(struct rtl8169_private *tp, u32 reg) (RTL_R32(tp, GPHY_OCP) & 0xffff) : -ETIMEDOUT; } -static void __r8168_mac_ocp_write(struct rtl8169_private *tp, u32 reg, u32 data) +static inline void __r8168_mac_ocp_write(struct rtl8169_private *tp, u32 reg, u32 data) { if (rtl_ocp_reg_failure(reg)) return; @@ -905,7 +905,7 @@ static void r8168_mac_ocp_write(struct rtl8169_private *tp, u32 reg, u32 data) raw_spin_unlock_irqrestore(&tp->mac_ocp_lock, flags); } -static u16 __r8168_mac_ocp_read(struct rtl8169_private *tp, u32 reg) +static inline u16 __r8168_mac_ocp_read(struct rtl8169_private *tp, u32 reg) { if (rtl_ocp_reg_failure(reg)) return 0; @@ -927,18 +927,79 @@ static u16 r8168_mac_ocp_read(struct rtl8169_private *tp, u32 reg) return val; } +static inline void __r8168_mac_ocp_modify(struct rtl8169_private *tp, u32 reg, u16 mask, + u16 set) +{ + u16 data; + + data = __r8168_mac_ocp_read(tp, reg); + __r8168_mac_ocp_write(tp, reg, (data & ~mask) | set); +} + static void r8168_mac_ocp_modify(struct rtl8169_private *tp, u32 reg, u16 mask, u16 set) { unsigned long flags; - u16 data; raw_spin_lock_irqsave(&tp->mac_ocp_lock, flags); - data = __r8168_mac_ocp_read(tp, reg); - __r8168_mac_ocp_write(tp, reg, (data & ~mask) | set); + __r8168_mac_ocp_modify(tp, reg, mask, set); raw_spin_unlock_irqrestore(&tp->mac_ocp_lock, flags); } +struct e_info_regdata { + u32 reg; + u32 data; +}; + +struct e_info_regmaskset { + u32 reg; + u16 mask; + u16 set; +}; + +static void __r8168_mac_ocp_write_seqlen(struct rtl8169_private *tp, + const struct e_info_regdata *array, int len) +{ + struct e_info_regdata const *p; + + for (p = array; len--; p++) + __r8168_mac_ocp_write(tp, p->reg, p->data); +} + +static void r8168_mac_ocp_write_seqlen(struct rtl8169_private *tp, + const struct e_info_regdata *array, int len) +{ + unsigned long flags; + + raw_spin_lock_irqsave(&tp->mac_ocp_lock, flags); + __r8168_mac_ocp_write_seqlen(tp, array, len); + raw_spin_unlock_irqrestore(&tp->mac_ocp_lock, flags); +} + +static void __r8168_mac_ocp_modify_seqlen(struct rtl8169_private *tp, + const struct e_info_regmaskset *array, int len) +{ + struct e_info_regmaskset const *p; + + for (p = array; len--; p++) + __r8168_mac_ocp_modify(tp, p->reg, p->mask, p->set); +} + +static void r8168_mac_ocp_modify_seqlen(struct rtl8169_private *tp, + const struct e_info_regmaskset *array, int len) +{ + unsigned long flags; + + raw_spin_lock_irqsave(&tp->mac_ocp_lock, flags); + __r8168_mac_ocp_modify_seqlen(tp, array, len); + raw_spin_unlock_irqrestore(&tp->mac_ocp_lock, flags); +} + +#define r8168_mac_ocp_write_seq(tp, a) r8168_mac_ocp_write_seqlen(tp, a, ARRAY_SIZE(a)) +#define r8168_mac_ocp_modify_seq(tp, a) r8168_mac_ocp_modify_seqlen(tp, a, ARRAY_SIZE(a)) +#define __r8168_mac_ocp_write_seq(tp, a) __r8168_mac_ocp_write_seqlen(tp, a, ARRAY_SIZE(a)) +#define __r8168_mac_ocp_modify_seq(tp, a) __r8168_mac_ocp_modify_seqlen(tp, a, ARRAY_SIZE(a)) + /* Work around a hw issue with RTL8168g PHY, the quirk disables * PHY MCU interrupts before PHY power-down. */ From patchwork Sun Oct 29 18:35:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mirsad Todorovac X-Patchwork-Id: 13439857 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BD28AF9EF for ; Sun, 29 Oct 2023 18:42:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=alu.unizg.hr header.i=@alu.unizg.hr header.b="cKsg9u6N"; dkim=pass (2048-bit key) header.d=alu.unizg.hr header.i=@alu.unizg.hr header.b="yttPFTlD" Received: from domac.alu.hr (domac.alu.unizg.hr [161.53.235.3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F26B8F1; Sun, 29 Oct 2023 11:42:53 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by domac.alu.hr (Postfix) with ESMTP id 67A5660182; Sun, 29 Oct 2023 19:42:52 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698604972; bh=B8EyPD13t0g4ymcG9S08wKsseiSx/0aDmoV19bBvwDc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cKsg9u6NH2Iai9IcC6r9Yw5jWWhcb6csGeQafdHi/p1i1+/nimcnyYMCFSiVYJL+J 4Yr5BwZe3Lx3hEKXzQmjlY8AtDLWBOt8j7N7W6zQ7mkjf1qb2hLG3fnUlPyU3WjIVg 9FzK1bJFNzgfuLHYhYH1IHrgGo6mo2Focv/f/jE1wNYy8PfnKxroO6QMyxZvFrTidA HU8L2QiN703KC72OB9M/03mEvG0hOy6ziqlSgRim1mhJTDpdk3RC51j+RLNUX8DPS2 9k0XiR9j6zPjEy9sloumBNIa1V84UpOZq264iwTlPIFlqZHzngFIF6GlTofKA4hglW pquFESRDYFhVA== X-Virus-Scanned: Debian amavisd-new at domac.alu.hr Received: from domac.alu.hr ([127.0.0.1]) by localhost (domac.alu.hr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wLjCM5r6q6f3; Sun, 29 Oct 2023 19:42:49 +0100 (CET) Received: from defiant.home (78-3-40-247.adsl.net.t-com.hr [78.3.40.247]) by domac.alu.hr (Postfix) with ESMTPSA id 5C64C60173; Sun, 29 Oct 2023 19:42:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698604969; bh=B8EyPD13t0g4ymcG9S08wKsseiSx/0aDmoV19bBvwDc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=yttPFTlD3pLdXVG2omdqsYw0aZSZioMM62UzKHraLYmh0CDCkpex9C9kMfh63gory wy8RHw4ROSL3+DlO+u5jfOgrVikc5dNdY/jp5naOBFVai3kUVnZ3OO+d/EmxvPqkBQ UhApBSkc7S84pSWlw5xh4w53tvZvy0mR8LMPKifaXUts/d9J2bu5dOKwn9oTNOQi8N fG00ZLbBM5/yWYvXYnt2zj/rs6wNoNlc81iSQDE6cNd9+qvSPEd8xvmA6nsESBK/sX WhhQt521Nny2hvT6PWVu365n/jOr6Bz+pU/M8XiyyTMvaZ5wNHFkAXN+69POERhkyM /zxRHxc7LMX2Q== From: Mirsad Goran Todorovac To: Jason Gunthorpe , Joerg Roedel , Lu Baolu , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Heiner Kallweit , nic_swsd@realtek.com, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirsad Goran Todorovac , Marco Elver Subject: [PATCH v5 2/7] r8169: Coalesce RTL8411b PHY power-down recovery calls to reduce spinlock contention Date: Sun, 29 Oct 2023 19:35:57 +0100 Message-Id: <20231029183600.451694-2-mirsad.todorovac@alu.unizg.hr> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231029183600.451694-1-mirsad.todorovac@alu.unizg.hr> References: <20231029183600.451694-1-mirsad.todorovac@alu.unizg.hr> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org On RTL8411b the RX unit gets confused if the PHY is powered-down. This was reported in [0] and confirmed by Realtek. Realtek provided a sequence to fix the RX unit after PHY wakeup. A series of about 130 r8168_mac_ocp_write() calls is performed to program the RTL registers for recovery. With about 130 of these sequential calls to r8168_mac_ocp_write() this looks like a lock storm that will stall all of the cores and CPUs on the same memory controller for certain time I/O takes to finish. In a sequential case of RTL register programming, a sequence of writes to the RTL registers can be coalesced under a same raw spinlock. This can dramatically decrease the number of bus stalls in a multicore or multi-CPU system: static void rtl_hw_start_8411_2(struct rtl8169_private *tp) { ... /* The following Realtek-provided magic fixes an issue with the RX unit * getting confused after the PHY having been powered-down. */ static const struct recover_8411b_info init_zero_seq[] = { { 0xFC28, 0x0000 }, { 0xFC2A, 0x0000 }, { 0xFC2C, 0x0000 }, ... }; static const struct recover_8411b_info recover_seq[] = { { 0xF800, 0xE008 }, { 0xF802, 0xE00A }, { 0xF804, 0xE00C }, ... }; static const struct recover_8411b_info final_seq[] = { { 0xFC2A, 0x0743 }, { 0xFC2C, 0x0801 }, { 0xFC2E, 0x0BE9 }, ... }; r8168_mac_ocp_write_seq(tp, init_zero_seq); mdelay(3); r8168_mac_ocp_write(tp, 0xFC26, 0x0000); r8168_mac_ocp_write_seq(tp, recover_seq); r8168_mac_ocp_write(tp, 0xFC26, 0x8000); r8168_mac_ocp_write_seq(tp, final_seq); } The hex data is preserved intact through s/r8168_mac_ocp_write[(]tp,/{ / and s/[)];/ },/ functions that only changed the function names and the ending of the line, so the actual hex data is unchanged. Note that the reason for the introduction of the original commit was to enable recovery of the RX unit on the RTL8411b which was confused by the powered-down PHY. This sequence of r8168_mac_ocp_write() calls amplifies the problem into a series of about 500+ memory bus locks, most waiting for the main MMIO memory read-modify-write under a LOCK. The memory barrier in RTL_W32 should suffice for the programming sequence to reach RTL NIC registers. [0] https://bugzilla.redhat.com/show_bug.cgi?id=1692075 Fixes: fe4e8db0392a6 ("r8169: fix issue with confused RX unit after PHY power-down on RTL8411b") Cc: Heiner Kallweit Cc: Marco Elver Cc: nic_swsd@realtek.com Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org Link: https://lore.kernel.org/lkml/20231028005153.2180411-1-mirsad.todorovac@alu.unizg.hr/ Link: https://lore.kernel.org/lkml/20231028110459.2644926-1-mirsad.todorovac@alu.unizg.hr/ Signed-off-by: Mirsad Goran Todorovac --- v5: added unlocked primitives to allow mac ocs modify grouping applied coalescing of mac ocp writes/modifies for 8168ep and 8117 some formatting fixes to please checkpatch.pl v4: fixed complaints as advised by Heiner and checkpatch.pl split the patch into five sections to be more easily manipulated and reviewed introduced r8168_mac_ocp_write_seq() applied coalescing of mac ocp writes/modifies for 8168H, 8125 and 8125B v3: removed register/mask pair array sentinels, so using ARRAY_SIZE(). avoided duplication of RTL_W32() call code as advised by Heiner. drivers/net/ethernet/realtek/r8169_main.c | 173 ++++++---------------- 1 file changed, 46 insertions(+), 127 deletions(-) diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index da1f5d1b4fd5..f4a1d1a74b8b 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -3161,145 +3161,64 @@ static void rtl_hw_start_8411_2(struct rtl8169_private *tp) { 0x1d, 0x0000, 0x4000 }, }; - rtl_hw_start_8168g(tp); + static const struct e_info_regdata init_zero_seq[] = { + { 0xFC28, 0x0000 }, { 0xFC2A, 0x0000 }, { 0xFC2C, 0x0000 }, { 0xFC2E, 0x0000 }, + { 0xFC30, 0x0000 }, { 0xFC32, 0x0000 }, { 0xFC34, 0x0000 }, { 0xFC36, 0x0000 }, + }; + static const struct e_info_regdata recover_seq[] = { + { 0xF800, 0xE008 }, { 0xF802, 0xE00A }, { 0xF804, 0xE00C }, { 0xF806, 0xE00E }, + { 0xF808, 0xE027 }, { 0xF80A, 0xE04F }, { 0xF80C, 0xE05E }, { 0xF80E, 0xE065 }, + { 0xF810, 0xC602 }, { 0xF812, 0xBE00 }, { 0xF814, 0x0000 }, { 0xF816, 0xC502 }, + { 0xF818, 0xBD00 }, { 0xF81A, 0x074C }, { 0xF81C, 0xC302 }, { 0xF81E, 0xBB00 }, + { 0xF820, 0x080A }, { 0xF822, 0x6420 }, { 0xF824, 0x48C2 }, { 0xF826, 0x8C20 }, + { 0xF828, 0xC516 }, { 0xF82A, 0x64A4 }, { 0xF82C, 0x49C0 }, { 0xF82E, 0xF009 }, + { 0xF830, 0x74A2 }, { 0xF832, 0x8CA5 }, { 0xF834, 0x74A0 }, { 0xF836, 0xC50E }, + { 0xF838, 0x9CA2 }, { 0xF83A, 0x1C11 }, { 0xF83C, 0x9CA0 }, { 0xF83E, 0xE006 }, + { 0xF840, 0x74F8 }, { 0xF842, 0x48C4 }, { 0xF844, 0x8CF8 }, { 0xF846, 0xC404 }, + { 0xF848, 0xBC00 }, { 0xF84A, 0xC403 }, { 0xF84C, 0xBC00 }, { 0xF84E, 0x0BF2 }, + { 0xF850, 0x0C0A }, { 0xF852, 0xE434 }, { 0xF854, 0xD3C0 }, { 0xF856, 0x49D9 }, + { 0xF858, 0xF01F }, { 0xF85A, 0xC526 }, { 0xF85C, 0x64A5 }, { 0xF85E, 0x1400 }, + { 0xF860, 0xF007 }, { 0xF862, 0x0C01 }, { 0xF864, 0x8CA5 }, { 0xF866, 0x1C15 }, + { 0xF868, 0xC51B }, { 0xF86A, 0x9CA0 }, { 0xF86C, 0xE013 }, { 0xF86E, 0xC519 }, + { 0xF870, 0x74A0 }, { 0xF872, 0x48C4 }, { 0xF874, 0x8CA0 }, { 0xF876, 0xC516 }, + { 0xF878, 0x74A4 }, { 0xF87A, 0x48C8 }, { 0xF87C, 0x48CA }, { 0xF87E, 0x9CA4 }, + { 0xF880, 0xC512 }, { 0xF882, 0x1B00 }, { 0xF884, 0x9BA0 }, { 0xF886, 0x1B1C }, + { 0xF888, 0x483F }, { 0xF88A, 0x9BA2 }, { 0xF88C, 0x1B04 }, { 0xF88E, 0xC508 }, + { 0xF890, 0x9BA0 }, { 0xF892, 0xC505 }, { 0xF894, 0xBD00 }, { 0xF896, 0xC502 }, + { 0xF898, 0xBD00 }, { 0xF89A, 0x0300 }, { 0xF89C, 0x051E }, { 0xF89E, 0xE434 }, + { 0xF8A0, 0xE018 }, { 0xF8A2, 0xE092 }, { 0xF8A4, 0xDE20 }, { 0xF8A6, 0xD3C0 }, + { 0xF8A8, 0xC50F }, { 0xF8AA, 0x76A4 }, { 0xF8AC, 0x49E3 }, { 0xF8AE, 0xF007 }, + { 0xF8B0, 0x49C0 }, { 0xF8B2, 0xF103 }, { 0xF8B4, 0xC607 }, { 0xF8B6, 0xBE00 }, + { 0xF8B8, 0xC606 }, { 0xF8BA, 0xBE00 }, { 0xF8BC, 0xC602 }, { 0xF8BE, 0xBE00 }, + { 0xF8C0, 0x0C4C }, { 0xF8C2, 0x0C28 }, { 0xF8C4, 0x0C2C }, { 0xF8C6, 0xDC00 }, + { 0xF8C8, 0xC707 }, { 0xF8CA, 0x1D00 }, { 0xF8CC, 0x8DE2 }, { 0xF8CE, 0x48C1 }, + { 0xF8D0, 0xC502 }, { 0xF8D2, 0xBD00 }, { 0xF8D4, 0x00AA }, { 0xF8D6, 0xE0C0 }, + { 0xF8D8, 0xC502 }, { 0xF8DA, 0xBD00 }, { 0xF8DC, 0x0132 }, + }; + + static const struct e_info_regdata final_seq[] = { + { 0xFC2A, 0x0743 }, { 0xFC2C, 0x0801 }, { 0xFC2E, 0x0BE9 }, { 0xFC30, 0x02FD }, + { 0xFC32, 0x0C25 }, { 0xFC34, 0x00A9 }, { 0xFC36, 0x012D }, + }; + + rtl_hw_start_8168g(tp); rtl_ephy_init(tp, e_info_8411_2); /* The following Realtek-provided magic fixes an issue with the RX unit * getting confused after the PHY having been powered-down. */ - r8168_mac_ocp_write(tp, 0xFC28, 0x0000); - r8168_mac_ocp_write(tp, 0xFC2A, 0x0000); - r8168_mac_ocp_write(tp, 0xFC2C, 0x0000); - r8168_mac_ocp_write(tp, 0xFC2E, 0x0000); - r8168_mac_ocp_write(tp, 0xFC30, 0x0000); - r8168_mac_ocp_write(tp, 0xFC32, 0x0000); - r8168_mac_ocp_write(tp, 0xFC34, 0x0000); - r8168_mac_ocp_write(tp, 0xFC36, 0x0000); + + r8168_mac_ocp_write_seq(tp, init_zero_seq); mdelay(3); r8168_mac_ocp_write(tp, 0xFC26, 0x0000); - r8168_mac_ocp_write(tp, 0xF800, 0xE008); - r8168_mac_ocp_write(tp, 0xF802, 0xE00A); - r8168_mac_ocp_write(tp, 0xF804, 0xE00C); - r8168_mac_ocp_write(tp, 0xF806, 0xE00E); - r8168_mac_ocp_write(tp, 0xF808, 0xE027); - r8168_mac_ocp_write(tp, 0xF80A, 0xE04F); - r8168_mac_ocp_write(tp, 0xF80C, 0xE05E); - r8168_mac_ocp_write(tp, 0xF80E, 0xE065); - r8168_mac_ocp_write(tp, 0xF810, 0xC602); - r8168_mac_ocp_write(tp, 0xF812, 0xBE00); - r8168_mac_ocp_write(tp, 0xF814, 0x0000); - r8168_mac_ocp_write(tp, 0xF816, 0xC502); - r8168_mac_ocp_write(tp, 0xF818, 0xBD00); - r8168_mac_ocp_write(tp, 0xF81A, 0x074C); - r8168_mac_ocp_write(tp, 0xF81C, 0xC302); - r8168_mac_ocp_write(tp, 0xF81E, 0xBB00); - r8168_mac_ocp_write(tp, 0xF820, 0x080A); - r8168_mac_ocp_write(tp, 0xF822, 0x6420); - r8168_mac_ocp_write(tp, 0xF824, 0x48C2); - r8168_mac_ocp_write(tp, 0xF826, 0x8C20); - r8168_mac_ocp_write(tp, 0xF828, 0xC516); - r8168_mac_ocp_write(tp, 0xF82A, 0x64A4); - r8168_mac_ocp_write(tp, 0xF82C, 0x49C0); - r8168_mac_ocp_write(tp, 0xF82E, 0xF009); - r8168_mac_ocp_write(tp, 0xF830, 0x74A2); - r8168_mac_ocp_write(tp, 0xF832, 0x8CA5); - r8168_mac_ocp_write(tp, 0xF834, 0x74A0); - r8168_mac_ocp_write(tp, 0xF836, 0xC50E); - r8168_mac_ocp_write(tp, 0xF838, 0x9CA2); - r8168_mac_ocp_write(tp, 0xF83A, 0x1C11); - r8168_mac_ocp_write(tp, 0xF83C, 0x9CA0); - r8168_mac_ocp_write(tp, 0xF83E, 0xE006); - r8168_mac_ocp_write(tp, 0xF840, 0x74F8); - r8168_mac_ocp_write(tp, 0xF842, 0x48C4); - r8168_mac_ocp_write(tp, 0xF844, 0x8CF8); - r8168_mac_ocp_write(tp, 0xF846, 0xC404); - r8168_mac_ocp_write(tp, 0xF848, 0xBC00); - r8168_mac_ocp_write(tp, 0xF84A, 0xC403); - r8168_mac_ocp_write(tp, 0xF84C, 0xBC00); - r8168_mac_ocp_write(tp, 0xF84E, 0x0BF2); - r8168_mac_ocp_write(tp, 0xF850, 0x0C0A); - r8168_mac_ocp_write(tp, 0xF852, 0xE434); - r8168_mac_ocp_write(tp, 0xF854, 0xD3C0); - r8168_mac_ocp_write(tp, 0xF856, 0x49D9); - r8168_mac_ocp_write(tp, 0xF858, 0xF01F); - r8168_mac_ocp_write(tp, 0xF85A, 0xC526); - r8168_mac_ocp_write(tp, 0xF85C, 0x64A5); - r8168_mac_ocp_write(tp, 0xF85E, 0x1400); - r8168_mac_ocp_write(tp, 0xF860, 0xF007); - r8168_mac_ocp_write(tp, 0xF862, 0x0C01); - r8168_mac_ocp_write(tp, 0xF864, 0x8CA5); - r8168_mac_ocp_write(tp, 0xF866, 0x1C15); - r8168_mac_ocp_write(tp, 0xF868, 0xC51B); - r8168_mac_ocp_write(tp, 0xF86A, 0x9CA0); - r8168_mac_ocp_write(tp, 0xF86C, 0xE013); - r8168_mac_ocp_write(tp, 0xF86E, 0xC519); - r8168_mac_ocp_write(tp, 0xF870, 0x74A0); - r8168_mac_ocp_write(tp, 0xF872, 0x48C4); - r8168_mac_ocp_write(tp, 0xF874, 0x8CA0); - r8168_mac_ocp_write(tp, 0xF876, 0xC516); - r8168_mac_ocp_write(tp, 0xF878, 0x74A4); - r8168_mac_ocp_write(tp, 0xF87A, 0x48C8); - r8168_mac_ocp_write(tp, 0xF87C, 0x48CA); - r8168_mac_ocp_write(tp, 0xF87E, 0x9CA4); - r8168_mac_ocp_write(tp, 0xF880, 0xC512); - r8168_mac_ocp_write(tp, 0xF882, 0x1B00); - r8168_mac_ocp_write(tp, 0xF884, 0x9BA0); - r8168_mac_ocp_write(tp, 0xF886, 0x1B1C); - r8168_mac_ocp_write(tp, 0xF888, 0x483F); - r8168_mac_ocp_write(tp, 0xF88A, 0x9BA2); - r8168_mac_ocp_write(tp, 0xF88C, 0x1B04); - r8168_mac_ocp_write(tp, 0xF88E, 0xC508); - r8168_mac_ocp_write(tp, 0xF890, 0x9BA0); - r8168_mac_ocp_write(tp, 0xF892, 0xC505); - r8168_mac_ocp_write(tp, 0xF894, 0xBD00); - r8168_mac_ocp_write(tp, 0xF896, 0xC502); - r8168_mac_ocp_write(tp, 0xF898, 0xBD00); - r8168_mac_ocp_write(tp, 0xF89A, 0x0300); - r8168_mac_ocp_write(tp, 0xF89C, 0x051E); - r8168_mac_ocp_write(tp, 0xF89E, 0xE434); - r8168_mac_ocp_write(tp, 0xF8A0, 0xE018); - r8168_mac_ocp_write(tp, 0xF8A2, 0xE092); - r8168_mac_ocp_write(tp, 0xF8A4, 0xDE20); - r8168_mac_ocp_write(tp, 0xF8A6, 0xD3C0); - r8168_mac_ocp_write(tp, 0xF8A8, 0xC50F); - r8168_mac_ocp_write(tp, 0xF8AA, 0x76A4); - r8168_mac_ocp_write(tp, 0xF8AC, 0x49E3); - r8168_mac_ocp_write(tp, 0xF8AE, 0xF007); - r8168_mac_ocp_write(tp, 0xF8B0, 0x49C0); - r8168_mac_ocp_write(tp, 0xF8B2, 0xF103); - r8168_mac_ocp_write(tp, 0xF8B4, 0xC607); - r8168_mac_ocp_write(tp, 0xF8B6, 0xBE00); - r8168_mac_ocp_write(tp, 0xF8B8, 0xC606); - r8168_mac_ocp_write(tp, 0xF8BA, 0xBE00); - r8168_mac_ocp_write(tp, 0xF8BC, 0xC602); - r8168_mac_ocp_write(tp, 0xF8BE, 0xBE00); - r8168_mac_ocp_write(tp, 0xF8C0, 0x0C4C); - r8168_mac_ocp_write(tp, 0xF8C2, 0x0C28); - r8168_mac_ocp_write(tp, 0xF8C4, 0x0C2C); - r8168_mac_ocp_write(tp, 0xF8C6, 0xDC00); - r8168_mac_ocp_write(tp, 0xF8C8, 0xC707); - r8168_mac_ocp_write(tp, 0xF8CA, 0x1D00); - r8168_mac_ocp_write(tp, 0xF8CC, 0x8DE2); - r8168_mac_ocp_write(tp, 0xF8CE, 0x48C1); - r8168_mac_ocp_write(tp, 0xF8D0, 0xC502); - r8168_mac_ocp_write(tp, 0xF8D2, 0xBD00); - r8168_mac_ocp_write(tp, 0xF8D4, 0x00AA); - r8168_mac_ocp_write(tp, 0xF8D6, 0xE0C0); - r8168_mac_ocp_write(tp, 0xF8D8, 0xC502); - r8168_mac_ocp_write(tp, 0xF8DA, 0xBD00); - r8168_mac_ocp_write(tp, 0xF8DC, 0x0132); + r8168_mac_ocp_write_seq(tp, recover_seq); r8168_mac_ocp_write(tp, 0xFC26, 0x8000); - r8168_mac_ocp_write(tp, 0xFC2A, 0x0743); - r8168_mac_ocp_write(tp, 0xFC2C, 0x0801); - r8168_mac_ocp_write(tp, 0xFC2E, 0x0BE9); - r8168_mac_ocp_write(tp, 0xFC30, 0x02FD); - r8168_mac_ocp_write(tp, 0xFC32, 0x0C25); - r8168_mac_ocp_write(tp, 0xFC34, 0x00A9); - r8168_mac_ocp_write(tp, 0xFC36, 0x012D); + r8168_mac_ocp_write_seq(tp, final_seq); + } static void rtl_hw_start_8168h_1(struct rtl8169_private *tp) From patchwork Sun Oct 29 18:35:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mirsad Todorovac X-Patchwork-Id: 13439858 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C5F5FBE8 for ; Sun, 29 Oct 2023 18:43:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=alu.unizg.hr header.i=@alu.unizg.hr header.b="0ljv846l"; dkim=pass (2048-bit key) header.d=alu.unizg.hr header.i=@alu.unizg.hr header.b="OKK/UwJe" Received: from domac.alu.hr (domac.alu.unizg.hr [161.53.235.3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F4B9C6; Sun, 29 Oct 2023 11:43:12 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by domac.alu.hr (Postfix) with ESMTP id 0392960186; Sun, 29 Oct 2023 19:43:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698604991; bh=T8Jr3u654cavid8IDVFmmKq+08kkDjPH+bbS9lNgtgk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=0ljv846lwP1hOLoB47JTOa7IKp9pR15VJPfr6bQKDDmmtk9LzOckGYnMl34MtOJDT J2z3pDXSsA5ATZdthiWlu4NokhSAea/CudjF0/wxZt/38gZRwjSL1g2zB1NS/GgGZ8 2kau8k0Hp2NMwY5LJjqSQ8GcYy+PpUhkgFvUnSYZ2quovcfEHNSlwySWJ4aheVEnFq mRTvzw0qxtVKOEIYmzSKtphPvLilg7dtm/k7HKPWl6HRNExQz3D393LJrErVvREeZW pjcqK1J5WRiZ3uS+vFU6d1/0dLtKqpY+K1LpSPjY90MJtUCojiLSskmp9Qen5saRV4 3qpg8ghU91KYw== X-Virus-Scanned: Debian amavisd-new at domac.alu.hr Received: from domac.alu.hr ([127.0.0.1]) by localhost (domac.alu.hr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nVi7RQTlmsdk; Sun, 29 Oct 2023 19:43:08 +0100 (CET) Received: from defiant.home (78-3-40-247.adsl.net.t-com.hr [78.3.40.247]) by domac.alu.hr (Postfix) with ESMTPSA id 6C0E760173; Sun, 29 Oct 2023 19:43:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698604988; bh=T8Jr3u654cavid8IDVFmmKq+08kkDjPH+bbS9lNgtgk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OKK/UwJeXihynVGyrKpyFYQsTtLflf2VGiQAKcCD+AOvlV3iyLhQYFGNF4G5mS4ay lrGQwkvrySCtQBLBQL1zYAtkG5s+BRUUBPut+aizbngoqYqK2NoETYyA6axPF0300a +DrQk3ifh+hRLGo4WhYXXEO6PDhvokt5M99HCYDWZRAa4hwWi38dsfONqEZtvRqBuy FjjzmYgKCWv+tBADwa7RZoedOqPap3o4c9rLKxkCQ7Lrnrk6wwkDNmyuJxJBPUm0Qa 9lza6G66GboTI3ykyaRfqVCuVnWCNMhIGjCU12dQJ7HQquTOd/ejphSdm5rziC2lZJ LJDqeQbR8X99g== From: Mirsad Goran Todorovac To: Jason Gunthorpe , Joerg Roedel , Lu Baolu , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Heiner Kallweit , nic_swsd@realtek.com, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirsad Goran Todorovac , Marco Elver Subject: [PATCH v5 3/7] r8169: Coalesce mac ocp write and modify for 8168H start to reduce spinlock contention Date: Sun, 29 Oct 2023 19:35:59 +0100 Message-Id: <20231029183600.451694-3-mirsad.todorovac@alu.unizg.hr> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231029183600.451694-1-mirsad.todorovac@alu.unizg.hr> References: <20231029183600.451694-1-mirsad.todorovac@alu.unizg.hr> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Repeated calls to r8168_mac_ocp_write() and r8168_mac_ocp_modify() in the startup of 8168H involve implicit spin_lock_irqsave() and spin_unlock_irqrestore() on each invocation. Coalesced with the corresponding helpers into r8168_mac_ocp_write_seq() and r8168_mac_ocp_modify_seq() with a sinqle paired spin_lock_irqsave() and spin_unlock_irqrestore(), these calls help reduce overall spinlock contention. Fixes: ef712ede3541d ("r8169: add helper r8168_mac_ocp_modify") Fixes: 6e1d0b8988188 ("r8169:add support for RTL8168H and RTL8107E") Cc: Heiner Kallweit Cc: Marco Elver Cc: nic_swsd@realtek.com Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org Link: https://lore.kernel.org/lkml/20231028005153.2180411-1-mirsad.todorovac@alu.unizg.hr/ Link: https://lore.kernel.org/lkml/20231028110459.2644926-1-mirsad.todorovac@alu.unizg.hr/ Signed-off-by: Mirsad Goran Todorovac --- v5: added unlocked primitives to allow mac ocs modify grouping applied coalescing of mac ocp writes/modifies for 8168ep and 8117 some formatting fixes to please checkpatch.pl v4: fixed complaints as advised by Heiner and checkpatch.pl split the patch into five sections to be more easily manipulated and reviewed introduced r8168_mac_ocp_write_seq() applied coalescing of mac ocp writes/modifies for 8168H, 8125 and 8125B v3: removed register/mask pair array sentinels, so using ARRAY_SIZE(). avoided duplication of RTL_W32() call code as advised by Heiner. drivers/net/ethernet/realtek/r8169_main.c | 26 +++++++++++++++-------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index f4a1d1a74b8b..29ee93b8b702 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -3231,6 +3231,21 @@ static void rtl_hw_start_8168h_1(struct rtl8169_private *tp) { 0x04, 0xffff, 0x854a }, { 0x01, 0xffff, 0x068b } }; + + static const struct e_info_regmaskset e_info_regmaskset_8168h_1[] = { + { 0xe056, 0x00f0, 0x0070 }, + { 0xe052, 0x6000, 0x8008 }, + { 0xe0d6, 0x01ff, 0x017f }, + { 0xd420, 0x0fff, 0x047f }, + }; + + static const struct e_info_regdata e_info_regdata_8168h_1[] = { + { 0xe63e, 0x0001 }, + { 0xe63e, 0x0000 }, + { 0xc094, 0x0000 }, + { 0xc09e, 0x0000 }, + }; + int rg_saw_cnt; rtl_ephy_init(tp, e_info_8168h_1); @@ -3271,15 +3286,8 @@ static void rtl_hw_start_8168h_1(struct rtl8169_private *tp) r8168_mac_ocp_modify(tp, 0xd412, 0x0fff, sw_cnt_1ms_ini); } - r8168_mac_ocp_modify(tp, 0xe056, 0x00f0, 0x0070); - r8168_mac_ocp_modify(tp, 0xe052, 0x6000, 0x8008); - r8168_mac_ocp_modify(tp, 0xe0d6, 0x01ff, 0x017f); - r8168_mac_ocp_modify(tp, 0xd420, 0x0fff, 0x047f); - - r8168_mac_ocp_write(tp, 0xe63e, 0x0001); - r8168_mac_ocp_write(tp, 0xe63e, 0x0000); - r8168_mac_ocp_write(tp, 0xc094, 0x0000); - r8168_mac_ocp_write(tp, 0xc09e, 0x0000); + r8168_mac_ocp_modify_seq(tp, e_info_regmaskset_8168h_1); + r8168_mac_ocp_write_seq(tp, e_info_regdata_8168h_1); } static void rtl_hw_start_8168ep(struct rtl8169_private *tp) From patchwork Sun Oct 29 18:36:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mirsad Todorovac X-Patchwork-Id: 13439859 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB928FC1E for ; Sun, 29 Oct 2023 18:43:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=alu.unizg.hr header.i=@alu.unizg.hr header.b="sujq5HcF"; dkim=pass (2048-bit key) header.d=alu.unizg.hr header.i=@alu.unizg.hr header.b="WbPLYZpf" Received: from domac.alu.hr (domac.alu.unizg.hr [IPv6:2001:b68:2:2800::3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E7956CC; Sun, 29 Oct 2023 11:43:28 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by domac.alu.hr (Postfix) with ESMTP id C63F160187; Sun, 29 Oct 2023 19:43:26 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698605006; bh=C0fbJhCelN3HUXSd6OtmKYAgitc2PZFM3gqVXrhWFL4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sujq5HcFHVcb7rJtKdra3UexWxdEQgN79FBkZ+idn+zw2xm6JOSgenw1eP83f3riX KhCvgni8iSzh+xU6lR/K+v3YclDPzE2UbQ66SGX1z0s2bo9QJKkH3w4gThuk2EDPEQ wFNWb+rJs3QbJq6V1Diu8hYcctdztxeEINeTAjROffMrcPc5K3DXpvkRGkq5ewnQ6a 0jIm6/5Rl6AYdVlP4Ofqw8q+/L+TmAE6XPqcXl+dtEuywrVz3ulXawRFixKc4ykgXH ojH5pt19eoc/AdH2b15c9f5t7mc0LDbmZYipZfwMnwW1CpOsGBjyt3skBib7jyU/Xm 7FiRBNZlWyF9g== X-Virus-Scanned: Debian amavisd-new at domac.alu.hr Received: from domac.alu.hr ([127.0.0.1]) by localhost (domac.alu.hr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hjd0BdmdBeQa; Sun, 29 Oct 2023 19:43:24 +0100 (CET) Received: from defiant.home (78-3-40-247.adsl.net.t-com.hr [78.3.40.247]) by domac.alu.hr (Postfix) with ESMTPSA id 3B5A160173; Sun, 29 Oct 2023 19:43:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698605004; bh=C0fbJhCelN3HUXSd6OtmKYAgitc2PZFM3gqVXrhWFL4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WbPLYZpfeA7P3IpyqtWcs4mD1SaB6HjXHzCX6Az/u/OHJzb1XUQJ1Z8jeSIFQ7N9M VgzqnObyyJ2XVZ2sJO1zi7dzpNcsrSwAHI5jiFWu/GQoBgl0HDrHGZ93KlOYhmE67I aohbFwCw87xblI9OjJh5/04S51hNfH3PX/gzOPuQP5tFWgXidKdf6kYmrI1Q9MQePU vj/2WCItMuHbmZ+pFqxolcDateFkM7To0U2wczC4tCzqly5U06uUxz7HXCCKoFLbWV TjNvAM0lKT2afVbWRbeHLGp7i1p8Ar4dwiV8FZDcX7bquCyamnZ7KNoqztUea+p6sE Tf/xOZJoMlaHg== From: Mirsad Goran Todorovac To: Jason Gunthorpe , Joerg Roedel , Lu Baolu , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Heiner Kallweit , nic_swsd@realtek.com, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirsad Goran Todorovac , Marco Elver Subject: [PATCH v5 4/7] r8169: Coalesce mac ocp write and modify for 8168ep start to reduce spinlock contention Date: Sun, 29 Oct 2023 19:36:01 +0100 Message-Id: <20231029183600.451694-4-mirsad.todorovac@alu.unizg.hr> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231029183600.451694-1-mirsad.todorovac@alu.unizg.hr> References: <20231029183600.451694-1-mirsad.todorovac@alu.unizg.hr> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Repeated calls to r8168_mac_ocp_write() and r8168_mac_ocp_modify() in the startup of RTL8168ep involve implicit spin_lock_irqsave() and spin_unlock_irqrestore() on each invocation. Coalesced with the corresponding helpers into r8168_mac_ocp_write_seq() and r8168_mac_ocp_modify_seq() with a sinqle paired spin_lock_irqsave() and spin_unlock_irqrestore(), these calls help reduce overall spinlock contention. Fixes: ef712ede3541d ("r8169: add helper r8168_mac_ocp_modify") Cc: Heiner Kallweit Cc: Marco Elver Cc: nic_swsd@realtek.com Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org Link: https://lore.kernel.org/lkml/20231028005153.2180411-1-mirsad.todorovac@alu.unizg.hr/ Link: https://lore.kernel.org/lkml/20231028110459.2644926-1-mirsad.todorovac@alu.unizg.hr/ Signed-off-by: Mirsad Goran Todorovac --- v5: added unlocked primitives to allow mac ocs modify grouping applied coalescing of mac ocp writes/modifies for 8168ep and 8117 some formatting fixes to please checkpatch.pl v4: fixed complaints as advised by Heiner and checkpatch.pl split the patch into five sections to be more easily manipulated and reviewed introduced r8168_mac_ocp_write_seq() applied coalescing of mac ocp writes/modifies for 8168H, 8125 and 8125B v3: removed register/mask pair array sentinels, so using ARRAY_SIZE(). avoided duplication of RTL_W32() call code as advised by Heiner. drivers/net/ethernet/realtek/r8169_main.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index 29ee93b8b702..27016aaeb6a0 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -3326,6 +3326,12 @@ static void rtl_hw_start_8168ep_3(struct rtl8169_private *tp) { 0x1e, 0x0000, 0x2000 }, }; + static const struct e_info_regmaskset e_info_8168ep_3_mod_1[] = { + { 0xd3e2, 0x0fff, 0x0271 }, + { 0xd3e4, 0x00ff, 0x0000 }, + { 0xe860, 0x0000, 0x0080 }, + }; + rtl_ephy_init(tp, e_info_8168ep_3); rtl_hw_start_8168ep(tp); @@ -3333,9 +3339,7 @@ static void rtl_hw_start_8168ep_3(struct rtl8169_private *tp) RTL_W8(tp, DLLPR, RTL_R8(tp, DLLPR) & ~PFM_EN); RTL_W8(tp, MISC_1, RTL_R8(tp, MISC_1) & ~PFM_D3COLD_EN); - r8168_mac_ocp_modify(tp, 0xd3e2, 0x0fff, 0x0271); - r8168_mac_ocp_modify(tp, 0xd3e4, 0x00ff, 0x0000); - r8168_mac_ocp_modify(tp, 0xe860, 0x0000, 0x0080); + r8168_mac_ocp_modify_seq(tp, e_info_8168ep_3_mod_1); } static void rtl_hw_start_8117(struct rtl8169_private *tp) From patchwork Sun Oct 29 18:36:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mirsad Todorovac X-Patchwork-Id: 13439860 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C949CFC1E for ; Sun, 29 Oct 2023 18:43:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=alu.unizg.hr header.i=@alu.unizg.hr header.b="FmpvM7s8"; dkim=pass (2048-bit key) header.d=alu.unizg.hr header.i=@alu.unizg.hr header.b="GTrGCIbM" Received: from domac.alu.hr (domac.alu.unizg.hr [IPv6:2001:b68:2:2800::3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 233B7D3; Sun, 29 Oct 2023 11:43:46 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by domac.alu.hr (Postfix) with ESMTP id B166F60182; Sun, 29 Oct 2023 19:43:45 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698605025; bh=WtQ0K7++ytbFBVwISz9KquDCSJEBxV2WigbuQjUKAto=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FmpvM7s8By0xhPMAWEDqKCod/3e0d+u/QW5C1Gci8lviQyMiNdnS2+gqErs2C7ol8 zygD0ZJB1cJ3OxBJuDBBygedJHUpqwlYnYxqfFonIVbI07PD7rDpdKgBYXBqPCiqpP KHyr6fYaGRUC7MwZlQmBgP85K+kri1Mr/SkEZirJkrxt9KJ/2K4ezIk/624O5v1Jjb rrBzh5fZtg1cZYA6QhR/OF8zlRPoW+Yr97IArS5/vZuKGIi+vOTL7jl9pErbBBs5W1 h/3PASI2AkLyCoqD24Hujne0ibc5DGlvfo0oS6uclSetLgjR+Nwm8wuYsTQsnF2QzP 9QprPCb2VguIQ== X-Virus-Scanned: Debian amavisd-new at domac.alu.hr Received: from domac.alu.hr ([127.0.0.1]) by localhost (domac.alu.hr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6UjbwLRJhPwx; Sun, 29 Oct 2023 19:43:43 +0100 (CET) Received: from defiant.home (78-3-40-247.adsl.net.t-com.hr [78.3.40.247]) by domac.alu.hr (Postfix) with ESMTPSA id 2608E60173; Sun, 29 Oct 2023 19:43:43 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698605023; bh=WtQ0K7++ytbFBVwISz9KquDCSJEBxV2WigbuQjUKAto=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GTrGCIbME3W1vrfK1FslEJ4+oAaUJvWh2dClppr3l1KGBmx+QtQRPPCPCckoJOpp3 3QWgPPz4ZT/eO4ABt429pIj9jgKyWgmYHDS+PrpDBAwx93N6V/ji6yO0cj9GliWlhb CXLrsTOdaIhg3WHpAM/+PoxJMMr+XbV9MUxLXafRre6ekM+O9GFwbP8xCM8ynEIabi jW9JKorSpICtAVzhQycyUV03SLWltj0wJmpvcATl+N8iv7RfLVgsZZ7YCm8x3m445E k+jMQcphJrt93BP0C/OtxBfoQC+IwuLBn8zUCLtesxsINMz3weHCv3gtqtQyptlGhJ W02IYziuhuQEA== From: Mirsad Goran Todorovac To: Jason Gunthorpe , Joerg Roedel , Lu Baolu , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Heiner Kallweit , nic_swsd@realtek.com, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirsad Goran Todorovac , Marco Elver Subject: [PATCH v5 5/7] r8169: Reduce spinlock contention for the start of RTL8117 Date: Sun, 29 Oct 2023 19:36:03 +0100 Message-Id: <20231029183600.451694-5-mirsad.todorovac@alu.unizg.hr> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231029183600.451694-1-mirsad.todorovac@alu.unizg.hr> References: <20231029183600.451694-1-mirsad.todorovac@alu.unizg.hr> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Repeated calls to r8168_mac_ocp_write() and r8168_mac_ocp_modify() in the startup of RTL8168ep involve implicit spin_lock_irqsave() and spin_unlock_irqrestore() on each invocation. This is avoided by grouping unlocked __r8168_mac_ocp_write() and __r8168_mac_ocp_modify() primitives enclosed in the explicit spin_lock_irqsave()/spin_unlock_irqrestore() pair. Even if the lock is not contended, the check requires a LOCK CMPXCHG or a similar instruction, which prevents all cores from accessing the memory bus. Fixes: 1287723aa139b ("r8169: add support for RTL8117") Cc: Heiner Kallweit Cc: Marco Elver Cc: nic_swsd@realtek.com Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org Link: https://lore.kernel.org/lkml/20231028005153.2180411-1-mirsad.todorovac@alu.unizg.hr/ Link: https://lore.kernel.org/lkml/20231028110459.2644926-1-mirsad.todorovac@alu.unizg.hr/ Signed-off-by: Mirsad Goran Todorovac --- v5: added unlocked primitives to allow mac ocs modify grouping applied coalescing of mac ocp writes/modifies for 8168ep and 8117 some formatting fixes to please checkpatch.pl v4: fixed complaints as advised by Heiner and checkpatch.pl split the patch into five sections to be more easily manipulated and reviewed introduced r8168_mac_ocp_write_seq() applied coalescing of mac ocp writes/modifies for 8168H, 8125 and 8125B v3: removed register/mask pair array sentinels, so using ARRAY_SIZE(). avoided duplication of RTL_W32() call code as advised by Heiner. drivers/net/ethernet/realtek/r8169_main.c | 24 ++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index 27016aaeb6a0..50fbacb05953 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -3348,7 +3348,15 @@ static void rtl_hw_start_8117(struct rtl8169_private *tp) { 0x19, 0x0040, 0x1100 }, { 0x59, 0x0040, 0x1100 }, }; + + static const struct e_info_regdata e_info_8117_wr_1[] = { + { 0xe63e, 0x0001 }, + { 0xe63e, 0x0000 }, + { 0xc094, 0x0000 }, + { 0xc09e, 0x0000 }, + }; int rg_saw_cnt; + unsigned long flags; rtl8168ep_stop_cmac(tp); rtl_ephy_init(tp, e_info_8117); @@ -3388,15 +3396,13 @@ static void rtl_hw_start_8117(struct rtl8169_private *tp) r8168_mac_ocp_modify(tp, 0xd412, 0x0fff, sw_cnt_1ms_ini); } - r8168_mac_ocp_modify(tp, 0xe056, 0x00f0, 0x0070); - r8168_mac_ocp_write(tp, 0xea80, 0x0003); - r8168_mac_ocp_modify(tp, 0xe052, 0x0000, 0x0009); - r8168_mac_ocp_modify(tp, 0xd420, 0x0fff, 0x047f); - - r8168_mac_ocp_write(tp, 0xe63e, 0x0001); - r8168_mac_ocp_write(tp, 0xe63e, 0x0000); - r8168_mac_ocp_write(tp, 0xc094, 0x0000); - r8168_mac_ocp_write(tp, 0xc09e, 0x0000); + raw_spin_lock_irqsave(&tp->mac_ocp_lock, flags); + __r8168_mac_ocp_modify(tp, 0xe056, 0x00f0, 0x0070); + __r8168_mac_ocp_write(tp, 0xea80, 0x0003); + __r8168_mac_ocp_modify(tp, 0xe052, 0x0000, 0x0009); + __r8168_mac_ocp_modify(tp, 0xd420, 0x0fff, 0x047f); + __r8168_mac_ocp_write_seq(tp, e_info_8117_wr_1); + raw_spin_unlock_irqrestore(&tp->mac_ocp_lock, flags); /* firmware is for MAC only */ r8169_apply_firmware(tp); From patchwork Sun Oct 29 18:36:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mirsad Todorovac X-Patchwork-Id: 13439861 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 799961094A for ; Sun, 29 Oct 2023 18:44:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=alu.unizg.hr header.i=@alu.unizg.hr header.b="I4oglD+g"; dkim=pass (2048-bit key) header.d=alu.unizg.hr header.i=@alu.unizg.hr header.b="VGwNHyBR" Received: from domac.alu.hr (domac.alu.unizg.hr [IPv6:2001:b68:2:2800::3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5A8D8BD; Sun, 29 Oct 2023 11:43:59 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by domac.alu.hr (Postfix) with ESMTP id 16B6860173; Sun, 29 Oct 2023 19:43:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698605038; bh=kTrKI+RYDHwLvsOjwX6EC4uYyc8dF8fc3KgwnN4pF48=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=I4oglD+gqpjI8GSCMer9UN+MuG+hXIz8uXZFJJEw3nnWIkPXGyW2n8wjDEb8RgUjx huZ1sKRup2wQgqUoj/BZClV74pLhoxZrxz2PNI0h67YQ3cq42uN5qy2/G8+Kec6TXW rc6LkYoCauAeYpYSERe3YJ34vehPhQvauhbI7UNto+Dpvg+j67Q2neuVMOyCSwYamG FrXZ1HsI+wlpjYeGfRyPAchQMlqG+tIBET+3cdqTBAyw0b9ZOCGWZvDqrr2N55UlhN 4gkdROdFyS2I7DSllbDltaSNFsL9nc4wutWOZtMnKIVGBWgWqqL5YvfqjbJpB1wfeM pmNxhR48JCzjw== X-Virus-Scanned: Debian amavisd-new at domac.alu.hr Received: from domac.alu.hr ([127.0.0.1]) by localhost (domac.alu.hr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yBSS37-Hzjdw; Sun, 29 Oct 2023 19:43:55 +0100 (CET) Received: from defiant.home (78-3-40-247.adsl.net.t-com.hr [78.3.40.247]) by domac.alu.hr (Postfix) with ESMTPSA id 6A74A60182; Sun, 29 Oct 2023 19:43:55 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698605035; bh=kTrKI+RYDHwLvsOjwX6EC4uYyc8dF8fc3KgwnN4pF48=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VGwNHyBRNsEywjflWi+92J6548s7A+Jtky6QkTblA5IKLHHu2reR8JaahD3hERPhf 9kYXlgVxy6ZOZhEg4HUckw7qMcxIggx25rgp6A9M+dznlRwgHPwTlleEv5BWcz+UQr 2aGJKQGNe+tB1oR2bGB3u3sHhAfsd9JA4DAv6Gwi3bzy1WM2BV1tDEQmZF5nugXFzx 0dczvI/gF8Oh57W3h7gUBOAL2J2YUMJmEZE0yzSMzkWZhA+oCoVTOqjTh2ayO8AQo3 Z+Tv1RPONMRfgPzYFjuI586MJBmSV5fkZpM8oXZ0DoXsyGntAeVnGbcNpvRb8Gkp9u AIeH/GtHsWUTQ== From: Mirsad Goran Todorovac To: Jason Gunthorpe , Joerg Roedel , Lu Baolu , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Heiner Kallweit , nic_swsd@realtek.com, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirsad Goran Todorovac , Marco Elver Subject: [PATCH v5 6/7] r8169: Coalesce mac ocp write and modify for 8125 and 8125B start to reduce spinlocks Date: Sun, 29 Oct 2023 19:36:05 +0100 Message-Id: <20231029183600.451694-6-mirsad.todorovac@alu.unizg.hr> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231029183600.451694-1-mirsad.todorovac@alu.unizg.hr> References: <20231029183600.451694-1-mirsad.todorovac@alu.unizg.hr> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Repeated calls to r8168_mac_ocp_write() and r8168_mac_ocp_modify() in the startup of 8125 and 8125B involve implicit spin_lock_irqsave() and spin_unlock_irqrestore() on each invocation. Coalesced with the corresponding helpers r8168_mac_ocp_write_seq() and r8168_mac_ocp_modify_seq() into sequential write or modidy with a sinqle pair of spin_lock_irqsave() and spin_unlock_irqrestore(), these calls reduce overall lock contention. Fixes: f1bce4ad2f1ce ("r8169: add support for RTL8125") Fixes: 0439297be9511 ("r8169: add support for RTL8125B") Cc: Heiner Kallweit Cc: Marco Elver Cc: nic_swsd@realtek.com Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org Link: https://lore.kernel.org/lkml/20231028005153.2180411-1-mirsad.todorovac@alu.unizg.hr/ Link: https://lore.kernel.org/lkml/20231028110459.2644926-1-mirsad.todorovac@alu.unizg.hr/ Signed-off-by: Mirsad Goran Todorovac --- v5: added unlocked primitives to allow mac ocs modify grouping applied coalescing of mac ocp writes/modifies for 8168ep and 8117 some formatting fixes to please checkpatch.pl v4: fixed complaints as advised by Heiner and checkpatch.pl split the patch into five sections to be more easily manipulated and reviewed introduced r8168_mac_ocp_write_seq() applied coalescing of mac ocp writes/modifies for 8168H, 8125 and 8125B v3: removed register/mask pair array sentinels, so using ARRAY_SIZE(). avoided duplication of RTL_W32() call code as advised by Heiner. drivers/net/ethernet/realtek/r8169_main.c | 75 +++++++++++++---------- 1 file changed, 44 insertions(+), 31 deletions(-) diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index 50fbacb05953..0778cd0ba2e0 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -3553,6 +3553,28 @@ DECLARE_RTL_COND(rtl_mac_ocp_e00e_cond) static void rtl_hw_start_8125_common(struct rtl8169_private *tp) { + static const struct e_info_regmaskset e_info_8125_common_1[] = { + { 0xd3e2, 0x0fff, 0x03a9 }, + { 0xd3e4, 0x00ff, 0x0000 }, + { 0xe860, 0x0000, 0x0080 }, + }; + + static const struct e_info_regmaskset e_info_8125_common_2[] = { + { 0xc0b4, 0x0000, 0x000c }, + { 0xeb6a, 0x00ff, 0x0033 }, + { 0xeb50, 0x03e0, 0x0040 }, + { 0xe056, 0x00f0, 0x0030 }, + { 0xe040, 0x1000, 0x0000 }, + { 0xea1c, 0x0003, 0x0001 }, + { 0xe0c0, 0x4f0f, 0x4403 }, + { 0xe052, 0x0080, 0x0068 }, + { 0xd430, 0x0fff, 0x047f }, + { 0xea1c, 0x0004, 0x0000 }, + { 0xeb54, 0x0000, 0x0001 }, + }; + + unsigned long flags; + rtl_pcie_state_l2l3_disable(tp); RTL_W16(tp, 0x382, 0x221b); @@ -3560,47 +3582,38 @@ static void rtl_hw_start_8125_common(struct rtl8169_private *tp) RTL_W16(tp, 0x4800, 0); /* disable UPS */ - r8168_mac_ocp_modify(tp, 0xd40a, 0x0010, 0x0000); + + raw_spin_lock_irqsave(&tp->mac_ocp_lock, flags); + __r8168_mac_ocp_modify(tp, 0xd40a, 0x0010, 0x0000); RTL_W8(tp, Config1, RTL_R8(tp, Config1) & ~0x10); - r8168_mac_ocp_write(tp, 0xc140, 0xffff); - r8168_mac_ocp_write(tp, 0xc142, 0xffff); + __r8168_mac_ocp_write(tp, 0xc140, 0xffff); + __r8168_mac_ocp_write(tp, 0xc142, 0xffff); - r8168_mac_ocp_modify(tp, 0xd3e2, 0x0fff, 0x03a9); - r8168_mac_ocp_modify(tp, 0xd3e4, 0x00ff, 0x0000); - r8168_mac_ocp_modify(tp, 0xe860, 0x0000, 0x0080); + __r8168_mac_ocp_modify_seq(tp, e_info_8125_common_1); /* disable new tx descriptor format */ - r8168_mac_ocp_modify(tp, 0xeb58, 0x0001, 0x0000); + __r8168_mac_ocp_modify(tp, 0xeb58, 0x0001, 0x0000); - if (tp->mac_version == RTL_GIGA_MAC_VER_63) - r8168_mac_ocp_modify(tp, 0xe614, 0x0700, 0x0200); - else - r8168_mac_ocp_modify(tp, 0xe614, 0x0700, 0x0400); + if (tp->mac_version == RTL_GIGA_MAC_VER_63) { + __r8168_mac_ocp_modify(tp, 0xe614, 0x0700, 0x0200); + __r8168_mac_ocp_modify(tp, 0xe63e, 0x0c30, 0x0000); + } else { + __r8168_mac_ocp_modify(tp, 0xe614, 0x0700, 0x0400); + __r8168_mac_ocp_modify(tp, 0xe63e, 0x0c30, 0x0020); + } + + __r8168_mac_ocp_modify_seq(tp, e_info_8125_common_2); + raw_spin_unlock_irqrestore(&tp->mac_ocp_lock, flags); - if (tp->mac_version == RTL_GIGA_MAC_VER_63) - r8168_mac_ocp_modify(tp, 0xe63e, 0x0c30, 0x0000); - else - r8168_mac_ocp_modify(tp, 0xe63e, 0x0c30, 0x0020); - - r8168_mac_ocp_modify(tp, 0xc0b4, 0x0000, 0x000c); - r8168_mac_ocp_modify(tp, 0xeb6a, 0x00ff, 0x0033); - r8168_mac_ocp_modify(tp, 0xeb50, 0x03e0, 0x0040); - r8168_mac_ocp_modify(tp, 0xe056, 0x00f0, 0x0030); - r8168_mac_ocp_modify(tp, 0xe040, 0x1000, 0x0000); - r8168_mac_ocp_modify(tp, 0xea1c, 0x0003, 0x0001); - r8168_mac_ocp_modify(tp, 0xe0c0, 0x4f0f, 0x4403); - r8168_mac_ocp_modify(tp, 0xe052, 0x0080, 0x0068); - r8168_mac_ocp_modify(tp, 0xd430, 0x0fff, 0x047f); - - r8168_mac_ocp_modify(tp, 0xea1c, 0x0004, 0x0000); - r8168_mac_ocp_modify(tp, 0xeb54, 0x0000, 0x0001); udelay(1); - r8168_mac_ocp_modify(tp, 0xeb54, 0x0001, 0x0000); - RTL_W16(tp, 0x1880, RTL_R16(tp, 0x1880) & ~0x0030); - r8168_mac_ocp_write(tp, 0xe098, 0xc302); + raw_spin_lock_irqsave(&tp->mac_ocp_lock, flags); + __r8168_mac_ocp_modify(tp, 0xeb54, 0x0001, 0x0000); + RTL_W16(tp, 0x1880, RTL_R16(tp, 0x1880) & ~0x0030); + __r8168_mac_ocp_write(tp, 0xe098, 0xc302); + raw_spin_unlock_irqrestore(&tp->mac_ocp_lock, flags); rtl_loop_wait_low(tp, &rtl_mac_ocp_e00e_cond, 1000, 10); From patchwork Sun Oct 29 18:36:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mirsad Todorovac X-Patchwork-Id: 13439862 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 928B61078D for ; Sun, 29 Oct 2023 18:44:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=alu.unizg.hr header.i=@alu.unizg.hr header.b="ItLZKwEh"; dkim=pass (2048-bit key) header.d=alu.unizg.hr header.i=@alu.unizg.hr header.b="HxnHPYwq" Received: from domac.alu.hr (domac.alu.unizg.hr [IPv6:2001:b68:2:2800::3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 580BDF4; Sun, 29 Oct 2023 11:44:12 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by domac.alu.hr (Postfix) with ESMTP id A332560186; Sun, 29 Oct 2023 19:44:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698605050; bh=lWCoCvbUwweuJllhVc2Ec84nBLwDbguF5xGDvVkqiA4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ItLZKwEh5QuJQNa21MHspcbicxZ1+MvUztWkwbqtHypzFHAhdJUTfhCfcUPTCtDch OtDEP/cymvaZd4mkI57TSrDr7TNamyi+YFE56FafqhKlD0207S2gkPQYHhAuTuvDrO gYnLeDCIefuDYO2dfW/wWz8wX8hwvEHX4h7wTn8b2SI4kx5135P0Z6td5OQ+v34m7q WAmKvTacH6SmawU2Q+AaO0TyO3MaxxSR8bYc3SGFBywZr4//RiRgKmTe6JGz5fysQr 0c6/yb5Ior4xkCzCf8brBibhSsP8uH4wEZa/WC7PaLb9PBTdM2PpVObK16lFC/7EHC XVDjOuGHs45WA== X-Virus-Scanned: Debian amavisd-new at domac.alu.hr Received: from domac.alu.hr ([127.0.0.1]) by localhost (domac.alu.hr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qBPRaj9kwNHs; Sun, 29 Oct 2023 19:44:08 +0100 (CET) Received: from defiant.home (78-3-40-247.adsl.net.t-com.hr [78.3.40.247]) by domac.alu.hr (Postfix) with ESMTPSA id 2427160182; Sun, 29 Oct 2023 19:44:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1698605048; bh=lWCoCvbUwweuJllhVc2Ec84nBLwDbguF5xGDvVkqiA4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HxnHPYwqS+I/I2+wn2jv1AIZiOgO2Z9+UthCCwYm3ryH1ttBb61EGGCA1e3alUJEa pPNSWPVWp+Rf/Au6alR6nfkdaLhiaOwOffpV6N8Mon9zwT8EGoXIHxaVeQGEv2KJNW Y34F+RGpyxneER1gkJeg9i40fRRWX915Jgz9Y2SQSVnrh5Rn0L9nsSqI5DtrsEYYT1 ZRSdIN+q2iXOcn3kXGW7qqLJssrtT5gYajtnIF5JKrv1DmQPgEmj8uPaaJx6KvVR2I t3RHhyUcVWxQBExcY5t4/dl9xl+/nZaiaT6y4i/jkJfLV/t8ZhKmxXYaaSy4jfyCO6 TcLKyZavVu0lw== From: Mirsad Goran Todorovac To: Jason Gunthorpe , Joerg Roedel , Lu Baolu , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: Joerg Roedel , Will Deacon , Robin Murphy , Heiner Kallweit , nic_swsd@realtek.com, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirsad Goran Todorovac , Marco Elver Subject: [PATCH v5 7/7] r8169: Coalesce mac ocp write and modify for rtl_hw_init_8125 to reduce spinlock contention Date: Sun, 29 Oct 2023 19:36:07 +0100 Message-Id: <20231029183600.451694-7-mirsad.todorovac@alu.unizg.hr> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231029183600.451694-1-mirsad.todorovac@alu.unizg.hr> References: <20231029183600.451694-1-mirsad.todorovac@alu.unizg.hr> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Repeated calls to r8168_mac_ocp_write() and r8168_mac_ocp_modify() in the init sequence of the 8125 involve implicit spin_lock_irqsave() and spin_unlock_irqrestore() on each invocation. Coalesced with the corresponding helpers r8168_mac_ocp_write_seq() and r8168_mac_ocp_modify_seq() into sequential write or modidy with a sinqle pair of spin_lock_irqsave() and spin_unlock_irqrestore(), these calls reduce overall lock contention. Fixes: f1bce4ad2f1ce ("r8169: add support for RTL8125") Cc: Heiner Kallweit Cc: Marco Elver Cc: nic_swsd@realtek.com Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org Link: https://lore.kernel.org/lkml/20231028005153.2180411-1-mirsad.todorovac@alu.unizg.hr/ Link: https://lore.kernel.org/lkml/20231028110459.2644926-1-mirsad.todorovac@alu.unizg.hr/ Signed-off-by: Mirsad Goran Todorovac --- v5: added unlocked primitives to allow mac ocs modify grouping applied coalescing of mac ocp writes/modifies for 8168ep and 8117 some formatting fixes to please checkpatch.pl v4: fixed complaints as advised by Heiner and checkpatch.pl split the patch into five sections to be more easily manipulated and reviewed introduced r8168_mac_ocp_write_seq() applied coalescing of mac ocp writes/modifies for 8168H, 8125 and 8125B v3: removed register/mask pair array sentinels, so using ARRAY_SIZE(). avoided duplication of RTL_W32() call code as advised by Heiner. drivers/net/ethernet/realtek/r8169_main.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index 0778cd0ba2e0..76f0f1e13909 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -5089,6 +5089,12 @@ static void rtl_hw_init_8168g(struct rtl8169_private *tp) static void rtl_hw_init_8125(struct rtl8169_private *tp) { + static const struct e_info_regdata hw_init_8125_1[] = { + { 0xc0aa, 0x07d0 }, + { 0xc0a6, 0x0150 }, + { 0xc01e, 0x5555 }, + }; + rtl_enable_rxdvgate(tp); RTL_W8(tp, ChipCmd, RTL_R8(tp, ChipCmd) & ~(CmdTxEnb | CmdRxEnb)); @@ -5098,9 +5104,7 @@ static void rtl_hw_init_8125(struct rtl8169_private *tp) r8168_mac_ocp_modify(tp, 0xe8de, BIT(14), 0); r8168g_wait_ll_share_fifo_ready(tp); - r8168_mac_ocp_write(tp, 0xc0aa, 0x07d0); - r8168_mac_ocp_write(tp, 0xc0a6, 0x0150); - r8168_mac_ocp_write(tp, 0xc01e, 0x5555); + r8168_mac_ocp_write_seq(tp, hw_init_8125_1); r8168g_wait_ll_share_fifo_ready(tp); }