From patchwork Tue Sep 26 21:27:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Anderson X-Patchwork-Id: 13399691 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 636AC4A52F for ; Tue, 26 Sep 2023 21:28:58 +0000 (UTC) Received: from mail-oi1-x233.google.com (mail-oi1-x233.google.com [IPv6:2607:f8b0:4864:20::233]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 017BFA266 for ; Tue, 26 Sep 2023 14:28:57 -0700 (PDT) Received: by mail-oi1-x233.google.com with SMTP id 5614622812f47-3adc9770737so6012346b6e.3 for ; Tue, 26 Sep 2023 14:28:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1695763736; x=1696368536; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1loE5eg1Eoj4ZTfC+vU3O0r4gZ2OLkuivrhZ+A+bBkA=; b=n0iEUIJZLZHal0tpBXmBPg99RnWiFacLJT5nsc//If9A9v3R1yk7gDIrkhtqPg0n5j XcT9T4gNsx0Z/ZE6YUyTcdNHquUmYPhwCsD/yA5FCj2LF8xD40/I80ycF0NRx4iOR5Vb Nix8n4jULZdUHcGa+soPXgh69sMfdAvDZ4lZc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695763736; x=1696368536; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1loE5eg1Eoj4ZTfC+vU3O0r4gZ2OLkuivrhZ+A+bBkA=; b=flE7uk8aTgQ5QP0fGFcTCycQqEZRG8I/ntOMK+v057E/GdIiuOFPh4H2PM93sR8Esq X4CJ8pWDhIb3RLGaHkK2yTNqGC01SaX2sUYLb9DRL2E2W/Umyf4/A9KD9tQd0ZbbZVWg pWbkjEtTSoQVx2eKdEbugbJscuA5a7I4I0gbegZFM5j1S5dMcNn2qjSvq0DC+x1HeDTE yTK0isYqtWfqGWWNwlx+h/Emn1vH03lkO1VxTOwvJAGluiV0CN9LmbDElWOfhk7fSf06 rJSgcr6nm9OudiRV2VqCjCOrhsTedVEJA2+CzGkedTT9Jr7zCC5YSa9mXX92i1d25FDp 0Jsw== X-Gm-Message-State: AOJu0YwHodSfRnZLeNAODdF1DkN7WRV4nX01ZADM55vAEd7nrdq67ZSu t6vJ/u8Nre8Nc6IFZtSPZ01Gwg== X-Google-Smtp-Source: AGHT+IF9qNMEYob7OXrxHyrG09tHkFsXpQZG6RDgC/gNKP4FupaZmh2+AHPaaHZkN4OnVPiNuR9dNA== X-Received: by 2002:aca:1218:0:b0:3a7:6d64:aa68 with SMTP id 24-20020aca1218000000b003a76d64aa68mr232514ois.18.1695763736267; Tue, 26 Sep 2023 14:28:56 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:f39:c3f2:a3b:4fcd]) by smtp.gmail.com with ESMTPSA id f15-20020aa78b0f000000b0068fece2c190sm10337251pfd.70.2023.09.26.14.28.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Sep 2023 14:28:55 -0700 (PDT) From: Douglas Anderson To: Jakub Kicinski , Hayes Wang , "David S . Miller" Cc: linux-usb@vger.kernel.org, Grant Grundler , Edward Hill , Douglas Anderson , anton@polit.no, bjorn@mork.no, edumazet@google.com, horms@kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, pabeni@redhat.com Subject: [PATCH 1/3] r8152: Increase USB control msg timeout to 5000ms as per spec Date: Tue, 26 Sep 2023 14:27:26 -0700 Message-ID: <20230926142724.1.I6e4fb5ae61b4c6ab32058cb12228fd5bd32da676@changeid> X-Mailer: git-send-email 2.42.0.515.g380fc7ccd1-goog In-Reply-To: <20230926212824.1512665-1-dianders@chromium.org> References: <20230926212824.1512665-1-dianders@chromium.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org According to the comment next to USB_CTRL_GET_TIMEOUT and USB_CTRL_SET_TIMEOUT, although sending/receiving control messages is usually quite fast, the spec allows them to take up to 5 seconds. Let's increase the timeout in the Realtek driver from 500ms to 5000ms (using the #defines) to account for this. This is not just a theoretical change. The need for the longer timeout was seen in testing. Specifically, if you drop a sc7180-trogdor based Chromebook into the kdb debugger and then "go" again after sitting in the debugger for a while, the next USB control message takes a long time. Out of ~40 tests the slowest USB control message was 4.5 seconds. While dropping into kdb is not exactly an end-user scenario, the above is similar to what could happen due to an temporary interrupt storm, what could happen if there was a host controller (HW or SW) issue, or what could happen if the Realtek device got into a confused state and needed time to recover. This change is fairly critical since the r8152 driver in Linux doesn't expect register reads/writes (which are backed by USB control messages) to fail. Fixes: ac718b69301c ("net/usb: new driver for RTL8152") Suggested-by: Hayes Wang Signed-off-by: Douglas Anderson --- drivers/net/usb/r8152.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c index 0c13d9950cd8..482957beae66 100644 --- a/drivers/net/usb/r8152.c +++ b/drivers/net/usb/r8152.c @@ -1212,7 +1212,7 @@ int get_registers(struct r8152 *tp, u16 value, u16 index, u16 size, void *data) ret = usb_control_msg(tp->udev, tp->pipe_ctrl_in, RTL8152_REQ_GET_REGS, RTL8152_REQT_READ, - value, index, tmp, size, 500); + value, index, tmp, size, USB_CTRL_GET_TIMEOUT); if (ret < 0) memset(data, 0xff, size); else @@ -1235,7 +1235,7 @@ int set_registers(struct r8152 *tp, u16 value, u16 index, u16 size, void *data) ret = usb_control_msg(tp->udev, tp->pipe_ctrl_out, RTL8152_REQ_SET_REGS, RTL8152_REQT_WRITE, - value, index, tmp, size, 500); + value, index, tmp, size, USB_CTRL_SET_TIMEOUT); kfree(tmp); @@ -9494,7 +9494,8 @@ static u8 __rtl_get_hw_ver(struct usb_device *udev) ret = usb_control_msg(udev, usb_rcvctrlpipe(udev, 0), RTL8152_REQ_GET_REGS, RTL8152_REQT_READ, - PLA_TCR0, MCU_TYPE_PLA, tmp, sizeof(*tmp), 500); + PLA_TCR0, MCU_TYPE_PLA, tmp, sizeof(*tmp), + USB_CTRL_GET_TIMEOUT); if (ret > 0) ocp_data = (__le32_to_cpu(*tmp) >> 16) & VERSION_MASK; From patchwork Tue Sep 26 21:27:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Anderson X-Patchwork-Id: 13399692 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72D8E4B200 for ; Tue, 26 Sep 2023 21:29:00 +0000 (UTC) Received: from mail-pf1-x432.google.com (mail-pf1-x432.google.com [IPv6:2607:f8b0:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7F44A25F for ; Tue, 26 Sep 2023 14:28:58 -0700 (PDT) Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-692ada71d79so6295229b3a.1 for ; Tue, 26 Sep 2023 14:28:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1695763738; x=1696368538; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qVCYC0vkrZ6zXTzyEFnI7OnTxuwy99dqGDu6Ppi9+fs=; b=SJGNymYYAvCwIq47qwJNzf0kqY+QCYUyEqgOttrW+4HnQ2tokbmtWl1GicjlIg6y5a c5oIwCoVzEch2FvoWvgJEqYP72kB4KQfWVzqKhRxi9IKpdGl3oimCzrtXzu1yV7UtLgD wggXHhpwkNilkn+lNkZx5hi84jB3z/7qUaJFg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695763738; x=1696368538; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qVCYC0vkrZ6zXTzyEFnI7OnTxuwy99dqGDu6Ppi9+fs=; b=jgz7YV4MmcVQ/MffuThgy5IYOpwpxCUf3iygqsqDXIKg1pkhbRGjHAu8MK0sbGaO99 CPTKpzMBWxska0BvuGdcInfdK28I3lNehJbn1aqhdj/x2uP135xr+dbuDpkSb6soz9nV JIkMiAn9w9MWlkrhN/UYOuVCaOPbyPWW7LI+7bj213/20tewb9B0iNEcHIrq4urx0v3X mQ3JaZ0vrw6+Hkhqc3slsB9q3CCjGx0HqqewwuVrLC0mtX0f2nTINgH4BzZCHqeP50sj DSRUu/S+LnwVJXWTWT/6RjbRKh6D8g2Ysz6d5TqJr37dUeHzHk4HgECZW5OnmiUIoYdk C61Q== X-Gm-Message-State: AOJu0YwT7xOX65cQPSxI/jdlbdbvdQ0xbMzw38+cGOHT3DlxR52qJo7Z wGdsdT6ZSKiIRl7o2PhRtuD0UQ== X-Google-Smtp-Source: AGHT+IHG1iNVZ2QkEqvmteXELKf2TZQxaAuiCqsMdO2hn0iNnxPeLxG8BUAPLN5Qx+Z+M0jVxMDFxg== X-Received: by 2002:a05:6a00:1acb:b0:68e:29a6:e247 with SMTP id f11-20020a056a001acb00b0068e29a6e247mr233576pfv.10.1695763738175; Tue, 26 Sep 2023 14:28:58 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:f39:c3f2:a3b:4fcd]) by smtp.gmail.com with ESMTPSA id f15-20020aa78b0f000000b0068fece2c190sm10337251pfd.70.2023.09.26.14.28.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Sep 2023 14:28:57 -0700 (PDT) From: Douglas Anderson To: Jakub Kicinski , Hayes Wang , "David S . Miller" Cc: linux-usb@vger.kernel.org, Grant Grundler , Edward Hill , Douglas Anderson , andre.przywara@arm.com, bjorn@mork.no, edumazet@google.com, gaul@gaul.org, horms@kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, pabeni@redhat.com Subject: [PATCH 2/3] r8152: Retry register reads/writes Date: Tue, 26 Sep 2023 14:27:27 -0700 Message-ID: <20230926142724.2.I65ea4ac938a55877dc99fdf5b3883ad92d8abce2@changeid> X-Mailer: git-send-email 2.42.0.515.g380fc7ccd1-goog In-Reply-To: <20230926212824.1512665-1-dianders@chromium.org> References: <20230926212824.1512665-1-dianders@chromium.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org Even though the functions to read/write registers can fail, most of the places in the r8152 driver that read/write register values don't check error codes. The lack of error code checking is problematic in at least two ways. The first problem is that the r8152 driver often uses code patterns similar to this: x = read_register() x = x | SOME_BIT; write_register(x); ...with the above pattern, if the read_register() fails and returns garbage then we'll end up trying to write modified garbage back to the Realtek adapter. If the write_register() succeeds that's bad. Note that as of commit f53a7ad18959 ("r8152: Set memory to all 0xFFs on failed reg reads") the "garbage" returned by read_register() will at least be consistent garbage, but it is still garbage. It turns out that this problem is very serious. Writing garbage to some of the hardware registers on the Ethernet adapter can put the adapter in such a bad state that it needs to be power cycled (fully unplugged and plugged in again) before it can enumerate again. The second problem is that the r8152 driver generally has functions that are long sequences of register writes. Assuming everything will be OK if a random register write fails in the middle isn't a great assumption. One might wonder if the above two problems are real. You could ask if we would really have a successful write after a failed read. It turns out that the answer appears to be "yes, this can happen". In fact, we've seen at least two distinct failure modes where this happens. On a sc7180-trogdor Chromebook if you drop into kdb for a while and then resume, you can see: 1. We get a "Tx timeout" 2. The "Tx timeout" queues up a USB reset. 3. In rtl8152_pre_reset() we try to reinit the hardware. 4. The first several (2-9) register accesses fail with a timeout, then things recover. The above test case was actually fixed by the patch ("r8152: Increase USB control msg timeout to 5000ms as per spec") but at least shows that we really can see successful calls after failed ones. On a different (AMD) based Chromebook, we found that during reboot tests we'd also sometimes get a transitory failure. In this case we saw -EPIPE being returned sometimes. Retrying one time worked fine. To keep things robust, let's try register access up to 3 times. If we get 3 5-second timeouts in a row this could block things for 15 seconds but that hasn't been seen in practice. If we see that happening and there is a better way to solve it then we can add a special case for that later. Signed-off-by: Douglas Anderson --- Originally when looking at this problem I thought that the obvious solution was to "just" add better error handling to the driver. This _sounds_ appealing, but it's a massive change and touches a significant portion of the lines in this driver. It's also not always obvious what the driver should be doing to handle errors. drivers/net/usb/r8152.c | 67 +++++++++++++++++++++++++++++++++++------ 1 file changed, 57 insertions(+), 10 deletions(-) diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c index 482957beae66..976d6caf2f04 100644 --- a/drivers/net/usb/r8152.c +++ b/drivers/net/usb/r8152.c @@ -1200,6 +1200,52 @@ static unsigned int agg_buf_sz = 16384; #define RTL_LIMITED_TSO_SIZE (size_to_mtu(agg_buf_sz) - sizeof(struct tx_desc)) +/* If we get a failure and the USB device is still attached when trying to read + * or write registers then we'll retry a few times. Failures accessing registers + * shouldn't be common and this adds robustness. Much code in the driver doesn't + * check for errors. Notably, many parts of the driver do a read/modify/write + * of a register value without confirming that the read succeeded. Writing back + * modified garbage like this can fully wedge the adapter, requiring a power + * cycle. + */ +#define REGISTER_ACCESS_TRIES 3 + +static +int r8152_control_msg(struct usb_device *udev, unsigned int pipe, __u8 request, + __u8 requesttype, __u16 value, __u16 index, void *data, + __u16 size, const char *msg_tag) +{ + int i; + int ret; + + for (i = 0; i < REGISTER_ACCESS_TRIES; i++) { + ret = usb_control_msg(udev, pipe, request, requesttype, + value, index, data, size, + USB_CTRL_GET_TIMEOUT); + + /* No need to retry or spam errors if the USB device got + * unplugged; just return immediately. + */ + if (udev->state == USB_STATE_NOTATTACHED) + return ret; + + if (ret >= 0) + break; + } + + if (ret < 0) { + dev_err(&udev->dev, + "Failed to %s %d bytes at %#06x/%#06x (%d)\n", + msg_tag, size, value, index, ret); + } else if (i != 0) { + dev_warn(&udev->dev, + "Needed %d tries to %s %d bytes at %#06x/%#06x\n", + i + 1, msg_tag, size, value, index); + } + + return ret; +} + static int get_registers(struct r8152 *tp, u16 value, u16 index, u16 size, void *data) { @@ -1210,9 +1256,10 @@ int get_registers(struct r8152 *tp, u16 value, u16 index, u16 size, void *data) if (!tmp) return -ENOMEM; - ret = usb_control_msg(tp->udev, tp->pipe_ctrl_in, - RTL8152_REQ_GET_REGS, RTL8152_REQT_READ, - value, index, tmp, size, USB_CTRL_GET_TIMEOUT); + ret = r8152_control_msg(tp->udev, tp->pipe_ctrl_in, + RTL8152_REQ_GET_REGS, RTL8152_REQT_READ, + value, index, tmp, size, "read"); + if (ret < 0) memset(data, 0xff, size); else @@ -1233,9 +1280,9 @@ int set_registers(struct r8152 *tp, u16 value, u16 index, u16 size, void *data) if (!tmp) return -ENOMEM; - ret = usb_control_msg(tp->udev, tp->pipe_ctrl_out, - RTL8152_REQ_SET_REGS, RTL8152_REQT_WRITE, - value, index, tmp, size, USB_CTRL_SET_TIMEOUT); + ret = r8152_control_msg(tp->udev, tp->pipe_ctrl_out, + RTL8152_REQ_SET_REGS, RTL8152_REQT_WRITE, + value, index, tmp, size, "write"); kfree(tmp); @@ -9492,10 +9539,10 @@ static u8 __rtl_get_hw_ver(struct usb_device *udev) if (!tmp) return 0; - ret = usb_control_msg(udev, usb_rcvctrlpipe(udev, 0), - RTL8152_REQ_GET_REGS, RTL8152_REQT_READ, - PLA_TCR0, MCU_TYPE_PLA, tmp, sizeof(*tmp), - USB_CTRL_GET_TIMEOUT); + ret = r8152_control_msg(udev, usb_rcvctrlpipe(udev, 0), + RTL8152_REQ_GET_REGS, RTL8152_REQT_READ, + PLA_TCR0, MCU_TYPE_PLA, tmp, sizeof(*tmp), + "read"); if (ret > 0) ocp_data = (__le32_to_cpu(*tmp) >> 16) & VERSION_MASK; From patchwork Tue Sep 26 21:27:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Anderson X-Patchwork-Id: 13399693 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D6EA54A552 for ; Tue, 26 Sep 2023 21:29:02 +0000 (UTC) Received: from mail-pf1-x432.google.com (mail-pf1-x432.google.com [IPv6:2607:f8b0:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6972BA275 for ; Tue, 26 Sep 2023 14:29:00 -0700 (PDT) Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-690b7cb71aeso7340916b3a.0 for ; Tue, 26 Sep 2023 14:29:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1695763740; x=1696368540; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JZ7x1sGJs3IwI2EHRWherpwDYB2HwErhY7l2CYJSHl8=; b=VJIuaHiWKyIqjyfefgBHHnrzx1bPs6hpgjKgZkpdlvzWpBF1epF9oc6phvRSxezdsF 2wnyEHTfBcq9kjMIivQIxor8T/f84K53NS8ky5knTzHpKLhghakZ2tXKeRZ4yD+Fz+E3 zJYrcsrG2SEUInFEiXGd6DYDpAiPd+5Yd2BK4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695763740; x=1696368540; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JZ7x1sGJs3IwI2EHRWherpwDYB2HwErhY7l2CYJSHl8=; b=JL/DsieLWe65Trjl1Dps9Jz96bb9y87f19RNzn2zaYDY7exD3U3EhVD83Ko+JCxaU/ LXGd+hUNSqLxAP6CsW2v+xJe6ubAx6QIrbw7AfhBRFJZMJ6lCRNgvp36yb1OqchG0ToO R1L4ZhZfpQ58VQE7o0Cb6rwgNwL6kQW5CNHLNp8uF3yzGG4Ns/3v1WydCF+m22zEFHgc KYeN2XWMTw/ZaTnAOKi6DOBHSPj68yVTmi0ss3bm45OrRzMBwrg1FLCPN9ALBZYQ6I7d RcEu1Xn7pyth1J+Ouwso2VyzrhnrERySL27Q8sf7e3ubdpH9sRpr3XN08AH93y72ZXqI B3Yg== X-Gm-Message-State: AOJu0YynbkCSOZnD5oGKYjXu+RXnu8hfeX5zm4yUI2FF5/pi5msWrcfX mKLNdbbGDnTaEjO9NYjKf/C9bA== X-Google-Smtp-Source: AGHT+IGliQVWRvnnGx8nZVCufY9sGes+j8Ixl6hIyxdTACxOhy8mzzlzOZKuwqC7TvNTBKSNs5S2GQ== X-Received: by 2002:a62:ee06:0:b0:68f:a92a:8509 with SMTP id e6-20020a62ee06000000b0068fa92a8509mr4687125pfi.7.1695763739823; Tue, 26 Sep 2023 14:28:59 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:f39:c3f2:a3b:4fcd]) by smtp.gmail.com with ESMTPSA id f15-20020aa78b0f000000b0068fece2c190sm10337251pfd.70.2023.09.26.14.28.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Sep 2023 14:28:59 -0700 (PDT) From: Douglas Anderson To: Jakub Kicinski , Hayes Wang , "David S . Miller" Cc: linux-usb@vger.kernel.org, Grant Grundler , Edward Hill , Douglas Anderson , bjorn@mork.no, edumazet@google.com, horms@kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, pabeni@redhat.com Subject: [PATCH 3/3] r8152: Block future register access if register access fails Date: Tue, 26 Sep 2023 14:27:28 -0700 Message-ID: <20230926142724.3.Ib2affdbfdc2527aaeef9b46d4f23f7c04147faeb@changeid> X-Mailer: git-send-email 2.42.0.515.g380fc7ccd1-goog In-Reply-To: <20230926212824.1512665-1-dianders@chromium.org> References: <20230926212824.1512665-1-dianders@chromium.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org As talked about in the patch ("r8152: Retry register reads/writes"), much of the r8152 driver doesn't check for error codes when modifying the adapter's registers. Specifically, the worst part of this appears to be when the driver does a read-modify-write of a register. If the read is garbage (because the read failed) and the write succeeds, we end up writing modified garbage to the adapter and this appears to be able to hard wedge it. After we added the retries, it should be nearly impossible to see this problem. After all, the problem shows up if the register read failed and _then_ the register write worked. If the read fails for many times in a row, chances are things are really dead and no future writes will succeed. However, nearly impossible isn't actually impossible. To be safe, let's fully lock out all register access if all of our retries fail. When we do this, we'll try to queue up a USB reset and try to unlock register access after the reset. To avoid looping forever, we'll only attempt this a handful of times. Signed-off-by: Douglas Anderson --- This patch was tested with REGISTER_ACCESS_TRIES set to 1, effectively disabling the patch ("r8152: Retry register reads/writes"). I also decreased the timeout back to 500ms so I could use kgdb to reproduce failures. To confirm that the reset queuing worked, I allowed queueing a reset even if pre-reset was running. That worked fine, but did cause a double-reset. drivers/net/usb/r8152.c | 80 +++++++++++++++++++++++++++++++++++------ 1 file changed, 69 insertions(+), 11 deletions(-) diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c index 976d6caf2f04..a7a55849fa1a 100644 --- a/drivers/net/usb/r8152.c +++ b/drivers/net/usb/r8152.c @@ -953,6 +953,10 @@ struct r8152 { u8 version; u8 duplex; u8 autoneg; + + bool reg_access_fail; + bool in_pre_reset; + unsigned int reg_access_reset_count; }; /** @@ -1210,14 +1214,22 @@ static unsigned int agg_buf_sz = 16384; */ #define REGISTER_ACCESS_TRIES 3 +/* If register access fails too many times, stop trying to access the registers + * and issue a reset to fix the issue. + */ +#define REGISTER_ACCESS_MAX_RESETS 3 + static int r8152_control_msg(struct usb_device *udev, unsigned int pipe, __u8 request, __u8 requesttype, __u16 value, __u16 index, void *data, - __u16 size, const char *msg_tag) + __u16 size, const char *msg_tag, struct r8152 *tp) { int i; int ret; + if (tp && tp->reg_access_fail) + return -EIO; + for (i = 0; i < REGISTER_ACCESS_TRIES; i++) { ret = usb_control_msg(udev, pipe, request, requesttype, value, index, data, size, @@ -1233,14 +1245,54 @@ int r8152_control_msg(struct usb_device *udev, unsigned int pipe, __u8 request, break; } - if (ret < 0) { + if (ret >= 0) { + if (tp) + tp->reg_access_reset_count = 0; + + if (i != 0) + dev_warn(&udev->dev, + "Needed %d tries to %s %d bytes at %#06x/%#06x\n", + i + 1, msg_tag, size, value, index); + + return ret; + } + + dev_err(&udev->dev, + "Failed to %s %d bytes at %#06x/%#06x (%d)\n", + msg_tag, size, value, index, ret); + + /* If tp is NULL then we're called from early probe or from the + * r8153_ecm module. In either case, the retries above are pretty much + * all we can do. Bail. + */ + if (!tp) + return ret; + + /* Block all future register access until we reset. This prevents the + * parts of the driver that do read-modify-write without checking for + * errors from writing back modified garbage to the adapter. + */ + tp->reg_access_fail = true; + + /* Failing to access registers in pre-reset is not surprising since we + * wouldn't be resetting if things were behaving normally. The register + * access we do in pre-reset isn't truly mandatory--we're just reusing + * the disable() function and trying to be nice by powering the + * adapter down before resetting it. + * + * If we're in pre-reset, we'll return right away and not try to queue + * up yet another reset. We know the post-reset is already coming. + */ + if (tp->in_pre_reset) + return ret; + + if (tp->reg_access_reset_count < REGISTER_ACCESS_MAX_RESETS) { + usb_queue_reset_device(tp->intf); + tp->reg_access_reset_count++; + } else if (tp->reg_access_reset_count == REGISTER_ACCESS_MAX_RESETS) { dev_err(&udev->dev, - "Failed to %s %d bytes at %#06x/%#06x (%d)\n", - msg_tag, size, value, index, ret); - } else if (i != 0) { - dev_warn(&udev->dev, - "Needed %d tries to %s %d bytes at %#06x/%#06x\n", - i + 1, msg_tag, size, value, index); + "Tried to reset %d times; giving up.\n", + REGISTER_ACCESS_MAX_RESETS); } return ret; @@ -1258,7 +1310,7 @@ int get_registers(struct r8152 *tp, u16 value, u16 index, u16 size, void *data) ret = r8152_control_msg(tp->udev, tp->pipe_ctrl_in, RTL8152_REQ_GET_REGS, RTL8152_REQT_READ, - value, index, tmp, size, "read"); + value, index, tmp, size, "read", tp); if (ret < 0) memset(data, 0xff, size); @@ -1282,7 +1334,7 @@ int set_registers(struct r8152 *tp, u16 value, u16 index, u16 size, void *data) ret = r8152_control_msg(tp->udev, tp->pipe_ctrl_out, RTL8152_REQ_SET_REGS, RTL8152_REQT_WRITE, - value, index, tmp, size, "write"); + value, index, tmp, size, "write", tp); kfree(tmp); @@ -8317,7 +8369,9 @@ static int rtl8152_pre_reset(struct usb_interface *intf) napi_disable(&tp->napi); if (netif_carrier_ok(netdev)) { mutex_lock(&tp->control); + tp->in_pre_reset = true; tp->rtl_ops.disable(tp); + tp->in_pre_reset = false; mutex_unlock(&tp->control); } @@ -8333,6 +8387,10 @@ static int rtl8152_post_reset(struct usb_interface *intf) if (!tp) return 0; + mutex_lock(&tp->control); + tp->reg_access_fail = false; + mutex_unlock(&tp->control); + /* reset the MAC address in case of policy change */ if (determine_ethernet_addr(tp, &sa) >= 0) { rtnl_lock(); @@ -9542,7 +9600,7 @@ static u8 __rtl_get_hw_ver(struct usb_device *udev) ret = r8152_control_msg(udev, usb_rcvctrlpipe(udev, 0), RTL8152_REQ_GET_REGS, RTL8152_REQT_READ, PLA_TCR0, MCU_TYPE_PLA, tmp, sizeof(*tmp), - "read"); + "read", NULL); if (ret > 0) ocp_data = (__le32_to_cpu(*tmp) >> 16) & VERSION_MASK;