From patchwork Sat Sep 25 11:22:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 12517359 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8251C433EF for ; Sat, 25 Sep 2021 11:23:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D0E6161351 for ; Sat, 25 Sep 2021 11:23:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244595AbhIYLZI (ORCPT ); Sat, 25 Sep 2021 07:25:08 -0400 Received: from mail.kernel.org ([198.145.29.99]:55792 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244574AbhIYLY6 (ORCPT ); Sat, 25 Sep 2021 07:24:58 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 853346128B; Sat, 25 Sep 2021 11:23:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1632569003; bh=02xRTwet1c5l8qRgcXs4VCpKQJvqfYPSwFUrWIoc2cM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eBdkOeQZAzQfx/m/yxFbsJIGBvkhRpE2hSfkJbHrHeRhC5LOGgZSfk4qdoFd5jXLC 4cajFeDHROqHcKL6Uxo3t/x66TBMHadOtMGgbKbHUJLVc+EoKzRywI13DAUuwL4Nmy XdaiN+sF1pwhdlQ+uSs0w4bITD8GdZlM7wYUfcvrz8HaBW66RC6ITexIAdOoJUX2tS NdMaGkjmUePmj+YdIJxYmXTz6HRjqnWB052FPKJjeF22nVurS6AbIJA54sOsNoQIvo MozNtLmeX9dcJkGV28/X2nazhnG4nqa89dbG8o8cBxCkZF4jyGYcMvQtjkCR91Gadf 06/PrR1PnbLsA== From: Leon Romanovsky To: "David S . Miller" , Jakub Kicinski Cc: Leon Romanovsky , Alexandre Belloni , Andrew Lunn , Ariel Elior , Bin Luo , Claudiu Manoil , Coiby Xu , Derek Chickles , drivers@pensando.io, Felix Manlunas , Florian Fainelli , Geetha sowjanya , Greg Kroah-Hartman , GR-everest-linux-l2@marvell.com, GR-Linux-NIC-Dev@marvell.com, hariprasad , Ido Schimmel , Intel Corporation , intel-wired-lan@lists.osuosl.org, Ioana Ciornei , Jerin Jacob , Jesse Brandeburg , Jiri Pirko , Jonathan Lemon , Linu Cherian , linux-kernel@vger.kernel.org, linux-omap@vger.kernel.org, linux-rdma@vger.kernel.org, linux-staging@lists.linux.dev, Loic Poulain , Manish Chopra , M Chetan Kumar , Michael Chan , Michael Guralnik , netdev@vger.kernel.org, oss-drivers@corigine.com, Richard Cochran , Saeed Mahameed , Satanand Burla , Sergey Ryazanov , Shannon Nelson , Simon Horman , Subbaraya Sundeep , Sunil Goutham , Taras Chornyi , Tariq Toukan , Tony Nguyen , UNGLinuxDriver@microchip.com, Vadym Kochan , Vivien Didelot , Vladimir Oltean Subject: [PATCH net-next v1 03/21] liquidio: Overcome missing device lock protection in init/remove flows Date: Sat, 25 Sep 2021 14:22:43 +0300 Message-Id: <4e78bd8c6f45eef8093547a37c4d00ef0d1fef56.1632565508.git.leonro@nvidia.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-omap@vger.kernel.org From: Leon Romanovsky The liquidio driver is broken by design. It initialize PCI devices in separate delayed works. It causes to the situation where device lock is dropped during initialize and remove sequences. That lock is part of driver/core and needed to protect from races during init, destroy and bus invocations. In addition to lack of locking protection, it has incorrect order of destroy flows and very questionable synchronization scheme based on atomic_t. This change doesn't fix that driver but makes sure that rest of the netdev subsystem doesn't suffer from such basic protection by adding device_lock over devlink_*() APIs and by moving devlink_register() to be last command in setup_nic_devices(). Signed-off-by: Leon Romanovsky --- .../net/ethernet/cavium/liquidio/lio_main.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c index a34b3bb2dd4f..dafc79bd34f4 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_main.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c @@ -1279,6 +1279,14 @@ static int liquidio_stop_nic_module(struct octeon_device *oct) struct lio *lio; dev_dbg(&oct->pci_dev->dev, "Stopping network interfaces\n"); + device_lock(&oct->pci_dev->dev); + if (oct->devlink) { + devlink_unregister(oct->devlink); + devlink_free(oct->devlink); + oct->devlink = NULL; + } + device_unlock(&oct->pci_dev->dev); + if (!oct->ifcount) { dev_err(&oct->pci_dev->dev, "Init for Octeon was not completed\n"); return 1; @@ -1300,12 +1308,6 @@ static int liquidio_stop_nic_module(struct octeon_device *oct) for (i = 0; i < oct->ifcount; i++) liquidio_destroy_nic_device(oct, i); - if (oct->devlink) { - devlink_unregister(oct->devlink); - devlink_free(oct->devlink); - oct->devlink = NULL; - } - dev_dbg(&oct->pci_dev->dev, "Network interfaces stopped\n"); return 0; } @@ -3749,10 +3751,12 @@ static int setup_nic_devices(struct octeon_device *octeon_dev) } } + device_lock(&octeon_dev->pci_dev->dev); devlink = devlink_alloc(&liquidio_devlink_ops, sizeof(struct lio_devlink_priv), &octeon_dev->pci_dev->dev); if (!devlink) { + device_unlock(&octeon_dev->pci_dev->dev); dev_err(&octeon_dev->pci_dev->dev, "devlink alloc failed\n"); goto setup_nic_dev_free; } @@ -3760,9 +3764,10 @@ static int setup_nic_devices(struct octeon_device *octeon_dev) lio_devlink = devlink_priv(devlink); lio_devlink->oct = octeon_dev; - devlink_register(devlink); octeon_dev->devlink = devlink; octeon_dev->eswitch_mode = DEVLINK_ESWITCH_MODE_LEGACY; + devlink_register(devlink); + device_unlock(&octeon_dev->pci_dev->dev); return 0;