From patchwork Fri Jun 29 11:52:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neil Horman X-Patchwork-Id: 10496483 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 77CD2601C7 for ; Fri, 29 Jun 2018 11:52:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 61B6329839 for ; Fri, 29 Jun 2018 11:52:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 53AE72983B; Fri, 29 Jun 2018 11:52:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B962629839 for ; Fri, 29 Jun 2018 11:52:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753518AbeF2Lwz (ORCPT ); Fri, 29 Jun 2018 07:52:55 -0400 Received: from charlotte.tuxdriver.com ([70.61.120.58]:51247 "EHLO smtp.tuxdriver.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752804AbeF2Lwy (ORCPT ); Fri, 29 Jun 2018 07:52:54 -0400 Received: from cpe-2606-a000-111b-40b7-640c-26a-4e16-9225.dyn6.twc.com ([2606:a000:111b:40b7:640c:26a:4e16:9225] helo=localhost) by smtp.tuxdriver.com with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1fYrxH-0005tU-AL; Fri, 29 Jun 2018 07:52:52 -0400 From: Neil Horman To: linux-rdma@vger.kernel.org Cc: Neil Horman , Adit Ranadive , VMware PV-Drivers , Doug Ledford , Jason Gunthorpe , linux-kernel@vger.kernel.org Subject: [PATCH v2] vmw_pvrdma: Release netdev when vmxnet3 module is removed Date: Fri, 29 Jun 2018 07:52:06 -0400 Message-Id: <20180629115206.18787-1-nhorman@tuxdriver.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180628135938.19625-1-nhorman@tuxdriver.com> References: <20180628135938.19625-1-nhorman@tuxdriver.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On repeated module load/unload cycles, its possible for the pvrmda driver to encounter this crash: ... 297.032448] RIP: 0010:[] [] netdev_walk_all_upper_dev_rcu+0x50/0xb0 [ 297.034078] RSP: 0018:ffff95087780bd08 EFLAGS: 00010286 [ 297.034986] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff95087a0c0000 [ 297.036196] RDX: ffff95087a0c0000 RSI: ffffffff839e44e0 RDI: ffff950835d0c000 [ 297.037421] RBP: ffff95087780bd40 R08: ffff95087a0e0ea0 R09: abddacd03f8e0ea0 [ 297.038636] R10: abddacd03f8e0ea0 R11: ffffef5901e9dbc0 R12: ffff95087a0c0000 [ 297.039854] R13: ffffffff839e44e0 R14: ffff95087a0c0000 R15: ffff950835d0c828 [ 297.041071] FS: 0000000000000000(0000) GS:ffff95087fc00000(0000) knlGS:0000000000000000 [ 297.042443] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 297.043429] CR2: ffffffffffffffe8 CR3: 000000007a652000 CR4: 00000000003607f0 [ 297.044674] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 297.045893] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 297.047109] Call Trace: [ 297.047545] [] netdev_has_upper_dev_all_rcu+0x18/0x20 [ 297.048691] [] is_eth_port_of_netdev+0x2f/0xa0 [ib_core] [ 297.049886] [] ? is_eth_active_slave_of_bonding_rcu+0x70/0x70 [ib_core] ... This occurs because vmw_pvrdma on probe stores a pointer to the netdev that exists on function 0 of the same bus/device/slot (which represents the vmxnet3 ethernet driver). However, it never removes this pointer if the vmxnet3 module is removed, leading to crashes resulting from use after free dereferencing incidents like the one above. The fix is pretty straightforward. vmw_pvrdma should listen for NETDEV_REGISTER and NETDEV_UNREGISTER events in its event listener code block, and update the stored netdev pointer accordingly. This solution has been tested by myself and the reporter with successful results. This fix also allows the pvrdma driver to find its underlying ethernet device in the event that vmxnet3 is loaded after pvrdma, which it was not able to do before. Signed-off-by: Neil Horman Reported-by: ruquin@redhat.com CC: Adit Ranadive CC: VMware PV-Drivers CC: Doug Ledford CC: Jason Gunthorpe CC: linux-kernel@vger.kernel.org Tested-by: Adit Ranadive Acked-by: Adit Ranadive --- Change notes v2) * Move dev_hold in pvrda_pci_probe to below null check (aditr) * Add dev_puts to probe error path and pvrda_pci_remove (jgg) * Cleaned up some checkpatch warnings (nhorman) --- .../infiniband/hw/vmw_pvrdma/pvrdma_main.c | 39 ++++++++++++++++++- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c index 0be33a81bbe6..970d24d887c2 100644 --- a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c +++ b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c @@ -699,8 +699,12 @@ static int pvrdma_del_gid(const struct ib_gid_attr *attr, void **context) } static void pvrdma_netdevice_event_handle(struct pvrdma_dev *dev, + struct net_device *ndev, unsigned long event) { + struct pci_dev *pdev_net; + unsigned int slot; + switch (event) { case NETDEV_REBOOT: case NETDEV_DOWN: @@ -718,6 +722,24 @@ static void pvrdma_netdevice_event_handle(struct pvrdma_dev *dev, else pvrdma_dispatch_event(dev, 1, IB_EVENT_PORT_ACTIVE); break; + case NETDEV_UNREGISTER: + dev_put(dev->netdev); + dev->netdev = NULL; + break; + case NETDEV_REGISTER: + /* vmxnet3 will have same bus, slot. But func will be 0 */ + slot = PCI_SLOT(dev->pdev->devfn); + pdev_net = pci_get_slot(dev->pdev->bus, + PCI_DEVFN(slot, 0)); + if ((dev->netdev == NULL) && + (pci_get_drvdata(pdev_net) == ndev)) { + /* this is our netdev */ + dev->netdev = ndev; + dev_hold(ndev); + } + pci_dev_put(pdev_net); + break; + default: dev_dbg(&dev->pdev->dev, "ignore netdevice event %ld on %s\n", event, dev->ib_dev.name); @@ -734,8 +756,11 @@ static void pvrdma_netdevice_event_work(struct work_struct *work) mutex_lock(&pvrdma_device_list_lock); list_for_each_entry(dev, &pvrdma_device_list, device_link) { - if (dev->netdev == netdev_work->event_netdev) { - pvrdma_netdevice_event_handle(dev, netdev_work->event); + if ((netdev_work->event == NETDEV_REGISTER) || + (dev->netdev == netdev_work->event_netdev)) { + pvrdma_netdevice_event_handle(dev, + netdev_work->event_netdev, + netdev_work->event); break; } } @@ -968,6 +993,7 @@ static int pvrdma_pci_probe(struct pci_dev *pdev, ret = -ENODEV; goto err_free_cq_ring; } + dev_hold(dev->netdev); dev_info(&pdev->dev, "paired device to %s\n", dev->netdev->name); @@ -1040,6 +1066,10 @@ static int pvrdma_pci_probe(struct pci_dev *pdev, pvrdma_free_irq(dev); pci_free_irq_vectors(pdev); err_free_cq_ring: + if (dev->netdev) { + dev_put(dev->netdev); + dev->netdev = NULL; + } pvrdma_page_dir_cleanup(dev, &dev->cq_pdir); err_free_async_ring: pvrdma_page_dir_cleanup(dev, &dev->async_pdir); @@ -1079,6 +1109,11 @@ static void pvrdma_pci_remove(struct pci_dev *pdev) flush_workqueue(event_wq); + if (dev->netdev) { + dev_put(dev->netdev); + dev->netdev = NULL; + } + /* Unregister ib device */ ib_unregister_device(&dev->ib_dev);