From patchwork Tue Jan 10 19:15:39 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adit Ranadive X-Patchwork-Id: 9508415 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7475460231 for ; Tue, 10 Jan 2017 19:17:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6530427F89 for ; Tue, 10 Jan 2017 19:17:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 56F9028553; Tue, 10 Jan 2017 19:17:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3306627F89 for ; Tue, 10 Jan 2017 19:17:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S942474AbdAJTRI (ORCPT ); Tue, 10 Jan 2017 14:17:08 -0500 Received: from mail-sn1nam01on0083.outbound.protection.outlook.com ([104.47.32.83]:28692 "EHLO NAM01-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S940646AbdAJTQL (ORCPT ); Tue, 10 Jan 2017 14:16:11 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=onevmw.onmicrosoft.com; s=selector1-vmware-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=lZFt5qG5pTMjBzgEbUk134bZIehuEZwOsGaGsmT7xA0=; b=YefH19scd80t7XPuj8s5JNTiCektIJEXx7rMp0iF3GT2TwrIrWWrN298e1QckpNA468jTHrODG1+ko/0LELeeKyN9XpVlfTyw+UEj0GpJsipoAHlU4m8f9l0fFP7+Op4mm+swWDwjUPK/LBRwO/rd+iDfMuIxiF1TDX2IyFHNqo= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=aditr@vmware.com; Received: from promb-2s-dhcp95-136.eng.vmware.com (208.91.1.34) by CO2PR0501MB838.namprd05.prod.outlook.com (10.141.244.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.845.6; Tue, 10 Jan 2017 19:16:02 +0000 From: Adit Ranadive To: , CC: Adit Ranadive , , Aditya Sarwade Subject: [PATCH 1/3] IB/vmw_pvrdma: Defer activating device until vmxnet3 link is up Date: Tue, 10 Jan 2017 11:15:39 -0800 Message-ID: <4b515ded56300f12cedc67253d42ab8fbc52134e.1484075557.git.aditr@vmware.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [208.91.1.34] X-ClientProxiedBy: DM5PR21CA0037.namprd21.prod.outlook.com (10.172.92.151) To CO2PR0501MB838.namprd05.prod.outlook.com (10.141.244.148) X-MS-Office365-Filtering-Correlation-Id: 4bd423fe-46da-445d-cd2d-08d4398d1e88 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001); SRVR:CO2PR0501MB838; X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB838; 3:heaxgFRQ6AqP7+7i6EFTyqVH03EGZbePeQoZFKWb93d8ZazSuQBFJVkXLlRZ9Ps7F0NouckV709jfVGIb9LccnYh63DMJKBm8Zopzz+colDa6xhG6/OLh90cPqKzuZ/bTTWYRZfyFCNuSTRGT55BVXAVzNCllL8de1E7EN+4Gb6abnNe6DXPBhvuJhdRSm30Var3t8EXE1LuhoxiH0xhKgLa9DvdrMmruqpKzmNa+aByyAV/4YdBdfaX/QaZL7qYMIP4FTngUKDnw/+m1INSHA== X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB838; 25:s0rK3z8IWRJTGpeMwgV7TFqEhI1xsjOVxJnIR9Gd3jvl/6T9PhmM0q7PzDr4Y4qEFJ4wvRvZ7VLY0V67xxyg6iBLwroZoyeFFlzXpNWPB6lr1B9ZK96b2aHUNVqOnyxrFEjJs+IDLsgF2koHnPyJ44lLwRfKFD2bt8V8fDXtqOJy7be6vj4oFegPROdYf3KSgcds2pp52LN6kDYEkPw9klgKMYCS5oEFm6+9DpOCCqVbF0jydtrtWJYt2fpw24kXPIf55ufwnC+oDDyJHnTu6JvO8CQYhKAdAuW2Wd3SZN5W0afLSdRclX/ibCeaRYDe2zZsRblpKSPVRUROJr7ynsXQ6eQF2EtB0YpdXZRUQlXkcbRUh/kEfnCu2c4ccveMXSwVT+4S7AdKYbOcWxnktHpjBEd21gobk9X+/LLMvgVgrCzY5Jy5yYr5kB8ouxd3UVLRmJDPGizdBUGpkrp2KLQxxzquN2h7iRog/H8MtkTOS0zJ3Mu6RGRhR+x18JR6jEnaCo6ICG3YnKO6AxxdhPHWDaicYDoa80hSHrg7YE+TXYZHGDpgKQkmhOteHmAhXm2ie6mYjmhR3klCmoIzLXK+SsQAaEaJWU8DygUeBfCt3cNtxyux2cwrAq750D066Q1CU4cy2/vmdkA9Ru7tjwUT6p8UvzWeyubgbJnMZSe0gTl3xMqP9CBKAeQVa9iDX4gOFetlfUKGAkz0L3KQkwzgWNWeye6aBpgPpU8WfnHnPwRnng/v1viBvyr22AxdNzXADhaiFAFDWmNYtMKdlDcIlJd3vv9MYxrQF4zTBunw99ZrxowTUmG9MSKn1wXn X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB838; 31:8/cKF1WsfrAzY7QYS4aY7DBCP842r/XvkVMZUkSlunOwVZ20SkXHq74vk68HMAFYnHURzpAuEBr/mB0regDl+rlBaN5qrC5PVGa9CV5u2fxWYzLROXqSjNXYbmT+TfkJlLRWJTOp/qKeNoB4Mc2HY86K76rGPgmwJc/9qJFYltPc36GAjDhv0ITLZIe9cGoKrEszQAhghIuqnhprD0jaZA7bxDyemXNfsYY88jnW9gHXGcyrQsbJYJvw74vjlMRUt2Cs2k6t9HHM0ev8FENzJkg7jY38oDWvfQ+EjyPA19c=; 20:7gmzGbQ23hKakWyN2fSmOlxJZMD5fhHjSHRBVdfxAT2j08WYq4j7bnuc9vmAGr7ESu7MO4nD+B46csPejsccmC2+iX2cMd+LoFsBPoVQyOpLtlqZfer8MzrAnrH15qvl9bHmrK2EJfpL6T3YVTOZJnRRmRihxqwMtFtB2fKUzD13KZgt1b0Sx88dfVwKK1ij8Z/G2mJCie4R78EZ33rYk6BArmTVMmWrvoaEyqAi6IWiMhGY5akn+6pgWsRM53bgIkxlql4PY3EetuOyo9iIRJ2xrihquXCHLh2E2tYCGGqU/Q/g72fLQK0m6s/zZfAs3nJ+XtwswxaJ4mbaV3+nB7MaN9BQCBkZ6O15n74eU1QFCv2TLJBpXN8thiiQe5giy/cj/1ggI/ePJx8p7pd8fKui39qy2nsfxUGcmSwan+Wq4kteQ8le2MhasypEsPYzcJ7pOjvyGYV8IGU2+7n79upXwB9pNttSNmXzuGDz63k9gDvlbvMz92Wdh8rKpGlQ X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(61668805478150); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040375)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6041248)(20161123562025)(20161123555025)(20161123564025)(20161123560025)(6072148); SRVR:CO2PR0501MB838; BCL:0; PCL:0; RULEID:; SRVR:CO2PR0501MB838; X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB838; 4:vPJEr7IgHGANc8nF1EA2Z5TB0K/mAsctoRVa9Ll9Kffv1VDKeXmbEIaCN+3nZS1xYOYV6y5T7JDWNEkf3Aitz42qJKvBwvQQ8gndzAtCrqmFkMIRAOIjFcSDpaTi0ReqjXMCUq8FoA7SXErZ1EdFCHnLjOLLGEzf3y6jDAWGCWCc4oIubbmHThqohSCerluaEzw7kI517E6mTEgtEeqtDA7G6y56iIVOSP7vBOZjpRXxCNuq7SWYlti/VsStdGbGjkyxU2Qo9S1he42dHz/rELMPf52kVfXX6GTPwLDaLpJLRf5yf/NdhqF/zt6o10S7XeIHRUwzyXnv/m30mvoeiMknYtg2018sOtGIdTVDhXZVJ5Frt12weiT673Fbcx3A6zCnm2AtqG2GRRgl2tpdGqXJKZaQ0NX27euJTSMW2hbMg4IWerKeyCu/CNhzeqnLFN/CmrUgJLyPWQ2o8Imk1SSzaetuhl2KekgnTrGCyunUTCBdbVZ59Aju+hl06H/5B1yb1c3ECDZZ9c1poWMaU/rxoMASTe0L+jTVkOWfXevmrmrKK07siV1Jh/sHoMi0qkGrdP/JOwA/9dREFRGFA6SjzUYuyCsu4vRYv96PVjCSWnugsc7fmR/yZd/3ySAv X-Forefront-PRVS: 01834E39B7 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(7916002)(39450400003)(189002)(199003)(7736002)(101416001)(48376002)(50986999)(81166006)(76176999)(50466002)(4001430100002)(5890100001)(42186005)(92566002)(106356001)(6116002)(2906002)(4326007)(105586002)(47776003)(118296001)(66066001)(6666003)(8676002)(33646002)(36756003)(50226002)(3846002)(2950100002)(189998001)(38730400001)(25786008)(68736007)(5003940100001)(97736004)(5660300001)(305945005)(107886002)(86362001)(54906002)(6486002)(81156014)(5001770100001)(7099028); DIR:OUT; SFP:1101; SCL:1; SRVR:CO2PR0501MB838; H:promb-2s-dhcp95-136.eng.vmware.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: vmware.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CO2PR0501MB838; 23:jqYjj4HgRBdmTTtQM39QffXChsD2CXUo4RxztDxZ?= =?us-ascii?Q?uQlONfT600LUTPfjTK7jh3BLRnR4f0riCxKRV0nuwKV8ho2xCMgI2ftoUZ2T?= =?us-ascii?Q?hIB2ptye5UXGgSy0/3uE+A8bq9SPRrZT5ldD+Vs6bHpdbrK8nZyaQilY+mq3?= =?us-ascii?Q?8u/KTKcKSoPmRBFh2Kxig8Hy406dRl1XA/rG2TMpyBPA2UfosvX30uh7CEi1?= =?us-ascii?Q?QSzMVfNR6BG0FLtxr47efN2tIK47rh7+DfTNwQoTls83HKxIWMdkbrBif2nr?= =?us-ascii?Q?snQWM1eUjemBeZf6TfrpR0r7Lji8GUHfmP51ANHwKQd/jIC5shkmn06Yuz/1?= =?us-ascii?Q?ScAAOfKa1cX+rDFIWzaHbOVkgUe6OWro7RlnW7qbO0nGuUNrRk//dRVqtn1K?= =?us-ascii?Q?grCA7OVS+U+LQigEAbZVhRkqC8bG8rx/v1RNbMXQqXXMySsISaKLOw9XQeG7?= =?us-ascii?Q?IuzBSF6az+Gefu9D/xyJ4RppJPXo8b0Xnjh44JXrfkeBGnnUtrKxMEsfTdwx?= =?us-ascii?Q?43bG1v/AGg72F1XuyO5DO8rScAZPEJiMRfJvXC2uosTivdHbDQBu/gLeuYua?= =?us-ascii?Q?i8QBugUQszwJa73JDxUD7C3Cnb/PQ/xRQK/yFqT+/M5qDYE5USEFDxZmV/qB?= =?us-ascii?Q?o9CuL8kNst7o8baWc/OEB+Xc9jc321Mfu+YiMu9pSigmu8YFj6cbuNKlAHAQ?= =?us-ascii?Q?gdzXUGlNy3MYewLUOFV+XV7HKYrlJlI1batgSSR7J28vcW3Tg0s6QESBx3Xx?= =?us-ascii?Q?RyKFi1TeDZLYQOwJ/2P2lX9FC9q3QI4vW65dpG10FWhdO8gP/XOsBw5mTExu?= =?us-ascii?Q?N54zFc3kTFaEQwjyF45uwX4uB+4/L1cvdjnKo4dCS+xs6IgNfDhDpry7tdjA?= =?us-ascii?Q?zfhgHG949xi//5Rfznveh179QNn2k5r28v5aaHQk8lK7tWUExN8fLwbYoznJ?= =?us-ascii?Q?2wqnjuOfVNoyR93YEBb6LvCvJzWJ/ymVlNbMaL8TVayPXkbnl26CchEZhr34?= =?us-ascii?Q?Ewkvn+mrr6cpYr7vCPv10whA7xKEHuxraOqlGXVojhND6WJcK0qKuqipWtrr?= =?us-ascii?Q?cgG0winzqagBZvvNCYODrwo7kfATMTafgdlGBNqgKvnPreXL1EYjC7u5ZLb6?= =?us-ascii?Q?kATeXy6OjkR/koZY7yWXe9pwwzxBMY0x9zAJy2a43mF9Bw8XvMHYpg=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB838; 6:esQKKpiSt8okRGZRN7P3+X0s75fEzWxIGjJiXhRPK0gIDDiYQVQV4C0HKvqLQmd6fixDBDaEPxxi7tvZ6ApFVtECLsqeN5Yv8hK/qzunY+OPhN/0NcIkRFJ4DOHwJkBgWmA+0L8v+GGTU8f5yylTxXL4kBMjf3VxzWpfq2ihGcLxMO4r7pXD5Cm2vTnAf99Srj7SWhyQgTr7CDahJ0AHFq+krHLt442b7mv2BrOjleB8gxBza8eeFkQ+NbHRe58jnSM3iRL/IwGn4Zsr9iz9OjB5U8tOwHVj5YsMMS0WaEVYmiIesgcK8tcg/OvQL7VqS0XmC0xrMMgsXdlUefYgeEsMIBfb2tBDorgIKkRUHlUhth00ZRUMwZxVW9TUQIGG9vfIQilXqGHlrf2WumVS2/WxhKbifAEu2/b4QVJHYxQ=; 5:2IX+m57/0O9z/uyk/Ji+J7bf48Az/7pajvf+AgvWbBAb9YIjSC7ZLI7VWZGRQYcvKtTV15t/XrDOKxnpPxjK0Y8wxNHtk8KjnO6gJ9rlmJZfOT10MEChpcbNzvhEiQtQueHi9k/D4MuetvZLgDGoXWyXKxGhDLyh2xj2PP/gPww=; 24:GmIQ8o8SCdVBixf+0L5aXHSHH2Ojga7cjZPquQTzr0W2kJ6ykD9vPPvta03D7Rhptd1eKUGj2Feig/FwEzS5Y1i5lT47RLpap2ReFIWVJ2c= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB838; 7:n9UBKiNj12y7bM/dYgn5pjgHtu6p6MWUrfLI0yAThizzKsOK0ZbbbX4agcx4JQc/WdioHPzxmmHURh1MNckgJmsjvvRPu4pbYMr6uB4DKWvUvJwWq7mD73hTKcH0/KP7p7CeimPa9mep3uQV4KHxR2HqH0G1YpIp49GUp1IaP0YgeRxI3A/RuC81x5P0Hv6w252d2JzmlYtn6V758sDCPb4YcHPTcjyHxZ0QLI+gNinIb3Mq5QFITBnbEMFaR0KQNe/iPF/VulFKM3tH5yTRPQSB95WspdFLvRPDJ3iuy8VT6Btl/gwfI5xDi2kDx9QDd9ym7r78iQS/VMo0JM62O6S+L6PugB8WIfpOWnwY5Gsh/XErwrAR9oTow2+cz3L2YBW73quzTNVn7uMhmpdgx7Re6Oj8FEtWl/N8w8VPW1BmwWYkhDvHZrRMSDREo01nlRRVL3hL4tkkyO9EZUZDgw==; 20:ehSsVVQMluwjsHmlN0ZNF67vHSn5Rd1vTSqpDtdtwVZhYOUIR2ZYkec2HOdBZ1z9GEg47xhCciz5ZDudbK0c44ZxOB7YJc8c4b+Are1AkzfSFt2AQqOWMpA5xhslMB9RxKbi+X4t2eRfi7uPmij4BnkMvxBn7+PKM97N9qCyHo4= X-OriginatorOrg: vmware.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Jan 2017 19:16:02.3820 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO2PR0501MB838 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Aditya Sarwade Currently on bootup, ethernet drivers seem to load before RDMA ones in Linux. So while the vmxnet3 module is loaded before vmw_pvrdma, the vmxnet3 link state may not necessarily be enabled by the stack before pvrdma is loaded. This is a problem because if the pvrdma module is loaded on bootup (by installing it in /lib/modules/*), the pvrdma device comes up in a port down state. Since this is the most common use case scenario, defer the activation of the device till the paired vmxnet3 link actually comes up. One downside of doing this is, if a user doesn't have the vmxnet3 link up when the pvrdma driver is loaded, they may not see any output for ibv_devinfo until the paired vmxnet3 link is enabled too. The users somehow need to be aware of this. This only changes how the device is activated the first time. Once enabled if the link goes down, a pvrdma driver reload is still required after link up. Fixes: 29c8d9eba550 ("IB: Add vmw_pvrdma driver") Signed-off-by: Aditya Sarwade Reviewed-by: Bryan Tan Signed-off-by: Adit Ranadive Reviewed-by: Yuval Shaia --- drivers/infiniband/hw/vmw_pvrdma/pvrdma.h | 1 + drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c | 99 +++++++++++++++++--------- 2 files changed, 65 insertions(+), 35 deletions(-) diff --git a/drivers/infiniband/hw/vmw_pvrdma/pvrdma.h b/drivers/infiniband/hw/vmw_pvrdma/pvrdma.h index 71e1d55..540a54b 100644 --- a/drivers/infiniband/hw/vmw_pvrdma/pvrdma.h +++ b/drivers/infiniband/hw/vmw_pvrdma/pvrdma.h @@ -221,6 +221,7 @@ struct pvrdma_dev { u32 port_cap_mask; struct mutex port_mutex; /* Port modification mutex. */ bool ib_active; + bool enabled; atomic_t num_qps; atomic_t num_cqs; atomic_t num_pds; diff --git a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c index 231a1ce..b57132f 100644 --- a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c +++ b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c @@ -74,6 +74,8 @@ static int pvrdma_del_gid(struct ib_device *ibdev, void **context); +static int pvrdma_enable_dev(struct pvrdma_dev *dev); + static ssize_t show_hca(struct device *device, struct device_attribute *attr, char *buf) { @@ -755,6 +757,10 @@ static void pvrdma_netdevice_event_handle(struct pvrdma_dev *dev, pvrdma_dispatch_event(dev, 1, IB_EVENT_PORT_ERR); break; case NETDEV_UP: + if (!dev->enabled && pvrdma_enable_dev(dev)) { + dev_err(&dev->pdev->dev, "failed to enable device\n"); + break; + } pvrdma_dispatch_event(dev, 1, IB_EVENT_PORT_ACTIVE); break; default: @@ -801,6 +807,48 @@ static int pvrdma_netdevice_event(struct notifier_block *this, return NOTIFY_DONE; } +static void pvrdma_disable_dev(struct pvrdma_dev *dev) +{ + if (dev->enabled) { + ib_unregister_device(&dev->ib_dev); + dev->enabled = false; + } +} + +static int pvrdma_enable_dev(struct pvrdma_dev *dev) +{ + int ret; + pvrdma_enable_intrs(dev); + + /* Activate pvrdma device */ + pvrdma_write_reg(dev, PVRDMA_REG_CTL, PVRDMA_DEVICE_CTL_ACTIVATE); + + /* Make sure the write is complete before reading status. */ + mb(); + + /* Check if device was successfully activated */ + ret = pvrdma_read_reg(dev, PVRDMA_REG_ERR); + if (ret) { + dev_err(&dev->pdev->dev, "failed to activate device\n"); + ret = -EFAULT; + goto err_disable_intrs; + } + + /* Register IB device */ + ret = pvrdma_register_device(dev); + if (ret) { + dev_err(&dev->pdev->dev, "failed to register IB device\n"); + goto err_disable_intrs; + } + + dev->enabled = true; + return 0; + +err_disable_intrs: + pvrdma_disable_intrs(dev); + return ret; +} + static int pvrdma_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) { @@ -867,14 +915,14 @@ static int pvrdma_pci_probe(struct pci_dev *pdev, /* Enable 64-Bit DMA */ if (pci_set_dma_mask(pdev, DMA_BIT_MASK(64)) == 0) { ret = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64)); - if (ret != 0) { + if (ret) { dev_err(&pdev->dev, "pci_set_consistent_dma_mask failed\n"); goto err_free_resource; } } else { ret = pci_set_dma_mask(pdev, DMA_BIT_MASK(32)); - if (ret != 0) { + if (ret) { dev_err(&pdev->dev, "pci_set_dma_mask failed\n"); goto err_free_resource; @@ -1029,7 +1077,7 @@ static int pvrdma_pci_probe(struct pci_dev *pdev, if (ret) { dev_err(&pdev->dev, "failed to allocate interrupts\n"); ret = -ENOMEM; - goto err_netdevice; + goto err_free_cq_ring; } /* Allocate UAR table. */ @@ -1049,51 +1097,35 @@ static int pvrdma_pci_probe(struct pci_dev *pdev, } dev_dbg(&pdev->dev, "gid table len %d\n", dev->dsr->caps.gid_tbl_len); - pvrdma_enable_intrs(dev); - - /* Activate pvrdma device */ - pvrdma_write_reg(dev, PVRDMA_REG_CTL, PVRDMA_DEVICE_CTL_ACTIVATE); - - /* Make sure the write is complete before reading status. */ - mb(); - - /* Check if device was successfully activated */ - ret = pvrdma_read_reg(dev, PVRDMA_REG_ERR); - if (ret != 0) { - dev_err(&pdev->dev, "failed to activate device\n"); - ret = -EFAULT; - goto err_disable_intr; - } - - /* Register IB device */ - ret = pvrdma_register_device(dev); - if (ret) { - dev_err(&pdev->dev, "failed to register IB device\n"); - goto err_disable_intr; + if (netif_running(dev->netdev) && netif_carrier_ok(dev->netdev)) { + ret = pvrdma_enable_dev(dev); + if (ret) { + dev_err(&pdev->dev, "failed to enable device\n"); + goto err_free_sgid_tbl; + } + } else { + dev_info(&pdev->dev, "pvrdma netdev link is down\n"); } dev->nb_netdev.notifier_call = pvrdma_netdevice_event; ret = register_netdevice_notifier(&dev->nb_netdev); if (ret) { dev_err(&pdev->dev, "failed to register netdevice events\n"); - goto err_unreg_ibdev; + goto err_disable_dev; } dev_info(&pdev->dev, "attached to device\n"); return 0; -err_unreg_ibdev: - ib_unregister_device(&dev->ib_dev); -err_disable_intr: - pvrdma_disable_intrs(dev); +err_disable_dev: + pvrdma_disable_dev(dev); +err_free_sgid_tbl: kfree(dev->sgid_tbl); err_free_uar_table: pvrdma_uar_table_cleanup(dev); err_free_intrs: pvrdma_free_irq(dev); pvrdma_disable_msi_all(dev); -err_netdevice: - unregister_netdevice_notifier(&dev->nb_netdev); err_free_cq_ring: pvrdma_page_dir_cleanup(dev, &dev->cq_pdir); err_free_async_ring: @@ -1132,10 +1164,7 @@ static void pvrdma_pci_remove(struct pci_dev *pdev) unregister_netdevice_notifier(&dev->nb_netdev); dev->nb_netdev.notifier_call = NULL; - flush_workqueue(event_wq); - - /* Unregister ib device */ - ib_unregister_device(&dev->ib_dev); + pvrdma_disable_dev(dev); mutex_lock(&pvrdma_device_list_lock); list_del(&dev->device_link);