From patchwork Thu Jan 17 20:41:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 10768999 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EA5ED13BF for ; Thu, 17 Jan 2019 20:41:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D8162300E9 for ; Thu, 17 Jan 2019 20:41:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CC730300ED; Thu, 17 Jan 2019 20:41:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C0501300E9 for ; Thu, 17 Jan 2019 20:41:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728588AbfAQUlO (ORCPT ); Thu, 17 Jan 2019 15:41:14 -0500 Received: from mga02.intel.com ([134.134.136.20]:20859 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727070AbfAQUlO (ORCPT ); Thu, 17 Jan 2019 15:41:14 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Jan 2019 12:41:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,489,1539673200"; d="scan'208";a="126693037" Received: from scymds01.sc.intel.com ([10.82.194.37]) by FMSMGA003.fm.intel.com with ESMTP; 17 Jan 2019 12:41:12 -0800 Received: from scvm10.sc.intel.com (scvm10.sc.intel.com [10.82.195.27]) by scymds01.sc.intel.com with ESMTP id x0HKfCrC029763; Thu, 17 Jan 2019 12:41:12 -0800 Received: from scvm10.sc.intel.com (localhost [127.0.0.1]) by scvm10.sc.intel.com with ESMTP id x0HKfC2k032221; Thu, 17 Jan 2019 12:41:12 -0800 Subject: [PATCH for-rc 1/7] IB/{hfi1, qib, rdmavt}: Do not depend on IB Verbs name for driver logging From: Dennis Dalessandro To: jgg@ziepe.ca, dledford@redhat.com Cc: linux-rdma@vger.kernel.org, "Michael J. Ruhl" , Mike Marciniszyn Date: Thu, 17 Jan 2019 12:41:12 -0800 Message-ID: <20190117204102.30826.18387.stgit@scvm10.sc.intel.com> In-Reply-To: <20190117203920.30826.76720.stgit@scvm10.sc.intel.com> References: <20190117203920.30826.76720.stgit@scvm10.sc.intel.com> User-Agent: StGit/0.17.1-18-g2e886-dirty MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Michael J. Ruhl Commit 896de0090a85 ("RDMA/core: Use dev_name instead of ibdev->name"), changes the API for setting the IB device name. HFI, QIB, and RDMAVT make assumptions with regards to this API (i.e. as to when the device name string is available). This assumption is no longer valid. Update the drivers to reflect the commit by storing the specific device name in device local information. Create a calldown for the RVT driver to access the drivers name information. Rework the logging macros to use the best device name. Fixes: 11f0e89710eb ("IB/{hfi1, qib}: Fix a concurrency issue with device name in logging") Reviewed-by: Mike Marciniszyn Reviewed-by: Dennis Dalessandro Signed-off-by: Michael J. Ruhl Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/affinity.c | 6 +++-- drivers/infiniband/hw/hfi1/driver.c | 8 +++++++ drivers/infiniband/hw/hfi1/hfi.h | 22 ++++++++----------- drivers/infiniband/hw/hfi1/init.c | 2 +- drivers/infiniband/hw/hfi1/verbs.c | 1 + drivers/infiniband/hw/qib/qib.h | 12 +++++----- drivers/infiniband/hw/qib/qib_driver.c | 8 +++++++ drivers/infiniband/hw/qib/qib_init.c | 10 ++++++-- drivers/infiniband/hw/qib/qib_verbs.c | 1 + drivers/infiniband/sw/rdmavt/trace.h | 6 +++-- drivers/infiniband/sw/rdmavt/vt.c | 2 +- drivers/infiniband/sw/rdmavt/vt.h | 10 ++++++-- include/rdma/rdma_vt.h | 38 ++++++-------------------------- 13 files changed, 64 insertions(+), 62 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/affinity.c b/drivers/infiniband/hw/hfi1/affinity.c index 2baf38c..b85a096 100644 --- a/drivers/infiniband/hw/hfi1/affinity.c +++ b/drivers/infiniband/hw/hfi1/affinity.c @@ -425,7 +425,7 @@ static void _dev_comp_vect_mappings_destroy(struct hfi1_devdata *dd) dd->comp_vect_mappings[i] = -1; hfi1_cdbg(AFFINITY, "[%s] Release CPU %d from completion vector %d", - rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), cpu, i); + kobject_name(&dd->kobj), cpu, i); } kfree(dd->comp_vect_mappings); @@ -475,7 +475,7 @@ static int _dev_comp_vect_mappings_create(struct hfi1_devdata *dd, dd->comp_vect_mappings[i] = cpu; hfi1_cdbg(AFFINITY, "[%s] Completion Vector %d -> CPU %d", - rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), i, cpu); + kobject_name(&dd->kobj), i, cpu); } return 0; @@ -578,7 +578,7 @@ static int _dev_comp_vect_cpu_mask_init(struct hfi1_devdata *dd, hfi1_cdbg(AFFINITY, "[%s] Completion vector affinity CPU set(s) %*pbl", - rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), + kobject_name(&dd->kobj), cpumask_pr_args(dev_comp_vect_mask)); return 0; diff --git a/drivers/infiniband/hw/hfi1/driver.c b/drivers/infiniband/hw/hfi1/driver.c index a8ad707..8aaa4a6 100644 --- a/drivers/infiniband/hw/hfi1/driver.c +++ b/drivers/infiniband/hw/hfi1/driver.c @@ -160,6 +160,14 @@ static int hfi1_caps_get(char *buffer, const struct kernel_param *kp) return scnprintf(buffer, PAGE_SIZE, "0x%lx", cap_mask); } +const char *hfi1_get_device_name(struct rvt_dev_info *rdi) +{ + struct hfi1_ibdev *ibdev = container_of(rdi, struct hfi1_ibdev, rdi); + struct hfi1_devdata *dd = container_of(ibdev, + struct hfi1_devdata, verbs_dev); + return kobject_name(&dd->kobj); +} + struct pci_dev *get_pci_dev(struct rvt_dev_info *rdi) { struct hfi1_ibdev *ibdev = container_of(rdi, struct hfi1_ibdev, rdi); diff --git a/drivers/infiniband/hw/hfi1/hfi.h b/drivers/infiniband/hw/hfi1/hfi.h index 6db2276..dc8ba43 100644 --- a/drivers/infiniband/hw/hfi1/hfi.h +++ b/drivers/infiniband/hw/hfi1/hfi.h @@ -2012,6 +2012,7 @@ int get_platform_config_field(struct hfi1_devdata *dd, table_type, int table_index, int field_index, u32 *data, u32 len); +const char *hfi1_get_device_name(struct rvt_dev_info *rdi); struct pci_dev *get_pci_dev(struct rvt_dev_info *rdi); /* @@ -2148,42 +2149,39 @@ static inline u64 hfi1_pkt_base_sdma_integrity(struct hfi1_devdata *dd) #define dd_dev_emerg(dd, fmt, ...) \ dev_emerg(&(dd)->pcidev->dev, "%s: " fmt, \ - rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), ##__VA_ARGS__) + kobject_name(&(dd)->kobj), ##__VA_ARGS__) #define dd_dev_err(dd, fmt, ...) \ dev_err(&(dd)->pcidev->dev, "%s: " fmt, \ - rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), ##__VA_ARGS__) + kobject_name(&(dd)->kobj), ##__VA_ARGS__) #define dd_dev_err_ratelimited(dd, fmt, ...) \ dev_err_ratelimited(&(dd)->pcidev->dev, "%s: " fmt, \ - rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), \ - ##__VA_ARGS__) + kobject_name(&(dd)->kobj), ##__VA_ARGS__) #define dd_dev_warn(dd, fmt, ...) \ dev_warn(&(dd)->pcidev->dev, "%s: " fmt, \ - rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), ##__VA_ARGS__) + kobject_name(&(dd)->kobj), ##__VA_ARGS__) #define dd_dev_warn_ratelimited(dd, fmt, ...) \ dev_warn_ratelimited(&(dd)->pcidev->dev, "%s: " fmt, \ - rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), \ - ##__VA_ARGS__) + kobject_name(&(dd)->kobj), ##__VA_ARGS__) #define dd_dev_info(dd, fmt, ...) \ dev_info(&(dd)->pcidev->dev, "%s: " fmt, \ - rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), ##__VA_ARGS__) + kobject_name(&(dd)->kobj), ##__VA_ARGS__) #define dd_dev_info_ratelimited(dd, fmt, ...) \ dev_info_ratelimited(&(dd)->pcidev->dev, "%s: " fmt, \ - rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), \ - ##__VA_ARGS__) + kobject_name(&(dd)->kobj), ##__VA_ARGS__) #define dd_dev_dbg(dd, fmt, ...) \ dev_dbg(&(dd)->pcidev->dev, "%s: " fmt, \ - rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), ##__VA_ARGS__) + kobject_name(&(dd)->kobj), ##__VA_ARGS__) #define hfi1_dev_porterr(dd, port, fmt, ...) \ dev_err(&(dd)->pcidev->dev, "%s: port %u: " fmt, \ - rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), (port), ##__VA_ARGS__) + kobject_name(&(dd)->kobj), (port), ##__VA_ARGS__) /* * this is used for formatting hw error messages... diff --git a/drivers/infiniband/hw/hfi1/init.c b/drivers/infiniband/hw/hfi1/init.c index 0904490..044d9a8 100644 --- a/drivers/infiniband/hw/hfi1/init.c +++ b/drivers/infiniband/hw/hfi1/init.c @@ -1313,7 +1313,7 @@ void hfi1_free_devdata(struct hfi1_devdata *dd) "Could not allocate unit ID: error %d\n", -ret); goto bail; } - rvt_set_ibdev_name(&dd->verbs_dev.rdi, "%s_%d", class_name(), dd->unit); + kobject_set_name(&dd->kobj, "%s_%d", class_name(), dd->unit); /* * Initialize all locks for the device. This needs to be as early as diff --git a/drivers/infiniband/hw/hfi1/verbs.c b/drivers/infiniband/hw/hfi1/verbs.c index ec582d8..6546cba 100644 --- a/drivers/infiniband/hw/hfi1/verbs.c +++ b/drivers/infiniband/hw/hfi1/verbs.c @@ -1680,6 +1680,7 @@ int hfi1_register_ib_device(struct hfi1_devdata *dd) * Fill in rvt info object. */ dd->verbs_dev.rdi.driver_f.port_callback = hfi1_create_port_files; + dd->verbs_dev.rdi.driver_f.get_device_name = hfi1_get_device_name; dd->verbs_dev.rdi.driver_f.get_pci_dev = get_pci_dev; dd->verbs_dev.rdi.driver_f.check_ah = hfi1_check_ah; dd->verbs_dev.rdi.driver_f.notify_new_ah = hfi1_notify_new_ah; diff --git a/drivers/infiniband/hw/qib/qib.h b/drivers/infiniband/hw/qib/qib.h index 83d2349..3681701 100644 --- a/drivers/infiniband/hw/qib/qib.h +++ b/drivers/infiniband/hw/qib/qib.h @@ -1075,6 +1075,8 @@ struct qib_devdata { struct tasklet_struct error_tasklet; int assigned_node_id; /* NUMA node closest to HCA */ + /* device name */ + char *name; }; /* hol_state values */ @@ -1425,6 +1427,7 @@ int qib_pcie_ddinit(struct qib_devdata *, struct pci_dev *, * dma_addr wrappers - all 0's invalid for hw */ int qib_map_page(struct pci_dev *d, struct page *p, dma_addr_t *daddr); +const char *qib_get_device_name(struct rvt_dev_info *rdi); struct pci_dev *qib_get_pci_dev(struct rvt_dev_info *rdi); /* @@ -1482,17 +1485,14 @@ static inline void qib_flush_wc(void) dev_err(dev, fmt, ##__VA_ARGS__) #define qib_dev_err(dd, fmt, ...) \ - dev_err(&(dd)->pcidev->dev, "%s: " fmt, \ - rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), ##__VA_ARGS__) + dev_err(&(dd)->pcidev->dev, "%s: " fmt, (dd)->name, ##__VA_ARGS__) #define qib_dev_warn(dd, fmt, ...) \ - dev_warn(&(dd)->pcidev->dev, "%s: " fmt, \ - rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), ##__VA_ARGS__) + dev_warn(&(dd)->pcidev->dev, "%s: " fmt, (dd)->name, ##__VA_ARGS__) #define qib_dev_porterr(dd, port, fmt, ...) \ dev_err(&(dd)->pcidev->dev, "%s: IB%u:%u " fmt, \ - rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), (dd)->unit, (port), \ - ##__VA_ARGS__) + (dd)->name, (dd)->unit, (port), ##__VA_ARGS__) #define qib_devinfo(pcidev, fmt, ...) \ dev_info(&(pcidev)->dev, fmt, ##__VA_ARGS__) diff --git a/drivers/infiniband/hw/qib/qib_driver.c b/drivers/infiniband/hw/qib/qib_driver.c index 3117cc5..f66d7e3 100644 --- a/drivers/infiniband/hw/qib/qib_driver.c +++ b/drivers/infiniband/hw/qib/qib_driver.c @@ -81,6 +81,14 @@ struct qlogic_ib_stats qib_stats; +const char *qib_get_device_name(struct rvt_dev_info *rdi) +{ + struct qib_ibdev *ibdev = container_of(rdi, struct qib_ibdev, rdi); + struct qib_devdata *dd = container_of(ibdev, + struct qib_devdata, verbs_dev); + return dd->name; +} + struct pci_dev *qib_get_pci_dev(struct rvt_dev_info *rdi) { struct qib_ibdev *ibdev = container_of(rdi, struct qib_ibdev, rdi); diff --git a/drivers/infiniband/hw/qib/qib_init.c b/drivers/infiniband/hw/qib/qib_init.c index 9fd6990..9896eae 100644 --- a/drivers/infiniband/hw/qib/qib_init.c +++ b/drivers/infiniband/hw/qib/qib_init.c @@ -1055,6 +1055,7 @@ void qib_free_devdata(struct qib_devdata *dd) qib_dbg_ibdev_exit(&dd->verbs_dev); #endif free_percpu(dd->int_counter); + kfree(dd->name); rvt_dealloc_device(&dd->verbs_dev.rdi); } @@ -1122,13 +1123,16 @@ struct qib_devdata *qib_alloc_devdata(struct pci_dev *pdev, size_t extra) "Could not allocate unit ID: error %d\n", -ret); goto bail; } - rvt_set_ibdev_name(&dd->verbs_dev.rdi, "%s%d", "qib", dd->unit); + dd->name = kasprintf(GFP_KERNEL, "qib%d", dd->unit); + if (!dd->name) { + qib_early_err(&pdev->dev, "Failed to allocate device name\n"); + goto bail; + } dd->int_counter = alloc_percpu(u64); if (!dd->int_counter) { ret = -ENOMEM; - qib_early_err(&pdev->dev, - "Could not allocate per-cpu int_counter\n"); + qib_dev_err(dd, "Could not allocate per-cpu int_counter\n"); goto bail; } diff --git a/drivers/infiniband/hw/qib/qib_verbs.c b/drivers/infiniband/hw/qib/qib_verbs.c index 276304f..b116c6d 100644 --- a/drivers/infiniband/hw/qib/qib_verbs.c +++ b/drivers/infiniband/hw/qib/qib_verbs.c @@ -1568,6 +1568,7 @@ int qib_register_ib_device(struct qib_devdata *dd) * Fill in rvt info object. */ dd->verbs_dev.rdi.driver_f.port_callback = qib_create_port_files; + dd->verbs_dev.rdi.driver_f.get_device_name = qib_get_device_name; dd->verbs_dev.rdi.driver_f.get_pci_dev = qib_get_pci_dev; dd->verbs_dev.rdi.driver_f.check_ah = qib_check_ah; dd->verbs_dev.rdi.driver_f.setup_wqe = qib_check_send_wqe; diff --git a/drivers/infiniband/sw/rdmavt/trace.h b/drivers/infiniband/sw/rdmavt/trace.h index 36ddbd2..74eac77 100644 --- a/drivers/infiniband/sw/rdmavt/trace.h +++ b/drivers/infiniband/sw/rdmavt/trace.h @@ -45,8 +45,10 @@ * */ -#define RDI_DEV_ENTRY(rdi) __string(dev, rvt_get_ibdev_name(rdi)) -#define RDI_DEV_ASSIGN(rdi) __assign_str(dev, rvt_get_ibdev_name(rdi)) +#include "vt.h" + +#define RDI_DEV_ENTRY(rdi) __string(dev, get_device_name(rdi)) +#define RDI_DEV_ASSIGN(rdi) __assign_str(dev, get_device_name(rdi)) #include "trace_rvt.h" #include "trace_qp.h" diff --git a/drivers/infiniband/sw/rdmavt/vt.c b/drivers/infiniband/sw/rdmavt/vt.c index aef3aa3..91b5c53 100644 --- a/drivers/infiniband/sw/rdmavt/vt.c +++ b/drivers/infiniband/sw/rdmavt/vt.c @@ -644,7 +644,7 @@ int rvt_register_device(struct rvt_dev_info *rdi, u32 driver_id) rdi->ibdev.driver_id = driver_id; /* We are now good to announce we exist */ - ret = ib_register_device(&rdi->ibdev, dev_name(&rdi->ibdev.dev), + ret = ib_register_device(&rdi->ibdev, get_device_name(rdi), rdi->driver_f.port_callback); if (ret) { rvt_pr_err(rdi, "Failed to register driver with ib core.\n"); diff --git a/drivers/infiniband/sw/rdmavt/vt.h b/drivers/infiniband/sw/rdmavt/vt.h index 0675ea6..513dd5f 100644 --- a/drivers/infiniband/sw/rdmavt/vt.h +++ b/drivers/infiniband/sw/rdmavt/vt.h @@ -62,19 +62,19 @@ #define rvt_pr_info(rdi, fmt, ...) \ __rvt_pr_info(rdi->driver_f.get_pci_dev(rdi), \ - rvt_get_ibdev_name(rdi), \ + get_device_name(rdi), \ fmt, \ ##__VA_ARGS__) #define rvt_pr_warn(rdi, fmt, ...) \ __rvt_pr_warn(rdi->driver_f.get_pci_dev(rdi), \ - rvt_get_ibdev_name(rdi), \ + get_device_name(rdi), \ fmt, \ ##__VA_ARGS__) #define rvt_pr_err(rdi, fmt, ...) \ __rvt_pr_err(rdi->driver_f.get_pci_dev(rdi), \ - rvt_get_ibdev_name(rdi), \ + get_device_name(rdi), \ fmt, \ ##__VA_ARGS__) @@ -99,4 +99,8 @@ static inline int ibport_num_to_idx(struct ib_device *ibdev, u8 port_num) return port_index; } +static inline const char *get_device_name(struct rvt_dev_info *rdi) +{ + return rdi->driver_f.get_device_name(rdi); +} #endif /* DEF_RDMAVT_H */ diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h index dd0ed80..c038552 100644 --- a/include/rdma/rdma_vt.h +++ b/include/rdma/rdma_vt.h @@ -254,6 +254,13 @@ struct rvt_driver_provided { int (*port_callback)(struct ib_device *, u8, struct kobject *); /* + * Returns a string to represent the device for which is being + * registered. This is primarily used for error and debug messages on + * the console. + */ + const char * (*get_device_name)(struct rvt_dev_info *rdi); + + /* * Returns a pointer to the undelying hardware's PCI device. This is * used to display information as to what hardware is being referenced * in an output message @@ -452,37 +459,6 @@ struct rvt_dev_info { struct rvt_wss *wss; }; -/** - * rvt_set_ibdev_name - Craft an IB device name from client info - * @rdi: pointer to the client rvt_dev_info structure - * @name: client specific name - * @unit: client specific unit number. - */ -static inline void rvt_set_ibdev_name(struct rvt_dev_info *rdi, - const char *fmt, const char *name, - const int unit) -{ - /* - * FIXME: rvt and its users want to touch the ibdev before - * registration and have things like the name work. We don't have the - * infrastructure in the core to support this directly today, hack it - * to work by setting the name manually here. - */ - dev_set_name(&rdi->ibdev.dev, fmt, name, unit); - strlcpy(rdi->ibdev.name, dev_name(&rdi->ibdev.dev), IB_DEVICE_NAME_MAX); -} - -/** - * rvt_get_ibdev_name - return the IB name - * @rdi: rdmavt device - * - * Return the registered name of the device. - */ -static inline const char *rvt_get_ibdev_name(const struct rvt_dev_info *rdi) -{ - return dev_name(&rdi->ibdev.dev); -} - static inline struct rvt_pd *ibpd_to_rvtpd(struct ib_pd *ibpd) { return container_of(ibpd, struct rvt_pd, ibpd); From patchwork Thu Jan 17 20:41:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 10768997 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9FA211390 for ; Thu, 17 Jan 2019 20:41:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8E959300EA for ; Thu, 17 Jan 2019 20:41:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 82BDA300F1; Thu, 17 Jan 2019 20:41:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 11537300EA for ; Thu, 17 Jan 2019 20:41:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728616AbfAQUlX (ORCPT ); Thu, 17 Jan 2019 15:41:23 -0500 Received: from mga07.intel.com ([134.134.136.100]:1213 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727402AbfAQUlX (ORCPT ); Thu, 17 Jan 2019 15:41:23 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Jan 2019 12:41:23 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,489,1539673200"; d="scan'208";a="139197629" Received: from scymds01.sc.intel.com ([10.82.194.37]) by fmsmga001.fm.intel.com with ESMTP; 17 Jan 2019 12:41:22 -0800 Received: from scvm10.sc.intel.com (scvm10.sc.intel.com [10.82.195.27]) by scymds01.sc.intel.com with ESMTP id x0HKfMAY029805; Thu, 17 Jan 2019 12:41:22 -0800 Received: from scvm10.sc.intel.com (localhost [127.0.0.1]) by scvm10.sc.intel.com with ESMTP id x0HKfMFn032296; Thu, 17 Jan 2019 12:41:22 -0800 Subject: [PATCH for-rc 2/7] IB/hfi1: Use new API to deallocate vnic rdma-netdev in hfi1 driver From: Dennis Dalessandro To: jgg@ziepe.ca, dledford@redhat.com Cc: linux-rdma@vger.kernel.org, "Michael J. Ruhl" , Andrzej Witkowski Date: Thu, 17 Jan 2019 12:41:22 -0800 Message-ID: <20190117204117.30826.2346.stgit@scvm10.sc.intel.com> In-Reply-To: <20190117203920.30826.76720.stgit@scvm10.sc.intel.com> References: <20190117203920.30826.76720.stgit@scvm10.sc.intel.com> User-Agent: StGit/0.17.1-18-g2e886-dirty MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Andrzej Witkowski VNIC uses the old deprecated path for freeing netdev private device information. This will lead to a memory leak (of netdev devices) and possibly incorrect cleanup behavior. Fix by using the new API in the VNIC code path. Fixes: 9f49a5b5c21d ("RDMA/netdev: Use priv_destructor for netdev cleanup") Reviewed-by: Michael J. Ruhl Signed-off-by: Andrzej Witkowski Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/vnic_main.c | 4 ++-- drivers/infiniband/ulp/opa_vnic/opa_vnic_netdev.c | 5 ++--- include/rdma/ib_verbs.h | 7 ------- 3 files changed, 4 insertions(+), 12 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/vnic_main.c b/drivers/infiniband/hw/hfi1/vnic_main.c index a922db5..1d66c43 100644 --- a/drivers/infiniband/hw/hfi1/vnic_main.c +++ b/drivers/infiniband/hw/hfi1/vnic_main.c @@ -789,7 +789,6 @@ static void hfi1_vnic_free_rn(struct net_device *netdev) hfi1_vnic_deinit(vinfo); mutex_destroy(&vinfo->lock); - free_netdev(netdev); } struct net_device *hfi1_vnic_alloc_rn(struct ib_device *device, @@ -826,7 +825,6 @@ struct net_device *hfi1_vnic_alloc_rn(struct ib_device *device, vinfo->num_tx_q = dd->num_sdma; vinfo->num_rx_q = dd->num_vnic_contexts; vinfo->netdev = netdev; - rn->free_rdma_netdev = hfi1_vnic_free_rn; rn->set_id = hfi1_vnic_set_vesw_id; netdev->features = NETIF_F_HIGHDMA | NETIF_F_SG; @@ -834,6 +832,8 @@ struct net_device *hfi1_vnic_alloc_rn(struct ib_device *device, netdev->vlan_features = netdev->features; netdev->watchdog_timeo = msecs_to_jiffies(HFI_TX_TIMEOUT_MS); netdev->netdev_ops = &hfi1_netdev_ops; + netdev->priv_destructor = hfi1_vnic_free_rn; + netdev->needs_free_netdev = true; mutex_init(&vinfo->lock); for (i = 0; i < vinfo->num_rx_q; i++) { diff --git a/drivers/infiniband/ulp/opa_vnic/opa_vnic_netdev.c b/drivers/infiniband/ulp/opa_vnic/opa_vnic_netdev.c index ae70cd1..e21dc73 100644 --- a/drivers/infiniband/ulp/opa_vnic/opa_vnic_netdev.c +++ b/drivers/infiniband/ulp/opa_vnic/opa_vnic_netdev.c @@ -382,7 +382,8 @@ struct opa_vnic_adapter *opa_vnic_add_netdev(struct ib_device *ibdev, mutex_destroy(&adapter->mactbl_lock); kfree(adapter); adapter_err: - rn->free_rdma_netdev(netdev); + netdev->priv_destructor(netdev); + free_netdev(netdev); return ERR_PTR(rc); } @@ -391,7 +392,6 @@ struct opa_vnic_adapter *opa_vnic_add_netdev(struct ib_device *ibdev, void opa_vnic_rem_netdev(struct opa_vnic_adapter *adapter) { struct net_device *netdev = adapter->netdev; - struct rdma_netdev *rn = netdev_priv(netdev); v_info("removing\n"); unregister_netdev(netdev); @@ -399,5 +399,4 @@ void opa_vnic_rem_netdev(struct opa_vnic_adapter *adapter) mutex_destroy(&adapter->lock); mutex_destroy(&adapter->mactbl_lock); kfree(adapter); - rn->free_rdma_netdev(netdev); } diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index a3ceed3..7539030 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -2213,13 +2213,6 @@ struct rdma_netdev { struct ib_device *hca; u8 port_num; - /* - * cleanup function must be specified. - * FIXME: This is only used for OPA_VNIC and that usage should be - * removed too. - */ - void (*free_rdma_netdev)(struct net_device *netdev); - /* control functions */ void (*set_id)(struct net_device *netdev, int id); /* send packet */ From patchwork Thu Jan 17 20:41:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 10769001 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0F16313BF for ; Thu, 17 Jan 2019 20:41:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F1F31300F2 for ; Thu, 17 Jan 2019 20:41:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E6A16300F7; Thu, 17 Jan 2019 20:41:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8D20F300F2 for ; Thu, 17 Jan 2019 20:41:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727070AbfAQUlf (ORCPT ); Thu, 17 Jan 2019 15:41:35 -0500 Received: from mga11.intel.com ([192.55.52.93]:9440 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726826AbfAQUlf (ORCPT ); Thu, 17 Jan 2019 15:41:35 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Jan 2019 12:41:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,489,1539673200"; d="scan'208";a="135558198" Received: from scymds01.sc.intel.com ([10.82.194.37]) by fmsmga002.fm.intel.com with ESMTP; 17 Jan 2019 12:41:33 -0800 Received: from scvm10.sc.intel.com (scvm10.sc.intel.com [10.82.195.27]) by scymds01.sc.intel.com with ESMTP id x0HKfW9E029827; Thu, 17 Jan 2019 12:41:32 -0800 Received: from scvm10.sc.intel.com (localhost [127.0.0.1]) by scvm10.sc.intel.com with ESMTP id x0HKfWh2032361; Thu, 17 Jan 2019 12:41:32 -0800 Subject: [PATCH for-rc 3/7] IB/{hfi1, qib}: Fix WC.byte_len calculation for UD_SEND_WITH_IMM From: Dennis Dalessandro To: jgg@ziepe.ca, dledford@redhat.com Cc: linux-rdma@vger.kernel.org, "Michael J. Ruhl" , Mike Marciniszyn , Brian Welty Date: Thu, 17 Jan 2019 12:41:32 -0800 Message-ID: <20190117204127.30826.96869.stgit@scvm10.sc.intel.com> In-Reply-To: <20190117203920.30826.76720.stgit@scvm10.sc.intel.com> References: <20190117203920.30826.76720.stgit@scvm10.sc.intel.com> User-Agent: StGit/0.17.1-18-g2e886-dirty MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Brian Welty The work completion length for a receiving a UD send with immediate is short by 4 bytes. The UD receive logic incorrectly subtracts 4 bytes for immediate value. These bytes are already included in header length and are used to calculate header/payload split, so the result is these 4 bytes are subtracted twice, once when the header length subtracted from the overall length and once again in the UD opcode specific path. Remove the extra subtraction when handling the opcode. Fixes: 7724105686e7 ("IB/hfi1: add driver files") Reviewed-by: Michael J. Ruhl Signed-off-by: Brian Welty Signed-off-by: Mike Marciniszyn Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/ud.c | 1 - drivers/infiniband/hw/qib/qib_ud.c | 1 - 2 files changed, 0 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/ud.c b/drivers/infiniband/hw/hfi1/ud.c index 88242fe..bf96067 100644 --- a/drivers/infiniband/hw/hfi1/ud.c +++ b/drivers/infiniband/hw/hfi1/ud.c @@ -987,7 +987,6 @@ void hfi1_ud_rcv(struct hfi1_packet *packet) opcode == IB_OPCODE_UD_SEND_ONLY_WITH_IMMEDIATE) { wc.ex.imm_data = packet->ohdr->u.ud.imm_data; wc.wc_flags = IB_WC_WITH_IMM; - tlen -= sizeof(u32); } else if (opcode == IB_OPCODE_UD_SEND_ONLY) { wc.ex.imm_data = 0; wc.wc_flags = 0; diff --git a/drivers/infiniband/hw/qib/qib_ud.c b/drivers/infiniband/hw/qib/qib_ud.c index 868da0e..445ea19 100644 --- a/drivers/infiniband/hw/qib/qib_ud.c +++ b/drivers/infiniband/hw/qib/qib_ud.c @@ -512,7 +512,6 @@ void qib_ud_rcv(struct qib_ibport *ibp, struct ib_header *hdr, opcode == IB_OPCODE_UD_SEND_ONLY_WITH_IMMEDIATE) { wc.ex.imm_data = ohdr->u.ud.imm_data; wc.wc_flags = IB_WC_WITH_IMM; - tlen -= sizeof(u32); } else if (opcode == IB_OPCODE_UD_SEND_ONLY) { wc.ex.imm_data = 0; wc.wc_flags = 0; From patchwork Thu Jan 17 20:41:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 10769003 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3B13D1390 for ; Thu, 17 Jan 2019 20:41:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2A7CE300F2 for ; Thu, 17 Jan 2019 20:41:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1EC3D300F7; Thu, 17 Jan 2019 20:41:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C1A80300F2 for ; Thu, 17 Jan 2019 20:41:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727402AbfAQUlo (ORCPT ); Thu, 17 Jan 2019 15:41:44 -0500 Received: from mga18.intel.com ([134.134.136.126]:56253 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726826AbfAQUlo (ORCPT ); Thu, 17 Jan 2019 15:41:44 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Jan 2019 12:41:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,489,1539673200"; d="scan'208";a="292417428" Received: from scymds01.sc.intel.com ([10.82.194.37]) by orsmga005.jf.intel.com with ESMTP; 17 Jan 2019 12:41:43 -0800 Received: from scvm10.sc.intel.com (scvm10.sc.intel.com [10.82.195.27]) by scymds01.sc.intel.com with ESMTP id x0HKfhh4030636; Thu, 17 Jan 2019 12:41:43 -0800 Received: from scvm10.sc.intel.com (localhost [127.0.0.1]) by scvm10.sc.intel.com with ESMTP id x0HKfhV2032529; Thu, 17 Jan 2019 12:41:43 -0800 Subject: [PATCH for-rc 4/7] IB/rdmavt: Add wc_flags and wc_immdata to cq entry trace From: Dennis Dalessandro To: jgg@ziepe.ca, dledford@redhat.com Cc: linux-rdma@vger.kernel.org, "Michael J. Ruhl" , Mike Marciniszyn Date: Thu, 17 Jan 2019 12:41:43 -0800 Message-ID: <20190117204137.30826.50398.stgit@scvm10.sc.intel.com> In-Reply-To: <20190117203920.30826.76720.stgit@scvm10.sc.intel.com> References: <20190117203920.30826.76720.stgit@scvm10.sc.intel.com> User-Agent: StGit/0.17.1-18-g2e886-dirty MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Mike Marciniszyn These fields were missing from the trace. Add them. Fixes: c6ad9482fcb8 ("IB/rdmavt: Add tracing for cq entry and poll") Reviewed-by: Michael J. Ruhl Signed-off-by: Mike Marciniszyn Signed-off-by: Dennis Dalessandro --- drivers/infiniband/sw/rdmavt/trace_cq.h | 10 ++++++++-- 1 files changed, 8 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/sw/rdmavt/trace_cq.h b/drivers/infiniband/sw/rdmavt/trace_cq.h index df8e1ad..e3c416c 100644 --- a/drivers/infiniband/sw/rdmavt/trace_cq.h +++ b/drivers/infiniband/sw/rdmavt/trace_cq.h @@ -105,7 +105,7 @@ TP_ARGS(cq, attr)); #define CQ_PRN \ -"[%s] idx %u wr_id %llx status %u opcode %u,%s length %u qpn %x" +"[%s] idx %u wr_id %llx status %u opcode %u,%s length %u qpn %x flags %x imm %x" DECLARE_EVENT_CLASS( rvt_cq_entry_template, @@ -119,6 +119,8 @@ __field(u32, qpn) __field(u32, length) __field(u32, idx) + __field(u32, flags) + __field(u32, imm) ), TP_fast_assign( RDI_DEV_ASSIGN(cq->rdi) @@ -128,6 +130,8 @@ __entry->length = wc->byte_len; __entry->qpn = wc->qp->qp_num; __entry->idx = idx; + __entry->flags = wc->wc_flags; + __entry->imm = be32_to_cpu(wc->ex.imm_data); ), TP_printk( CQ_PRN, @@ -137,7 +141,9 @@ __entry->status, __entry->opcode, show_wc_opcode(__entry->opcode), __entry->length, - __entry->qpn + __entry->qpn, + __entry->flags, + __entry->imm ) ); From patchwork Thu Jan 17 20:41:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 10769005 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5093213BF for ; Thu, 17 Jan 2019 20:41:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E02D300F2 for ; Thu, 17 Jan 2019 20:41:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 30DBA300F7; Thu, 17 Jan 2019 20:41:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A6B70300F2 for ; Thu, 17 Jan 2019 20:41:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727776AbfAQUl4 (ORCPT ); Thu, 17 Jan 2019 15:41:56 -0500 Received: from mga14.intel.com ([192.55.52.115]:4384 "EHLO mga14.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726826AbfAQUl4 (ORCPT ); Thu, 17 Jan 2019 15:41:56 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Jan 2019 12:41:54 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,489,1539673200"; d="scan'208";a="109095156" Received: from scymds01.sc.intel.com ([10.82.194.37]) by orsmga006.jf.intel.com with ESMTP; 17 Jan 2019 12:41:54 -0800 Received: from scvm10.sc.intel.com (scvm10.sc.intel.com [10.82.195.27]) by scymds01.sc.intel.com with ESMTP id x0HKfsBA030828; Thu, 17 Jan 2019 12:41:54 -0800 Received: from scvm10.sc.intel.com (localhost [127.0.0.1]) by scvm10.sc.intel.com with ESMTP id x0HKfsfY032594; Thu, 17 Jan 2019 12:41:54 -0800 Subject: [PATCH for-rc 5/7] IB/hfi1: Close race condition on user context disable and close From: Dennis Dalessandro To: jgg@ziepe.ca, dledford@redhat.com Cc: linux-rdma@vger.kernel.org, "Michael J. Ruhl" , Mike Marciniszyn , stable@vger.kernel.org Date: Thu, 17 Jan 2019 12:41:54 -0800 Message-ID: <20190117204148.30826.53799.stgit@scvm10.sc.intel.com> In-Reply-To: <20190117203920.30826.76720.stgit@scvm10.sc.intel.com> References: <20190117203920.30826.76720.stgit@scvm10.sc.intel.com> User-Agent: StGit/0.17.1-18-g2e886-dirty MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Michael J. Ruhl When disabling and removing a receive context, it is possible for an asynchronous event (i.e IRQ) to occur. Because of this there is a race between cleaning up the context, and the context being used by the asynchronous event. cpu 0 (context cleanup) rc->ref_count-- (ref_count == 0) hfi1_rcd_free() cpu 1 (IRQ (with rcd index)) rcd_get_by_index() lock ref_count+++ <-- reference count race (WARNING) return rcd unlock cpu 0 hfi1_free_ctxtdata() <-- incorrect free location lock remove rcd from array unlock free rcd This race will cause the following WARNING trace: WARNING: CPU: 0 PID: 175027 at include/linux/kref.h:52 hfi1_rcd_get_by_index+0x84/0xa0 [hfi1] CPU: 0 PID: 175027 Comm: IMB-MPI1 Kdump: loaded Tainted: G OE ------------ 3.10.0-957.el7.x86_64 #1 Hardware name: Intel Corporation S2600KP/S2600KP, BIOS SE5C610.86B.11.01.0076.C4.111920150602 11/19/2015 Call Trace: dump_stack+0x19/0x1b __warn+0xd8/0x100 warn_slowpath_null+0x1d/0x20 hfi1_rcd_get_by_index+0x84/0xa0 [hfi1] is_rcv_urgent_int+0x24/0x90 [hfi1] general_interrupt+0x1b6/0x210 [hfi1] __handle_irq_event_percpu+0x44/0x1c0 handle_irq_event_percpu+0x32/0x80 handle_irq_event+0x3c/0x60 handle_edge_irq+0x7f/0x150 handle_irq+0xe4/0x1a0 do_IRQ+0x4d/0xf0 common_interrupt+0x162/0x162 The race can also lead to a use after free which could be similar to: general protection fault: 0000 1 SMP CPU: 71 PID: 177147 Comm: IMB-MPI1 Kdump: loaded Tainted: G W OE ------------ 3.10.0-957.el7.x86_64 #1 Hardware name: Intel Corporation S2600KP/S2600KP, BIOS SE5C610.86B.11.01.0076.C4.111920150602 11/19/2015 task: ffff9962a8098000 ti: ffff99717a508000 task.ti: ffff99717a508000 __kmalloc+0x94/0x230 Call Trace: ? hfi1_user_sdma_process_request+0x9c8/0x1250 [hfi1] hfi1_user_sdma_process_request+0x9c8/0x1250 [hfi1] hfi1_aio_write+0xba/0x110 [hfi1] do_sync_readv_writev+0x7b/0xd0 do_readv_writev+0xce/0x260 ? handle_mm_fault+0x39d/0x9b0 ? pick_next_task_fair+0x5f/0x1b0 ? sched_clock_cpu+0x85/0xc0 ? __schedule+0x13a/0x890 vfs_writev+0x35/0x60 SyS_writev+0x7f/0x110 system_call_fastpath+0x22/0x27 Introduce a context enable flag to indicate if the context is enabled. Reorder context cleanup to ensure context removal before cleanup occurs correctly. Cc: stable@vger.kernel.org # v4.14.0+ Fixes: f683c80ca68e ("IB/hfi1: Resolve kernel panics by reference counting receive contexts") Reviewed-by: Mike Marciniszyn Signed-off-by: Michael J. Ruhl Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/file_ops.c | 2 ++ drivers/infiniband/hw/hfi1/hfi.h | 3 ++- drivers/infiniband/hw/hfi1/init.c | 8 +++++--- 3 files changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/file_ops.c b/drivers/infiniband/hw/hfi1/file_ops.c index c22ebc7..0ba0cf5 100644 --- a/drivers/infiniband/hw/hfi1/file_ops.c +++ b/drivers/infiniband/hw/hfi1/file_ops.c @@ -671,6 +671,8 @@ static int hfi1_file_close(struct inode *inode, struct file *fp) } spin_unlock_irqrestore(&dd->uctxt_lock, flags); + uctxt->del_pend = 1; + /* * Disable receive context and interrupt available, reset all * RcvCtxtCtrl bits to default values. diff --git a/drivers/infiniband/hw/hfi1/hfi.h b/drivers/infiniband/hw/hfi1/hfi.h index dc8ba43..ea99768 100644 --- a/drivers/infiniband/hw/hfi1/hfi.h +++ b/drivers/infiniband/hw/hfi1/hfi.h @@ -284,7 +284,8 @@ struct hfi1_ctxtdata { u16 expected_base; /* Device context index */ u8 ctxt; - + /* ctxt deletion is pending */ + u8 del_pend; /* PSM Specific fields */ /* lock protecting all Expected TID data */ struct mutex exp_mutex; diff --git a/drivers/infiniband/hw/hfi1/init.c b/drivers/infiniband/hw/hfi1/init.c index 044d9a8..d2ac3ef 100644 --- a/drivers/infiniband/hw/hfi1/init.c +++ b/drivers/infiniband/hw/hfi1/init.c @@ -215,12 +215,11 @@ static void hfi1_rcd_free(struct kref *kref) struct hfi1_ctxtdata *rcd = container_of(kref, struct hfi1_ctxtdata, kref); - hfi1_free_ctxtdata(rcd->dd, rcd); - spin_lock_irqsave(&rcd->dd->uctxt_lock, flags); rcd->dd->rcd[rcd->ctxt] = NULL; spin_unlock_irqrestore(&rcd->dd->uctxt_lock, flags); + hfi1_free_ctxtdata(rcd->dd, rcd); kfree(rcd); } @@ -326,7 +325,10 @@ struct hfi1_ctxtdata *hfi1_rcd_get_by_index(struct hfi1_devdata *dd, u16 ctxt) spin_lock_irqsave(&dd->uctxt_lock, flags); if (dd->rcd[ctxt]) { rcd = dd->rcd[ctxt]; - hfi1_rcd_get(rcd); + if (rcd->del_pend) + rcd = NULL; + else + hfi1_rcd_get(rcd); } spin_unlock_irqrestore(&dd->uctxt_lock, flags); From patchwork Thu Jan 17 20:42:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 10769007 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 25EE613BF for ; Thu, 17 Jan 2019 20:42:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 15844300F2 for ; Thu, 17 Jan 2019 20:42:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0961C300F9; Thu, 17 Jan 2019 20:42:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 42BB2300F2 for ; Thu, 17 Jan 2019 20:42:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727402AbfAQUmF (ORCPT ); Thu, 17 Jan 2019 15:42:05 -0500 Received: from mga11.intel.com ([192.55.52.93]:9490 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726159AbfAQUmF (ORCPT ); Thu, 17 Jan 2019 15:42:05 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Jan 2019 12:42:05 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,489,1539673200"; d="scan'208";a="136687637" Received: from scymds01.sc.intel.com ([10.82.194.37]) by fmsmga004.fm.intel.com with ESMTP; 17 Jan 2019 12:42:05 -0800 Received: from scvm10.sc.intel.com (scvm10.sc.intel.com [10.82.195.27]) by scymds01.sc.intel.com with ESMTP id x0HKg48B030855; Thu, 17 Jan 2019 12:42:04 -0800 Received: from scvm10.sc.intel.com (localhost [127.0.0.1]) by scvm10.sc.intel.com with ESMTP id x0HKg4a6032663; Thu, 17 Jan 2019 12:42:04 -0800 Subject: [PATCH for-rc 6/7] IB/hfi1: Remove overly conservative VM_EXEC flag check From: Dennis Dalessandro To: jgg@ziepe.ca, dledford@redhat.com Cc: linux-rdma@vger.kernel.org, "Michael J. Ruhl" , Mike Marciniszyn , Ira Weiny , stable@vger.kernel.org Date: Thu, 17 Jan 2019 12:42:04 -0800 Message-ID: <20190117204159.30826.92063.stgit@scvm10.sc.intel.com> In-Reply-To: <20190117203920.30826.76720.stgit@scvm10.sc.intel.com> References: <20190117203920.30826.76720.stgit@scvm10.sc.intel.com> User-Agent: StGit/0.17.1-18-g2e886-dirty MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Michael J. Ruhl Applications that use the stack for execution purposes cause PSM jobs to fail during mmap(). Both Fortran (non-standard format parsing) and C (callback functions located in the stack) applications can be written such that stack execution is required. Because of this the EXECSTACK bit can be automatically set at link time for any application. On application load, the ELF loader evaluates the EXECSTACK bit for the application and it's linked libraries. It will set the process VM flags to allow the stack to include the VM_EXEC bit if the EXECSTACK bit is set. This flag is propagated to the driver during the mmap() call in the vma flag bits. Checking for this bit and failing the request with EPERM is overly conservative and will break any PSM application that has the bit set. Remove the VM_EXEC flag from the check. Cc: #v4.14+ Fixes: 12220267645c ("IB/hfi: Protect against writable mmap") Reviewed-by: Mike Marciniszyn Reviewed-by: Dennis Dalessandro Reviewed-by: Ira Weiny Signed-off-by: Michael J. Ruhl Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/file_ops.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/file_ops.c b/drivers/infiniband/hw/hfi1/file_ops.c index 0ba0cf5..68c397a 100644 --- a/drivers/infiniband/hw/hfi1/file_ops.c +++ b/drivers/infiniband/hw/hfi1/file_ops.c @@ -488,7 +488,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma) vmf = 1; break; case STATUS: - if (flags & (unsigned long)(VM_WRITE | VM_EXEC)) { + if (flags & VM_WRITE) { ret = -EPERM; goto done; } From patchwork Thu Jan 17 20:42:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 10769009 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A41D31390 for ; Thu, 17 Jan 2019 20:42:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 93B95300F2 for ; Thu, 17 Jan 2019 20:42:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 87CC8300F7; Thu, 17 Jan 2019 20:42:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 273FF300F2 for ; Thu, 17 Jan 2019 20:42:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727776AbfAQUmR (ORCPT ); Thu, 17 Jan 2019 15:42:17 -0500 Received: from mga12.intel.com ([192.55.52.136]:53031 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726159AbfAQUmR (ORCPT ); Thu, 17 Jan 2019 15:42:17 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Jan 2019 12:42:17 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,489,1539673200"; d="scan'208";a="109095213" Received: from scymds01.sc.intel.com ([10.82.194.37]) by orsmga006.jf.intel.com with ESMTP; 17 Jan 2019 12:42:16 -0800 Received: from scvm10.sc.intel.com (scvm10.sc.intel.com [10.82.195.27]) by scymds01.sc.intel.com with ESMTP id x0HKgGUI030901; Thu, 17 Jan 2019 12:42:16 -0800 Received: from scvm10.sc.intel.com (localhost [127.0.0.1]) by scvm10.sc.intel.com with ESMTP id x0HKgGoV032755; Thu, 17 Jan 2019 12:42:16 -0800 Subject: [PATCH for-rc 7/7] IB/hfi1: Add limit test for RC/UC send via loopback From: Dennis Dalessandro To: jgg@ziepe.ca, dledford@redhat.com Cc: linux-rdma@vger.kernel.org, "Michael J. Ruhl" , Mike Marciniszyn , stable@vger.kernel.org Date: Thu, 17 Jan 2019 12:42:16 -0800 Message-ID: <20190117204209.30826.62040.stgit@scvm10.sc.intel.com> In-Reply-To: <20190117203920.30826.76720.stgit@scvm10.sc.intel.com> References: <20190117203920.30826.76720.stgit@scvm10.sc.intel.com> User-Agent: StGit/0.17.1-18-g2e886-dirty MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Mike Marciniszyn Fix potential memory corruption and panic in loopback for IB_WR_SEND variants. The code blindly assumes the posted length will fit in the fetched rwqe, which is not a valid assumption. Fix by adding a limit test, and triggering the appropriate send completion and putting the QP in an error state. This mimics the handling for non-loopback QPs. Fixes: 15703461533a ("IB/{hfi1, qib, rdmavt}: Move ruc_loopback to rdmavt") Cc: #v4.20+ Reviewed-by: Michael J. Ruhl Signed-off-by: Mike Marciniszyn Signed-off-by: Dennis Dalessandro --- drivers/infiniband/sw/rdmavt/qp.c | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/sw/rdmavt/qp.c b/drivers/infiniband/sw/rdmavt/qp.c index a1bd8cf..c6cc3e4 100644 --- a/drivers/infiniband/sw/rdmavt/qp.c +++ b/drivers/infiniband/sw/rdmavt/qp.c @@ -2910,6 +2910,8 @@ void rvt_ruc_loopback(struct rvt_qp *sqp) goto op_err; if (!ret) goto rnr_nak; + if (wqe->length > qp->r_len) + goto inv_err; break; case IB_WR_RDMA_WRITE_WITH_IMM: @@ -3078,7 +3080,10 @@ void rvt_ruc_loopback(struct rvt_qp *sqp) goto err; inv_err: - send_status = IB_WC_REM_INV_REQ_ERR; + send_status = + sqp->ibqp.qp_type == IB_QPT_RC ? + IB_WC_REM_INV_REQ_ERR : + IB_WC_SUCCESS; wc.status = IB_WC_LOC_QP_OP_ERR; goto err;