From patchwork Tue Jun 25 20:57:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 11016409 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A56E376 for ; Tue, 25 Jun 2019 20:58:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 93BAD205FD for ; Tue, 25 Jun 2019 20:58:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8792E28498; Tue, 25 Jun 2019 20:58:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EEC4A205FD for ; Tue, 25 Jun 2019 20:58:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726476AbfFYU62 (ORCPT ); Tue, 25 Jun 2019 16:58:28 -0400 Received: from mail-eopbgr140054.outbound.protection.outlook.com ([40.107.14.54]:20873 "EHLO EUR01-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726486AbfFYU61 (ORCPT ); Tue, 25 Jun 2019 16:58:27 -0400 ARC-Seal: i=1; a=rsa-sha256; s=testarcselector01; d=microsoft.com; cv=none; b=bkJM41qwONOPFnHRU13H72pbBGs5ix4Leosj7Jv2iPbYpu1c8ss+PuOqYjL9a95RhuAqv9Onxfeveasg7O3VBG2hRF1MGnHPLWZGIbunbPVgjuDsBniV+kbkzUHOayIF7RLl7MCpmlPDBuTJu0ivE3tZrHyrCz6pCFOvfRRB8F0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=testarcselector01; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tj9EptZI4FC4/HROQQpXLF65m0A2wIEtuOg8wp25WIg=; b=BqzBmkhTkvEByAhRTygNjhsBs774qF+bMQMStAa63GSGaavaDwb0ltkp7VOiYVBTKuCrt3npCKRd93PwYM3PJlfshB5p3S+yFYcsia3qUr8Bz+g+aeK8Reir0tMDfrbOgINjfxiKk97eCKgKemc1hAOSo1ikc6frpLJUpC2By54= ARC-Authentication-Results: i=1; test.office365.com 1;spf=none;dmarc=none;dkim=none;arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tj9EptZI4FC4/HROQQpXLF65m0A2wIEtuOg8wp25WIg=; b=AJLsPj4Po84WuFcVTc0y1LD/OdDCJx16LXwpZiNdSGtuBupNj43laVB1yxD2GaKPXBk/ehq4l3iUHxOAI+EQ13tx3Om3ny3jVLe8HGe07J302wy9Jk9goSie3VkfWSeeQry7YyD/uYO2sOfiWmrgBqL+mSzrPYrBjZeaX+jQRWE= Received: from DB6PR0501MB2759.eurprd05.prod.outlook.com (10.172.227.7) by DB6PR0501MB2696.eurprd05.prod.outlook.com (10.172.225.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2008.16; Tue, 25 Jun 2019 20:57:51 +0000 Received: from DB6PR0501MB2759.eurprd05.prod.outlook.com ([fe80::a901:6951:59de:3278]) by DB6PR0501MB2759.eurprd05.prod.outlook.com ([fe80::a901:6951:59de:3278%2]) with mapi id 15.20.2008.014; Tue, 25 Jun 2019 20:57:51 +0000 From: Saeed Mahameed To: "David S. Miller" , Doug Ledford , Jason Gunthorpe CC: Leon Romanovsky , Or Gerlitz , Sagi Grimberg , Tal Gilboa , "netdev@vger.kernel.org" , "linux-rdma@vger.kernel.org" , Yamin Friedman , Saeed Mahameed Subject: [for-next V2 09/10] RDMA/nldev: Added configuration of RDMA dynamic interrupt moderation to netlink Thread-Topic: [for-next V2 09/10] RDMA/nldev: Added configuration of RDMA dynamic interrupt moderation to netlink Thread-Index: AQHVK5inSPo/jIeedkunXG0ciooV4w== Date: Tue, 25 Jun 2019 20:57:51 +0000 Message-ID: <20190625205701.17849-10-saeedm@mellanox.com> References: <20190625205701.17849-1-saeedm@mellanox.com> In-Reply-To: <20190625205701.17849-1-saeedm@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.21.0 x-originating-ip: [209.116.155.178] x-clientproxiedby: BYAPR03CA0025.namprd03.prod.outlook.com (2603:10b6:a02:a8::38) To DB6PR0501MB2759.eurprd05.prod.outlook.com (2603:10a6:4:84::7) authentication-results: spf=none (sender IP is ) smtp.mailfrom=saeedm@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: fb7e8f3d-8654-4460-f111-08d6f9afc967 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020);SRVR:DB6PR0501MB2696; x-ms-traffictypediagnostic: DB6PR0501MB2696: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-forefront-prvs: 0079056367 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(39860400002)(396003)(136003)(376002)(366004)(346002)(199004)(189003)(6512007)(66476007)(186003)(86362001)(305945005)(3846002)(26005)(6116002)(68736007)(7736002)(102836004)(50226002)(99286004)(14444005)(53936002)(6636002)(76176011)(1076003)(2906002)(5660300002)(6436002)(64756008)(14454004)(66556008)(25786009)(486006)(52116002)(73956011)(71190400001)(6486002)(6506007)(11346002)(81166006)(71200400001)(446003)(386003)(66946007)(256004)(81156014)(66446008)(478600001)(4326008)(66066001)(8676002)(107886003)(2616005)(36756003)(316002)(476003)(8936002)(110136005)(54906003);DIR:OUT;SFP:1101;SCL:1;SRVR:DB6PR0501MB2696;H:DB6PR0501MB2759.eurprd05.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: bEuXDZX6HdZ+C6hRHWg3bPPbvjvXBiRHIniY1KCh5qgCw4H7tvv5yJJfKNrmF/bRLLqhWDlgo4dHggYQHlCZsFnvhJyXecp1WoSXFGLbq7w6tfoMpRbhevinvGILZjDOORtOwNN052ffkZLVUy6EfOpT7yKhSXdl41gZcl2a/3vg8Wt5aBoY8gny8+OAlUHucxndmpghZ5sobZKc7aKvHc4Ejr/9ytDXdKXdAcZbVD2Ip14dyQ7KSEVNG1vOqDgEeGdvEAKTbiWUuCVkhBZiibi0JTXwLXaL0J6GorhFFJ16pDC+PMfRLc5n9X1nn2akbNxdsDphIE+euLcPbQqVIHhWL59QL5DkBDZwcFr79k9Dw6SG/CBxlCwxcXLoTwEP64HCPlwfXGE1cBOT2gImPd2jSdCWLEmI0Ri7Yk5nDCI= MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: fb7e8f3d-8654-4460-f111-08d6f9afc967 X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Jun 2019 20:57:51.7172 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: saeedm@mellanox.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0501MB2696 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Yamin Friedman Added parameter in ib_device for enabling dynamic interrupt moderation so that it can be configured in userspace using rdma tool. In order to set dim for an ib device the command is: rdma dev set [DEV] dim [on|off] Please set on/off. rdma dev show 0: mlx5_0: node_type ca fw 16.26.0055 node_guid 248a:0703:00a5:29d0 sys_image_guid 248a:0703:00a5:29d0 dim on rdma resource show cq dev mlx5_0 cqn 0 cqe 1023 users 4 poll-ctx UNBOUND_WORKQUEUE dim off comm [ib_core] Signed-off-by: Yamin Friedman Reviewed-by: Leon Romanovsky Signed-off-by: Saeed Mahameed --- drivers/infiniband/Kconfig | 1 + drivers/infiniband/core/core_priv.h | 1 + drivers/infiniband/core/device.c | 9 +++++++++ drivers/infiniband/core/nldev.c | 14 ++++++++++++++ include/rdma/ib_verbs.h | 4 ++++ include/uapi/rdma/rdma_netlink.h | 5 +++++ 6 files changed, 34 insertions(+) diff --git a/drivers/infiniband/Kconfig b/drivers/infiniband/Kconfig index 8ba41cbf1869..060649093ee1 100644 --- a/drivers/infiniband/Kconfig +++ b/drivers/infiniband/Kconfig @@ -7,6 +7,7 @@ menuconfig INFINIBAND depends on m || IPV6 != m depends on !ALPHA select IRQ_POLL + select DIMLIB ---help--- Core support for InfiniBand (IB). Make sure to also select any protocols you wish to use as well as drivers for your diff --git a/drivers/infiniband/core/core_priv.h b/drivers/infiniband/core/core_priv.h index ff40a450b5d2..9724179a7d7b 100644 --- a/drivers/infiniband/core/core_priv.h +++ b/drivers/infiniband/core/core_priv.h @@ -60,6 +60,7 @@ extern bool ib_devices_shared_netns; int ib_device_register_sysfs(struct ib_device *device); void ib_device_unregister_sysfs(struct ib_device *device); int ib_device_rename(struct ib_device *ibdev, const char *name); +int ib_device_set_dim(struct ib_device *ibdev, u8 use_dim); typedef void (*roce_netdev_callback)(struct ib_device *device, u8 port, struct net_device *idev, void *cookie); diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 78dc07c6ac4b..7da149f1afe2 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -432,6 +432,15 @@ int ib_device_rename(struct ib_device *ibdev, const char *name) return ret; } +int ib_device_set_dim(struct ib_device *ibdev, u8 use_dim) +{ + if (use_dim > 1) + return -EINVAL; + ibdev->use_cq_dim = use_dim; + + return 0; +} + static int alloc_name(struct ib_device *ibdev, const char *name) { struct ib_device *device; diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c index 69188cbbd99b..71d1ec0e43cb 100644 --- a/drivers/infiniband/core/nldev.c +++ b/drivers/infiniband/core/nldev.c @@ -120,6 +120,7 @@ static const struct nla_policy nldev_policy[RDMA_NLDEV_ATTR_MAX] = { [RDMA_NLDEV_ATTR_DEV_PROTOCOL] = { .type = NLA_NUL_STRING, .len = RDMA_NLDEV_ATTR_ENTRY_STRLEN }, [RDMA_NLDEV_NET_NS_FD] = { .type = NLA_U32 }, + [RDMA_NLDEV_ATTR_DEV_DIM] = { .type = NLA_U8 }, }; static int put_driver_name_print_type(struct sk_buff *msg, const char *name, @@ -232,6 +233,8 @@ static int fill_dev_info(struct sk_buff *msg, struct ib_device *device) return -EMSGSIZE; if (nla_put_u8(msg, RDMA_NLDEV_ATTR_DEV_NODE_TYPE, device->node_type)) return -EMSGSIZE; + if (nla_put_u8(msg, RDMA_NLDEV_ATTR_DEV_DIM, device->use_cq_dim)) + return -EMSGSIZE; /* * Link type is determined on first port and mlx4 device @@ -532,6 +535,9 @@ static int fill_res_cq_entry(struct sk_buff *msg, bool has_cap_net_admin, nla_put_u8(msg, RDMA_NLDEV_ATTR_RES_POLL_CTX, cq->poll_ctx)) goto err; + if (nla_put_u8(msg, RDMA_NLDEV_ATTR_DEV_DIM, (cq->dim != NULL))) + goto err; + if (nla_put_u32(msg, RDMA_NLDEV_ATTR_RES_CQN, res->id)) goto err; if (!rdma_is_kernel_res(res) && @@ -704,6 +710,14 @@ static int nldev_set_doit(struct sk_buff *skb, struct nlmsghdr *nlh, goto put_done; } + if (tb[RDMA_NLDEV_ATTR_DEV_DIM]) { + u8 use_dim; + + use_dim = nla_get_u8(tb[RDMA_NLDEV_ATTR_DEV_DIM]); + err = ib_device_set_dim(device, use_dim); + goto done; + } + done: ib_device_put(device); put_done: diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 0742095355f2..074d7f4bc8a6 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -65,6 +65,7 @@ #include #include #include +#include #define IB_FW_VERSION_NAME_MAX ETHTOOL_FWVERS_LEN @@ -1638,6 +1639,7 @@ struct ib_cq { * Implementation details of the RDMA core, don't use in drivers: */ struct rdma_restrack_entry res; + struct dim *dim; }; struct ib_srq { @@ -2692,6 +2694,8 @@ struct ib_device { /* Used by iWarp CM */ char iw_ifname[IFNAMSIZ]; u32 iw_driver_flags; + + bool use_cq_dim; }; struct ib_client { diff --git a/include/uapi/rdma/rdma_netlink.h b/include/uapi/rdma/rdma_netlink.h index 41db51367efa..6050c7daee83 100644 --- a/include/uapi/rdma/rdma_netlink.h +++ b/include/uapi/rdma/rdma_netlink.h @@ -479,6 +479,11 @@ enum rdma_nldev_attr { */ RDMA_NLDEV_NET_NS_FD, /* u32 */ + /* + * Setting of dynamic interrupt moderation + */ + RDMA_NLDEV_ATTR_DEV_DIM, /* u8 */ + /* * Always the end */