From patchwork Mon Feb 26 20:36:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 10243405 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 8704760208 for ; Mon, 26 Feb 2018 20:38:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 75ED32A3BB for ; Mon, 26 Feb 2018 20:38:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6A28E2A3AE; Mon, 26 Feb 2018 20:38:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1011C2A3AF for ; Mon, 26 Feb 2018 20:38:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753874AbeBZUhc (ORCPT ); Mon, 26 Feb 2018 15:37:32 -0500 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:57553 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753825AbeBZUh3 (ORCPT ); Mon, 26 Feb 2018 15:37:29 -0500 Received: from Internal Mail-Server by MTLPINE1 (envelope-from saeedm@mellanox.com) with ESMTPS (AES256-SHA encrypted); 26 Feb 2018 22:37:42 +0200 Received: from stpd.mtl.com ([172.16.5.78]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id w1QKabKY002827; Mon, 26 Feb 2018 22:37:20 +0200 From: Saeed Mahameed To: "David S. Miller" , Doug Ledford , Jason Gunthorpe Cc: Leon Romanovsky , Mark Bloch , Or Gerlitz , netdev@vger.kernel.org, linux-rdma@vger.kernel.org, Leon Romanovsky , Saeed Mahameed Subject: [for-next 11/15] IB/mlx5: Don't expose MR cache in switchdev mode Date: Mon, 26 Feb 2018 12:36:13 -0800 Message-Id: <20180226203617.10242-12-saeedm@mellanox.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180226203617.10242-1-saeedm@mellanox.com> References: <20180226203617.10242-1-saeedm@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Mark Bloch When enabling many VFs and switching to switchdev mode, the total amount of mkeys we try to allocate when loading representors is very large and may cause timeouts on allocations, the same issues was observed on VFs and we employ the same fix that was done for them. We avoid allocating the full MR cache on load but still allow it to be manipulated once the IB device is loaded. Signed-off-by: Mark Bloch Signed-off-by: Leon Romanovsky Signed-off-by: Saeed Mahameed --- drivers/infiniband/hw/mlx5/mr.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c index 556e015678de..a5fad3e87ff7 100644 --- a/drivers/infiniband/hw/mlx5/mr.c +++ b/drivers/infiniband/hw/mlx5/mr.c @@ -587,7 +587,7 @@ static void clean_keys(struct mlx5_ib_dev *dev, int c) static void mlx5_mr_cache_debugfs_cleanup(struct mlx5_ib_dev *dev) { - if (!mlx5_debugfs_root) + if (!mlx5_debugfs_root || dev->rep) return; debugfs_remove_recursive(dev->cache.root); @@ -600,7 +600,7 @@ static int mlx5_mr_cache_debugfs_init(struct mlx5_ib_dev *dev) struct mlx5_cache_ent *ent; int i; - if (!mlx5_debugfs_root) + if (!mlx5_debugfs_root || dev->rep) return 0; cache->root = debugfs_create_dir("mr_cache", dev->mdev->priv.dbg_root); @@ -690,6 +690,7 @@ int mlx5_mr_cache_init(struct mlx5_ib_dev *dev) MLX5_IB_UMR_OCTOWORD; ent->access_mode = MLX5_MKC_ACCESS_MODE_MTT; if ((dev->mdev->profile->mask & MLX5_PROF_MASK_MR_CACHE) && + !dev->rep && mlx5_core_is_pf(dev->mdev)) ent->limit = dev->mdev->profile->mr_cache[i].limit; else