From patchwork Mon Nov 8 15:07:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12608603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A9E9C433EF for ; Mon, 8 Nov 2021 15:08:14 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0B47F61027 for ; Mon, 8 Nov 2021 15:08:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 0B47F61027 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8935721E013; Mon, 8 Nov 2021 07:08:06 -0800 (PST) Received: from smtp3.ccs.ornl.gov (SMTP3.CCS.ORNL.GOV [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A815E21C967 for ; Mon, 8 Nov 2021 07:07:51 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 367F3222C; Mon, 8 Nov 2021 10:07:46 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 338E4E07E7; Mon, 8 Nov 2021 10:07:46 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 8 Nov 2021 10:07:41 -0500 Message-Id: <1636384063-13838-14-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1636384063-13838-1-git-send-email-jsimmons@infradead.org> References: <1636384063-13838-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 13/15] lnet: socklnd: default conns_per_peer to 0 X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Serguei Smirnov , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Serguei Smirnov Setting conns_per_peer to 0 triggers socklnd to choose the (heuristically) optimal setting for the interface given its speed. Make 0 the default for socklnd conns_per_peer. Fixes: 6374d25cfe ("lnet: socklnd: set conns_per_peer based on link speed") WC-bug-id: https://jira.whamcloud.com/browse/LU-15136 Lustre-commit: 30a028e2ee2b3eead ("U-15136 socklnd: default conns_per_peer to 0") Signed-off-by: Serguei Smirnov Reviewed-on: https://review.whamcloud.com/45319 Reviewed-by: Andreas Dilger Reviewed-by: Chris Horn Reviewed-by: James Simmons Tested-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/linux/lnet/lib-lnet.h | 2 +- net/lnet/klnds/socklnd/socklnd_modparams.c | 10 ++++++++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/include/linux/lnet/lib-lnet.h b/include/linux/lnet/lib-lnet.h index 890f61a..104c98d 100644 --- a/include/linux/lnet/lib-lnet.h +++ b/include/linux/lnet/lib-lnet.h @@ -87,7 +87,7 @@ #define DEFAULT_CREDITS 256 /* default number of connections per peer */ -#define DEFAULT_CONNS_PER_PEER 1 +#define DEFAULT_CONNS_PER_PEER 0 int choose_ipv4_src(u32 *ret, int interface, u32 dst_ipaddr, struct net *ns); diff --git a/net/lnet/klnds/socklnd/socklnd_modparams.c b/net/lnet/klnds/socklnd/socklnd_modparams.c index 72f9df2..c00ea49 100644 --- a/net/lnet/klnds/socklnd/socklnd_modparams.c +++ b/net/lnet/klnds/socklnd/socklnd_modparams.c @@ -163,6 +163,10 @@ static int ksocklnd_ni_get_eth_intf_speed(struct lnet_ni *ni) int intf_idx = -1; int ret = -1; + /* check if ni has interface assigned */ + if (!ni->ni_net_ns || !ni->ni_interface) + return 0; + rtnl_lock(); for_each_netdev(ni->ni_net_ns, dev) { int flags = dev_get_flags(dev); @@ -215,10 +219,12 @@ static int ksocklnd_speed2cpp(int speed) static int ksocklnd_lookup_conns_per_peer(struct lnet_ni *ni) { - int cpp = DEFAULT_CONNS_PER_PEER; + int cpp = 1; int speed = ksocklnd_ni_get_eth_intf_speed(ni); - CDEBUG(D_NET, "intf %s speed %d\n", ni->ni_interface, speed); + if (ni->ni_interface) + CDEBUG(D_NET, "intf %s speed %d\n", ni->ni_interface, speed); + if (speed > 0) cpp = ksocklnd_speed2cpp(speed);