From patchwork Tue Oct 21 08:26:02 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Koushnir X-Patchwork-Id: 5111261 X-Patchwork-Delegate: hal@mellanox.com Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id C067D9F349 for ; Tue, 21 Oct 2014 08:26:24 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id CC97B20149 for ; Tue, 21 Oct 2014 08:26:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D6B4C20148 for ; Tue, 21 Oct 2014 08:26:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751143AbaJUI0R (ORCPT ); Tue, 21 Oct 2014 04:26:17 -0400 Received: from mailp.voltaire.com ([193.47.165.129]:51662 "EHLO mellanox.co.il" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1751005AbaJUI0Q (ORCPT ); Tue, 21 Oct 2014 04:26:16 -0400 Received: from Internal Mail-Server by MTLPINE2 (envelope-from vladimirk@mellanox.com) with SMTP; 21 Oct 2014 10:26:15 +0200 Received: from r-ufm99.mtr.labs.mlnx. (r-ufm99.mtr.labs.mlnx [10.209.36.190]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id s9L8QDNl023175; Tue, 21 Oct 2014 11:26:13 +0300 From: Vladimir Koushnir To: linux-rdma@vger.kernel.org, hal@dev.mellanox.co.il Cc: Vladimir Koushnir Subject: [PATCH] send client reregistration in case of SM LID change Date: Tue, 21 Oct 2014 10:26:02 +0200 Message-Id: <1413879962-15383-1-git-send-email-vladimirk@mellanox.com> X-Mailer: git-send-email 1.7.1 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-8.3 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP if for some reason SM LID recorded in PortInfo of the node is different from our SM LID, it is safer to send client reregister notification to this node to trigger reregistration of the node in Master SM. This scenario may occur when Standby SM loses connectivity to the Master SM and becomes Master SM for some time period. If the connectivity is re-established but Handover sent by Standby SM is lost. the only indication about SM handovers in the fabric is SM LID mis-configuration in some nodes. It is important to note that this is a workaround. Some applications/ULPs only look at client reregistration and not at SM LID change. Most IB device drivers are not currently issuing SM LID change local event. The fix forces opensm to set client reregister bit in PortInfo(Set) in the scenario above. Signed-off-by: Vladimir Koushnir --- opensm/osm_lid_mgr.c | 28 +++++++++++++++++++--------- 1 files changed, 19 insertions(+), 9 deletions(-) diff --git a/opensm/osm_lid_mgr.c b/opensm/osm_lid_mgr.c index f52c770..9f4858d 100644 --- a/opensm/osm_lid_mgr.c +++ b/opensm/osm_lid_mgr.c @@ -806,6 +806,7 @@ static int lid_mgr_set_physp_pi(IN osm_lid_mgr_t * p_mgr, uint8_t op_vls; uint8_t port_num; boolean_t send_set = FALSE; + boolean_t send_client_rereg = FALSE; boolean_t update_mkey = FALSE; int ret = 0; @@ -892,11 +893,17 @@ static int lid_mgr_set_physp_pi(IN osm_lid_mgr_t * p_mgr, p_mgr->dirty = TRUE; } - /* we are updating the ports with our local sm_base_lid */ + /* + We are updating the ports with our local sm_base_lid + if for some reason currently received SM LID is different from our SM LID, + need to send client reregister to this port + */ p_pi->master_sm_base_lid = p_mgr->p_subn->sm_base_lid; if (memcmp(&p_pi->master_sm_base_lid, &p_old_pi->master_sm_base_lid, - sizeof(p_pi->master_sm_base_lid))) + sizeof(p_pi->master_sm_base_lid))) { + send_client_rereg = TRUE; send_set = TRUE; + } p_pi->m_key_lease_period = p_mgr->p_subn->opt.m_key_lease_period; if (memcmp(&p_pi->m_key_lease_period, &p_old_pi->m_key_lease_period, @@ -1029,13 +1036,16 @@ static int lid_mgr_set_physp_pi(IN osm_lid_mgr_t * p_mgr, context.pi_context.active_transition = FALSE; /* - We need to set the cli_rereg bit when we are in first_time_master_sweep - for ports supporting the ClientReregistration Vol1 (v1.2) p811 14.4.11 - Also, if this port was just now discovered, then we should also set - the cli_rereg bit. We know that the port was just discovered if its - is_new field is set. - */ - if ((p_mgr->p_subn->first_time_master_sweep == TRUE || p_port->is_new) + For ports supporting the ClientReregistration Vol1 (v1.2) p811 14.4.11: + need to set the cli_rereg bit when current SM LID at the Host + is different from our SM LID, + also if we are in first_time_master_sweep, + also if this port was just now discovered, then we should also set + the cli_rereg bit (we know that the port was just discovered + if its is_new field is set). + */ + if ((send_client_rereg || + p_mgr->p_subn->first_time_master_sweep == TRUE || p_port->is_new) && !p_mgr->p_subn->opt.no_clients_rereg && (p_old_pi->capability_mask & IB_PORT_CAP_HAS_CLIENT_REREG)) { OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG,