From patchwork Thu Mar 15 16:08:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10285069 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C30916061F for ; Thu, 15 Mar 2018 16:08:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B1A0A2895B for ; Thu, 15 Mar 2018 16:08:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A660328B45; Thu, 15 Mar 2018 16:08:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 892302895B for ; Thu, 15 Mar 2018 16:08:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751687AbeCOQIV (ORCPT ); Thu, 15 Mar 2018 12:08:21 -0400 Received: from mail-wm0-f52.google.com ([74.125.82.52]:53953 "EHLO mail-wm0-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751650AbeCOQIS (ORCPT ); Thu, 15 Mar 2018 12:08:18 -0400 Received: by mail-wm0-f52.google.com with SMTP id e194so11442434wmd.3 for ; Thu, 15 Mar 2018 09:08:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=hgSv1qrtNfSbypDD/28I1ZUhrRxXUbgFPYuIiyRwZPs=; b=mXYrMXiBiSW3He6YlnK7+2Idp+HW5FrxdxU72Bfnm4e7xMoR5yJJd51ms28QHlCUmG R/cnfixBCg4dQBhhSQIEZvyVQTuDTsdXY/j/oA4oAY95rmmuc07V0ZND22sVKrq1mY4i FfmrmNTLxcPmjyK16d5r0/zUGKqCfPGu4yBMmVZhWGXdHMozmOGCSgzmBSFaIzSjaOqS ZlXP3D5NuqPf5aq3oVYpGQ1Aj2RQxAvC0zPr3lwzcLlAMuKYPLK7XHv5/+/1VKH+uysw M6t9sU73WT2i3ExuQJZ3gKrnENZ5dzXCoMVoUbE8FrukqF4SxSTSisslgaf8A5meTWXc Os7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=hgSv1qrtNfSbypDD/28I1ZUhrRxXUbgFPYuIiyRwZPs=; b=Fq4eY9m1I5PRvD/JtF/vTbqFbRqQQWqTsmiHcCBV/lOSU538DROncCqU9qSRFHafXj pO2O+gEuDiDqo6UEpVd2jDSRFFUAUivprb+r9/YEXBUqDZSiFGjXJHivYxTg07njzxdL UTliRaALgKzLXFH/ScYBSMkW2vN9zQhQn13DdAtAOrfA9JJyhezzQJOtihZlGxFFMQs7 kqs4WI+gmkdgrSTvdpTDaRA28WvIgwmhenQzEa4yFf4UnsLtH7DbxXAA666q5InbOwG0 9s9uz5i2KoICfblBVjqTpAmaY4B/j57LmD7dfhHhS48mj+jf8mFfWRe1q/xavH90wZzS WTRA== X-Gm-Message-State: AElRT7FOvyybBrSN+rke67PBBJpMeh16a2SSfR/T03zWECYgiSZr0StX pZ2LAwnKFVDlYqZCeDKx198I8A== X-Google-Smtp-Source: AG47ELu+4AnxppBShwbvHb2HHaMtOmcDZ4B43HcfirAczB42euALuiHjq5AbX/MEhNZdqSABoYFN9g== X-Received: by 10.28.211.195 with SMTP id k186mr4887082wmg.106.1521130096595; Thu, 15 Mar 2018 09:08:16 -0700 (PDT) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id w29sm5537130wra.84.2018.03.15.09.08.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Mar 2018 09:08:11 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.86_2) (envelope-from ) id 1ewVQG-000797-Cv; Thu, 15 Mar 2018 10:08:08 -0600 Date: Thu, 15 Mar 2018 10:08:08 -0600 From: Jason Gunthorpe To: Erez Shitrit Cc: Alex Vesker , Leon Romanovsky , Doug Ledford , Shalom Lagziel , RDMA mailing list , Erez Shitrit Subject: Re: [PATCH rdma-next] IB/ipoib: Fix wqe initialized param on ipoib set mode to connected Message-ID: <20180315160808.GG21888@ziepe.ca> References: <20180311112823.3626-1-leon@kernel.org> <20180312153914.GE24759@mellanox.com> <74da7f31-7b07-344d-060d-0dca7af88ba4@mellanox.com> <20180314180941.GI1175@mellanox.com> <20180314205349.GG8073@mellanox.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Thu, Mar 15, 2018 at 05:06:08PM +0200, Erez Shitrit wrote: > The origin issue was that after changing to CM mode traffic might > stopped for very long time (depends of the arp time, at least 30 sec). > Now, if we move the line after the ipoib_flush_paths() call, the > problem is much smaller: > only while ipoib_flush_paths() runs, packet that sent to CM > connection after packet from UD/GSO will be dropped. sure, that is prett clear, and the above would be a much better commit message. > The question is does this something that we really need to handle? > the error flow in CM mode already does this (and for other more often > error flows like RNR etc.) > and also we are talking about a case that is unlikely in the real life > of ipoib driver, mode changing is something that done once at the > beginning. should we add a code for this rare case? If you don't want to deal with it then a big comment is needed here to explain that it is racey but we don't care for XYZ reasons. And only do that if fixing it properly is really hard.. but it doesn't look too hard.. What about something like this.. I didn't check too carefully but it seems to hold the right locks: Jason --- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c index 161ba8c76285cb..bdf7a723f52818 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c @@ -711,7 +711,7 @@ static void push_pseudo_header(struct sk_buff *skb, const char *daddr) memcpy(phdr->hwaddr, daddr, INFINIBAND_ALEN); } -void ipoib_flush_paths(struct net_device *dev) +void ipoib_flush_paths(struct net_device *dev, bool cm_mode) { struct ipoib_dev_priv *priv = ipoib_priv(dev); struct ipoib_path *path, *tp; @@ -737,6 +737,20 @@ void ipoib_flush_paths(struct net_device *dev) spin_lock_irqsave(&priv->lock, flags); } + /* + * Mode change is done atomically with list flushing, we stay in the + * old mode until the list is empty then atomically switch modes for + * new neighs from then on. + */ + if (cm_mode) { + priv->tx_wr.wr.send_flags &= ~IB_SEND_IP_CSUM; + priv->tx_wr.wr.opcode = IB_WR_SEND; + set_bit(IPOIB_FLAG_ADMIN_CM, &priv->flags); + } else { + // .. fixme set tx_wr properly for !CM mode + clear_bit(IPOIB_FLAG_ADMIN_CM, &priv->flags); + } + spin_unlock_irqrestore(&priv->lock, flags); netif_tx_unlock_bh(dev); }