From patchwork Mon Sep 28 17:11:50 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Christoph Lameter (Ampere)" X-Patchwork-Id: 7279121 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 5BCB7BEEA4 for ; Mon, 28 Sep 2015 17:11:55 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 5792B20712 for ; Mon, 28 Sep 2015 17:11:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3E9D520710 for ; Mon, 28 Sep 2015 17:11:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933540AbbI1RLw (ORCPT ); Mon, 28 Sep 2015 13:11:52 -0400 Received: from resqmta-ch2-04v.sys.comcast.net ([69.252.207.36]:48941 "EHLO resqmta-ch2-04v.sys.comcast.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933063AbbI1RLv (ORCPT ); Mon, 28 Sep 2015 13:11:51 -0400 Received: from resomta-ch2-11v.sys.comcast.net ([69.252.207.107]) by resqmta-ch2-04v.sys.comcast.net with comcast id NhBc1r0072Ka2Q501hBrZV; Mon, 28 Sep 2015 17:11:51 +0000 Received: from gentwo.org ([98.222.162.64]) by resomta-ch2-11v.sys.comcast.net with comcast id NhBq1r00V1PgSZ601hBqn9; Mon, 28 Sep 2015 17:11:51 +0000 Received: by gentwo.org (Postfix, from userid 1001) id 747671AED; Mon, 28 Sep 2015 12:11:50 -0500 (CDT) Received: from localhost (localhost [127.0.0.1]) by gentwo.org (Postfix) with ESMTP id 725D7473; Mon, 28 Sep 2015 12:11:50 -0500 (CDT) Date: Mon, 28 Sep 2015 12:11:50 -0500 (CDT) From: Christoph Lameter X-X-Sender: cl@east.gentwo.org To: Doug Ledford cc: Or Gerlitz , Or Gerlitz , "linux-rdma@vger.kernel.org" Subject: Re: [PATCH] Expire sendonly joins (was Re: [PATCH rdma-rc 0/2] Add mechanism for ipoib neigh state change notifications) In-Reply-To: <560971DC.500@redhat.com> Message-ID: References: <1442486283-9699-1-git-send-email-ogerlitz@mellanox.com> <5608282F.1020507@redhat.com> <56095E6B.60509@redhat.com> <560971DC.500@redhat.com> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=q20140121; t=1443460311; bh=rYF4VKd3IGXGrxCZ2Q4FZmi9/4PGxidu10pvlofafoI=; h=Received:Received:Received:Received:Date:From:To:Subject: Message-ID:Content-Type; b=GR+0eYooL157YpoXtqpKh0ebjK47R+6+kZF+Bk7a+jP2pxUvGccJjmAYErSVEyQLS WpaISP3D1wuOAK1bWpaog+92d5RBQsIzvyH+m0w4OtLEzdEV3tITeAf+86XdybdNNN sctI/TbKxyx8MjeRUk5COMwW+efMuq+zovURjKG2PUHXH2quklBUmk+2i9OE72YF1P LFTr3kvdl8+uKD6VIEHBbFOtiZMZPu3oVN4wbuw6/fOXOzTZMUBRMEsVcpSlXwgE42 t9mn9YVJtVR3mOAA2SMihEM0cHrL+nWYSDJUS13Q+a2H56Xxh05rJt02+PFzEKyN7y eQdouOUqGTqYw== Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, MIME_HEADER_CTYPE_ONLY, RCVD_IN_DNSWL_HI, T_DKIM_INVALID, T_RP_MATCHES_RCVD, T_TVD_MIME_EPI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Ok I refactored the whole thing to make it less invasive and keep more functionality in ipoib_multicast.c. Since you are working on it it would be best for you to have the newest version. I split this into two patches: One preparatory and one that implements the actual logic. Both attached. The patch that implements the join is inline here: Subject: ipoib multicast: Expire MC groups when the address expires Upon address expiration do the proper thing to also expire the sendonly multicast group. Signed-off-by: Christoph Lameter From: Christoph Lameter Subject: ipoib multicast: Extract two function from ipoib_mcast_flush We need these two functions later to do the implicit leave when the address handle expires so refactor the code. Signed-off-by: Christoph Lameter Index: linux/drivers/infiniband/ulp/ipoib/ipoib_multicast.c =================================================================== --- linux.orig/drivers/infiniband/ulp/ipoib/ipoib_multicast.c 2015-09-28 11:20:05.387701463 -0500 +++ linux/drivers/infiniband/ulp/ipoib/ipoib_multicast.c 2015-09-28 11:20:53.819702839 -0500 @@ -775,6 +775,31 @@ spin_unlock_irqrestore(&priv->lock, flags); } +/* + * Detach a multicast group from the devices multicast tree and move it + * to a list for future removal + */ +static void ipoib_detach_mc_group(struct ipoib_dev_priv *priv, + struct ipoib_mcast *mcast, struct list_head *remove_list) +{ + list_del(&mcast->list); + rb_erase(&mcast->rb_node, &priv->multicast_tree); + list_add_tail(&mcast->list, remove_list); +} + +/* + * Remove a list of multicast groups that has been detached and free them + */ +void ipoib_mcast_remove_mc_list(struct net_device *dev, struct list_head *remove_list) +{ + struct ipoib_mcast *mcast, *tmcast; + + list_for_each_entry_safe(mcast, tmcast, remove_list, list) { + ipoib_mcast_leave(dev, mcast); + ipoib_mcast_free(mcast); + } +} + void ipoib_mcast_dev_flush(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); @@ -786,11 +811,8 @@ spin_lock_irqsave(&priv->lock, flags); - list_for_each_entry_safe(mcast, tmcast, &priv->multicast_list, list) { - list_del(&mcast->list); - rb_erase(&mcast->rb_node, &priv->multicast_tree); - list_add_tail(&mcast->list, &remove_list); - } + list_for_each_entry_safe(mcast, tmcast, &priv->multicast_list, list) + ipoib_detach_mc_group(priv, mcast, &remove_list); if (priv->broadcast) { rb_erase(&priv->broadcast->rb_node, &priv->multicast_tree); @@ -808,10 +830,7 @@ if (test_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags)) wait_for_completion(&mcast->done); - list_for_each_entry_safe(mcast, tmcast, &remove_list, list) { - ipoib_mcast_leave(dev, mcast); - ipoib_mcast_free(mcast); - } + ipoib_mcast_remove_mc_list(dev, &remove_list); } static int ipoib_mcast_addr_is_valid(const u8 *addr, const u8 *broadcast) Subject: ipoib multicast: Expire MC groups when the address expires Upon address expiration do the proper thing to also expire the sendonly multicast group. Signed-off-by: Christoph Lameter Index: linux/drivers/infiniband/ulp/ipoib/ipoib.h =================================================================== --- linux.orig/drivers/infiniband/ulp/ipoib/ipoib.h 2015-09-28 11:56:59.779764388 -0500 +++ linux/drivers/infiniband/ulp/ipoib/ipoib.h 2015-09-28 11:57:16.291764857 -0500 @@ -548,6 +548,8 @@ int ipoib_mcast_attach(struct net_device *dev, u16 mlid, union ib_gid *mgid, int set_qkey); +void ipoib_mcast_remove_mc_list(struct net_device *dev, struct list_head *list); +void ipoib_mcast_detach_sendonly(struct ipoib_dev_priv *priv, u8 *mgid, struct list_head *list); int ipoib_init_qp(struct net_device *dev); int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca); Index: linux/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- linux.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c 2015-09-28 11:56:59.779764388 -0500 +++ linux/drivers/infiniband/ulp/ipoib/ipoib_main.c 2015-09-28 11:56:59.775764388 -0500 @@ -1149,6 +1149,8 @@ unsigned long dt; unsigned long flags; int i; + LIST_HEAD(remove_list); + struct net_device *dev = priv->dev; if (test_bit(IPOIB_STOP_NEIGH_GC, &priv->flags)) return; @@ -1176,6 +1178,9 @@ lockdep_is_held(&priv->lock))) != NULL) { /* was the neigh idle for two GC periods */ if (time_after(neigh_obsolete, neigh->alive)) { + + ipoib_mcast_detach_sendonly(priv, neigh->daddr + 4, &remove_list); + rcu_assign_pointer(*np, rcu_dereference_protected(neigh->hnext, lockdep_is_held(&priv->lock))); @@ -1191,6 +1196,7 @@ out_unlock: spin_unlock_irqrestore(&priv->lock, flags); + ipoib_mcast_remove_mc_list(dev, &remove_list); } static void ipoib_reap_neigh(struct work_struct *work) Index: linux/drivers/infiniband/ulp/ipoib/ipoib_multicast.c =================================================================== --- linux.orig/drivers/infiniband/ulp/ipoib/ipoib_multicast.c 2015-09-28 11:56:59.779764388 -0500 +++ linux/drivers/infiniband/ulp/ipoib/ipoib_multicast.c 2015-09-28 11:56:59.775764388 -0500 @@ -800,6 +800,23 @@ } } +/* + * Check if this is a sendonly multicast group. If so remove it from the list and put it + * onto the given list for final removal. + */ +void ipoib_mcast_detach_sendonly(struct ipoib_dev_priv *priv, u8 *mgid, struct list_head *remove_list) +{ + struct ipoib_mcast *mcast; + + /* Is this multicast ? */ + if (mcast_auto_create && *mgid == 0xff) { + mcast = __ipoib_mcast_find(priv->dev, mgid); + + if (mcast && test_bit(IPOIB_MCAST_FLAG_SENDONLY, &mcast->flags)) + ipoib_detach_mc_group(priv, mcast, remove_list); + } +} + void ipoib_mcast_dev_flush(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev); Index: linux/drivers/infiniband/ulp/ipoib/ipoib.h =================================================================== --- linux.orig/drivers/infiniband/ulp/ipoib/ipoib.h 2015-09-28 11:56:59.779764388 -0500 +++ linux/drivers/infiniband/ulp/ipoib/ipoib.h 2015-09-28 11:57:16.291764857 -0500 @@ -548,6 +548,8 @@ int ipoib_mcast_attach(struct net_device *dev, u16 mlid, union ib_gid *mgid, int set_qkey); +void ipoib_mcast_remove_mc_list(struct net_device *dev, struct list_head *list); +void ipoib_mcast_detach_sendonly(struct ipoib_dev_priv *priv, u8 *mgid, struct list_head *list); int ipoib_init_qp(struct net_device *dev); int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca); Index: linux/drivers/infiniband/ulp/ipoib/ipoib_main.c =================================================================== --- linux.orig/drivers/infiniband/ulp/ipoib/ipoib_main.c 2015-09-28 11:56:59.779764388 -0500 +++ linux/drivers/infiniband/ulp/ipoib/ipoib_main.c 2015-09-28 11:56:59.775764388 -0500 @@ -1149,6 +1149,8 @@ unsigned long dt; unsigned long flags; int i; + LIST_HEAD(remove_list); + struct net_device *dev = priv->dev; if (test_bit(IPOIB_STOP_NEIGH_GC, &priv->flags)) return; @@ -1176,6 +1178,9 @@ lockdep_is_held(&priv->lock))) != NULL) { /* was the neigh idle for two GC periods */ if (time_after(neigh_obsolete, neigh->alive)) { + + ipoib_mcast_detach_sendonly(priv, neigh->daddr + 4, &remove_list); + rcu_assign_pointer(*np, rcu_dereference_protected(neigh->hnext, lockdep_is_held(&priv->lock))); @@ -1191,6 +1196,7 @@ out_unlock: spin_unlock_irqrestore(&priv->lock, flags); + ipoib_mcast_remove_mc_list(dev, &remove_list); } static void ipoib_reap_neigh(struct work_struct *work) Index: linux/drivers/infiniband/ulp/ipoib/ipoib_multicast.c =================================================================== --- linux.orig/drivers/infiniband/ulp/ipoib/ipoib_multicast.c 2015-09-28 11:56:59.779764388 -0500 +++ linux/drivers/infiniband/ulp/ipoib/ipoib_multicast.c 2015-09-28 11:56:59.775764388 -0500 @@ -800,6 +800,23 @@ } } +/* + * Check if this is a sendonly multicast group. If so remove it from the list and put it + * onto the given list for final removal. + */ +void ipoib_mcast_detach_sendonly(struct ipoib_dev_priv *priv, u8 *mgid, struct list_head *remove_list) +{ + struct ipoib_mcast *mcast; + + /* Is this multicast ? */ + if (mcast_auto_create && *mgid == 0xff) { + mcast = __ipoib_mcast_find(priv->dev, mgid); + + if (mcast && test_bit(IPOIB_MCAST_FLAG_SENDONLY, &mcast->flags)) + ipoib_detach_mc_group(priv, mcast, remove_list); + } +} + void ipoib_mcast_dev_flush(struct net_device *dev) { struct ipoib_dev_priv *priv = netdev_priv(dev);