From patchwork Fri Mar 10 21:36:22 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Igor Druzhinin X-Patchwork-Id: 9618033 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0D40660414 for ; Fri, 10 Mar 2017 21:39:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E7076286E6 for ; Fri, 10 Mar 2017 21:39:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D9C21287AA; Fri, 10 Mar 2017 21:39:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 284FE27C0B for ; Fri, 10 Mar 2017 21:39:31 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cmSDi-000469-15; Fri, 10 Mar 2017 21:37:06 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cmSDg-00045z-D6 for xen-devel@lists.xenproject.org; Fri, 10 Mar 2017 21:37:04 +0000 Received: from [85.158.139.211] by server-6.bemta-5.messagelabs.com id 56/4B-16497-F7C13C85; Fri, 10 Mar 2017 21:37:03 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrOLMWRWlGSWpSXmKPExsXitHSDvW69zOE Ig7uf2S2+b5nM5MDocfjDFZYAxijWzLyk/IoE1oyJF26wFDxTq7i6YTtjA+MRhS5GDg4JAX+J NX/Suxg5OdgEDCRObVrEAmKLCNhJXNp8gB3EZhYokWg7uZ4VxBYW8JG42bIeLM4ioCpxv/EMG 4jNK+Al8WjGb7AaCQEFiSkP3zOD2EICahJHu3axQNQISpyc+YQFYqaExMEXL5gnMHLPQpKahS S1gJFpFaN6cWpRWWqRroVeUlFmekZJbmJmjq6hgalebmpxcWJ6ak5iUrFecn7uJkZgIDAAwQ7 Gg83OhxglOZiURHl/fzkUIcSXlJ9SmZFYnBFfVJqTWnyIUYaDQ0mC957U4QghwaLU9NSKtMwc YEjCpCU4eJREeD+CpHmLCxJzizPTIVKnGHU55sze/YZJiCUvPy9VSpx3J0iRAEhRRmke3AhYf FxilJUS5mUEOkqIpyC1KDezBFX+FaM4B6OSMO8FkCk8mXklcJteAR3BBHTENL6DIEeUJCKkpB oYjVdaTb6y+Xj0z4Ydi/K+1UvcWvNCX8LA3dH0XNMVwfNaqW+K1favMvrguSO1lCfgBPOugMC Ac8puErdYTfm2F6f1KW6awBd5evWzsvsLF8woktHPq2MtLl3seMncOO3N+yCb0o3vfypVd/8o e2MpEHJI/8Yf62NHiz4+SZjt0ehnmTV9yYl3SizFGYmGWsxFxYkAAT+WL4oCAAA= X-Env-Sender: prvs=235f9c656=igor.druzhinin@citrix.com X-Msg-Ref: server-8.tower-206.messagelabs.com!1489181821!88768782!1 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n, received_headers: No Received headers X-StarScan-Received: X-StarScan-Version: 9.2.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 20974 invoked from network); 10 Mar 2017 21:37:02 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-8.tower-206.messagelabs.com with RC4-SHA encrypted SMTP; 10 Mar 2017 21:37:02 -0000 X-IronPort-AV: E=Sophos;i="5.36,143,1486425600"; d="scan'208";a="421888923" From: Igor Druzhinin To: , Date: Fri, 10 Mar 2017 21:36:22 +0000 Message-ID: <1489181782-120847-1-git-send-email-igor.druzhinin@citrix.com> X-Mailer: git-send-email 1.8.3.1 MIME-Version: 1.0 Cc: jgross@suse.com, Igor Druzhinin , paul.durrant@citrix.com, wei.liu2@citrix.com Subject: [Xen-devel] [PATCH net v4] xen-netback: fix race condition on XenBus disconnect X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP In some cases during XenBus disconnect event handling and subsequent queue resource release there may be some TX handlers active on other processors. Use RCU in order to synchronize with them. Signed-off-by: Igor Druzhinin --- v4: * Use READ_ONCE instead of rcu_dereference to stop sparse complaining v3: * Fix unintended semantic change in xenvif_get_ethtool_stats * Dropped extra code v2: * Add protection for xenvif_get_ethtool_stats * Additional comments and fixes --- drivers/net/xen-netback/interface.c | 26 +++++++++++++++++--------- drivers/net/xen-netback/netback.c | 2 +- drivers/net/xen-netback/xenbus.c | 20 ++++++++++---------- 3 files changed, 28 insertions(+), 20 deletions(-) diff --git a/drivers/net/xen-netback/interface.c b/drivers/net/xen-netback/interface.c index 829b26c..8397f6c 100644 --- a/drivers/net/xen-netback/interface.c +++ b/drivers/net/xen-netback/interface.c @@ -165,13 +165,17 @@ static int xenvif_start_xmit(struct sk_buff *skb, struct net_device *dev) { struct xenvif *vif = netdev_priv(dev); struct xenvif_queue *queue = NULL; - unsigned int num_queues = vif->num_queues; + unsigned int num_queues; u16 index; struct xenvif_rx_cb *cb; BUG_ON(skb->dev != dev); - /* Drop the packet if queues are not set up */ + /* Drop the packet if queues are not set up. + * This handler should be called inside an RCU read section + * so we don't need to enter it here explicitly. + */ + num_queues = READ_ONCE(vif->num_queues); if (num_queues < 1) goto drop; @@ -222,18 +226,18 @@ static struct net_device_stats *xenvif_get_stats(struct net_device *dev) { struct xenvif *vif = netdev_priv(dev); struct xenvif_queue *queue = NULL; + unsigned int num_queues; u64 rx_bytes = 0; u64 rx_packets = 0; u64 tx_bytes = 0; u64 tx_packets = 0; unsigned int index; - spin_lock(&vif->lock); - if (vif->queues == NULL) - goto out; + rcu_read_lock(); + num_queues = READ_ONCE(vif->num_queues); /* Aggregate tx and rx stats from each queue */ - for (index = 0; index < vif->num_queues; ++index) { + for (index = 0; index < num_queues; ++index) { queue = &vif->queues[index]; rx_bytes += queue->stats.rx_bytes; rx_packets += queue->stats.rx_packets; @@ -241,8 +245,7 @@ static struct net_device_stats *xenvif_get_stats(struct net_device *dev) tx_packets += queue->stats.tx_packets; } -out: - spin_unlock(&vif->lock); + rcu_read_unlock(); vif->dev->stats.rx_bytes = rx_bytes; vif->dev->stats.rx_packets = rx_packets; @@ -378,10 +381,13 @@ static void xenvif_get_ethtool_stats(struct net_device *dev, struct ethtool_stats *stats, u64 * data) { struct xenvif *vif = netdev_priv(dev); - unsigned int num_queues = vif->num_queues; + unsigned int num_queues; int i; unsigned int queue_index; + rcu_read_lock(); + num_queues = READ_ONCE(vif->num_queues); + for (i = 0; i < ARRAY_SIZE(xenvif_stats); i++) { unsigned long accum = 0; for (queue_index = 0; queue_index < num_queues; ++queue_index) { @@ -390,6 +396,8 @@ static void xenvif_get_ethtool_stats(struct net_device *dev, } data[i] = accum; } + + rcu_read_unlock(); } static void xenvif_get_strings(struct net_device *dev, u32 stringset, u8 * data) diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c index f9bcf4a..602d408 100644 --- a/drivers/net/xen-netback/netback.c +++ b/drivers/net/xen-netback/netback.c @@ -214,7 +214,7 @@ static void xenvif_fatal_tx_err(struct xenvif *vif) netdev_err(vif->dev, "fatal error; disabling device\n"); vif->disabled = true; /* Disable the vif from queue 0's kthread */ - if (vif->queues) + if (vif->num_queues) xenvif_kick_thread(&vif->queues[0]); } diff --git a/drivers/net/xen-netback/xenbus.c b/drivers/net/xen-netback/xenbus.c index d2d7cd9..a56d3ea 100644 --- a/drivers/net/xen-netback/xenbus.c +++ b/drivers/net/xen-netback/xenbus.c @@ -495,26 +495,26 @@ static void backend_disconnect(struct backend_info *be) struct xenvif *vif = be->vif; if (vif) { + unsigned int num_queues = vif->num_queues; unsigned int queue_index; - struct xenvif_queue *queues; xen_unregister_watchers(vif); #ifdef CONFIG_DEBUG_FS xenvif_debugfs_delif(vif); #endif /* CONFIG_DEBUG_FS */ xenvif_disconnect_data(vif); - for (queue_index = 0; - queue_index < vif->num_queues; - ++queue_index) - xenvif_deinit_queue(&vif->queues[queue_index]); - spin_lock(&vif->lock); - queues = vif->queues; + /* At this point some of the handlers may still be active + * so we need to have additional synchronization here. + */ vif->num_queues = 0; - vif->queues = NULL; - spin_unlock(&vif->lock); + synchronize_net(); - vfree(queues); + for (queue_index = 0; queue_index < num_queues; ++queue_index) + xenvif_deinit_queue(&vif->queues[queue_index]); + + vfree(vif->queues); + vif->queues = NULL; xenvif_disconnect_ctrl(vif); }