From patchwork Sat May 14 17:49:58 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Copeland X-Patchwork-Id: 9095201 X-Patchwork-Delegate: johannes@sipsolutions.net Return-Path: X-Original-To: patchwork-linux-wireless@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 6AA229F1D3 for ; Sat, 14 May 2016 17:51:12 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 1A9A82025A for ; Sat, 14 May 2016 17:51:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8BC4820218 for ; Sat, 14 May 2016 17:51:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753904AbcENRvG (ORCPT ); Sat, 14 May 2016 13:51:06 -0400 Received: from mail-io0-f177.google.com ([209.85.223.177]:35883 "EHLO mail-io0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753866AbcENRvE (ORCPT ); Sat, 14 May 2016 13:51:04 -0400 Received: by mail-io0-f177.google.com with SMTP id i75so162754383ioa.3 for ; Sat, 14 May 2016 10:51:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bobcopeland-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id; bh=Z2cOHb3H+hni17jAtCAx6+5I7VGMpomA2JHADBx1ubU=; b=Naco4ngZyoN+w234IyNJSu7J8TQxOE8dlOLEGi8suBkFIvS96Vgh2WG+VupaFFTl/w j2ivcPe8i0LTqFG72RZDKuLSAiyTpIaeWbDjO31NKz+3CzerjIpm4Xa0baGQt0kDcRPp jKP8dblgj3kBnP2wHi24MSUO5SlefZPvNXNG2T7qNCEIjsVOuUmz8qqUZ7giRKcsPawD DDvvA2c2geO5LjpRx4OBKWDy8NzIp3DeMbS7vjzky3Y9HfV07U+Cu/oQ70I5P7ZXWSqq kAcSTtobBDNbfF38kN8VziyH6TH2kBeh6ZZQzrzJup35rS48nWtguKzsvBHxbYaMBkaH 2LOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=Z2cOHb3H+hni17jAtCAx6+5I7VGMpomA2JHADBx1ubU=; b=ewp46O9hrLRN4NghEY7wir0bQBYH3BV/z2bws8b1Kin8v6sB0cq8s3DQIdWSkgiM2b CZiCH0CXxdCrlRiQ5KPBJMK+9yqvDUFf0asN8IZMLT9YLVYi4YwI/Ldi5V7O2B8n6SfH iednd19KcQ9i/JnYbe+65VYql9y1LUaOe2/k0o6AojaXwXN+/6ZpzszjYdh8cUuKjH6Z wUFoL3oAqIwK7NGdFe+UQmGaEsX0szO7ZIq8wDgz20otsA6LlrW1asjjJcZDAho0WuDW gYHsP+x5Gd9MO0LEVMOs2BJcmRoRXzNn3Yds16U8wbOsr0mjM5j+ZkgN55c/NVacm/nd t5lg== X-Gm-Message-State: AOPr4FWL8vyukDhu33enQIBIA1BiZVNv0UhyoEiYkiQ/oMyLzl0HJg9i030KTLMF1i+sVQ== X-Received: by 10.36.5.12 with SMTP id 12mr5587020itl.11.1463248262828; Sat, 14 May 2016 10:51:02 -0700 (PDT) Received: from hash (CPE0018e7fe5281-CM18593342f28f.cpe.net.cable.rogers.com. [99.254.238.186]) by smtp.gmail.com with ESMTPSA id u67sm2810365itd.1.2016.05.14.10.51.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 14 May 2016 10:51:02 -0700 (PDT) Received: from glass.lan ([192.168.1.51] helo=glass) by hash with esmtp (Exim 4.84_2) (envelope-from ) id 1b1diM-0002yd-Bo; Sat, 14 May 2016 13:50:58 -0400 Received: from bob by glass with local (Exim 4.86) (envelope-from ) id 1b1diP-000120-BH; Sat, 14 May 2016 13:51:01 -0400 From: Bob Copeland To: Johannes Berg Cc: linux-wireless@vger.kernel.org, fred.veldini@gmail.com, Bob Copeland , stable@vger.kernel.org Subject: [PATCH] mac80211: mesh: flush mesh paths early in sta destroy Date: Sat, 14 May 2016 13:49:58 -0400 Message-Id: <1463248198-3926-1-git-send-email-me@bobcopeland.com> X-Mailer: git-send-email 2.6.1 Sender: linux-wireless-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org X-Spam-Status: No, score=-8.2 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently, the mesh paths associated with a nexthop station are cleaned up in the following code path: __sta_info_destroy_part1 synchronize_net() __sta_info_destroy_part2 -> cleanup_single_sta -> mesh_sta_cleanup -> mesh_plink_deactivate -> mesh_path_flush_by_nexthop However, there are a couple of problems here: 1) the paths aren't flushed at all if the MPM is running in userspace (e.g. when using wpa_supplicant or authsae) 2) there is no synchronize_rcu between removing the path and readers accessing the nexthop, which means the following race is possible: CPU0 CPU1 ~~~~ ~~~~ sta_info_destroy_part1() synchronize_net() rcu_read_lock() mesh_nexthop_resolve() mpath = mesh_path_lookup() [...] -> mesh_path_flush_by_nexthop() sta = rcu_dereference( mpath->next_hop) kfree(sta) access sta <-- CRASH Fix both of these by unconditionally flushing paths in part1 of sta destroy. The synchronize_net() will then happen after paths are flushed, preventing mesh_path_lookup from finding a path with the to-be-freed nexthop sta. Also, add a new sta bit and a check in mesh_path_assign_nexthop() to avoid adding it back between the flush and the synchronize_net(). Fixes this crash: [ 348.529295] BUG: unable to handle kernel paging request at 00020040 [ 348.530014] IP: [] ieee80211_mps_set_frame_flags+0x40/0xaa [mac80211] [ 348.530014] *pde = 00000000 [ 348.530014] Oops: 0000 [#1] PREEMPT [ 348.530014] Modules linked in: drbg ansi_cprng ctr ccm ppp_generic slhc ipt_MASQUERADE nf_nat_masquerade_ipv4 8021q ] [ 348.530014] CPU: 0 PID: 20597 Comm: wget Tainted: G O 4.6.0-rc5-wt=V1 #1 [ 348.530014] Hardware name: To Be Filled By O.E.M./To be filled by O.E.M., BIOS 080016 11/07/2014 [ 348.530014] task: f64fa280 ti: f4f9c000 task.ti: f4f9c000 [ 348.530014] EIP: 0060:[] EFLAGS: 00010246 CPU: 0 [ 348.530014] EIP is at ieee80211_mps_set_frame_flags+0x40/0xaa [mac80211] [ 348.530014] EAX: f4ce63e0 EBX: 00000088 ECX: f3788416 EDX: 00020008 [ 348.530014] ESI: 00000000 EDI: 00000088 EBP: f6409a4c ESP: f6409a40 [ 348.530014] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068 [ 348.530014] CR0: 80050033 CR2: 00020040 CR3: 33190000 CR4: 00000690 [ 348.530014] Stack: [ 348.530014] 00000000 f4ce63e0 f5f9bd80 f6409a64 f9291d80 0000ce67 f5d51e00 f4ce63e0 [ 348.530014] f3788416 f6409a80 f9291dc1 f4ce8320 f4ce63e0 f5d51e00 f4ce63e0 f4ce8320 [ 348.530014] f6409a98 f9277f6f 00000000 00000000 0000007c 00000000 f6409b2c f9278dd1 [ 348.530014] Call Trace: [ 348.530014] [] mesh_nexthop_lookup+0xbb/0xc8 [mac80211] [ 348.530014] [] mesh_nexthop_resolve+0x34/0xd8 [mac80211] [ 348.530014] [] ieee80211_xmit+0x92/0xc1 [mac80211] [ 348.530014] [] __ieee80211_subif_start_xmit+0x807/0x83c [mac80211] [ 348.530014] [] ? sch_direct_xmit+0xd7/0x1b3 [ 348.530014] [] ? __local_bh_enable_ip+0x5d/0x7b [ 348.530014] [] ? nf_nat_ipv4_out+0x4c/0xd0 [nf_nat_ipv4] [ 348.530014] [] ? iptable_nat_ipv4_fn+0xf/0xf [iptable_nat] [ 348.530014] [] ? netif_skb_features+0x14d/0x30a [ 348.530014] [] ieee80211_subif_start_xmit+0xa/0xe [mac80211] [ 348.530014] [] dev_hard_start_xmit+0x1f8/0x267 [ 348.530014] [] ? validate_xmit_skb.isra.120.part.121+0x10/0x253 [ 348.530014] [] sch_direct_xmit+0x8b/0x1b3 [ 348.530014] [] __dev_queue_xmit+0x2c8/0x513 [ 348.530014] [] dev_queue_xmit+0xa/0xc [ 348.530014] [] batadv_send_skb_packet+0xd6/0xec [batman_adv] [ 348.530014] [] batadv_send_unicast_skb+0x15/0x4a [batman_adv] [ 348.530014] [] batadv_dat_send_data+0x27e/0x310 [batman_adv] [ 348.530014] [] ? batadv_tt_global_hash_find.isra.11+0x8/0xa [batman_adv] [ 348.530014] [] batadv_dat_snoop_outgoing_arp_request+0x208/0x23d [batman_adv] [ 348.530014] [] batadv_interface_tx+0x206/0x385 [batman_adv] [ 348.530014] [] dev_hard_start_xmit+0x1f8/0x267 [ 348.530014] [] ? validate_xmit_skb.isra.120.part.121+0x10/0x253 [ 348.530014] [] sch_direct_xmit+0x8b/0x1b3 [ 348.530014] [] __dev_queue_xmit+0x2c8/0x513 [ 348.530014] [] ? igb_xmit_frame+0x57/0x72 [igb] [ 348.530014] [] dev_queue_xmit+0xa/0xc [ 348.530014] [] br_dev_queue_push_xmit+0xeb/0xfb [bridge] [ 348.530014] [] br_forward_finish+0x29/0x74 [bridge] [ 348.530014] [] ? deliver_clone+0x3b/0x3b [bridge] [ 348.530014] [] __br_forward+0x89/0xe7 [bridge] [ 348.530014] [] ? br_dev_queue_push_xmit+0xfb/0xfb [bridge] [ 348.530014] [] deliver_clone+0x34/0x3b [bridge] [ 348.530014] [] ? br_flood+0x95/0x95 [bridge] [ 348.530014] [] br_flood+0x77/0x95 [bridge] [ 348.530014] [] br_flood_forward+0x13/0x1a [bridge] [ 348.530014] [] ? br_flood+0x95/0x95 [bridge] [ 348.530014] [] br_handle_frame_finish+0x392/0x3db [bridge] [ 348.530014] [] ? nf_iterate+0x2b/0x6b [ 348.530014] [] br_handle_frame+0x1e6/0x240 [bridge] [ 348.530014] [] ? br_handle_local_finish+0x6a/0x6a [bridge] [ 348.530014] [] __netif_receive_skb_core+0x43a/0x66b [ 348.530014] [] ? br_handle_frame_finish+0x3db/0x3db [bridge] [ 348.530014] [] ? resched_curr+0x19/0x37 [ 348.530014] [] ? check_preempt_wakeup+0xbf/0xfe [ 348.530014] [] ? ktime_get_with_offset+0x5c/0xfc [ 348.530014] [] __netif_receive_skb+0x47/0x55 [ 348.530014] [] netif_receive_skb_internal+0x40/0x5a [ 348.530014] [] napi_gro_receive+0x3a/0x94 [ 348.530014] [] igb_poll+0x6fd/0x9ad [igb] [ 348.530014] [] ? swake_up_locked+0x14/0x26 [ 348.530014] [] net_rx_action+0xde/0x250 [ 348.530014] [] __do_softirq+0x8a/0x163 [ 348.530014] [] ? __hrtimer_tasklet_trampoline+0x19/0x19 [ 348.530014] [] do_softirq_own_stack+0x26/0x2c [ 348.530014] [ 348.530014] [] irq_exit+0x31/0x6f [ 348.530014] [] do_IRQ+0x8d/0xa0 [ 348.530014] [] common_interrupt+0x2c/0x40 [ 348.530014] Code: e7 8c 00 66 81 ff 88 00 75 12 85 d2 75 0e b2 c3 b8 83 e9 29 f9 e8 a7 5f f9 c6 eb 74 66 81 e3 8c 005 [ 348.530014] EIP: [] ieee80211_mps_set_frame_flags+0x40/0xaa [mac80211] SS:ESP 0068:f6409a40 [ 348.530014] CR2: 0000000000020040 [ 348.530014] ---[ end trace 48556ac26779732e ]--- [ 348.530014] Kernel panic - not syncing: Fatal exception in interrupt [ 348.530014] Kernel Offset: disabled Cc: stable@vger.kernel.org Reported-by: Fred Veldini Tested-by: Fred Veldini Signed-off-by: Bob Copeland --- From git history, this appears to go back to earliest commits adding authsae support. I reproduced in a 4.4 kernel, which predates all of the mesh pathtable rework, though it is harder to hit there than in the current code. This is slightly different from the version Fred tested (CCed) -- the version I sent him didn't have the BLOCK_MPATH part, but that seems necessary to me to prevent re-adding in hwmp_route_info_get(). But I can reproduce the above crash and this patch fixes it for me. --- net/mac80211/mesh_pathtbl.c | 4 ++++ net/mac80211/sta_info.c | 10 ++++++++++ net/mac80211/sta_info.h | 2 ++ 3 files changed, 16 insertions(+) diff --git a/net/mac80211/mesh_pathtbl.c b/net/mac80211/mesh_pathtbl.c index 6db2ddfa0695..798c522b6522 100644 --- a/net/mac80211/mesh_pathtbl.c +++ b/net/mac80211/mesh_pathtbl.c @@ -87,6 +87,10 @@ void mesh_path_assign_nexthop(struct mesh_path *mpath, struct sta_info *sta) struct ieee80211_hdr *hdr; unsigned long flags; + /* sta is being deleted, don't add back */ + if (test_sta_flag(sta, WLAN_STA_BLOCK_MPATH)) + return; + rcu_assign_pointer(mpath->next_hop, sta); spin_lock_irqsave(&mpath->frame_queue.lock, flags); diff --git a/net/mac80211/sta_info.c b/net/mac80211/sta_info.c index 5ccfdbd406bd..f2a1bfa53c04 100644 --- a/net/mac80211/sta_info.c +++ b/net/mac80211/sta_info.c @@ -914,6 +914,16 @@ static int __must_check __sta_info_destroy_part1(struct sta_info *sta) list_del_rcu(&sta->list); sta->removed = true; + /* + * Remove all path table references to the station (and prevent + * adding new ones) so that new mesh transmissions won't use it + * as a next hop. + */ + if (ieee80211_vif_is_mesh(&sdata->vif)) { + set_sta_flag(sta, WLAN_STA_BLOCK_MPATH); + mesh_path_flush_by_nexthop(sta); + } + drv_sta_pre_rcu_remove(local, sta->sdata, sta); if (sdata->vif.type == NL80211_IFTYPE_AP_VLAN && diff --git a/net/mac80211/sta_info.h b/net/mac80211/sta_info.h index c8b8ccc370eb..65ecd3746c3b 100644 --- a/net/mac80211/sta_info.h +++ b/net/mac80211/sta_info.h @@ -70,6 +70,7 @@ * @WLAN_STA_MPSP_RECIPIENT: local STA is recipient of a MPSP. * @WLAN_STA_PS_DELIVER: station woke up, but we're still blocking TX * until pending frames are delivered + * @WLAN_STA_BLOCK_MPATH: block adding this station as a mesh path * * @NUM_WLAN_STA_FLAGS: number of defined flags */ @@ -100,6 +101,7 @@ enum ieee80211_sta_info_flags { WLAN_STA_MPSP_OWNER, WLAN_STA_MPSP_RECIPIENT, WLAN_STA_PS_DELIVER, + WLAN_STA_BLOCK_MPATH, NUM_WLAN_STA_FLAGS, };