| Message ID | 20210810034305.63997-1-mbloch@nvidia.com (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | a5397d68b2dbdb8f725ab2ff420171fbde39dbe5 |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | [net-next,v2] net/sched: cls_api, reset flags on replay | expand |
| Context | Check | Description |
|---|---|---|
| netdev/cover_letter | success | Link |
| netdev/fixes_present | success | Link |
| netdev/patch_count | success | Link |
| netdev/tree_selection | success | Clearly marked for net-next |
| netdev/subject_prefix | success | Link |
| netdev/cc_maintainers | fail | 1 blamed authors not CCed: davem@davemloft.net; 2 maintainers not CCed: davem@davemloft.net kuba@kernel.org |
| netdev/source_inline | success | Was 0 now: 0 |
| netdev/verify_signedoff | success | Link |
| netdev/module_param | success | Was 0 now: 0 |
| netdev/build_32bit | success | Errors and warnings before: 8 this patch: 8 |
| netdev/kdoc | success | Errors and warnings before: 0 this patch: 0 |
| netdev/verify_fixes | success | Link |
| netdev/checkpatch | success | total: 0 errors, 0 warnings, 0 checks, 15 lines checked |
| netdev/build_allmodconfig_warn | success | Errors and warnings before: 8 this patch: 8 |
| netdev/header_inline | success | Link |
On Tue, Aug 10, 2021 at 03:43:05AM +0000, Mark Bloch wrote: > tc_new_tfilter() can replay a request if it got EAGAIN. The cited commit > didn't account for this when it converted TC action ->init() API > to use flags instead of parameters. This can lead to passing stale flags > down the call chain which results in trying to lock rtnl when it's > already locked, deadlocking the entire system. > > Fix by making sure to reset flags on each replay. [...] > Fixes: 695176bfe5de ("net_sched: refactor TC action init API") > Signed-off-by: Mark Bloch <mbloch@nvidia.com> > Reviewed-by: Vlad Buslov <vladbu@nvidia.com> Reviewed-by: Ido Schimmel <idosch@nvidia.com> Tested-by: Ido Schimmel <idosch@nvidia.com>
Hello: This patch was applied to netdev/net-next.git (refs/heads/master): On Tue, 10 Aug 2021 03:43:05 +0000 you wrote: > tc_new_tfilter() can replay a request if it got EAGAIN. The cited commit > didn't account for this when it converted TC action ->init() API > to use flags instead of parameters. This can lead to passing stale flags > down the call chain which results in trying to lock rtnl when it's > already locked, deadlocking the entire system. > > Fix by making sure to reset flags on each replay. > > [...] Here is the summary with links: - [net-next,v2] net/sched: cls_api, reset flags on replay https://git.kernel.org/netdev/net-next/c/a5397d68b2db You are awesome, thank you! -- Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html
diff --git a/net/sched/cls_api.c b/net/sched/cls_api.c index 69185e311422..4a7043a4e5d6 100644 --- a/net/sched/cls_api.c +++ b/net/sched/cls_api.c @@ -1949,7 +1949,7 @@ static int tc_new_tfilter(struct sk_buff *skb, struct nlmsghdr *n, int err; int tp_created; bool rtnl_held = false; - u32 flags = 0; + u32 flags; if (!netlink_ns_capable(skb, net->user_ns, CAP_NET_ADMIN)) return -EPERM; @@ -1970,6 +1970,7 @@ static int tc_new_tfilter(struct sk_buff *skb, struct nlmsghdr *n, tp = NULL; cl = 0; block = NULL; + flags = 0; if (prio == 0) { /* If no priority is provided by the user,
tc_new_tfilter() can replay a request if it got EAGAIN. The cited commit didn't account for this when it converted TC action ->init() API to use flags instead of parameters. This can lead to passing stale flags down the call chain which results in trying to lock rtnl when it's already locked, deadlocking the entire system. Fix by making sure to reset flags on each replay. ============================================ WARNING: possible recursive locking detected 5.14.0-rc3-custom-49011-g3d2bbb4f104d #447 Not tainted -------------------------------------------- tc/37605 is trying to acquire lock: ffffffff841df2f0 (rtnl_mutex){+.+.}-{3:3}, at: tc_setup_cb_add+0x14b/0x4d0 but task is already holding lock: ffffffff841df2f0 (rtnl_mutex){+.+.}-{3:3}, at: tc_new_tfilter+0xb12/0x22e0 other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock(rtnl_mutex); lock(rtnl_mutex); *** DEADLOCK *** May be due to missing lock nesting notation 1 lock held by tc/37605: #0: ffffffff841df2f0 (rtnl_mutex){+.+.}-{3:3}, at: tc_new_tfilter+0xb12/0x22e0 stack backtrace: CPU: 0 PID: 37605 Comm: tc Not tainted 5.14.0-rc3-custom-49011-g3d2bbb4f104d #447 Hardware name: Mellanox Technologies Ltd. MSN2010/SA002610, BIOS 5.6.5 08/24/2017 Call Trace: dump_stack_lvl+0x8b/0xb3 __lock_acquire.cold+0x175/0x3cb lock_acquire+0x1a4/0x4f0 __mutex_lock+0x136/0x10d0 fl_hw_replace_filter+0x458/0x630 [cls_flower] fl_change+0x25f2/0x4a64 [cls_flower] tc_new_tfilter+0xa65/0x22e0 rtnetlink_rcv_msg+0x86c/0xc60 netlink_rcv_skb+0x14d/0x430 netlink_unicast+0x539/0x7e0 netlink_sendmsg+0x84d/0xd80 ____sys_sendmsg+0x7ff/0x970 ___sys_sendmsg+0xf8/0x170 __sys_sendmsg+0xea/0x1b0 do_syscall_64+0x35/0x80 entry_SYSCALL_64_after_hwframe+0x44/0xae RIP: 0033:0x7f7b93b6c0a7 Code: 0c 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> RSP: 002b:00007ffe365b3818 EFLAGS: 00000246 ORIG_RAX: 000000000000002e RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f7b93b6c0a7 RDX: 0000000000000000 RSI: 00007ffe365b3880 RDI: 0000000000000003 RBP: 00000000610a75f6 R08: 0000000000000001 R09: 0000000000000000 R10: fffffffffffff3a9 R11: 0000000000000246 R12: 0000000000000001 R13: 0000000000000000 R14: 00007ffe365b7b58 R15: 00000000004822c0 Fixes: 695176bfe5de ("net_sched: refactor TC action init API") Signed-off-by: Mark Bloch <mbloch@nvidia.com> Reviewed-by: Vlad Buslov <vladbu@nvidia.com> --- v2: Removed Change-Id tag. --- net/sched/cls_api.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)