From patchwork Wed Feb 15 00:25:52 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 9573135 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 8702B600F6 for ; Wed, 15 Feb 2017 00:27:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7A3AA205D6 for ; Wed, 15 Feb 2017 00:27:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6F6202841A; Wed, 15 Feb 2017 00:27:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 01BD32842E for ; Wed, 15 Feb 2017 00:27:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751452AbdBOA1a (ORCPT ); Tue, 14 Feb 2017 19:27:30 -0500 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:36970 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750923AbdBOA12 (ORCPT ); Tue, 14 Feb 2017 19:27:28 -0500 X-IronPort-AV: E=Sophos;i="5.35,163,1483977600"; d="scan'208";a="81774864" Received: from unknown (HELO milsmgep15.sandisk.com) ([63.163.107.21]) by ob1.hgst.iphmx.com with ESMTP; 15 Feb 2017 08:43:12 +0800 Received: from MILHUBIP04.sdcorp.global.sandisk.com (Unknown_Domain [10.201.67.162]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id 99.E4.57638.B20A3A85; Tue, 14 Feb 2017 16:26:19 -0800 (PST) Received: from milsmgip11.sandisk.com (10.177.8.100) by MILHUBIP04.sdcorp.global.sandisk.com (10.177.9.97) with Microsoft SMTP Server id 14.3.319.2; Tue, 14 Feb 2017 16:26:15 -0800 X-AuditID: 0ac94369-26dee9800001e126-70-58a3a02b16ff Received: from exp-402881.sdcorp.global.sandisk.com ( [10.177.9.6]) by (Symantec Messaging Gateway) with SMTP id FB.7D.11415.720A3A85; Tue, 14 Feb 2017 16:26:15 -0800 (PST) From: Bart Van Assche To: "Nicholas A . Bellinger" CC: , Bart Van Assche , Hannes Reinecke , David Disseldorp Subject: [PATCH v6 13/33] target: Avoid that XCOPY commands trigger a deadlock Date: Tue, 14 Feb 2017 16:25:52 -0800 Message-ID: <20170215002612.14566-14-bart.vanassche@sandisk.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170215002612.14566-1-bart.vanassche@sandisk.com> References: <20170215002612.14566-1-bart.vanassche@sandisk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrCLMWRmVeSWpSXmKPExsXCddJ5ka72gsURBk07dSy+/p/OYrHgzV42 i7bVZxgtWpe+ZXJg8bi//QiTx/otV1k8Np+u9vi8SS6AJYrLJiU1J7MstUjfLoEr4/nRwoKn thUr/xxlbGD8ZdrFyMkhIWAisefBA8YuRi4OIYGlTBLfpl6AcnYwSvQv7GSEqVr5YikbiC0k sJFR4t12dxCbTcBI4tv7mSwgtgiQvbpnGRNIM7PAdEaJze2bWEESwgJ+EnfObWIGsVkEVCV+ 9f8Ds3kFHCQW7uxihlggL7Gr7SJYPSdQfMaOk0BDOYCW2Us8OmMNMlNCoIlVYtubRywQvYIS J2c+AbOZBSQkDr54wQxxnLrEySXzmSYwCs1CUjYLSdkCRqZVjGK5mTnFuempBYamesWJeSmZ xdl6yfm5mxghYZ25g/HuE+9DjAIcjEo8vCdkF0cIsSaWFVfmHmKU4GBWEuFlqwcK8aYkVlal FuXHF5XmpBYfYpTmYFES5z0nMzVCSCA9sSQ1OzW1ILUIJsvEwSnVwJgUKFc06++2tsLiWq16 vfRf1ZEOS08fXiL78PDurmMeb652inhvMH+Z+Wt9grtGa0hB8HyzpV6qUz9d/9f0eftD0+dv 17yU39Xd8dpie1bRd+H+nS1rJ8S3nd/7IMBm6yc5UX9//U2X5WSSdlcXSlzq9TwyJ/lEhFRF fsSRne6TNVpSuG7x7FZiKc5INNRiLipOBAAM1JwYZwIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrCJMWRmVeSWpSXmKPExsXCtZGTTVd9weIIg3OTJSwO/mxjtPj6fzqL xYI3e9ks2lafYbRoXfqWyYHV4/72I0we09acZ/JYv+Uqi8fm09UenzfJBbBGcdmkpOZklqUW 6dslcGU8P1pY8NS2YuWfo4wNjL9Muxg5OSQETCRWvljK1sXIxSEksJ5R4vjBLewgCTYBI4lv 72eygNgiQPbqnmVMIEXMAtMZJU6d3QGWEBbwk7hzbhMziM0ioCrxq/8fmM0r4CBxde8lVogN 8hK72i6C2ZxA8Rk7TgL1cgBts5d4dMZ6AiP3AkaGVYxiuZk5xbnpmQWGhnrFiXkpmcXZesn5 uZsYIeEQuYPx6UTzQ4xMHJxSDYyJc59G7r6+TipJ6dm3F7rMDFpOKsFrOfa3f/LbcG2S1Jsr GovXWKi47tjjPjd8UdTq9Fr1m/9Z8nqFjvDaGJ/+Ub+b/9faXKujP/kjvsj6f9zJJmPNUL/A eZYDk+7uN2vd2iKWhE2f3fvgese7v+vl30ZPPH1bI+Dj/GVrUq53MpoEKC5IYEhTYinOSDTU Yi4qTgQAOC2mP7cBAAA= MIME-Version: 1.0 Sender: target-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Move the target_depend_item() call out of the critical section. This patch avoids that lockdep reports the following: ====================================================== [ INFO: possible circular locking dependency detected ] 4.10.0-rc4-debug+ #1 Not tainted ------------------------------------------------------- rmdir/1799 is trying to acquire lock: (&sess->cmdsn_mutex){+.+.+.}, at: [] iscsit_free_all_ooo_cmdsns+0x2d/0xb0 [iscsi_target_mod] but task is already holding lock: (&sb->s_type->i_mutex_key#15){++++++}, at: [] vfs_rmdir+0x50/0x140 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #2 (&sb->s_type->i_mutex_key#15){++++++}: [] lock_acquire+0x71/0x90 [] down_write+0x3f/0x70 [] configfs_depend_item+0x3a/0xb0 [configfs] [] target_depend_item+0x13/0x20 [target_core_mod] [] target_xcopy_locate_se_dev_e4+0xdb/0x1a0 [target_core_mod] [] target_do_xcopy+0x31c/0x8b0 [target_core_mod] [] __target_execute_cmd+0x22/0xa0 [target_core_mod] [] target_execute_cmd.part.20+0x1e9/0x290 [target_core_mod] [] target_execute_cmd+0x13/0x20 [target_core_mod] [] iscsit_execute_cmd+0x1d2/0x230 [iscsi_target_mod] [] iscsit_sequence_cmd+0x10b/0x190 [iscsi_target_mod] [] iscsit_get_rx_pdu+0x37d/0xcd0 [iscsi_target_mod] [] iscsi_target_rx_thread+0x6e/0xa0 [iscsi_target_mod] [] kthread+0x102/0x140 [] ret_from_fork+0x31/0x40 -> #1 (g_device_mutex){+.+...}: [] lock_acquire+0x71/0x90 [] mutex_lock_nested+0x5f/0x670 [] target_xcopy_locate_se_dev_e4+0x23/0x1a0 [target_core_mod] [] target_do_xcopy+0x31c/0x8b0 [target_core_mod] [] __target_execute_cmd+0x22/0xa0 [target_core_mod] [] target_execute_cmd.part.20+0x1e9/0x290 [target_core_mod] [] target_execute_cmd+0x13/0x20 [target_core_mod] [] iscsit_execute_cmd+0x1d2/0x230 [iscsi_target_mod] [] iscsit_sequence_cmd+0x10b/0x190 [iscsi_target_mod] [] iscsit_get_rx_pdu+0x37d/0xcd0 [iscsi_target_mod] [] iscsi_target_rx_thread+0x6e/0xa0 [iscsi_target_mod] [] kthread+0x102/0x140 [] ret_from_fork+0x31/0x40 -> #0 (&sess->cmdsn_mutex){+.+.+.}: [] __lock_acquire+0x10e6/0x1260 [] lock_acquire+0x71/0x90 [] mutex_lock_nested+0x5f/0x670 [] iscsit_free_all_ooo_cmdsns+0x2d/0xb0 [iscsi_target_mod] [] iscsit_close_session+0xac/0x200 [iscsi_target_mod] [] lio_tpg_close_session+0x9f/0xb0 [iscsi_target_mod] [] target_shutdown_sessions+0xc3/0xd0 [target_core_mod] [] core_tpg_del_initiator_node_acl+0x91/0x140 [target_core_mod] [] target_fabric_nacl_base_release+0x20/0x30 [target_core_mod] [] config_item_release+0x5a/0xc0 [configfs] [] config_item_put+0x1d/0x1f [configfs] [] configfs_rmdir+0x1a6/0x300 [configfs] [] vfs_rmdir+0xb7/0x140 [] do_rmdir+0x1f4/0x200 [] SyS_rmdir+0x11/0x20 [] entry_SYSCALL_64_fastpath+0x23/0xc6 other info that might help us debug this: Chain exists of: &sess->cmdsn_mutex --> g_device_mutex --> &sb->s_type->i_mutex_key#15 Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&sb->s_type->i_mutex_key#15); lock(g_device_mutex); lock(&sb->s_type->i_mutex_key#15); lock(&sess->cmdsn_mutex); *** DEADLOCK *** 3 locks held by rmdir/1799: #0: (sb_writers#10){.+.+.+}, at: [] mnt_want_write+0x1f/0x50 #1: (&default_group_class[depth - 1]#2/1){+.+.+.}, at: [] do_rmdir+0x15e/0x200 #2: (&sb->s_type->i_mutex_key#15){++++++}, at: [] vfs_rmdir+0x50/0x140 stack backtrace: CPU: 1 PID: 1799 Comm: rmdir Not tainted 4.10.0-rc4-debug+ #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.9.1-0-gb3ef39f-prebuilt.qemu-project.org 04/01/2014 Call Trace: dump_stack+0x86/0xc3 print_circular_bug+0x1c7/0x220 __lock_acquire+0x10e6/0x1260 lock_acquire+0x71/0x90 mutex_lock_nested+0x5f/0x670 iscsit_free_all_ooo_cmdsns+0x2d/0xb0 [iscsi_target_mod] iscsit_close_session+0xac/0x200 [iscsi_target_mod] lio_tpg_close_session+0x9f/0xb0 [iscsi_target_mod] target_shutdown_sessions+0xc3/0xd0 [target_core_mod] core_tpg_del_initiator_node_acl+0x91/0x140 [target_core_mod] target_fabric_nacl_base_release+0x20/0x30 [target_core_mod] config_item_release+0x5a/0xc0 [configfs] config_item_put+0x1d/0x1f [configfs] configfs_rmdir+0x1a6/0x300 [configfs] vfs_rmdir+0xb7/0x140 do_rmdir+0x1f4/0x200 SyS_rmdir+0x11/0x20 entry_SYSCALL_64_fastpath+0x23/0xc6 Signed-off-by: Bart Van Assche Reviewed-by: Christoph Hellwig Cc: Hannes Reinecke Cc: David Disseldorp --- drivers/target/target_core_xcopy.c | 43 ++++++++++++++++++++------------------ 1 file changed, 23 insertions(+), 20 deletions(-) diff --git a/drivers/target/target_core_xcopy.c b/drivers/target/target_core_xcopy.c index cac5a20a4de0..0355dced0682 100644 --- a/drivers/target/target_core_xcopy.c +++ b/drivers/target/target_core_xcopy.c @@ -60,6 +60,8 @@ static int target_xcopy_locate_se_dev_e4(const unsigned char *dev_wwn, unsigned char tmp_dev_wwn[XCOPY_NAA_IEEE_REGEX_LEN]; int rc; + *found_dev = NULL; + mutex_lock(&g_device_mutex); list_for_each_entry(se_dev, &g_device_list, g_dev_node) { @@ -69,32 +71,33 @@ static int target_xcopy_locate_se_dev_e4(const unsigned char *dev_wwn, memset(&tmp_dev_wwn[0], 0, XCOPY_NAA_IEEE_REGEX_LEN); target_xcopy_gen_naa_ieee(se_dev, &tmp_dev_wwn[0]); - rc = memcmp(&tmp_dev_wwn[0], dev_wwn, XCOPY_NAA_IEEE_REGEX_LEN); - if (rc != 0) + if (memcmp(&tmp_dev_wwn[0], dev_wwn, XCOPY_NAA_IEEE_REGEX_LEN)) continue; - *found_dev = se_dev; - pr_debug("XCOPY 0xe4: located se_dev: %p\n", se_dev); + if (target_get_device(se_dev)) + *found_dev = se_dev; + break; + } + mutex_unlock(&g_device_mutex); - rc = target_depend_item(&se_dev->dev_group.cg_item); - if (rc != 0) { - pr_err("configfs_depend_item attempt failed:" - " %d for se_dev: %p\n", rc, se_dev); - mutex_unlock(&g_device_mutex); - return rc; - } + if (*found_dev == NULL) { + pr_debug_ratelimited("Unable to locate 0xe4 descriptor for EXTENDED_COPY\n"); + return -EINVAL; + } - pr_debug("Called configfs_depend_item for se_dev: %p" - " se_dev->se_dev_group: %p\n", se_dev, - &se_dev->dev_group); + pr_debug("XCOPY 0xe4: located se_dev: %p\n", se_dev); - mutex_unlock(&g_device_mutex); - return 0; - } - mutex_unlock(&g_device_mutex); + rc = target_depend_item(&se_dev->dev_group.cg_item); + if (rc != 0) + pr_err("configfs_depend_item attempt failed: %d for se_dev: %p\n", + rc, se_dev); + else + pr_debug("Called configfs_depend_item for se_dev: %p se_dev->se_dev_group: %p\n", + se_dev, &se_dev->dev_group); - pr_debug_ratelimited("Unable to locate 0xe4 descriptor for EXTENDED_COPY\n"); - return -EINVAL; + target_put_device(se_dev); + + return rc; } static int target_xcopy_parse_tiddesc_e4(struct se_cmd *se_cmd, struct xcopy_op *xop,