From patchwork Wed Feb 8 22:25:07 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 9563657 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0288B6020F for ; Wed, 8 Feb 2017 22:28:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C99B7284F2 for ; Wed, 8 Feb 2017 22:28:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BE5752852C; Wed, 8 Feb 2017 22:28:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 045E12852B for ; Wed, 8 Feb 2017 22:28:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752086AbdBHW2J (ORCPT ); Wed, 8 Feb 2017 17:28:09 -0500 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:63863 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752008AbdBHW2D (ORCPT ); Wed, 8 Feb 2017 17:28:03 -0500 X-IronPort-AV: E=Sophos;i="5.35,348,1483977600"; d="scan'208";a="76618203" Received: from unknown (HELO milsmgep15.sandisk.com) ([63.163.107.21]) by ob1.hgst.iphmx.com with ESMTP; 09 Feb 2017 06:28:22 +0800 Received: from MILHUBIP03.sdcorp.global.sandisk.com (Unknown_Domain [10.201.67.162]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id 47.54.57638.BCA9B985; Wed, 8 Feb 2017 14:25:16 -0800 (PST) Received: from milsmgip12.sandisk.com (10.177.8.100) by MILHUBIP03.sdcorp.global.sandisk.com (10.177.9.96) with Microsoft SMTP Server id 14.3.319.2; Wed, 8 Feb 2017 14:25:11 -0800 X-AuditID: 0ac94369-26dee9800001e126-fe-589b9acb4769 Received: from exp-402881.sdcorp.global.sandisk.com ( [10.177.9.6]) by (Symantec Messaging Gateway) with SMTP id 4A.42.18148.8CA9B985; Wed, 8 Feb 2017 14:25:12 -0800 (PST) From: Bart Van Assche To: Bart Van Assche CC: , Hannes Reinecke , David Disseldorp Subject: [PATCH v4 37/37] target: Avoid that XCOPY commands trigger a deadlock Date: Wed, 8 Feb 2017 14:25:07 -0800 Message-ID: <20170208222507.25715-38-bart.vanassche@sandisk.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170208222507.25715-1-bart.vanassche@sandisk.com> References: <20170208222507.25715-1-bart.vanassche@sandisk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrNLMWRmVeSWpSXmKPExsXCddJ5ke6ZWbMjDLbs5bH4+n86i8WCN3vZ LFqXvmVyYPZYv+Uqi8fm09UenzfJBTBHcdmkpOZklqUW6dslcGWs3PeQueCpbcW0/c/YGhh/ mXYxcnJICJhIrHnyiqmLkYtDSGApk0TDs52MEM52Ronlk7YxdzFygFV179eFiG9klLh09jYT SDebgJHEt/czWUBsEQEDid+9FxhBbGaBdIkNu3vZQWxhAT+Jq2t2g9ksAioSr7dcZwOxeQUc JE6f2cQEcYW8xK62i6wgNidQfMLLZ2BzhATsJV4d+AJ2kITALxaJja83sUI0C0qcnPmEBWKZ hMTBFy+YIRrUJU4umc80gVFoFpKyWUjKFjAyrWIUy83MKc5NTy0wNNUrTsxLySzO1kvOz93E CAnizB2Md594H2IU4GBU4uGtsJwdIcSaWFZcmXuIUYKDWUmEN2UaUIg3JbGyKrUoP76oNCe1 +BCjNAeLkjjvOZmpEUIC6YklqdmpqQWpRTBZJg5OqQZG9VfZ6XtC96z+tEdiowPTiR5BBYmq hw5Sbc/Wsgc8bDhku/zxXSfx6fO3er6SKTq9emX7LBXVlk8Hly9iaNi0PqUq3uLy++3R5WFr PD33+c0Jk5UsDPAqLGGzOR9oqyZcb/zyV3yhtlL1hn/su2XMXsq/irkWdEHgcuXDHV6XVy+z ZEz1W++lxFKckWioxVxUnAgAVblpe14CAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprJJMWRmVeSWpSXmKPExsXCtZGTTffErNkRBo/mM1sc/NnGaPH1/3QW iwVv9rJZtC59y+TA4jFtzXkmj/VbrrJ4bD5d7fF5k1wASxSXTUpqTmZZapG+XQJXxsp9D5kL ntpWTNv/jK2B8ZdpFyMHh4SAiUT3ft0uRi4OIYH1jBL3Lx9k72Lk5GATMJL49n4mC4gtImAg 8bv3AiOIzSyQLvFs9h0mEFtYwE/i6prd7CBzWARUJTY/tQMJ8wo4SKz6ewqsVUJAXmJX20VW EJsTKD7h5TOwMUIC9hKvDnxhnMDIvYCRYRWjWG5mTnFuemaBoZFecWJeSmZxtl5yfu4mRojn o3YwXp9ofoiRiYNTqoFR9LVrgPyu8kbGHtfpt67+XOQpkrhuXcwkL8GTL2xl5ZSfn3P/aflm e22m4UN100N9tYLphkLOKSm718Xkq3ua7slm65OtlKq8sJdvyroTnBvztPK6bygabNkzWUx3 z9wdXxoLslbs83j84g/PexO2RzvFAkIsHzqtbTpX9fff7rWrvS7n3KhSYinOSDTUYi4qTgQA jgvo1qwBAAA= MIME-Version: 1.0 Sender: target-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Move the target_depend_item() call out of the critical section. This patch avoids that lockdep reports the following: ====================================================== [ INFO: possible circular locking dependency detected ] 4.10.0-rc4-debug+ #1 Not tainted ------------------------------------------------------- rmdir/1799 is trying to acquire lock: (&sess->cmdsn_mutex){+.+.+.}, at: [] iscsit_free_all_ooo_cmdsns+0x2d/0xb0 [iscsi_target_mod] but task is already holding lock: (&sb->s_type->i_mutex_key#15){++++++}, at: [] vfs_rmdir+0x50/0x140 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #2 (&sb->s_type->i_mutex_key#15){++++++}: [] lock_acquire+0x71/0x90 [] down_write+0x3f/0x70 [] configfs_depend_item+0x3a/0xb0 [configfs] [] target_depend_item+0x13/0x20 [target_core_mod] [] target_xcopy_locate_se_dev_e4+0xdb/0x1a0 [target_core_mod] [] target_do_xcopy+0x31c/0x8b0 [target_core_mod] [] __target_execute_cmd+0x22/0xa0 [target_core_mod] [] target_execute_cmd.part.20+0x1e9/0x290 [target_core_mod] [] target_execute_cmd+0x13/0x20 [target_core_mod] [] iscsit_execute_cmd+0x1d2/0x230 [iscsi_target_mod] [] iscsit_sequence_cmd+0x10b/0x190 [iscsi_target_mod] [] iscsit_get_rx_pdu+0x37d/0xcd0 [iscsi_target_mod] [] iscsi_target_rx_thread+0x6e/0xa0 [iscsi_target_mod] [] kthread+0x102/0x140 [] ret_from_fork+0x31/0x40 -> #1 (g_device_mutex){+.+...}: [] lock_acquire+0x71/0x90 [] mutex_lock_nested+0x5f/0x670 [] target_xcopy_locate_se_dev_e4+0x23/0x1a0 [target_core_mod] [] target_do_xcopy+0x31c/0x8b0 [target_core_mod] [] __target_execute_cmd+0x22/0xa0 [target_core_mod] [] target_execute_cmd.part.20+0x1e9/0x290 [target_core_mod] [] target_execute_cmd+0x13/0x20 [target_core_mod] [] iscsit_execute_cmd+0x1d2/0x230 [iscsi_target_mod] [] iscsit_sequence_cmd+0x10b/0x190 [iscsi_target_mod] [] iscsit_get_rx_pdu+0x37d/0xcd0 [iscsi_target_mod] [] iscsi_target_rx_thread+0x6e/0xa0 [iscsi_target_mod] [] kthread+0x102/0x140 [] ret_from_fork+0x31/0x40 -> #0 (&sess->cmdsn_mutex){+.+.+.}: [] __lock_acquire+0x10e6/0x1260 [] lock_acquire+0x71/0x90 [] mutex_lock_nested+0x5f/0x670 [] iscsit_free_all_ooo_cmdsns+0x2d/0xb0 [iscsi_target_mod] [] iscsit_close_session+0xac/0x200 [iscsi_target_mod] [] lio_tpg_close_session+0x9f/0xb0 [iscsi_target_mod] [] target_shutdown_sessions+0xc3/0xd0 [target_core_mod] [] core_tpg_del_initiator_node_acl+0x91/0x140 [target_core_mod] [] target_fabric_nacl_base_release+0x20/0x30 [target_core_mod] [] config_item_release+0x5a/0xc0 [configfs] [] config_item_put+0x1d/0x1f [configfs] [] configfs_rmdir+0x1a6/0x300 [configfs] [] vfs_rmdir+0xb7/0x140 [] do_rmdir+0x1f4/0x200 [] SyS_rmdir+0x11/0x20 [] entry_SYSCALL_64_fastpath+0x23/0xc6 other info that might help us debug this: Chain exists of: &sess->cmdsn_mutex --> g_device_mutex --> &sb->s_type->i_mutex_key#15 Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&sb->s_type->i_mutex_key#15); lock(g_device_mutex); lock(&sb->s_type->i_mutex_key#15); lock(&sess->cmdsn_mutex); *** DEADLOCK *** 3 locks held by rmdir/1799: #0: (sb_writers#10){.+.+.+}, at: [] mnt_want_write+0x1f/0x50 #1: (&default_group_class[depth - 1]#2/1){+.+.+.}, at: [] do_rmdir+0x15e/0x200 #2: (&sb->s_type->i_mutex_key#15){++++++}, at: [] vfs_rmdir+0x50/0x140 stack backtrace: CPU: 1 PID: 1799 Comm: rmdir Not tainted 4.10.0-rc4-debug+ #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.9.1-0-gb3ef39f-prebuilt.qemu-project.org 04/01/2014 Call Trace: dump_stack+0x86/0xc3 print_circular_bug+0x1c7/0x220 __lock_acquire+0x10e6/0x1260 lock_acquire+0x71/0x90 mutex_lock_nested+0x5f/0x670 iscsit_free_all_ooo_cmdsns+0x2d/0xb0 [iscsi_target_mod] iscsit_close_session+0xac/0x200 [iscsi_target_mod] lio_tpg_close_session+0x9f/0xb0 [iscsi_target_mod] target_shutdown_sessions+0xc3/0xd0 [target_core_mod] core_tpg_del_initiator_node_acl+0x91/0x140 [target_core_mod] target_fabric_nacl_base_release+0x20/0x30 [target_core_mod] config_item_release+0x5a/0xc0 [configfs] config_item_put+0x1d/0x1f [configfs] configfs_rmdir+0x1a6/0x300 [configfs] vfs_rmdir+0xb7/0x140 do_rmdir+0x1f4/0x200 SyS_rmdir+0x11/0x20 entry_SYSCALL_64_fastpath+0x23/0xc6 Signed-off-by: Bart Van Assche Reviewed-by: Christoph Hellwig Cc: Hannes Reinecke Cc: David Disseldorp --- drivers/target/target_core_xcopy.c | 43 ++++++++++++++++++++------------------ 1 file changed, 23 insertions(+), 20 deletions(-) diff --git a/drivers/target/target_core_xcopy.c b/drivers/target/target_core_xcopy.c index 180d8d718db5..623835371f2f 100644 --- a/drivers/target/target_core_xcopy.c +++ b/drivers/target/target_core_xcopy.c @@ -60,6 +60,8 @@ static int target_xcopy_locate_se_dev_e4(const unsigned char *dev_wwn, unsigned char tmp_dev_wwn[XCOPY_NAA_IEEE_REGEX_LEN]; int rc; + *found_dev = NULL; + mutex_lock(&g_device_mutex); list_for_each_entry(se_dev, &g_device_list, g_dev_node) { @@ -69,32 +71,33 @@ static int target_xcopy_locate_se_dev_e4(const unsigned char *dev_wwn, memset(&tmp_dev_wwn[0], 0, XCOPY_NAA_IEEE_REGEX_LEN); target_xcopy_gen_naa_ieee(se_dev, &tmp_dev_wwn[0]); - rc = memcmp(&tmp_dev_wwn[0], dev_wwn, XCOPY_NAA_IEEE_REGEX_LEN); - if (rc != 0) + if (memcmp(&tmp_dev_wwn[0], dev_wwn, XCOPY_NAA_IEEE_REGEX_LEN)) continue; - *found_dev = se_dev; - pr_debug("XCOPY 0xe4: located se_dev: %p\n", se_dev); + if (target_get_device(se_dev)) + *found_dev = se_dev; + break; + } + mutex_unlock(&g_device_mutex); - rc = target_depend_item(&se_dev->dev_group.cg_item); - if (rc != 0) { - pr_err("configfs_depend_item attempt failed:" - " %d for se_dev: %p\n", rc, se_dev); - mutex_unlock(&g_device_mutex); - return rc; - } + if (*found_dev == NULL) { + pr_debug_ratelimited("Unable to locate 0xe4 descriptor for EXTENDED_COPY\n"); + return -EINVAL; + } - pr_debug("Called configfs_depend_item for se_dev: %p" - " se_dev->se_dev_group: %p\n", se_dev, - &se_dev->dev_group); + pr_debug("XCOPY 0xe4: located se_dev: %p\n", se_dev); - mutex_unlock(&g_device_mutex); - return 0; - } - mutex_unlock(&g_device_mutex); + rc = target_depend_item(&se_dev->dev_group.cg_item); + if (rc != 0) + pr_err("configfs_depend_item attempt failed: %d for se_dev: %p\n", + rc, se_dev); + else + pr_debug("Called configfs_depend_item for se_dev: %p se_dev->se_dev_group: %p\n", + se_dev, &se_dev->dev_group); - pr_debug_ratelimited("Unable to locate 0xe4 descriptor for EXTENDED_COPY\n"); - return -EINVAL; + target_put_device(se_dev); + + return rc; } static int target_xcopy_parse_tiddesc_e4(struct se_cmd *se_cmd, struct xcopy_op *xop,