From patchwork Tue Oct 31 18:03:12 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10035071 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B9F2460327 for ; Tue, 31 Oct 2017 18:03:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B18D82851D for ; Tue, 31 Oct 2017 18:03:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A66C72884E; Tue, 31 Oct 2017 18:03:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 08FA42851D for ; Tue, 31 Oct 2017 18:03:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932625AbdJaSD2 (ORCPT ); Tue, 31 Oct 2017 14:03:28 -0400 Received: from esa4.hgst.iphmx.com ([216.71.154.42]:38505 "EHLO esa4.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932637AbdJaSDZ (ORCPT ); Tue, 31 Oct 2017 14:03:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1509473005; x=1541009005; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=5H0uPf7jzOSDQe/ph3X2r4hlC0jHtmV5n3LEQVuaYeY=; b=pGBLhpSNXXKx6JuNVxH8WCITi9KxSS7oGDMRwJHMZMPun1LNXLN9N602 0mD2CZnqFiHsUwovEzu/VzEQbQ4whZBfkPGivc0JTXumIAdw0agVj4csM v7vtQQ6iEBw6qIyW8ehlI2ep0WoXvWwZAGsR0YzsVy3R597tPN7yAYkz1 NoHqFQIrPk4pwfRDavoCOdrWReUyQ2jzCSsNgrkn0XxffIJnAomfZRgw6 JAAEPe7/+F0QSNMzZhIX0Ems0kPx51NVAFPBgp3NWVJwU+GKzN7TPzC6J xmqRxlM4HeHClPhsog52GnW5ipH9YrLehKl3z9EBtWKLqEEfqEkSucUbz Q==; X-IronPort-AV: E=Sophos;i="5.44,325,1505750400"; d="scan'208";a="60328626" Received: from h199-255-45-15.hgst.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 01 Nov 2017 02:03:23 +0800 Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP; 31 Oct 2017 11:01:08 -0700 Received: from unknown (HELO MILHUBIP03.sdcorp.global.sandisk.com) ([10.177.9.96]) by uls-op-cesaip02.wdc.com with ESMTP; 31 Oct 2017 11:03:24 -0700 Received: from milsmgip12.sandisk.com (10.177.9.6) by MILHUBIP03.sdcorp.global.sandisk.com (10.177.9.96) with Microsoft SMTP Server id 14.3.319.2; Tue, 31 Oct 2017 11:03:23 -0700 X-AuditID: 0ab1095a-d99e3980000146e4-4f-59f8baeac1e6 Received: from thinkpad-bart.int.fusionio.com ( [10.177.8.100]) by (Symantec Messaging Gateway) with SMTP id 99.A9.18148.AEAB8F95; Tue, 31 Oct 2017 11:03:22 -0700 (PDT) From: Bart Van Assche To: "Martin K . Petersen" , "James E . J . Bottomley" CC: Nicholas Bellinger , Christoph Hellwig , , Bart Van Assche , Hannes Reinecke , Mike Christie Subject: [PATCH 04/11] target: Avoid that EXTENDED COPY commands triggers lock inversion Date: Tue, 31 Oct 2017 11:03:12 -0700 Message-ID: <20171031180319.19258-5-bart.vanassche@wdc.com> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171031180319.19258-1-bart.vanassche@wdc.com> References: <20171031180319.19258-1-bart.vanassche@wdc.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrMJMWRmVeSWpSXmKPExsXCtZEjRffVrh+RBq8Oq1pc+vOF0WLBm71s FitXH2WyWDUxz2L58X9MFuuPbGC0aFt9htGidelbJgcOj/vbjzB5PDi0mcVj980GNo+PT2+x eLzfd5XNY/2WqywenzfJebQf6GYK4IjisklJzcksSy3St0vgyljwagFbQY9mxfKXu5gaGL8r djFycEgImEj0fBTtYuTiEBJYzSgxtamRpYuRk4NNQE/i1Lx9TCC2iECuxKmHN5hAipgFnjBK tPzcxw6SEBaIkFjx+hsjiM0ioCrxeO0usGZeAWuJSe+XgNVICMhLvF9wH6yGU8BG4m3PbrAa IaCaa3uXME9g5F7AyLCKUSw3M6c4Nz2zwNBIrzgxLyWzOFsvOT93EyM4mDijdjBen2h+iJGJ g1OqgXHSrEV9BVYm17c+XvWL4Vpw0WfVP/mtPUtDX53Q5Xx5e3eCmuXtRS5pH5ccmcO4Tjch 2lZu+s70y5N3PvE4s8n1Uq7WfsW4+vTXzN4K195zTOv+Ffizrde4/vbiSI4YC+1Kn5aDhV0H 98znazo6NyDvRGfetDNmv53WnXwe2pjO++Ji28JsaV4lluKMREMt5qLiRAC16ysj1gEAAA== MIME-Version: 1.0 Sender: target-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch prevents that lockdep reports the following complaint: Reviewed-by: Hannes Reinecke ====================================================== WARNING: possible circular locking dependency detected 4.12.0-rc1-dbg+ #1 Not tainted ------------------------------------------------------ rmdir/12053 is trying to acquire lock: (device_mutex#2){+.+.+.}, at: [] target_free_device+0xae/0xf0 [target_core_mod] but task is already holding lock: (&sb->s_type->i_mutex_key#14){++++++}, at: [] vfs_rmdir+0x50/0x140 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #1 (&sb->s_type->i_mutex_key#14){++++++}: lock_acquire+0x59/0x80 down_write+0x36/0x70 configfs_depend_item+0x3a/0xb0 [configfs] target_depend_item+0x13/0x20 [target_core_mod] target_xcopy_locate_se_dev_e4_iter+0x87/0x100 [target_core_mod] target_devices_idr_iter+0x16/0x20 [target_core_mod] idr_for_each+0x39/0xc0 target_for_each_device+0x36/0x50 [target_core_mod] target_xcopy_locate_se_dev_e4+0x28/0x80 [target_core_mod] target_xcopy_do_work+0x2e9/0xdd0 [target_core_mod] process_one_work+0x1ca/0x3f0 worker_thread+0x49/0x3b0 kthread+0x109/0x140 ret_from_fork+0x31/0x40 -> #0 (device_mutex#2){+.+.+.}: __lock_acquire+0x101f/0x11d0 lock_acquire+0x59/0x80 __mutex_lock+0x7e/0x950 mutex_lock_nested+0x16/0x20 target_free_device+0xae/0xf0 [target_core_mod] target_core_dev_release+0x10/0x20 [target_core_mod] config_item_put+0x6e/0xb0 [configfs] configfs_rmdir+0x1a6/0x300 [configfs] vfs_rmdir+0xb7/0x140 do_rmdir+0x1f4/0x200 SyS_rmdir+0x11/0x20 entry_SYSCALL_64_fastpath+0x23/0xc2 other info that might help us debug this: Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&sb->s_type->i_mutex_key#14); lock(device_mutex#2); lock(&sb->s_type->i_mutex_key#14); lock(device_mutex#2); *** DEADLOCK *** 3 locks held by rmdir/12053: #0: (sb_writers#10){.+.+.+}, at: [] mnt_want_write+0x1f/0x50 #1: (&sb->s_type->i_mutex_key#14/1){+.+.+.}, at: [] do_rmdir+0x15e/0x200 #2: (&sb->s_type->i_mutex_key#14){++++++}, at: [] vfs_rmdir+0x50/0x140 stack backtrace: CPU: 3 PID: 12053 Comm: rmdir Not tainted 4.12.0-rc1-dbg+ #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.0.0-prebuilt.qemu-project.org 04/01/2014 Call Trace: dump_stack+0x86/0xcf print_circular_bug+0x1c7/0x220 __lock_acquire+0x101f/0x11d0 lock_acquire+0x59/0x80 __mutex_lock+0x7e/0x950 mutex_lock_nested+0x16/0x20 target_free_device+0xae/0xf0 [target_core_mod] target_core_dev_release+0x10/0x20 [target_core_mod] config_item_put+0x6e/0xb0 [configfs] configfs_rmdir+0x1a6/0x300 [configfs] vfs_rmdir+0xb7/0x140 do_rmdir+0x1f4/0x200 SyS_rmdir+0x11/0x20 entry_SYSCALL_64_fastpath+0x23/0xc2 Signed-off-by: Bart Van Assche Cc: Nicholas Bellinger Cc: Christoph Hellwig Cc: Hannes Reinecke Cc: Mike Christie --- drivers/target/target_core_configfs.c | 18 +++++++++++++++++- include/target/target_core_base.h | 1 + 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/drivers/target/target_core_configfs.c b/drivers/target/target_core_configfs.c index 7e87d952bb7a..7d0971789f12 100644 --- a/drivers/target/target_core_configfs.c +++ b/drivers/target/target_core_configfs.c @@ -2251,13 +2251,29 @@ static struct configfs_attribute *target_core_dev_attrs[] = { NULL, }; +static void target_core_dev_release_work(struct work_struct *release_work) +{ + struct se_device *dev = + container_of(release_work, struct se_device, release_work); + + target_free_device(dev); +} + static void target_core_dev_release(struct config_item *item) { struct config_group *dev_cg = to_config_group(item); struct se_device *dev = container_of(dev_cg, struct se_device, dev_group); - target_free_device(dev); + /* + * Call target_free_device() asynchronously to avoid lock inversion + * against the configfs_depend_item() call from + * target_xcopy_locate_se_dev_e4_iter(). That call namely occurs with + * device_mutex held. This function is called from inside vfs_rmdir + * and hence is called with i_rwsem held. + */ + INIT_WORK(&dev->release_work, target_core_dev_release_work); + schedule_work(&dev->release_work); } /* diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index 516764febeb7..19da905dd2ca 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -793,6 +793,7 @@ struct se_device { struct list_head dev_tmr_list; struct workqueue_struct *tmr_wq; struct work_struct qf_work_queue; + struct work_struct release_work; struct list_head delayed_cmd_list; struct list_head state_list; struct list_head qf_cmd_list;