diff mbox

[v4,09/37] target: Make it possible to specify I_T nexus for SCSI abort

Message ID 20170208222507.25715-10-bart.vanassche@sandisk.com (mailing list archive)
State Superseded
Headers show

Commit Message

Bart Van Assche Feb. 8, 2017, 10:24 p.m. UTC 
target_submit_tmr() only supports the I_T_L nexus for SCSI abort
and other task management functions. Make it possible for target
drivers to specify I_T nexus for SCSI abort by passing the
TARGET_SCF_IGNORE_TMR_LUN flag to target_submit_tmr().

Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Cc: Himanshu Madhani <himanshu.madhani@cavium.com>
Cc: Giridhar Malavali <giridhar.malavali@cavium.com>
---
 drivers/target/target_core_tmr.c       | 16 +++++++++++++++-
 drivers/target/target_core_transport.c | 30 +++++++++++++++++++++++-------
 include/target/target_core_base.h      |  1 +
 3 files changed, 39 insertions(+), 8 deletions(-)

Comments

Nicholas A. Bellinger Feb. 9, 2017, 9:39 a.m. UTC | #1 
On Wed, 2017-02-08 at 14:24 -0800, Bart Van Assche wrote:
> target_submit_tmr() only supports the I_T_L nexus for SCSI abort
> and other task management functions. Make it possible for target
> drivers to specify I_T nexus for SCSI abort by passing the
> TARGET_SCF_IGNORE_TMR_LUN flag to target_submit_tmr().
> 
> Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
> Reviewed-by: Hannes Reinecke <hare@suse.com>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> Cc: Himanshu Madhani <himanshu.madhani@cavium.com>
> Cc: Giridhar Malavali <giridhar.malavali@cavium.com>
> ---
>  drivers/target/target_core_tmr.c       | 16 +++++++++++++++-
>  drivers/target/target_core_transport.c | 30 +++++++++++++++++++++++-------
>  include/target/target_core_base.h      |  1 +
>  3 files changed, 39 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/target/target_core_tmr.c b/drivers/target/target_core_tmr.c
> index 311dc3c2f1dc..367799b4dde1 100644
> --- a/drivers/target/target_core_tmr.c
> +++ b/drivers/target/target_core_tmr.c
> @@ -149,6 +149,13 @@ static bool __target_check_io_state(struct se_cmd *se_cmd,
>  	return kref_get_unless_zero(&se_cmd->cmd_kref);
>  }
>  
> +/**
> + * core_tmr_abort_task - abort a SCSI command
> + * @dev: LUN specified in task management function or NULL if no LUN has been
> + *	 specified.
> + * @tmr: Task management function.
> + * @se_sess: Session a.k.a. I_T nexus.
> + */
>  void core_tmr_abort_task(
>  	struct se_device *dev,
>  	struct se_tmr_req *tmr,
> @@ -161,7 +168,7 @@ void core_tmr_abort_task(
>  	spin_lock_irqsave(&se_sess->sess_cmd_lock, flags);
>  	list_for_each_entry(se_cmd, &se_sess->sess_cmd_list, se_cmd_list) {
>  
> -		if (dev != se_cmd->se_dev)
> +		if (dev && dev != se_cmd->se_dev)
>  			continue;
>  
>  		/* skip task management functions, including tmr->task_cmd */
> @@ -178,6 +185,13 @@ void core_tmr_abort_task(
>  		if (!__target_check_io_state(se_cmd, se_sess, 0))
>  			continue;
>  
> +		if (!tmr->tmr_dev &&
> +		    transport_lookup_tmr_lun(tmr->task_cmd,
> +					     se_cmd->orig_fe_lun) < 0) {
> +			target_put_sess_cmd(se_cmd);
> +			continue;
> +		}
> +

There is a deadlock here.  While se_sess->sess_cmd_lock is held and
transport_lookup_tmr_lun() is called above, se_device->se_tmr_lock is
also taken.

However, core_tmr_drain_tmr_list() holds se_device->se_tmr_lock while
walking se_device->se_tmr_list and taking se_sess->sess_cmd_lock on each
se_tmr_req->task_cmd.

Dropping this patch for now.

--
To unsubscribe from this list: send the line "unsubscribe target-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Bart Van Assche Feb. 9, 2017, 5:24 p.m. UTC | #2 
On Thu, 2017-02-09 at 01:39 -0800, Nicholas A. Bellinger wrote:
> There is a deadlock here.  While se_sess->sess_cmd_lock is held and
> transport_lookup_tmr_lun() is called above, se_device->se_tmr_lock is
> also taken.

I'll move the transport_lookup_tmr_lun() further down.

Bart.--
To unsubscribe from this list: send the line "unsubscribe target-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Nicholas A. Bellinger Feb. 10, 2017, 1:07 a.m. UTC | #3 
On Thu, 2017-02-09 at 17:24 +0000, Bart Van Assche wrote:
> On Thu, 2017-02-09 at 01:39 -0800, Nicholas A. Bellinger wrote:
> > There is a deadlock here.  While se_sess->sess_cmd_lock is held and
> > transport_lookup_tmr_lun() is called above, se_device->se_tmr_lock is
> > also taken.
> 
> I'll move the transport_lookup_tmr_lun() further down.
> 

It should be done in transport_generic_handle_tmr().

--
To unsubscribe from this list: send the line "unsubscribe target-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Bart Van Assche Feb. 10, 2017, 1:13 a.m. UTC | #4 
On Thu, 2017-02-09 at 17:07 -0800, Nicholas A. Bellinger wrote:
> On Thu, 2017-02-09 at 17:24 +0000, Bart Van Assche wrote:
> > On Thu, 2017-02-09 at 01:39 -0800, Nicholas A. Bellinger wrote:
> > > There is a deadlock here.  While se_sess->sess_cmd_lock is held and
> > > transport_lookup_tmr_lun() is called above, se_device->se_tmr_lock is
> > > also taken.
> > 
> > I'll move the transport_lookup_tmr_lun() further down.
> 
> It should be done in transport_generic_handle_tmr().

Please explain why you think that's how it should be done. I'm not convinced.

Bart.--
To unsubscribe from this list: send the line "unsubscribe target-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Nicholas A. Bellinger Feb. 10, 2017, 7:20 a.m. UTC | #5 
On Fri, 2017-02-10 at 01:13 +0000, Bart Van Assche wrote:
> On Thu, 2017-02-09 at 17:07 -0800, Nicholas A. Bellinger wrote:
> > On Thu, 2017-02-09 at 17:24 +0000, Bart Van Assche wrote:
> > > On Thu, 2017-02-09 at 01:39 -0800, Nicholas A. Bellinger wrote:
> > > > There is a deadlock here.  While se_sess->sess_cmd_lock is held and
> > > > transport_lookup_tmr_lun() is called above, se_device->se_tmr_lock is
> > > > also taken.
> > > 
> > > I'll move the transport_lookup_tmr_lun() further down.
> > 
> > It should be done in transport_generic_handle_tmr().
> 
> Please explain why you think that's how it should be done. I'm not convinced.

Doing it 'further down' is totally broken for LUN != 0.

transport_lookup_tmr_lun() is responsible for adding se_tmr to
se_device->tmr_list, but it's also responsible for se_cmd->orig_fe_lun
assignment based on incoming unpacked_lun from fabric driver code.

Since this patch tries to call transport_lookup_tmr_lun() from within
tmr_wq context using a se_cmd->orig_fe_lun that is always zero, it's
always going to be issuing all TMRs to LUN=0.

--
To unsubscribe from this list: send the line "unsubscribe target-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Bart Van Assche Feb. 10, 2017, 6:51 p.m. UTC | #6 
On Thu, 2017-02-09 at 23:20 -0800, Nicholas A. Bellinger wrote:
> On Fri, 2017-02-10 at 01:13 +0000, Bart Van Assche wrote:
> > On Thu, 2017-02-09 at 17:07 -0800, Nicholas A. Bellinger wrote:
> > > On Thu, 2017-02-09 at 17:24 +0000, Bart Van Assche wrote:
> > > > On Thu, 2017-02-09 at 01:39 -0800, Nicholas A. Bellinger wrote:
> > > > > There is a deadlock here.  While se_sess->sess_cmd_lock is held and
> > > > > transport_lookup_tmr_lun() is called above, se_device->se_tmr_lock is
> > > > > also taken.
> > > > 
> > > > I'll move the transport_lookup_tmr_lun() further down.
> > > 
> > > It should be done in transport_generic_handle_tmr().
> > 
> > Please explain why you think that's how it should be done. I'm not convinced.
> 
> Doing it 'further down' is totally broken for LUN != 0.
> 
> transport_lookup_tmr_lun() is responsible for adding se_tmr to
> se_device->tmr_list, but it's also responsible for se_cmd->orig_fe_lun
> assignment based on incoming unpacked_lun from fabric driver code.
> 
> Since this patch tries to call transport_lookup_tmr_lun() from within
> tmr_wq context using a se_cmd->orig_fe_lun that is always zero, it's
> always going to be issuing all TMRs to LUN=0.

Your comment seems to apply to the se_cmd structure of the task management
function. But that's not what is passed to transport_lookup_tmr_lun(). The
LUN passed in my patch to that function is the LUN of the command that is
being aborted. Had you perhaps misinterpret my patch?

Bart.--
To unsubscribe from this list: send the line "unsubscribe target-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/drivers/target/target_core_tmr.c b/drivers/target/target_core_tmr.c
index 311dc3c2f1dc..367799b4dde1 100644
--- a/drivers/target/target_core_tmr.c
+++ b/drivers/target/target_core_tmr.c
@@ -149,6 +149,13 @@  static bool __target_check_io_state(struct se_cmd *se_cmd,
 	return kref_get_unless_zero(&se_cmd->cmd_kref);
 }
 
+/**
+ * core_tmr_abort_task - abort a SCSI command
+ * @dev: LUN specified in task management function or NULL if no LUN has been
+ *	 specified.
+ * @tmr: Task management function.
+ * @se_sess: Session a.k.a. I_T nexus.
+ */
 void core_tmr_abort_task(
 	struct se_device *dev,
 	struct se_tmr_req *tmr,
@@ -161,7 +168,7 @@  void core_tmr_abort_task(
 	spin_lock_irqsave(&se_sess->sess_cmd_lock, flags);
 	list_for_each_entry(se_cmd, &se_sess->sess_cmd_list, se_cmd_list) {
 
-		if (dev != se_cmd->se_dev)
+		if (dev && dev != se_cmd->se_dev)
 			continue;
 
 		/* skip task management functions, including tmr->task_cmd */
@@ -178,6 +185,13 @@  void core_tmr_abort_task(
 		if (!__target_check_io_state(se_cmd, se_sess, 0))
 			continue;
 
+		if (!tmr->tmr_dev &&
+		    transport_lookup_tmr_lun(tmr->task_cmd,
+					     se_cmd->orig_fe_lun) < 0) {
+			target_put_sess_cmd(se_cmd);
+			continue;
+		}
+
 		list_del_init(&se_cmd->se_cmd_list);
 		spin_unlock_irqrestore(&se_sess->sess_cmd_lock, flags);
 
diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c
index 084a7fbfc72e..72fd0b430380 100644
--- a/drivers/target/target_core_transport.c
+++ b/drivers/target/target_core_transport.c
@@ -1593,18 +1593,18 @@  static void target_complete_tmr_failure(struct work_struct *work)
 }
 
 /**
- * target_submit_tmr - lookup unpacked lun and submit uninitialized se_cmd
- *                     for TMR CDBs
+ * target_submit_tmr - submit a SCSI task management function to the target core
  *
  * @se_cmd: command descriptor to submit
  * @se_sess: associated se_sess for endpoint
  * @sense: pointer to SCSI sense buffer
- * @unpacked_lun: unpacked LUN to reference for struct se_lun
+ * @unpacked_lun: LUN the TMR applies to. Ignored if TARGET_SCF_IGNORE_TMR_LUN
+ *	has been set in @flags.
  * @fabric_context: fabric context for TMR req
  * @tm_type: Type of TM request
  * @gfp: gfp type for caller
  * @tag: referenced task tag for TMR_ABORT_TASK
- * @flags: submit cmd flags
+ * @flags: submit cmd flags (TARGET_SCF_*).
  *
  * Callable from all contexts.
  **/
@@ -1640,7 +1640,14 @@  int target_submit_tmr(struct se_cmd *se_cmd, struct se_session *se_sess,
 		return ret;
 	}
 
-	ret = transport_lookup_tmr_lun(se_cmd, unpacked_lun);
+	if (flags & TARGET_SCF_IGNORE_TMR_LUN) {
+		WARN_ON_ONCE(tm_type != TMR_ABORT_TASK);
+		se_cmd->se_lun = NULL;
+		se_cmd->se_dev = NULL;
+		se_cmd->se_tmr_req->tmr_dev = NULL;
+	} else {
+		ret = transport_lookup_tmr_lun(se_cmd, unpacked_lun);
+	}
 	if (ret) {
 		/*
 		 * For callback during failure handling, push this work off
@@ -1650,6 +1657,7 @@  int target_submit_tmr(struct se_cmd *se_cmd, struct se_session *se_sess,
 		schedule_work(&se_cmd->work);
 		return 0;
 	}
+	WARN_ON_ONCE(tm_type != TMR_ABORT_TASK && !se_cmd->se_dev);
 	transport_generic_handle_tmr(se_cmd);
 	return 0;
 }
@@ -3128,8 +3136,16 @@  static void target_tmr_work(struct work_struct *work)
 	transport_cmd_check_stop_to_fabric(cmd);
 }
 
-int transport_generic_handle_tmr(
-	struct se_cmd *cmd)
+/**
+ * transport_generic_handle_tmr - queue a task management function
+ *
+ * Note: task management functions for which flag TARGET_SCF_IGNORE_TMR_LUN
+ * has been set are queued on a separate workqueue. This may cause out-of-order
+ * execution of task management functions. That is fine though since the
+ * SCSI Architecture Manual does not require that task management functions
+ * are executed in order.
+ */
+int transport_generic_handle_tmr(struct se_cmd *cmd)
 {
 	unsigned long flags;
 
diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h
index 5cb7583ad701..fb87f0e5d0d6 100644
--- a/include/target/target_core_base.h
+++ b/include/target/target_core_base.h
@@ -186,6 +186,7 @@  enum target_sc_flags_table {
 	TARGET_SCF_ACK_KREF		= 0x02,
 	TARGET_SCF_UNKNOWN_SIZE		= 0x04,
 	TARGET_SCF_USE_CPUID	= 0x08,
+	TARGET_SCF_IGNORE_TMR_LUN	= 0x10,
 };
 
 /* fabric independent task management function values */