mbox series

[v5,00/21] ibmvfc: initial MQ development/enablement

Message ID 20210114203148.246656-1-tyreld@linux.ibm.com (mailing list archive)
Headers show
Series ibmvfc: initial MQ development/enablement | expand

Message

Tyrel Datwyler Jan. 14, 2021, 8:31 p.m. UTC
Recent updates in pHyp Firmware and VIOS releases provide new infrastructure
towards enabling Subordinate Command Response Queues (Sub-CRQs) such that each
Sub-CRQ is a channel backed by an actual hardware queue in the FC stack on the
partner VIOS. Sub-CRQs are registered with the firmware via hypercalls and then
negotiated with the VIOS via new Management Datagrams (MADs) for channel setup.

This initial implementation adds the necessary Sub-CRQ framework and implements
the new MADs for negotiating and assigning a set of Sub-CRQs to associated VIOS
HW backed channels.

This latest series is completely rebased and reimplemented on top of the recent
("ibmvfc: MQ prepartory locking work") series. [1]

[1] https://lore.kernel.org/linux-scsi/20210106201835.1053593-1-tyreld@linux.ibm.com/

changes in v5:
* Addressed comments from brking in following patches:
* Patch 18: Drop queue lock in loop after sending cancel event
	    Remove cancel event from list after completion
	    Return -EIO on unknown failure
* Patch 21: Removed can_queue rebase artifact and range check user supplied
            nr_scsi_hw_queue value

changes in v4:
* Series rebased and reworked on top of previous ibmvfc locking series
* Dropped all previous Reviewed-by tags

changes in v3:
* Patch 4: changed firmware support logging to dev_warn_once
* Patch 6: adjusted locking
* Patch 15: dropped logging verbosity, moved cancel event tracking into subqueue
* Patch 17: removed write permission for migration module parameters
	    drive hard reset after update to num of scsi channels

changes in v2:
* Patch 4: NULL'd scsi_scrq reference after deallocation
* Patch 6: Added switch case to handle XPORT event
* Patch 9: fixed ibmvfc_event leak and double free
* added support for cancel command with MQ
* added parameter toggles for MQ settings

Tyrel Datwyler (21):
  ibmvfc: add vhost fields and defaults for MQ enablement
  ibmvfc: move event pool init/free routines
  ibmvfc: init/free event pool during queue allocation/free
  ibmvfc: add size parameter to ibmvfc_init_event_pool
  ibmvfc: define hcall wrapper for registering a Sub-CRQ
  ibmvfc: add Subordinate CRQ definitions
  ibmvfc: add alloc/dealloc routines for SCSI Sub-CRQ Channels
  ibmvfc: add Sub-CRQ IRQ enable/disable routine
  ibmvfc: add handlers to drain and complete Sub-CRQ responses
  ibmvfc: define Sub-CRQ interrupt handler routine
  ibmvfc: map/request irq and register Sub-CRQ interrupt handler
  ibmvfc: implement channel enquiry and setup commands
  ibmvfc: advertise client support for using hardware channels
  ibmvfc: set and track hw queue in ibmvfc_event struct
  ibmvfc: send commands down HW Sub-CRQ when channelized
  ibmvfc: register Sub-CRQ handles with VIOS during channel setup
  ibmvfc: add cancel mad initialization helper
  ibmvfc: send Cancel MAD down each hw scsi channel
  ibmvfc: purge scsi channels after transport loss/reset
  ibmvfc: enable MQ and set reasonable defaults
  ibmvfc: provide modules parameters for MQ settings

 drivers/scsi/ibmvscsi/ibmvfc.c | 917 ++++++++++++++++++++++++++++-----
 drivers/scsi/ibmvscsi/ibmvfc.h |  39 ++
 2 files changed, 828 insertions(+), 128 deletions(-)

Comments

Brian King Jan. 14, 2021, 10:47 p.m. UTC | #1
Tyrel,

I think this patch series is looking pretty good. I don't think we need
to wait for resolution of the can_queue issue being discussed, since
that is an issue that exists prior to this patch series and this patch
series doesn't make the issue any worse. Let's work that separately.

Thanks,

Brian
Martin K. Petersen Jan. 15, 2021, 3:31 a.m. UTC | #2
Tyrel,

> Recent updates in pHyp Firmware and VIOS releases provide new
> infrastructure towards enabling Subordinate Command Response Queues
> (Sub-CRQs) such that each Sub-CRQ is a channel backed by an actual
> hardware queue in the FC stack on the partner VIOS. Sub-CRQs are
> registered with the firmware via hypercalls and then negotiated with
> the VIOS via new Management Datagrams (MADs) for channel setup.
>
> This initial implementation adds the necessary Sub-CRQ framework and
> implements the new MADs for negotiating and assigning a set of
> Sub-CRQs to associated VIOS HW backed channels.

Applied to 5.12/scsi-staging, thanks!
Martin K. Petersen Jan. 21, 2021, 3:34 a.m. UTC | #3
On Thu, 14 Jan 2021 14:31:27 -0600, Tyrel Datwyler wrote:

> Recent updates in pHyp Firmware and VIOS releases provide new infrastructure
> towards enabling Subordinate Command Response Queues (Sub-CRQs) such that each
> Sub-CRQ is a channel backed by an actual hardware queue in the FC stack on the
> partner VIOS. Sub-CRQs are registered with the firmware via hypercalls and then
> negotiated with the VIOS via new Management Datagrams (MADs) for channel setup.
> 
> This initial implementation adds the necessary Sub-CRQ framework and implements
> the new MADs for negotiating and assigning a set of Sub-CRQs to associated VIOS
> HW backed channels.
> 
> [...]

Applied to 5.12/scsi-queue, thanks!

[01/21] ibmvfc: add vhost fields and defaults for MQ enablement
        https://git.kernel.org/mkp/scsi/c/6ae208e5d2db
[02/21] ibmvfc: move event pool init/free routines
        https://git.kernel.org/mkp/scsi/c/225acf5f1aba
[03/21] ibmvfc: init/free event pool during queue allocation/free
        https://git.kernel.org/mkp/scsi/c/003d91a1393d
[04/21] ibmvfc: add size parameter to ibmvfc_init_event_pool
        https://git.kernel.org/mkp/scsi/c/bb35ecb2a949
[05/21] ibmvfc: define hcall wrapper for registering a Sub-CRQ
        https://git.kernel.org/mkp/scsi/c/9e6b6b81aafa
[06/21] ibmvfc: add Subordinate CRQ definitions
        https://git.kernel.org/mkp/scsi/c/6d07f129dce2
[07/21] ibmvfc: add alloc/dealloc routines for SCSI Sub-CRQ Channels
        https://git.kernel.org/mkp/scsi/c/3034ebe26389
[08/21] ibmvfc: add Sub-CRQ IRQ enable/disable routine
        https://git.kernel.org/mkp/scsi/c/d20046e64c09
[09/21] ibmvfc: add handlers to drain and complete Sub-CRQ responses
        https://git.kernel.org/mkp/scsi/c/1d956ad853fc
[10/21] ibmvfc: define Sub-CRQ interrupt handler routine
        https://git.kernel.org/mkp/scsi/c/80a9e8eaed63
[11/21] ibmvfc: map/request irq and register Sub-CRQ interrupt handler
        https://git.kernel.org/mkp/scsi/c/39e461fddff0
[12/21] ibmvfc: implement channel enquiry and setup commands
        https://git.kernel.org/mkp/scsi/c/e95eef3fc0bc
[13/21] ibmvfc: advertise client support for using hardware channels
        https://git.kernel.org/mkp/scsi/c/c53408baa502
[14/21] ibmvfc: set and track hw queue in ibmvfc_event struct
        https://git.kernel.org/mkp/scsi/c/cb72477be729
[15/21] ibmvfc: send commands down HW Sub-CRQ when channelized
        https://git.kernel.org/mkp/scsi/c/31750fbd7b6d
[16/21] ibmvfc: register Sub-CRQ handles with VIOS during channel setup
        https://git.kernel.org/mkp/scsi/c/b88a5d9b7f56
[17/21] ibmvfc: add cancel mad initialization helper
        https://git.kernel.org/mkp/scsi/c/a61236da7f9c
[18/21] ibmvfc: send Cancel MAD down each hw scsi channel
        https://git.kernel.org/mkp/scsi/c/a835f386f970
[19/21] ibmvfc: purge scsi channels after transport loss/reset
        https://git.kernel.org/mkp/scsi/c/7eb3ccd884ae
[20/21] ibmvfc: enable MQ and set reasonable defaults
        https://git.kernel.org/mkp/scsi/c/9000cb998bcf
[21/21] ibmvfc: provide modules parameters for MQ settings
        https://git.kernel.org/mkp/scsi/c/032d1900869f