diff mbox series

[v2,08/10] mpt3sas: Enable interrupt coalescing on high iops

Message ID 20190520102604.3466-9-suganath-prabu.subramani@broadcom.com (mailing list archive)
State Superseded
Headers show
Series mpt3sas: Aero/Sea HBA feature addition | expand

Commit Message

Suganath Prabu S May 20, 2019, 10:26 a.m. UTC 
From: Suganath Prabu <suganath-prabu.subramani@broadcom.com>

Enable interrupt coalescing only on high iops queues when
high iops queues are enabled.

In ioc config page 1, offset 0x14 (ProductSpecific field)
is used to determine interrupt coalescing enabled/disabled on per
reply descriptor post queue group(8) basis.
If 31st bit is zero then interrupt coalescing is enabled for
all reply descriptor post queues. If 31st bit is set to one
then user can enable/disable interrupt coalescing on per reply
descriptor post queue group(8) basis. So to enable interrupt
coalescing only on first reply descriptor post queue group
(i.e. on high iops queues) set bit 0 and 31.

This configuration should reset during driver unload or shutdown
to the default settings. For this driver takes copy of default ioc page 1
and copy backs the default or unmodified ioc page1 during unload and
shutdown. so that on next driver load (e.g. if older version driver is
loaded by user), current modified changes on ioc page1 won't take effect.

Signed-off-by: Suganath Prabu <suganath-prabu.subramani@broadcom.com>
---
 drivers/scsi/mpt3sas/mpi/mpi2_cnfg.h  |  2 +-
 drivers/scsi/mpt3sas/mpt3sas_base.c   | 16 ++++++++
 drivers/scsi/mpt3sas/mpt3sas_base.h   |  5 +++
 drivers/scsi/mpt3sas/mpt3sas_config.c | 71 +++++++++++++++++++++++++++++++++++
 drivers/scsi/mpt3sas/mpt3sas_scsih.c  | 17 ++++++++-
 5 files changed, 109 insertions(+), 2 deletions(-)

Comments

Martin K. Petersen May 30, 2019, 1:05 a.m. UTC | #1 
Suganath,

> This configuration should reset during driver unload or shutdown to
> the default settings. For this driver takes copy of default ioc page 1
> and copy backs the default or unmodified ioc page1 during unload and
> shutdown. so that on next driver load (e.g. if older version driver is
> loaded by user), current modified changes on ioc page1 won't take
> effect.

What happens if the system crashes before the old page is put back?
Sreekanth Reddy May 30, 2019, 1:07 p.m. UTC | #2 
On Thu, May 30, 2019 at 6:36 AM Martin K. Petersen
<martin.petersen@oracle.com> wrote:
>
>
> Suganath,
>
> > This configuration should reset during driver unload or shutdown to
> > the default settings. For this driver takes copy of default ioc page 1
> > and copy backs the default or unmodified ioc page1 during unload and
> > shutdown. so that on next driver load (e.g. if older version driver is
> > loaded by user), current modified changes on ioc page1 won't take
> > effect.
>
> What happens if the system crashes before the old page is put back?

Any changes done on the pages from 'current region' will persist until
HBA firmware is alive. if diag reset happens or cold boot/reboot
occurs then firmware will get reinitialize and it will overwrite
current region pages with NVRAM region pages. So the changes done on
the 'current region' will be erased.

Now when system crashes, kdump utility will trigger a warm reboot into
kdump kernel. when mpt3sas driver getting loaded then it observes that
 HBA is in OPERATIONAL state. So it issues the diag reset to move the
HBA state to READY state and hence the changes made in the 'current
region' will be erased. After memory is dumped into the pre-configured
location, kdump utility will do cold reboot to boot in to standard
kernel. As cold reboot occurs HBA firmware will initialize  from day
one and if any changes done on 'current region' will overwrite with
NVRAM region.

In normal warm reboot case, driver issues the Message Unit Reset (to
move HBA state to READY state) from shutdown callback before going for
reboot operation and hence in this case driver has to copy back the
default settings.

Thanks,
Sreekanth

>
> --
> Martin K. Petersen      Oracle Linux Engineering
diff mbox series

Patch

diff --git a/drivers/scsi/mpt3sas/mpi/mpi2_cnfg.h b/drivers/scsi/mpt3sas/mpi/mpi2_cnfg.h
index a2f4a55..167d79d 100644
--- a/drivers/scsi/mpt3sas/mpi/mpi2_cnfg.h
+++ b/drivers/scsi/mpt3sas/mpi/mpi2_cnfg.h
@@ -1398,7 +1398,7 @@  typedef struct _MPI2_CONFIG_PAGE_IOC_1 {
 	U8                      PCIBusNum;                  /*0x0E */
 	U8                      PCIDomainSegment;           /*0x0F */
 	U32                     Reserved1;                  /*0x10 */
-	U32                     Reserved2;                  /*0x14 */
+	U32                     ProductSpecific;            /* 0x14 */
 } MPI2_CONFIG_PAGE_IOC_1,
 	*PTR_MPI2_CONFIG_PAGE_IOC_1,
 	Mpi2IOCPage1_t, *pMpi2IOCPage1_t;
diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c
index 6620c49..abc783f 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_base.c
+++ b/drivers/scsi/mpt3sas/mpt3sas_base.c
@@ -4440,6 +4440,7 @@  _base_static_config_pages(struct MPT3SAS_ADAPTER *ioc)
 {
 	Mpi2ConfigReply_t mpi_reply;
 	u32 iounit_pg1_flags;
+	Mpi2IOCPage1_t ioc_pg1;
 
 	ioc->nvme_abort_timeout = 30;
 	mpt3sas_config_get_manufacturing_pg0(ioc, &mpi_reply, &ioc->manu_pg0);
@@ -4472,6 +4473,21 @@  _base_static_config_pages(struct MPT3SAS_ADAPTER *ioc)
 		else
 			ioc->nvme_abort_timeout = ioc->manu_pg11.NVMeAbortTO;
 	}
+	if (ioc->high_iops_queues) {
+		mpt3sas_config_get_ioc_pg1(ioc, &mpi_reply, &ioc_pg1);
+		pr_info(
+		"%s Enable interrupt coalescing only for first reply queue group(8)\n",
+		ioc->name);
+		/* If 31st bit is zero then interrupt coalescing is enabled
+		 * for all reply descriptor post queues. If 31st bit is set
+		 * to one then user can enable/disable interrupt coalescing
+		 * on per reply descriptor post queue group(8) basis. So to
+		 * enable interrupt coalescing only on first reply descriptor
+		 * post queue group 31st bit and zero th bit is enabled.
+		 */
+		ioc_pg1.ProductSpecific = cpu_to_le32(0x80000001);
+		mpt3sas_config_set_ioc_pg1(ioc, &mpi_reply, &ioc_pg1);
+	}
 
 	mpt3sas_config_get_bios_pg2(ioc, &mpi_reply, &ioc->bios_pg2);
 	mpt3sas_config_get_bios_pg3(ioc, &mpi_reply, &ioc->bios_pg3);
diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.h b/drivers/scsi/mpt3sas/mpt3sas_base.h
index f3818e3..b5a2071 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_base.h
+++ b/drivers/scsi/mpt3sas/mpt3sas_base.h
@@ -1286,6 +1286,7 @@  struct MPT3SAS_ADAPTER {
 	Mpi2IOUnitPage0_t iounit_pg0;
 	Mpi2IOUnitPage1_t iounit_pg1;
 	Mpi2IOUnitPage8_t iounit_pg8;
+	Mpi2IOCPage1_t	ioc_pg1_copy;
 
 	struct _boot_device req_boot_device;
 	struct _boot_device req_alt_boot_device;
@@ -1634,6 +1635,10 @@  int mpt3sas_config_get_sas_iounit_pg1(struct MPT3SAS_ADAPTER *ioc,
 int mpt3sas_config_set_sas_iounit_pg1(struct MPT3SAS_ADAPTER *ioc,
 	Mpi2ConfigReply_t *mpi_reply, Mpi2SasIOUnitPage1_t *config_page,
 	u16 sz);
+int mpt3sas_config_get_ioc_pg1(struct MPT3SAS_ADAPTER *ioc, Mpi2ConfigReply_t
+	*mpi_reply, Mpi2IOCPage1_t *config_page);
+int mpt3sas_config_set_ioc_pg1(struct MPT3SAS_ADAPTER *ioc, Mpi2ConfigReply_t
+	*mpi_reply, Mpi2IOCPage1_t *config_page);
 int mpt3sas_config_get_ioc_pg8(struct MPT3SAS_ADAPTER *ioc, Mpi2ConfigReply_t
 	*mpi_reply, Mpi2IOCPage8_t *config_page);
 int mpt3sas_config_get_expander_pg0(struct MPT3SAS_ADAPTER *ioc,
diff --git a/drivers/scsi/mpt3sas/mpt3sas_config.c b/drivers/scsi/mpt3sas/mpt3sas_config.c
index b18cbbc..14a1a27 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_config.c
+++ b/drivers/scsi/mpt3sas/mpt3sas_config.c
@@ -949,6 +949,77 @@  mpt3sas_config_get_ioc_pg8(struct MPT3SAS_ADAPTER *ioc,
  out:
 	return r;
 }
+/**
+ * mpt3sas_config_get_ioc_pg1 - obtain ioc page 1
+ * @ioc: per adapter object
+ * @mpi_reply: reply mf payload returned from firmware
+ * @config_page: contents of the config page
+ * Context: sleep.
+ *
+ * Return: 0 for success, non-zero for failure.
+ */
+int
+mpt3sas_config_get_ioc_pg1(struct MPT3SAS_ADAPTER *ioc,
+	Mpi2ConfigReply_t *mpi_reply, Mpi2IOCPage1_t *config_page)
+{
+	Mpi2ConfigRequest_t mpi_request;
+	int r;
+
+	memset(&mpi_request, 0, sizeof(Mpi2ConfigRequest_t));
+	mpi_request.Function = MPI2_FUNCTION_CONFIG;
+	mpi_request.Action = MPI2_CONFIG_ACTION_PAGE_HEADER;
+	mpi_request.Header.PageType = MPI2_CONFIG_PAGETYPE_IOC;
+	mpi_request.Header.PageNumber = 1;
+	mpi_request.Header.PageVersion = MPI2_IOCPAGE8_PAGEVERSION;
+	ioc->build_zero_len_sge_mpi(ioc, &mpi_request.PageBufferSGE);
+	r = _config_request(ioc, &mpi_request, mpi_reply,
+	    MPT3_CONFIG_PAGE_DEFAULT_TIMEOUT, NULL, 0);
+	if (r)
+		goto out;
+
+	mpi_request.Action = MPI2_CONFIG_ACTION_PAGE_READ_CURRENT;
+	r = _config_request(ioc, &mpi_request, mpi_reply,
+	    MPT3_CONFIG_PAGE_DEFAULT_TIMEOUT, config_page,
+	    sizeof(*config_page));
+ out:
+	return r;
+}
+
+/**
+ * mpt3sas_config_set_ioc_pg1 - modify ioc page 1
+ * @ioc: per adapter object
+ * @mpi_reply: reply mf payload returned from firmware
+ * @config_page: contents of the config page
+ * Context: sleep.
+ *
+ * Return: 0 for success, non-zero for failure.
+ */
+int
+mpt3sas_config_set_ioc_pg1(struct MPT3SAS_ADAPTER *ioc,
+	Mpi2ConfigReply_t *mpi_reply, Mpi2IOCPage1_t *config_page)
+{
+	Mpi2ConfigRequest_t mpi_request;
+	int r;
+
+	memset(&mpi_request, 0, sizeof(Mpi2ConfigRequest_t));
+	mpi_request.Function = MPI2_FUNCTION_CONFIG;
+	mpi_request.Action = MPI2_CONFIG_ACTION_PAGE_HEADER;
+	mpi_request.Header.PageType = MPI2_CONFIG_PAGETYPE_IOC;
+	mpi_request.Header.PageNumber = 1;
+	mpi_request.Header.PageVersion = MPI2_IOCPAGE8_PAGEVERSION;
+	ioc->build_zero_len_sge_mpi(ioc, &mpi_request.PageBufferSGE);
+	r = _config_request(ioc, &mpi_request, mpi_reply,
+	    MPT3_CONFIG_PAGE_DEFAULT_TIMEOUT, NULL, 0);
+	if (r)
+		goto out;
+
+	mpi_request.Action = MPI2_CONFIG_ACTION_PAGE_WRITE_CURRENT;
+	r = _config_request(ioc, &mpi_request, mpi_reply,
+	    MPT3_CONFIG_PAGE_DEFAULT_TIMEOUT, config_page,
+	    sizeof(*config_page));
+ out:
+	return r;
+}
 
 /**
  * mpt3sas_config_get_sas_device_pg0 - obtain sas device page 0
diff --git a/drivers/scsi/mpt3sas/mpt3sas_scsih.c b/drivers/scsi/mpt3sas/mpt3sas_scsih.c
index 3e93c4a..d957f78 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_scsih.c
+++ b/drivers/scsi/mpt3sas/mpt3sas_scsih.c
@@ -9671,6 +9671,7 @@  static void scsih_remove(struct pci_dev *pdev)
 	struct _pcie_device *pcie_device, *pcienext;
 	struct workqueue_struct	*wq;
 	unsigned long flags;
+	Mpi2ConfigReply_t mpi_reply;
 
 	ioc->remove_host = 1;
 
@@ -9685,7 +9686,13 @@  static void scsih_remove(struct pci_dev *pdev)
 	spin_unlock_irqrestore(&ioc->fw_event_lock, flags);
 	if (wq)
 		destroy_workqueue(wq);
-
+	/*
+	 * Copy back the unmodified ioc page1. so that on next driver load,
+	 * current modified changes on ioc page1 won't take effect.
+	 */
+	if (ioc->is_aero_ioc)
+		mpt3sas_config_set_ioc_pg1(ioc, &mpi_reply,
+				&ioc->ioc_pg1_copy);
 	/* release all the volumes */
 	_scsih_ir_shutdown(ioc);
 	sas_remove_host(shost);
@@ -9748,6 +9755,7 @@  scsih_shutdown(struct pci_dev *pdev)
 	struct MPT3SAS_ADAPTER *ioc = shost_priv(shost);
 	struct workqueue_struct	*wq;
 	unsigned long flags;
+	Mpi2ConfigReply_t mpi_reply;
 
 	ioc->remove_host = 1;
 
@@ -9762,6 +9770,13 @@  scsih_shutdown(struct pci_dev *pdev)
 	spin_unlock_irqrestore(&ioc->fw_event_lock, flags);
 	if (wq)
 		destroy_workqueue(wq);
+	/*
+	 * Copy back the unmodified ioc page1. so that on next driver load,
+	 * current modified changes on ioc page1 won't take effect.
+	 */
+	if (ioc->is_aero_ioc)
+		mpt3sas_config_set_ioc_pg1(ioc, &mpi_reply,
+				&ioc->ioc_pg1_copy);
 
 	_scsih_ir_shutdown(ioc);
 	mpt3sas_base_detach(ioc);