From patchwork Thu Feb 14 05:46:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suganath Prabu S X-Patchwork-Id: 10811827 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9CA2C1399 for ; Thu, 14 Feb 2019 05:46:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8CA282D390 for ; Thu, 14 Feb 2019 05:46:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 808B82D3CF; Thu, 14 Feb 2019 05:46:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 11F202D390 for ; Thu, 14 Feb 2019 05:46:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391109AbfBNFq3 (ORCPT ); Thu, 14 Feb 2019 00:46:29 -0500 Received: from mail-ed1-f67.google.com ([209.85.208.67]:45132 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388674AbfBNFq3 (ORCPT ); Thu, 14 Feb 2019 00:46:29 -0500 Received: by mail-ed1-f67.google.com with SMTP id d9so3965716edh.12 for ; Wed, 13 Feb 2019 21:46:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=H/l1PG98U0MbiOKPtTCGueW/3XAsuGEJqyt11DU17L8=; b=TF+7aYFWPvgZKltryHH1QEztbIIRLxrwclJJTF9iWrMHkwAVHVR59ULWnsO0Ujh0ky Zwrudnd5ztTwzJTmFstZVtw2D2/SZXVnQ5EYgBrS1ektdiI2GkGMdXwSTK0SXCTLcwDM 7m9OuF3710IUwO5TCTc4RLlh29AZCCyaQIQlE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=H/l1PG98U0MbiOKPtTCGueW/3XAsuGEJqyt11DU17L8=; b=fB+4gO3kQkKQ/FDrmxjeiQGRBOrWJcXETIkU8uR4feEXYyNcWkkxKzZG4TjLH0lOUX F9qiRxY7ep08Gw/gDuMeR5yBeoRZ3RPivbXtIg1t4h/NvlMvcmKveCjMvRnTjKAIvc/u ad8vVYpMX5mzz31f1k9ud0oO6gNTRdLVxRjMaNHGS05WIYUW+xkOvFChsUzj+Wta9UpF dm2VsP7cM0ARbGBmX0x/K6bH8bmdgj0wTxvs2PRF4hC7DW3uZRzD/Hrc5GI110nQJp4i /NMINwvF28c7lv7dKydiN9upRSK8Mzv04azKuD6pddsJjIqlxQaFx5mRcVydeMScPHJP 5JHA== X-Gm-Message-State: AHQUAuaoZL4StF0qntChP3RMDgU0sMANDer6heLusLGzTKmH+HqnyH/c pRmQ5rpmPbQ58etj17C9BjvtPUu5WRQEtzKmUwxnGDvkIqbQ93z/qpzmtzKhKPQrvcW/zjZUfIw dmF5hNNyt41OA3r7Hx807M7CtcbOowNlEo1t7Ii/I/Fo/lbG7hq+y0BoA+ejj49YiqdEdWmzuhD pc7r+fa9cX590A9LxWAqEf X-Google-Smtp-Source: AHgI3IaSUnB/KT7pfujOLFfaqjo6fF2mr6csXC0SFl2Wu8+ia8qj4akUXp6qJyquSzhvdr0ycfeNMg== X-Received: by 2002:a05:6402:120b:: with SMTP id c11mr1480014edw.257.1550123186632; Wed, 13 Feb 2019 21:46:26 -0800 (PST) Received: from dhcp-10-123-20-72.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id h1sm311791ejx.41.2019.02.13.21.46.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 Feb 2019 21:46:25 -0800 (PST) From: Suganath Prabu To: linux-scsi@vger.kernel.org Cc: Sathya.Prakash@broadcom.com, sreekanth.reddy@broadcom.com, Suganath Prabu Subject: [PATCH 1/6] mpt3sas: Fix typo in request_desript_type. Date: Thu, 14 Feb 2019 00:46:06 -0500 Message-Id: <1550123171-15993-2-git-send-email-suganath-prabu.subramani@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1550123171-15993-1-git-send-email-suganath-prabu.subramani@broadcom.com> References: <1550123171-15993-1-git-send-email-suganath-prabu.subramani@broadcom.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Fixed typo in request_desript_type. request_desript_type --> request_descript_type. Signed-off-by: Suganath Prabu --- drivers/scsi/mpt3sas/mpt3sas_base.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c index 0a6cb8f..4184d96 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.c +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c @@ -1395,7 +1395,7 @@ _base_interrupt(int irq, void *bus_id) struct adapter_reply_queue *reply_q = bus_id; union reply_descriptor rd; u32 completed_cmds; - u8 request_desript_type; + u8 request_descript_type; u16 smid; u8 cb_idx; u32 reply; @@ -1411,9 +1411,9 @@ _base_interrupt(int irq, void *bus_id) return IRQ_NONE; rpf = &reply_q->reply_post_free[reply_q->reply_post_host_index]; - request_desript_type = rpf->Default.ReplyFlags + request_descript_type = rpf->Default.ReplyFlags & MPI2_RPY_DESCRIPT_FLAGS_TYPE_MASK; - if (request_desript_type == MPI2_RPY_DESCRIPT_FLAGS_UNUSED) { + if (request_descript_type == MPI2_RPY_DESCRIPT_FLAGS_UNUSED) { atomic_dec(&reply_q->busy); return IRQ_NONE; } @@ -1426,11 +1426,11 @@ _base_interrupt(int irq, void *bus_id) goto out; reply = 0; smid = le16_to_cpu(rpf->Default.DescriptorTypeDependent1); - if (request_desript_type == + if (request_descript_type == MPI25_RPY_DESCRIPT_FLAGS_FAST_PATH_SCSI_IO_SUCCESS || - request_desript_type == + request_descript_type == MPI2_RPY_DESCRIPT_FLAGS_SCSI_IO_SUCCESS || - request_desript_type == + request_descript_type == MPI26_RPY_DESCRIPT_FLAGS_PCIE_ENCAPSULATED_SUCCESS) { cb_idx = _base_get_cb_idx(ioc, smid); if ((likely(cb_idx < MPT_MAX_CALLBACKS)) && @@ -1440,7 +1440,7 @@ _base_interrupt(int irq, void *bus_id) if (rc) mpt3sas_base_free_smid(ioc, smid); } - } else if (request_desript_type == + } else if (request_descript_type == MPI2_RPY_DESCRIPT_FLAGS_ADDRESS_REPLY) { reply = le32_to_cpu( rpf->AddressReply.ReplyFrameAddress); @@ -1486,7 +1486,7 @@ _base_interrupt(int irq, void *bus_id) (reply_q->reply_post_host_index == (ioc->reply_post_queue_depth - 1)) ? 0 : reply_q->reply_post_host_index + 1; - request_desript_type = + request_descript_type = reply_q->reply_post_free[reply_q->reply_post_host_index]. Default.ReplyFlags & MPI2_RPY_DESCRIPT_FLAGS_TYPE_MASK; completed_cmds++; @@ -1509,7 +1509,7 @@ _base_interrupt(int irq, void *bus_id) } completed_cmds = 1; } - if (request_desript_type == MPI2_RPY_DESCRIPT_FLAGS_UNUSED) + if (request_descript_type == MPI2_RPY_DESCRIPT_FLAGS_UNUSED) goto out; if (!reply_q->reply_post_host_index) rpf = reply_q->reply_post_free; From patchwork Thu Feb 14 05:46:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suganath Prabu S X-Patchwork-Id: 10811829 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 66A2D13A4 for ; Thu, 14 Feb 2019 05:46:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5573B2D38C for ; Thu, 14 Feb 2019 05:46:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4999D2D3AE; Thu, 14 Feb 2019 05:46:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DF9642D38C for ; Thu, 14 Feb 2019 05:46:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391265AbfBNFqc (ORCPT ); Thu, 14 Feb 2019 00:46:32 -0500 Received: from mail-ed1-f67.google.com ([209.85.208.67]:45135 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726129AbfBNFqc (ORCPT ); Thu, 14 Feb 2019 00:46:32 -0500 Received: by mail-ed1-f67.google.com with SMTP id d9so3965783edh.12 for ; Wed, 13 Feb 2019 21:46:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=LEd7pzhb3Qyzh2tAUBqKJkeSUbMBgDoqQjg3OAcFZKU=; b=h0CCNtrGIZ2O11wegm7zGl2l0ylE1hi7W/ud2uH9iBqkd6vYskeUpI13dJmBOkZ+TE 4dOEyi0ASHFh+HlRyTolIEBSlCBYr/ZwQ3zxQZ5kgAwZpsyV9F6TIgnChQ0ZH6Lad5gy svpH+ZpCh/YJJnNdcNkJFhVKyDA1zEFob1Zqk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=LEd7pzhb3Qyzh2tAUBqKJkeSUbMBgDoqQjg3OAcFZKU=; b=OWfsAUF1MUOKcQD+XBdlTmeJja67zocQ6Dc2nor4T68UvRrer6kL9yYsSFx9fZupZk 0kfW3TnULCLkiLjsvuwYmAiG9tGirfVsQdNIO3GbypFbtjSgOfT5Y9NztcKEKnn57e8q 2nvz4FNGKfVEjlbNqLfAobrh9hvgGGkf/E6lqB3ckJhe+8EVJYOVLzdwJ5I0AAYs/sRK PHPFeiYodGk/SBEz++sa2Pbylmm/GuygP7CW/E8cuyyhhacX+nMVl6V8My7fhGNYAHWj ynh8HnvDRBAVIi1+P2/m2WfhAyhSKRmnGwie87mEU6g0zBLt+qAbD2TGbSycBQm58uTZ RShA== X-Gm-Message-State: AHQUAuYlTl0APvwQ75dFYPVLWjxYjY1hYCLihh906NkHdT6nG9/jNrZ6 fcAaflg9i+uMXnTQdNLcp4Guz7EPqcskwBDSGDnpZ8i6+aI6CgF7j8Huj8qvJCIvYSTMOVj/N50 LpT4QyDrI6j4PUCTDZVuufXLmvHdpyofKkfGw4o7y6UDLoBbHB2TYDyl1OdBYWTZBajSsLyBnHi fLYxWykJy3FgiteYlQNxoC X-Google-Smtp-Source: AHgI3IbRtoCE0sBZH5b4a3d8JkAQvOvo4feLI1W+09ZPkggdnQwgFcWP4MOZkcp3IDM4Lcvb6pITaw== X-Received: by 2002:a50:9189:: with SMTP id g9mr1663606eda.32.1550123190005; Wed, 13 Feb 2019 21:46:30 -0800 (PST) Received: from dhcp-10-123-20-72.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id h1sm311791ejx.41.2019.02.13.21.46.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 Feb 2019 21:46:29 -0800 (PST) From: Suganath Prabu To: linux-scsi@vger.kernel.org Cc: Sathya.Prakash@broadcom.com, sreekanth.reddy@broadcom.com, Suganath Prabu Subject: [PATCH 2/6] mpt3sas: simplify interrupt handler. Date: Thu, 14 Feb 2019 00:46:07 -0500 Message-Id: <1550123171-15993-3-git-send-email-suganath-prabu.subramani@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1550123171-15993-1-git-send-email-suganath-prabu.subramani@broadcom.com> References: <1550123171-15993-1-git-send-email-suganath-prabu.subramani@broadcom.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Separate out processing of reply descriptor post queue from _base_interrupt to _base_process_reply_queue. Signed-off-by: Suganath Prabu --- drivers/scsi/mpt3sas/mpt3sas_base.c | 49 +++++++++++++++++++++++++------------ 1 file changed, 33 insertions(+), 16 deletions(-) diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c index 4184d96..076428d 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.c +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c @@ -1383,16 +1383,16 @@ union reply_descriptor { }; /** - * _base_interrupt - MPT adapter (IOC) specific interrupt handler. - * @irq: irq number (not used) - * @bus_id: bus identifier cookie == pointer to MPT_ADAPTER structure + * _base_process_reply_queue - Process reply descriptors from reply + * descriptor post queue. + * @reply_q: per IRQ's reply queue object. * - * Return: IRQ_HANDLED if processed, else IRQ_NONE. + * Return: number of reply descriptors processed from reply + * descriptor queue. */ -static irqreturn_t -_base_interrupt(int irq, void *bus_id) +static int +_base_process_reply_queue(struct adapter_reply_queue *reply_q) { - struct adapter_reply_queue *reply_q = bus_id; union reply_descriptor rd; u32 completed_cmds; u8 request_descript_type; @@ -1404,21 +1404,18 @@ _base_interrupt(int irq, void *bus_id) Mpi2ReplyDescriptorsUnion_t *rpf; u8 rc; - if (ioc->mask_interrupts) - return IRQ_NONE; - + completed_cmds = 0; if (!atomic_add_unless(&reply_q->busy, 1, 1)) - return IRQ_NONE; + return completed_cmds; rpf = &reply_q->reply_post_free[reply_q->reply_post_host_index]; request_descript_type = rpf->Default.ReplyFlags & MPI2_RPY_DESCRIPT_FLAGS_TYPE_MASK; if (request_descript_type == MPI2_RPY_DESCRIPT_FLAGS_UNUSED) { atomic_dec(&reply_q->busy); - return IRQ_NONE; + return completed_cmds; } - completed_cmds = 0; cb_idx = 0xFF; do { rd.word = le64_to_cpu(rpf->Words); @@ -1521,14 +1518,14 @@ _base_interrupt(int irq, void *bus_id) if (!completed_cmds) { atomic_dec(&reply_q->busy); - return IRQ_NONE; + return completed_cmds; } if (ioc->is_warpdrive) { writel(reply_q->reply_post_host_index, ioc->reply_post_host_index[msix_index]); atomic_dec(&reply_q->busy); - return IRQ_HANDLED; + return completed_cmds; } /* Update Reply Post Host Index. @@ -1555,7 +1552,27 @@ _base_interrupt(int irq, void *bus_id) MPI2_RPHI_MSIX_INDEX_SHIFT), &ioc->chip->ReplyPostHostIndex); atomic_dec(&reply_q->busy); - return IRQ_HANDLED; + return completed_cmds; +} + +/** + * _base_interrupt - MPT adapter (IOC) specific interrupt handler. + * @irq: irq number (not used) + * @bus_id: bus identifier cookie == pointer to MPT_ADAPTER structure + * + * Return: IRQ_HANDLED if processed, else IRQ_NONE. + */ +static irqreturn_t +_base_interrupt(int irq, void *bus_id) +{ + struct adapter_reply_queue *reply_q = bus_id; + struct MPT3SAS_ADAPTER *ioc = reply_q->ioc; + + if (ioc->mask_interrupts) + return IRQ_NONE; + + return ((_base_process_reply_queue(reply_q) > 0) ? + IRQ_HANDLED : IRQ_NONE); } /** From patchwork Thu Feb 14 05:46:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suganath Prabu S X-Patchwork-Id: 10811831 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5ED1313A4 for ; Thu, 14 Feb 2019 05:46:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4E20A2D37A for ; Thu, 14 Feb 2019 05:46:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 42BCF2D390; Thu, 14 Feb 2019 05:46:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6D5842D37A for ; Thu, 14 Feb 2019 05:46:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391959AbfBNFqh (ORCPT ); Thu, 14 Feb 2019 00:46:37 -0500 Received: from mail-ed1-f65.google.com ([209.85.208.65]:35535 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726129AbfBNFqg (ORCPT ); Thu, 14 Feb 2019 00:46:36 -0500 Received: by mail-ed1-f65.google.com with SMTP id b17so4006972eds.2 for ; Wed, 13 Feb 2019 21:46:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=T35PGQ+wRdgSJv96ODUtux7/oHYkThy3VTp+8fxkHhA=; b=UNQGT2NIg1vUWJOge4v6mlbJSf0pNtPVOeVyZidR+yOUY0HV99GIwn5KaK1gbb0ojf RjEZ/f/iJb45UQonRHARpZZLs85Gg0YvtULycbwq1ocseOgwUr9/fzNo0phgRAUu4X+V iVJrE8iQnriVoFdYKBWX646V59QqN6MeNolgo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=T35PGQ+wRdgSJv96ODUtux7/oHYkThy3VTp+8fxkHhA=; b=iKYB8prgvUJJ94d/0m9XstfhHDaoPmHHM/TXcDrLHs2QUxBBbnLtreSTLgnBdw48Eh //ElIGfAmaDIJfhXik/Y6AGdkpxkjzVsnxsEty46Lvtmxq86KYH16A3NZ43PH+FgFRNB AtKYz4/lqIAB5QfI5xC3jFr+MZZgfbZvFlO3gnVlzL9kt1JY1j1ohj5PKyWIEQUecVZJ OGZU797odG+gJOxEWnQu+l+7X7y2ypNPo6gRciW3b50AdnYTN45NaPLZiz5PvNTAFN65 L51mhJdZTqEr2gyYaGkUa7tJVZUMjtszpEU/89Cv1CUHTVi79NccnvoDTBI6kRMcmUB6 dpZA== X-Gm-Message-State: AHQUAuYO7DcKHvf43J+Espnqj/9GxVX8rZBri9ib9rEkSnseztoerEeo d/DvIVS7CimLTgMHrKZLHCwOGRrsWp9loQecrhqLj8hBkGPfzCKl6D4Vc9sn3uFXKpMcdsn2XdF M70kaY42XAzxuW2yKxezoKxXg4Ti0jeDiMouC6EJy4//eFqp8luSDwwFaSXMViMn/KuHGraDUDJ 92pm2RcU696N4pYba1ZtVt X-Google-Smtp-Source: AHgI3IbtlJO6Pv6EpuphyjWB+NodEUIg9taSRPpHIENWphH0RaHsvLt0cDODLUPlNVJ6FbR23vDkbA== X-Received: by 2002:a50:b7ad:: with SMTP id h42mr1585256ede.210.1550123193424; Wed, 13 Feb 2019 21:46:33 -0800 (PST) Received: from dhcp-10-123-20-72.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id h1sm311791ejx.41.2019.02.13.21.46.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 Feb 2019 21:46:32 -0800 (PST) From: Suganath Prabu To: linux-scsi@vger.kernel.org Cc: Sathya.Prakash@broadcom.com, sreekanth.reddy@broadcom.com, Suganath Prabu Subject: [PATCH 3/6] mpt3sas: Irq poll to avoid CPU hard lockups. Date: Thu, 14 Feb 2019 00:46:08 -0500 Message-Id: <1550123171-15993-4-git-send-email-suganath-prabu.subramani@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1550123171-15993-1-git-send-email-suganath-prabu.subramani@broadcom.com> References: <1550123171-15993-1-git-send-email-suganath-prabu.subramani@broadcom.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Issue Discription: We have seen cpu lock up issue from fields if system has greater (more than 96) logical cpu count. SAS3.0 controller (Invader series) supports at max 96 msix vector and SAS3.5 product (Ventura) supports at max 128 msix vectors. This may be a generic issue (if PCI device support completion on multiple reply queues). Let me explain it w.r.t to mpt3sas supported h/w just to simplify the problem and possible changes to handle such issues. IT HBA (mpt3sas) supports multiple reply queues in completion path. Driver creates MSI-x vectors for controller as "min of (FW supported Reply queue, Logical CPUs)". If submitter is not interrupted via completion on same CPU, there is a loop in the IO path. This behavior can cause hard/soft CPU lockups, IO timeout, system sluggish etc. Example - one CPU (e.g. CPU A) is busy submitting the IOs and another CPU (e.g. CPU B) is busy with processing the corresponding IO's reply descriptors from reply descriptor queue upon receiving the interrupts from HBA. If the CPU A is continuously pumping the IOs then always CPU B (which is executing the ISR) will see the valid reply descriptors in the reply descriptor queue and it will be continuously processing those reply descriptor in a loop without quitting the ISR handler. Mpt3sas driver will exit ISR handler if it finds unused reply descriptor in the reply descriptor queue. Since CPU A will be continuously sending the IOs, CPU B may always see a valid reply descriptor (posted by HBA Firmware after processing the IO) in the reply descriptor queue. In worst case, driver will not quit from this loop in the ISR handler. Eventually, CPU lockup will be detected by watchdog. Above mentioned behavior is not common if "rq_affinity" set to 2 or affinity_hint is honored by irqbalance as "exact". If rq_affinity is set to 2, submitter will be always interrupted via completion on same CPU. If irqbalance is using "exact" policy, interrupt will be delivered to submitter CPU. Problem statement - If CPU counts to MSI-X vectors (reply descriptor Queues) count ratio is not 1:1, we still have exposure of issue explained above and for that we don't have any solution. Exposure of soft/hard lockup if CPU count is more than MSI-x supported by device. If CPUs count to MSI-x vectors count ratio is not 1:1, (Other way, if CPU counts to MSI-x vector count ratio is something like X:1, where X > 1) then 'exact' irqbalance policy OR rq_affinity = 2 won't help to avoid CPU hard/soft lockups. There won't be any one to one mapping between CPU to MSI-x vector instead one MSI-x interrupt (or reply descriptor queue) is shared with group/set of CPUs and there is a possibility of having a loop in the IO path within that CPU group and may observe lockups. For example: Consider a system having two NUMA nodes and each node having four logical CPUs and also consider that number of MSI-x vectors enabled on the HBA is two, then CPUs count to MSI-x vector count ratio as 4:1. e.g. MSIx vector 0 is affinity to CPU 0, CPU 1, CPU 2 & CPU 3 of NUMA node 0 and MSI-x vector 1 is affinity to CPU 4, CPU 5, CPU 6 & CPU 7 of NUMA node 1. numactl --hardware available: 2 nodes (0-1) node 0 cpus: 0 1 2 3 --> MSI-x 0 node 0 size: 65536 MB node 0 free: 63176 MB node 1 cpus: 4 5 6 7 -->MSI-x 1 node 1 size: 65536 MB node 1 free: 63176 MB Assume that user started an application which uses all the CPUs of NUMA node 0 for issuing the IOs. Only one CPU from affinity list (it can be any cpu since this behavior depends upon irqbalance) CPU0 will receive the interrupts from MSIx vector 0 for all the IOs. Eventually, CPU 0 IO submission percentage will be decreasing and ISR processing percentage will be increasing as it is more busy with processing the interrupts. Gradually IO submission percentage on CPU 0 will be zero and it's ISR processing percentage will be 100 percentage as IO loop has already formed within the NUMA node 0, i.e. CPU 1, CPU 2 & CPU 3 will be continuously busy with submitting the heavy IOs and only CPU 0 is busy in the ISR path as it always find the valid reply descriptor in the reply descriptor queue. Eventually, we will observe the hard lockup here. Chances of occurring of hard/soft lockups are directly proportional to value of X. If value of X is high, then chances of observing CPU lockups is high. Solution - Fix - Use IRQ poll interface defined in " irq_poll.c". mpt3sas driver will execute ISR routine in Softirq context and it will always quit the loop based on budget provided in IRQ poll interface. In these scenarios( i.e. where CPUs count to MSI-X vectors count ratio is X:1 (where X > 1)), IRQ poll interface will avoid CPU hard lockups due to voluntary exit from the reply queue processing based on budget. Note - Only one MSI-x vector is busy doing processing. Irqstat output - IRQs / 1 second(s) IRQ# TOTAL NODE0 NODE1 NODE2 NODE3 NAME 44 122871 122871 0 0 0 IR-PCI-MSI-edge mpt3sas0-msix0 45 0 0 0 0 0 IR-PCI-MSI-edge mpt3sas0-msix1 We have used the fix only if cpu count is more than FW supported MSI-x vector Signed-off-by: Suganath Prabu --- drivers/scsi/mpt3sas/mpt3sas_base.c | 78 ++++++++++++++++++++++++++++++++++++- drivers/scsi/mpt3sas/mpt3sas_base.h | 8 ++++ 2 files changed, 84 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c index 076428d..1f358bc 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.c +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c @@ -1504,7 +1504,12 @@ _base_process_reply_queue(struct adapter_reply_queue *reply_q) MPI2_RPHI_MSIX_INDEX_SHIFT), &ioc->chip->ReplyPostHostIndex); } - completed_cmds = 1; + if (!reply_q->irq_poll_scheduled) { + reply_q->irq_poll_scheduled = true; + irq_poll_sched(&reply_q->irqpoll); + } + atomic_dec(&reply_q->busy); + return completed_cmds; } if (request_descript_type == MPI2_RPY_DESCRIPT_FLAGS_UNUSED) goto out; @@ -1570,12 +1575,67 @@ _base_interrupt(int irq, void *bus_id) if (ioc->mask_interrupts) return IRQ_NONE; - + if (reply_q->irq_poll_scheduled) + return IRQ_HANDLED; return ((_base_process_reply_queue(reply_q) > 0) ? IRQ_HANDLED : IRQ_NONE); } /** + * _base_irqpoll - IRQ poll callback handler + * @irqpoll - irq_poll object + * @budget - irq poll weight + * + * returns number of reply descriptors processed + */ +static int +_base_irqpoll(struct irq_poll *irqpoll, int budget) +{ + struct adapter_reply_queue *reply_q; + int num_entries = 0; + + reply_q = container_of(irqpoll, struct adapter_reply_queue, + irqpoll); + if (reply_q->irq_line_enable) { + disable_irq(reply_q->os_irq); + reply_q->irq_line_enable = false; + } + num_entries = _base_process_reply_queue(reply_q); + if (num_entries < budget) { + irq_poll_complete(irqpoll); + reply_q->irq_poll_scheduled = false; + reply_q->irq_line_enable = true; + enable_irq(reply_q->os_irq); + } + + return num_entries; +} + +/** + * _base_init_irqpolls - initliaze IRQ polls + * @ioc: per adapter object + * + * returns nothing + */ +static void +_base_init_irqpolls(struct MPT3SAS_ADAPTER *ioc) +{ + struct adapter_reply_queue *reply_q, *next; + + if (list_empty(&ioc->reply_queue_list)) + return; + + list_for_each_entry_safe(reply_q, next, &ioc->reply_queue_list, list) { + irq_poll_init(&reply_q->irqpoll, + ioc->hba_queue_depth/4, _base_irqpoll); + reply_q->irq_poll_scheduled = false; + reply_q->irq_line_enable = true; + reply_q->os_irq = pci_irq_vector(ioc->pdev, + reply_q->msix_index); + } +} + +/** * _base_is_controller_msix_enabled - is controller support muli-reply queues * @ioc: per adapter object * @@ -1613,6 +1673,17 @@ mpt3sas_base_sync_reply_irqs(struct MPT3SAS_ADAPTER *ioc) /* TMs are on msix_index == 0 */ if (reply_q->msix_index == 0) continue; + if (reply_q->irq_poll_scheduled) { + /* Calling irq_poll_disable will wait for any pending + * callbacks to have completed. + */ + irq_poll_disable(&reply_q->irqpoll); + irq_poll_enable(&reply_q->irqpoll); + reply_q->irq_poll_scheduled = false; + reply_q->irq_line_enable = true; + enable_irq(reply_q->os_irq); + continue; + } synchronize_irq(pci_irq_vector(ioc->pdev, reply_q->msix_index)); } } @@ -3032,6 +3103,8 @@ mpt3sas_base_map_resources(struct MPT3SAS_ADAPTER *ioc) if (r) goto out_fail; + if (!ioc->is_driver_loading) + _base_init_irqpolls(ioc); /* Use the Combined reply queue feature only for SAS3 C0 & higher * revision HBAs and also only when reply queue count is greater than 8 */ @@ -6514,6 +6587,7 @@ mpt3sas_base_attach(struct MPT3SAS_ADAPTER *ioc) if (r) goto out_free_resources; + _base_init_irqpolls(ioc); init_waitqueue_head(&ioc->reset_wq); /* allocate memory pd handle bitmask list */ diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.h b/drivers/scsi/mpt3sas/mpt3sas_base.h index 19158cb..fb572cd 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.h +++ b/drivers/scsi/mpt3sas/mpt3sas_base.h @@ -67,6 +67,7 @@ #include #include #include +#include #include "mpt3sas_debug.h" #include "mpt3sas_trigger_diag.h" @@ -882,6 +883,9 @@ struct _event_ack_list { * @reply_post_free: reply post base virt address * @name: the name registered to request_irq() * @busy: isr is actively processing replies on another cpu + * @os_irq: irq number + * @irqpoll: irq_poll object + * @irq_poll_scheduled: Tells whether irq poll is scheduled or not * @list: this list */ struct adapter_reply_queue { @@ -891,6 +895,10 @@ struct adapter_reply_queue { Mpi2ReplyDescriptorsUnion_t *reply_post_free; char name[MPT_NAME_LENGTH]; atomic_t busy; + u32 os_irq; + struct irq_poll irqpoll; + bool irq_poll_scheduled; + bool irq_line_enable; struct list_head list; }; From patchwork Thu Feb 14 05:46:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suganath Prabu S X-Patchwork-Id: 10811833 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4E1421399 for ; Thu, 14 Feb 2019 05:46:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 39E332D37A for ; Thu, 14 Feb 2019 05:46:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2B2FE2D390; Thu, 14 Feb 2019 05:46:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BE00E2D37A for ; Thu, 14 Feb 2019 05:46:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392594AbfBNFqj (ORCPT ); Thu, 14 Feb 2019 00:46:39 -0500 Received: from mail-ed1-f65.google.com ([209.85.208.65]:40250 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726129AbfBNFqj (ORCPT ); Thu, 14 Feb 2019 00:46:39 -0500 Received: by mail-ed1-f65.google.com with SMTP id 10so3987597eds.7 for ; Wed, 13 Feb 2019 21:46:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dy+e6oQ7CSinT9QlwhS+fuQA9alIwcaS7YY0JzYje+Q=; b=QE8aFb9LkErgAOO6P40MfZU3n1PBPm/sXyybAePvzs8YCjB1j3BMhs61/slsB+dsOJ JRwEo50wRcs43w/JthFj1CwMg8yI6EJhyDWH84Iie3tQsdDy+WHZbF5jD0SdG3G1PXmz kx9hLbg4SFqTq7qk4maWM/qBvMjtFOxXJ8CsM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dy+e6oQ7CSinT9QlwhS+fuQA9alIwcaS7YY0JzYje+Q=; b=tddHbpAfEnT3dKvbgbA1JxrqwHG9CJ1oFxKiCDiPEMg9mWpM3Y0SLTS7csoLra4NzV klkux/MPBQdfXFIWpenSV14y1/GgwsFXE60Qz3jd6rbr9WETFrSQ8fx5mwTC3YggfBIe soMOcp+B9S/RSw0BvjIsHvRMx4uK3ls1jFENpNEjVn+KYrUkK0FRIuFQyEkbwmzivmH4 2/B1s9IG7oC5Wpt+IKoYm1/0z34+47wUVOpkD90qofOaQ3quMZoC/CVZhkCsM6H2/xPj yuOi9zd5oZU6CiQ4pA5BXr/achZhT33/Rs08djg9T1oaT9X+7mc9sf6mbhBoON+8PC1q qY4w== X-Gm-Message-State: AHQUAuap6gozIthJsRpiG7f/C0MXOU7LoxXFDaCdV7mdkTJ8DrIfKLPe xaYGhnahB3+TCQck9BrTEspY7kMj9/lwxaC9+zR9iPAKfty9meCgXO4xxmDJgO1W9VIY4wKEVY7 nbqzk3DdrhN7GzEeQuSnUdSVVpXLuf3HLDlk8M9aK/nwjEyVAKXCrxa9zLeDQ20Q6FmQWRlk5Dk sozb5w8pXyULMOBrahBlLz X-Google-Smtp-Source: AHgI3IaBYpq++1xVaIRULqh8xli3bF/w2m6vPdJvG/wiExCOk3RuRbmuDJhTQx6LLLeglDdNGDLi3Q== X-Received: by 2002:a17:906:18f1:: with SMTP id e17mr1464025ejf.82.1550123196840; Wed, 13 Feb 2019 21:46:36 -0800 (PST) Received: from dhcp-10-123-20-72.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id h1sm311791ejx.41.2019.02.13.21.46.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 Feb 2019 21:46:36 -0800 (PST) From: Suganath Prabu To: linux-scsi@vger.kernel.org Cc: Sathya.Prakash@broadcom.com, sreekanth.reddy@broadcom.com, Suganath Prabu Subject: [PATCH 4/6] mpt3sas: Load balance to improve performance and avoid soft lockups. Date: Thu, 14 Feb 2019 00:46:09 -0500 Message-Id: <1550123171-15993-5-git-send-email-suganath-prabu.subramani@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1550123171-15993-1-git-send-email-suganath-prabu.subramani@broadcom.com> References: <1550123171-15993-1-git-send-email-suganath-prabu.subramani@broadcom.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Driver uses "reply descriptor post queues" in round robin fashion so that IO's are distributed to all the available reply descriptor post queues equally. With this each reply descriptor post queue load is balanced. This is enabled only if CPUs count to MSI-X vector count ratio is X:1 (where X > 1) This improves performance and also fixes soft lockups. Signed-off-by: Suganath Prabu --- drivers/scsi/mpt3sas/mpt3sas_base.c | 21 +++++++++++++++++++++ drivers/scsi/mpt3sas/mpt3sas_base.h | 5 +++++ 2 files changed, 26 insertions(+) diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c index 1f358bc..aadd9e2 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.c +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c @@ -1382,6 +1382,16 @@ union reply_descriptor { } u; }; +static u32 base_mod64(u64 dividend, u32 divisor) +{ + u32 remainder; + + if (!divisor) + pr_err("mpt3sas: DIVISOR is zero, in div fn\n"); + remainder = do_div(dividend, divisor); + return remainder; +} + /** * _base_process_reply_queue - Process reply descriptors from reply * descriptor post queue. @@ -2845,6 +2855,11 @@ _base_assign_reply_queues(struct MPT3SAS_ADAPTER *ioc) if (!_base_is_controller_msix_enabled(ioc)) return; + ioc->msix_load_balance = false; + if (ioc->reply_queue_count < num_online_cpus()) { + ioc->msix_load_balance = true; + return; + } memset(ioc->cpu_msix_table, 0, ioc->cpu_msix_table_sz); @@ -3248,6 +3263,12 @@ mpt3sas_base_get_reply_virt_addr(struct MPT3SAS_ADAPTER *ioc, u32 phys_addr) static inline u8 _base_get_msix_index(struct MPT3SAS_ADAPTER *ioc) { + /* Enables reply_queue load balancing */ + if (ioc->msix_load_balance) + return ioc->reply_queue_count ? + base_mod64(atomic64_add_return(1, + &ioc->total_io_cnt), ioc->reply_queue_count) : 0; + return ioc->cpu_msix_table[raw_smp_processor_id()]; } diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.h b/drivers/scsi/mpt3sas/mpt3sas_base.h index fb572cd..3895407 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.h +++ b/drivers/scsi/mpt3sas/mpt3sas_base.h @@ -1024,6 +1024,9 @@ typedef void (*MPT3SAS_FLUSH_RUNNING_CMDS)(struct MPT3SAS_ADAPTER *ioc); * @msix_vector_count: number msix vectors * @cpu_msix_table: table for mapping cpus to msix index * @cpu_msix_table_sz: table size + * @total_io_cnt: Gives total IO count, used to load balance the interrupts + * @msix_load_balance: Enables load balancing of interrupts across + * the multiple MSIXs * @schedule_dead_ioc_flush_running_cmds: callback to flush pending commands * @scsi_io_cb_idx: shost generated commands * @tm_cb_idx: task management commands @@ -1200,6 +1203,8 @@ struct MPT3SAS_ADAPTER { u32 ioc_reset_count; MPT3SAS_FLUSH_RUNNING_CMDS schedule_dead_ioc_flush_running_cmds; u32 non_operational_loop; + atomic64_t total_io_cnt; + bool msix_load_balance; /* internal commands, callback index */ u8 scsi_io_cb_idx; From patchwork Thu Feb 14 05:46:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suganath Prabu S X-Patchwork-Id: 10811835 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0BAE13A4 for ; Thu, 14 Feb 2019 05:46:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AED152D37A for ; Thu, 14 Feb 2019 05:46:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A2A312D390; Thu, 14 Feb 2019 05:46:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 399952D37A for ; Thu, 14 Feb 2019 05:46:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393072AbfBNFqm (ORCPT ); Thu, 14 Feb 2019 00:46:42 -0500 Received: from mail-ed1-f66.google.com ([209.85.208.66]:42133 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726129AbfBNFqm (ORCPT ); Thu, 14 Feb 2019 00:46:42 -0500 Received: by mail-ed1-f66.google.com with SMTP id r15so3973262eds.9 for ; Wed, 13 Feb 2019 21:46:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=T41ztuycnt/Lyvj02Hp0CQi6itCWV3yTIAhY51tzQNw=; b=BXTQvmqfud7JcPJ9/Z/osEwF6O4MlWXEJlcVMnxaZRyS9JOl0/rUYMyrN/5E6pVRSL tcitVVwEZid+OP4KWpRpE5shDDLmPcJsALDLi0BlP0thl9D9HSS0B5a4Um8cZmHu1rTN s3EWO+PWX7eo33o0z17T8wd00xJbb70pXYBrw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=T41ztuycnt/Lyvj02Hp0CQi6itCWV3yTIAhY51tzQNw=; b=KU+7XZlaOBerRWpEw5eTBtEv+tN9IidcdL1bw/MKbf4gSKmQ1EVnN+1sl02HlaKIKF R1qkZzw5Q6gQt0rsNXPwVP2pAft2QG1QLLxkxGuUQkEgk5UeVEWhpmAsaLtQq0j/h47L XO0tqJFm5+UfVi/FKlGMgAx9HotisbBSPWTB/dIUbRdTIZn5X05MgG421Zyg21AmGquH JAgniIMMSJWv1LLk+0ZAiTf7ix7R129B1625FnphzN95EzmPXcztIfo63HHn8+l+C40z JqlZvJAyr7omE52KdApmxo0IpzI8COFXhvIeKq7afxgH13CS2sJuo+Otvc1oPLq/NRJG bUGQ== X-Gm-Message-State: AHQUAuamijxj+Qnn0iZZzsMPOlIyzeQfb45iqWlaaCVsBMvib2Bf8BRL Hcq0eE4To0kQlbtXVzBPewi+fiuZvf4PLF4VwFYluewDKylIU/aYCFRxvPuDaTHoQDJAwk1iOBI X9vHIUKAOHNYPZr0R8lWbtDOOJ2K2B+Y8mfRcOZ+0HFvUHAiouOoakwu90l/g2VyCSp+AcfwLXO O/mX/bpmMdWpfPw03EzBsf X-Google-Smtp-Source: AHgI3IZ+kSmU7d1U+R4EZIxuqj12/xerUEShBiW0rRrJluFgC+1x8KGLOsjhW30ohYxY8BmodeEhtA== X-Received: by 2002:a17:906:bf2:: with SMTP id z18mr1432493ejg.52.1550123200247; Wed, 13 Feb 2019 21:46:40 -0800 (PST) Received: from dhcp-10-123-20-72.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id h1sm311791ejx.41.2019.02.13.21.46.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 Feb 2019 21:46:39 -0800 (PST) From: Suganath Prabu To: linux-scsi@vger.kernel.org Cc: Sathya.Prakash@broadcom.com, sreekanth.reddy@broadcom.com, Suganath Prabu Subject: [PATCH 5/6] mpt3sas: Improve the threshold value and introduce module param. Date: Thu, 14 Feb 2019 00:46:10 -0500 Message-Id: <1550123171-15993-6-git-send-email-suganath-prabu.subramani@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1550123171-15993-1-git-send-email-suganath-prabu.subramani@broadcom.com> References: <1550123171-15993-1-git-send-email-suganath-prabu.subramani@broadcom.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP * Reduce the threshold value to 1/4 of the queue depth. * With this FW can find enough entries to post the Reply Descriptors in the reply descriptor post queue. * With module param, user can play with threshold value, the same irqpoll_weight is used as the budget in processing of reply descriptor post queues in _base_process_reply_queue. Signed-off-by: Suganath Prabu --- drivers/scsi/mpt3sas/mpt3sas_base.c | 14 ++++++++++++-- drivers/scsi/mpt3sas/mpt3sas_base.h | 3 +++ 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c index aadd9e2..880bd22 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.c +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c @@ -94,6 +94,11 @@ module_param(max_msix_vectors, int, 0); MODULE_PARM_DESC(max_msix_vectors, " max msix vectors"); +static int irqpoll_weight = -1; +module_param(irqpoll_weight, int, 0); +MODULE_PARM_DESC(irqpoll_weight, + "irq poll weight (default= one fourth of HBA queue depth)"); + static int mpt3sas_fwfault_debug; MODULE_PARM_DESC(mpt3sas_fwfault_debug, " enable detection of firmware fault and halt firmware - (default=0)"); @@ -1404,7 +1409,7 @@ static int _base_process_reply_queue(struct adapter_reply_queue *reply_q) { union reply_descriptor rd; - u32 completed_cmds; + u64 completed_cmds; u8 request_descript_type; u16 smid; u8 cb_idx; @@ -1502,7 +1507,7 @@ _base_process_reply_queue(struct adapter_reply_queue *reply_q) * So that FW can find enough entries to post the Reply * Descriptors in the reply descriptor post queue. */ - if (completed_cmds > ioc->hba_queue_depth/3) { + if (!base_mod64(completed_cmds, ioc->thresh_hold)) { if (ioc->combined_reply_queue) { writel(reply_q->reply_post_host_index | ((msix_index & 7) << @@ -6608,6 +6613,11 @@ mpt3sas_base_attach(struct MPT3SAS_ADAPTER *ioc) if (r) goto out_free_resources; + if (irqpoll_weight > 0) + ioc->thresh_hold = irqpoll_weight; + else + ioc->thresh_hold = ioc->hba_queue_depth/4; + _base_init_irqpolls(ioc); init_waitqueue_head(&ioc->reset_wq); diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.h b/drivers/scsi/mpt3sas/mpt3sas_base.h index 3895407..dab64f3 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.h +++ b/drivers/scsi/mpt3sas/mpt3sas_base.h @@ -1028,6 +1028,8 @@ typedef void (*MPT3SAS_FLUSH_RUNNING_CMDS)(struct MPT3SAS_ADAPTER *ioc); * @msix_load_balance: Enables load balancing of interrupts across * the multiple MSIXs * @schedule_dead_ioc_flush_running_cmds: callback to flush pending commands + * @thresh_hold: Max number of reply descriptors processed + * before updating Host Index * @scsi_io_cb_idx: shost generated commands * @tm_cb_idx: task management commands * @scsih_cb_idx: scsih internal commands @@ -1205,6 +1207,7 @@ struct MPT3SAS_ADAPTER { u32 non_operational_loop; atomic64_t total_io_cnt; bool msix_load_balance; + u16 thresh_hold; /* internal commands, callback index */ u8 scsi_io_cb_idx; From patchwork Thu Feb 14 05:46:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suganath Prabu S X-Patchwork-Id: 10811837 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EF74B13A4 for ; Thu, 14 Feb 2019 05:46:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DF20D2D37A for ; Thu, 14 Feb 2019 05:46:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D3A752D390; Thu, 14 Feb 2019 05:46:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 836472D37A for ; Thu, 14 Feb 2019 05:46:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393953AbfBNFqq (ORCPT ); Thu, 14 Feb 2019 00:46:46 -0500 Received: from mail-ed1-f68.google.com ([209.85.208.68]:40261 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726129AbfBNFqp (ORCPT ); Thu, 14 Feb 2019 00:46:45 -0500 Received: by mail-ed1-f68.google.com with SMTP id 10so3987762eds.7 for ; Wed, 13 Feb 2019 21:46:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=18Ct0AMMAlksY/9mfDQX7szYwJalKF83DPthr/i+A2s=; b=LtjQZ/m2lGCSzM8XyLiZ4eD5nG3jRmP5+tsYjajt0dkvqhPyrz4EmzokJWW7arDMVK DEFCkJNdToi0/6qDUUXgYwonn87zVjLFLvESZo8xp5IJLZNe7gape1Nw4zYjhAtGRWPJ 7i6RkovU2R8vPF8e9rtclmjlDDA5v0TCjF2bo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=18Ct0AMMAlksY/9mfDQX7szYwJalKF83DPthr/i+A2s=; b=lBKH2f7bSd+VX2n+QBbcZ5v07DHW57xKvQXCF5HcuSqRBaqopD3tQjLzwdQqUdiWZB ZMCmJsdug0YAOMOV8rUKsNjlFgCyXVh8UMsGTTIvjJw+4gRgchIkFts/Riss6nXZjTTe G/U3jF0oVRjh5WjNNqO2WUPnLsHVjaZVZ1AN88UBN58dnnuFyZoH3agwiA8hMe5ki++x Ji5pOWVYZbV77CrDReeBxh5nSgKfwaEpweuxaJJU5EY5NgVA1HKQa7CC8JtAnfge+R7f C05Cf4mvmXRewUr8UAGfnax62VSoc3PUX9kzMbdaeIH3blHPVWne1nYu1gUb//LHK7oy ubLw== X-Gm-Message-State: AHQUAubJ+y3G0znT/iV7bYwiu0f+34WTobMYVt9StgrJfL5IxHgjMA9i gfL1l6ADdoRPcE3MKs6CgU8cq7BjWFVis3sXhvTgDdtiqZ+5GV3JQPWRJeQGWPPE5NLJSev7LKH Lrj0Vwr5FCRfy+4tK2vbop/23Z+HgagP/5MJEj45p8glttKYjdrOEnIRS9skeA/trUaZmTukPb6 fQUE4B01n4vaWq8Hx9+Qrh X-Google-Smtp-Source: AHgI3Ia/A0DkbJ6AxNgdfUI4FOuPnt2noxFDJwVkOYMnZ1oJ1M34sVXvsvGxs61O7P3Zmf3ksU31/Q== X-Received: by 2002:a50:fc12:: with SMTP id i18mr1562906edr.149.1550123203811; Wed, 13 Feb 2019 21:46:43 -0800 (PST) Received: from dhcp-10-123-20-72.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id h1sm311791ejx.41.2019.02.13.21.46.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 Feb 2019 21:46:43 -0800 (PST) From: Suganath Prabu To: linux-scsi@vger.kernel.org Cc: Sathya.Prakash@broadcom.com, sreekanth.reddy@broadcom.com, Suganath Prabu Subject: [PATCH 6/6] mpt3sas: Update mpt3sas driver version to 28.100.00.00 Date: Thu, 14 Feb 2019 00:46:11 -0500 Message-Id: <1550123171-15993-7-git-send-email-suganath-prabu.subramani@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1550123171-15993-1-git-send-email-suganath-prabu.subramani@broadcom.com> References: <1550123171-15993-1-git-send-email-suganath-prabu.subramani@broadcom.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Updated driver version to 28.100.00.00, which is equivalent to OOB Ph9 Signed-off-by: Suganath Prabu --- drivers/scsi/mpt3sas/mpt3sas_base.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.h b/drivers/scsi/mpt3sas/mpt3sas_base.h index dab64f3..480219f 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.h +++ b/drivers/scsi/mpt3sas/mpt3sas_base.h @@ -76,9 +76,9 @@ #define MPT3SAS_DRIVER_NAME "mpt3sas" #define MPT3SAS_AUTHOR "Avago Technologies " #define MPT3SAS_DESCRIPTION "LSI MPT Fusion SAS 3.0 Device Driver" -#define MPT3SAS_DRIVER_VERSION "27.102.00.00" -#define MPT3SAS_MAJOR_VERSION 27 -#define MPT3SAS_MINOR_VERSION 102 +#define MPT3SAS_DRIVER_VERSION "28.100.00.00" +#define MPT3SAS_MAJOR_VERSION 28 +#define MPT3SAS_MINOR_VERSION 100 #define MPT3SAS_BUILD_VERSION 0 #define MPT3SAS_RELEASE_VERSION 00