From patchwork Wed Oct 26 01:15:36 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Shan X-Patchwork-Id: 9395775 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5F0DF60231 for ; Wed, 26 Oct 2016 01:22:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0B66529721 for ; Wed, 26 Oct 2016 01:22:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F359429728; Wed, 26 Oct 2016 01:22:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8814D29722 for ; Wed, 26 Oct 2016 01:22:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753771AbcJZBWs (ORCPT ); Tue, 25 Oct 2016 21:22:48 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:52395 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753758AbcJZBWr (ORCPT ); Tue, 25 Oct 2016 21:22:47 -0400 Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id u9Q1J8dm036677 for ; Tue, 25 Oct 2016 21:22:47 -0400 Received: from e23smtp05.au.ibm.com (e23smtp05.au.ibm.com [202.81.31.147]) by mx0a-001b2d01.pphosted.com with ESMTP id 26ad0fs99h-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 25 Oct 2016 21:22:46 -0400 Received: from localhost by e23smtp05.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 26 Oct 2016 11:22:43 +1000 Received: from d23dlp01.au.ibm.com (202.81.31.203) by e23smtp05.au.ibm.com (202.81.31.211) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 26 Oct 2016 11:22:41 +1000 Received: from d23relay08.au.ibm.com (d23relay08.au.ibm.com [9.185.71.33]) by d23dlp01.au.ibm.com (Postfix) with ESMTP id C44DB2CE8062 for ; Wed, 26 Oct 2016 12:22:40 +1100 (EST) Received: from d23av05.au.ibm.com (d23av05.au.ibm.com [9.190.234.119]) by d23relay08.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id u9Q1MeQ015270134 for ; Wed, 26 Oct 2016 12:22:40 +1100 Received: from d23av05.au.ibm.com (localhost [127.0.0.1]) by d23av05.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id u9Q1MdJR000967 for ; Wed, 26 Oct 2016 12:22:40 +1100 Received: from ozlabs.au.ibm.com (ozlabs.au.ibm.com [9.192.253.14]) by d23av05.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id u9Q1MdS8000961; Wed, 26 Oct 2016 12:22:39 +1100 Received: from bran.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) by ozlabs.au.ibm.com (Postfix) with ESMTP id 8912AA0217; Wed, 26 Oct 2016 12:22:39 +1100 (AEDT) Received: from gwshan (shangw.ozlabs.ibm.com [10.61.2.199]) by bran.ozlabs.ibm.com (Postfix) with ESMTP id 755FBE3C65; Wed, 26 Oct 2016 12:22:39 +1100 (AEDT) Received: by gwshan (Postfix, from userid 1000) id A492D942CE3; Wed, 26 Oct 2016 12:15:38 +1100 (AEDT) From: Gavin Shan To: linux-pci@vger.kernel.org Cc: linuxppc-dev@lists.ozlabs.org, bhelgaas@google.com, benh@kernel.crashing.org, mpe@ellerman.id.au, clsoto@us.ibm.com, Gavin Shan Subject: [PATCH v3 2/2] PCI: Disable VF's memory space on updating IOV BAR in pci_update_resource() Date: Wed, 26 Oct 2016 12:15:36 +1100 X-Mailer: git-send-email 2.1.0 In-Reply-To: <1477444536-29612-1-git-send-email-gwshan@linux.vnet.ibm.com> References: <1477444536-29612-1-git-send-email-gwshan@linux.vnet.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16102601-0016-0000-0000-000001E32F0A X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 16102601-0017-0000-0000-000005B4B2C2 Message-Id: <1477444536-29612-3-git-send-email-gwshan@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2016-10-25_23:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=1 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1609300000 definitions=main-1610260020 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP pci_update_resource() might be called to update (shift) IOV BARs in PPC PowerNV specific pcibios_sriov_enable() when enabling PF's SRIOV capability. At that point, the PF have been functional if the SRIOV is enabled through sysfs entry "sriov_numvfs". The PF's memory decoding (0x2 in PCI_COMMAND) shouldn't be disabled when updating its IOV BARs with pci_update_resource(). Otherwise, we receives EEH error caused by MMIO access to PF's memory BARs during the window when PF's memory decoding is disabled. sriov_numvfs_store pdev->driver->sriov_configure mlx5_core_sriov_configure pci_enable_sriov sriov_enable pcibios_sriov_enable pnv_pci_sriov_enable pnv_pci_vf_resource_shift pci_update_resource This disables VF's memory space instead of PF's memory decoding when 64-bits IOV BARs are updated in pci_update_resource(). Reported-by: Carol Soto Suggested-by: Bjorn Helgaas Signed-off-by: Gavin Shan Tested-by: Carol Soto --- drivers/pci/setup-res.c | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c index 66c4d8f..1456896 100644 --- a/drivers/pci/setup-res.c +++ b/drivers/pci/setup-res.c @@ -29,10 +29,10 @@ void pci_update_resource(struct pci_dev *dev, int resno) { struct pci_bus_region region; - bool disable; - u16 cmd; + bool disable = false; + u16 cmd, bit; u32 new, check, mask; - int reg; + int reg, cmd_reg; enum pci_bar_type type; struct resource *res = dev->resource + resno; @@ -81,11 +81,23 @@ void pci_update_resource(struct pci_dev *dev, int resno) * disable decoding so that a half-updated BAR won't conflict * with another device. */ - disable = (res->flags & IORESOURCE_MEM_64) && !dev->mmio_always_on; + if (res->flags & IORESOURCE_MEM_64) { + if (resno <= PCI_ROM_RESOURCE) { + disable = !dev->mmio_always_on; + cmd_reg = PCI_COMMAND; + bit = PCI_COMMAND_MEMORY; + } else { +#ifdef CONFIG_PCI_IOV + disable = true; + cmd_reg = dev->sriov->pos + PCI_SRIOV_CTRL; + bit = PCI_SRIOV_CTRL_MSE; +#endif + } + } + if (disable) { - pci_read_config_word(dev, PCI_COMMAND, &cmd); - pci_write_config_word(dev, PCI_COMMAND, - cmd & ~PCI_COMMAND_MEMORY); + pci_read_config_word(dev, cmd_reg, &cmd); + pci_write_config_word(dev, cmd_reg, cmd & ~bit); } pci_write_config_dword(dev, reg, new); @@ -107,7 +119,7 @@ void pci_update_resource(struct pci_dev *dev, int resno) } if (disable) - pci_write_config_word(dev, PCI_COMMAND, cmd); + pci_write_config_word(dev, cmd_reg, cmd); } int pci_claim_resource(struct pci_dev *dev, int resource)