From patchwork Mon Mar 21 09:39:52 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Krzysztof_Ha=C5=82asa?= X-Patchwork-Id: 8630741 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 6BECCC0553 for ; Mon, 21 Mar 2016 09:40:01 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id A6DD8202D1 for ; Mon, 21 Mar 2016 09:39:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 34790202B8 for ; Mon, 21 Mar 2016 09:39:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751842AbcCUJj5 (ORCPT ); Mon, 21 Mar 2016 05:39:57 -0400 Received: from ni.piap.pl ([195.187.100.4]:55466 "EHLO ni.piap.pl" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751696AbcCUJj4 convert rfc822-to-8bit (ORCPT ); Mon, 21 Mar 2016 05:39:56 -0400 Received: from t19.piap.pl (OSB1819.piap.pl [10.0.9.19]) by ni.piap.pl (Postfix) with ESMTP id B5A93441B7C; Mon, 21 Mar 2016 10:39:52 +0100 (CET) From: khalasa@piap.pl (Krzysztof Halasa) To: Bjorn Helgaas Cc: Arnd Bergmann , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH] Extend PCIE_BUS_PEER2PEER to set MRSS=128 to fix CNS3xxx BM DMA. Date: Mon, 21 Mar 2016 10:39:52 +0100 Message-ID: MIME-Version: 1.0 X-KLMS-Rule-ID: 1 X-KLMS-Message-Action: clean X-KLMS-AntiSpam-Lua-Profiles: 93349 [Mar 21 2016] X-KLMS-AntiSpam-Version: 5.5.9.33 X-KLMS-AntiSpam-Envelope-From: khalasa@piap.pl X-KLMS-AntiSpam-Rate: 30 X-KLMS-AntiSpam-Status: not_detected X-KLMS-AntiSpam-Method: none X-KLMS-AntiSpam-Moebius-Timestamps: 4023087, 4023105, 4023017 X-KLMS-AntiSpam-Info: LuaCore: 415 415 56d27afa4611b5fc17406ce7708f83a66d615280, {FROM: brackets after email}, Auth:dkim=none X-KLMS-AntiSpam-Interceptor-Info: scan successful X-KLMS-AntiPhishing: Clean, 2016/03/17 15:17:21 X-KLMS-AntiVirus: Kaspersky Security 8.0 for Linux Mail Server, version 8.0.1.721, bases: 2016/03/21 01:19:00 #7288512 X-KLMS-AntiVirus-Status: Clean, skipped Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The platform in question is Cavium CNS3xxx, ARMv6. A recent patch by Arnd Bergmann (498a92d42596 "ARM: cns3xxx: pci: avoid potential stack overflow") converted an explicit setting of PCI_EXP_DEVCTL_READRQ = 0 (i.e., max 128 bytes for bus-mastering PCIe DMA read request) to: + pcie_bus_config = PCIE_BUS_PEER2PEER; with the following commentary: "The second part is how the driver sets up the Max_Read_Request_Size value for the first device/function on bus 1, i.e. the device plugged directly into the PCIe root port. For all I can tell, this is in fact incomplete, as it does not perform the same setting on devices attached to a PCIe switch, or multi-function devices. The solution for this part fortunately is even easier: if we just set the global pcie_bus_config variable to PCIE_BUS_PEER2PEER, all PCIe devices in the system are limited to 128 byte MPS, which in turn limits the MRRS to 128 bytes for all devices, and we no longer even need to touch any devices." The problem is the MRRS setting is never written to the hardware. I propose the following, though I'm not sure if we can do this safely, especially given the comments in probe.c. OTOH, this change may be required in other (all?) cases when the user requests PCIE_BUS_PEER2PEER. On this Laguna GW-2388 the following patch fixes BM DMA with: 0000:00:00.0 PCI bridge: Cavium Networks Device 3400 (rev 01) 0000:01:00.0 PCI bridge: Texas Instruments XIO2001 PCI Express-to-PCI Bridge 0000:02:0e.0 (PCI devices behind the bridge, these are doing actual BM xfers) 0001:00:00.0 PCI bridge: Cavium Networks Device 3400 (rev 01 - this is the second lane from the CPU) pci 0000:00:00.0: Max Payload Size set to 128/ 128 (was 128), Max Read Rq 128 pci 0000:01:00.0: Max Payload Size set to 128/ 512 (was 128), Max Read Rq 128 pci 0001:00:00.0: Max Payload Size set to 128/ 128 (was 128), Max Read Rq 128 Signed-off-by: Krzysztof Ha?asa Fixes: 498a92d42596 ("ARM: cns3xxx: pci: avoid potential stack overflow") --- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 6d7ab9b..91713b6 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1919,7 +1919,8 @@ static void pcie_write_mrrs(struct pci_dev *dev) /* In the "safe" case, do not configure the MRRS. There appear to be * issues with setting MRRS to 0 on a number of devices. */ - if (pcie_bus_config != PCIE_BUS_PERFORMANCE) + if (pcie_bus_config != PCIE_BUS_PERFORMANCE && + pcie_bus_config != PCIE_BUS_PEER2PEER) return; /* For Max performance, the MRRS must be set to the largest supported diff --git a/include/linux/pci.h b/include/linux/pci.h index 2771625..6f5088a 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -756,7 +756,7 @@ enum pcie_bus_config_types { PCIE_BUS_DEFAULT, /* ensure MPS matches upstream bridge */ PCIE_BUS_SAFE, /* use largest MPS boot-time devices support */ PCIE_BUS_PERFORMANCE, /* use MPS and MRRS for best performance */ - PCIE_BUS_PEER2PEER, /* set MPS = 128 for all devices */ + PCIE_BUS_PEER2PEER, /* set MPS and MRSS to 128 for all devices */ }; extern enum pcie_bus_config_types pcie_bus_config;