From patchwork Sat Dec 5 11:10:23 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 7775401 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 53FD59F1C2 for ; Sat, 5 Dec 2015 12:43:26 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 8926F204D9 for ; Sat, 5 Dec 2015 12:43:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D8FC12049E for ; Sat, 5 Dec 2015 12:43:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751983AbbLEMnL (ORCPT ); Sat, 5 Dec 2015 07:43:11 -0500 Received: from mail-bn1bon0064.outbound.protection.outlook.com ([157.56.111.64]:65072 "EHLO na01-bn1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751969AbbLEMnJ (ORCPT ); Sat, 5 Dec 2015 07:43:09 -0500 X-Greylist: delayed 4660 seconds by postgrey-1.27 at vger.kernel.org; Sat, 05 Dec 2015 07:43:08 EST Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Christian.Koenig@amd.com; Received: from [IPv6:2a02:908:f679:3281:6c35:9a0e:2627:da65] (2a02:908:f679:3281:6c35:9a0e:2627:da65) by CY1PR12MB0133.namprd12.prod.outlook.com (10.161.173.156) with Microsoft SMTP Server (TLS) id 15.1.337.19; Sat, 5 Dec 2015 11:10:39 +0000 To: Bjorn Helgaas , , "Alex Deucher" From: =?UTF-8?Q?Christian_K=c3=b6nig?= Subject: Resizeable PCI BAR support Message-ID: <5662C61F.50303@amd.com> Date: Sat, 5 Dec 2015 12:10:23 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 X-Originating-IP: [2a02:908:f679:3281:6c35:9a0e:2627:da65] X-ClientProxiedBy: VI1PR02CA0030.eurprd02.prod.outlook.com (25.162.7.168) To CY1PR12MB0133.namprd12.prod.outlook.com (25.161.173.156) X-Microsoft-Exchange-Diagnostics: 1; CY1PR12MB0133; 2:HhANfQoWVZ2pcOqs0uER/3tYSjabcJmYjb3KrSgvLTmYqzmbVHuXZ2MkkyEiEeruRIFSSYtGFEvr1PWGTwYUfMn4+Ntc/M6P6t/kHlgjCKhR8annCFuBaPlFGAgh4dDPBfFDsizinYfXqB4K1nA7Tg==; 3:gvyOOieZnjfe0/CAx6vfqVyjn6fMeDFMonfRSEg6hiQxaPFe1gOV+J+sLrnhCwFl6N5yD5zHYPT/w1XMOllpFVag2CsylZHhymDSDZIcCIZ0gNUqg513+FfrcolqwQw8; 25:2+Ii5BKBjUqm9jML8vbgTt0NwahQDu2tBas9HKBdSdSjw3R+HenHbPZ86AkXtBUPD6NBHZy97/WDO/etlx8IXSzX9coZdVOsr5cbUweTouEXEDffIn7j41yLB9g9PF3JJ60zR7XeVZove9Vfyd3+ewSHOaZ9MHdMzr5ApbkqR1ITUQvKrXgeFki6bKTeXvCki90meUkdm/FBNQhjoKZFB0idEa8RNTWE+e2VXHmznL250dG/DHDvbk9ZQ+mPPY53QsIKSDd2XIcBVYzP1chvyA== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY1PR12MB0133; X-Microsoft-Exchange-Diagnostics: 1; CY1PR12MB0133; 20:tDXJb59kBrOX6e5n95sN1JCqTBZ1Inu5gkYTrHj6kNZnntt3mVW25CMK6yAitUNI3bPJSEiE40rA5Mx9h0hL9NQ+wK8/kv3sDTnHJ6IBFy5L9X4+UJ604fntj+n13sk2fsWleVTpZdg7e5D8DQWYA2wI28YKVuTmP5EfZJcEyNvJYHM0d3tye6iG0VpsFFTM7d08Q7agbeHfItKmzG5mCuiZ1V3RSoYdp3apv0khCTXl3waGIRXYlkFr5DYGwvoCRP93J0UdOXoU8HH3oR26dCBdQcvR4MZAIEjaAWh0iCMOdNC/6lB4y33shv/UqSAzZEuCGdGfffEJhNu38bsiRpoRr9WQsQRms147h06GjWrJZfM/uOBkE2zPFiMfU3xTAXz3KZb1NzYllmpuPzOO6b1yMvWZsjEE5OxZcFmc8QlV2lmDASnbhq1bR7lJMQtu6Btfocv692H/s0rt30wXGRZg32FB0MDIKAYM2hTLgAFoSPl3AaaQuQfATGgmUnpu; 4:YjvEvPKBBNXQ1Q/1l5+9burMb6ICtNDK9jKRy5SBHSIELrbFCSQ80LpP+iqsIBNqYbxUnLW8WXWo6uBIuO8e1o4jomkWT4aV/7SK/i2FmYZ3OxT7nSB7ruNYuTVgpS2kDtOhnxIMHkTh5ww4MO3PoIFpJ5Y/jIwfdXHvsfoiqutpzJLGcHEOzJspWlUgZUDb8227MR3hFFlHcVkIuNs8gNo8fZ7G9eLYR7VHJpzKsF5Ha6beICDbrua4+Uam6n8JJYYnq1Azq59CllFXpYJxGowPj6b/cmR/YRJsYz0ShkAHzOPPaYnnBtG/y+IswmE7AW/dBIoqv0BvQaDYxpMToO/x4gkTrCh5AQ6XnI9rgxXgzHC/FIeVJXj+iQZiNvz2 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(520078)(8121501046)(3002001)(10201501046); SRVR:CY1PR12MB0133; BCL:0; PCL:0; RULEID:; SRVR:CY1PR12MB0133; X-Forefront-PRVS: 07817FCC2D X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(199003)(189002)(83506001)(65816999)(101416001)(84326002)(87976001)(65956001)(4001350100001)(65806001)(19580395003)(107886002)(5001960100002)(97736004)(5004730100002)(106356001)(81156007)(5890100001)(54356999)(5001770100001)(5008740100001)(4610100001)(1096002)(4810100001)(2476003)(50986999)(1706002)(105586002)(270700001)(586003)(6116002)(86362001)(568964002)(189998001)(33656002)(122386002)(42186005)(40100003)(92566002)(229853001)(15975445007)(64126003)(77096005)(512874002)(36756003)(3826002)(217873001); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR12MB0133; H:[IPv6:2a02:908:f679:3281:6c35:9a0e:2627:da65]; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtDWTFQUjEyTUIwMTMzOzIzOmVVcFM3V3ZwTFBxRiswaXZNTGhDK1FpTmJZ?= =?utf-8?B?cjBNdXhkeUUvNTJlVWxicVpNd0xCNlptZXN4U3ZHS0YwSDhKKzFOUzJuY3pG?= =?utf-8?B?YU9jNlBPL1hScVFTVlVudm00Sk1NSEU0MG9OL3lONnc5SUN0UnpIM2tmUlVJ?= =?utf-8?B?RjZWV2NJWDRtak1OQTk4QkI3UWRBU09lQVRvL1pzbW05VnN3VWVVdDk2MTNw?= =?utf-8?B?WWVTMlNBQVY0eU5RVU5GRHNBUGRTK3IvMEdlOVc2MjV2SzVNc25QZitVS3hM?= =?utf-8?B?MC9NVDNpN2FleVVyQmpZdCtDTzlIMk5lTGp5SXFrdkxSRkcwRnVzT0tDY3cy?= =?utf-8?B?TWpKTVZBeW56ZUdIQm1aeURwSWkyS0g0VFlRaENSS3BOWWJmRkdrMnVweFJi?= =?utf-8?B?V3FaU2p6VkFQS3FZRUl4Z1VycmtNYnFvUytFU2t0azhkdnJjWEpEbng0Wk9h?= =?utf-8?B?dmNGWEtOa1llbVdTRjMzQzljQ0I5Ly9VNFdSbytmZUJLSnhZdW5NNTRGZ1lO?= =?utf-8?B?cXB3U3ZveFV2dEpqZVdOQWdTQWVLaWVSOWY0dnV0OGFtaHhOUmFKZWhRd3BE?= =?utf-8?B?RDR2MFc3M3JERXRhbllDK0pkNHNBaHJuRGFRRUN3cXpIVE10OHBwWVllZVYx?= =?utf-8?B?UDREQ1lTREQzL0ErNjBUWlN1NnlXTXBZMmdSdGxOUDhDTGt2MlNEckFaWllO?= =?utf-8?B?TWxpQjRpVVJybFQraUhudGY0ZzF1SFhDVjBQeXI5NzVQbzAyRkhiK0pWK0pY?= =?utf-8?B?QnZoWEdSblVCejBna0x4UDRhU252b3pmK1F1d2VNUlhXYVVnYzJpUmFtbXNX?= =?utf-8?B?ei9yalhWaVE3UXFXR3ZCOVVMWTZscDV1RHF5TmI5Qjk0MnFiaXgwbEtIejNW?= =?utf-8?B?RDFmRHVFZXFteVlXVzVUenpzUGMyQTEyVkdrYS9zb3lhOVcwUGgyMGRISTE5?= =?utf-8?B?MnpvdEI4SzNpL3pmTkZyTVRmQ3hDWkRwczZKT2UrUXZGTlV0ZnBmY2RGN2Rk?= =?utf-8?B?NDU4RU5JY3R3QTdDaHB6eWNhUGxkVkRoNldwVk1hUlBXd2pYU0ZsWHlEWjBi?= =?utf-8?B?ckNMUTNrOC9mOGxoTEhwMnZNZk1oYU5uVHBCczNac1lFT3RDMTVwWW5jeDRM?= =?utf-8?B?SjVBaFlyczBvNkc2dnU4a2pJcUFqclpNM2ZqMENUS2U5ditQK21wZitTREZL?= =?utf-8?B?Ym9HRzBPUjl0RGNoUEYwaGpmZTcrZ1haOGE4SC9MZXRjeGpCZm9aQkpJUmtw?= =?utf-8?B?WThFVkxRT3lyN0dFeVRqckROTFFES2hTTGtPR2RNYyswNnZjSGdzSG4vVVQ4?= =?utf-8?B?cS9pVUhwTDdVTjl4dTdIeGNXb2NhdUkveHFWb2V3WU9VY3pMbmF0c2N1Vlhv?= =?utf-8?B?NzBZNEJVZlgvNjdvemhQY0E0RkdrYnhSUDBVN0RIOVg2YmdkWWRwWjRVSUdo?= =?utf-8?B?RGJuMzkrUDd0SkdzYUFxR21QMEZpQk9XcVdRLy9mNWhmQVFwNURmZjRIWlZk?= =?utf-8?B?WUFqYlZzL1ZvU05leXRBZlFJbkdDRXY2bHBxZnFPSVFFaUozZHI4aXNKNFFw?= =?utf-8?B?QVhteVpNRHdkdUhpQkVjVGgvaTV4b1JnZ21FSWI0TzVxbUlNVVJ2RHBpUmdq?= =?utf-8?B?WjZDNDRORGNDdDVWWlhOY01kVHZzcllyL0p6a1NWblRZQTc1ZkRsMncyV1dv?= =?utf-8?Q?/T6/W9Q91LSEU70ZZjM/HuSoeW6nKtw6XuYG5Lw?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR12MB0133; 5:r7CjsxmPzL+vGlJ9+5L3JkLSKapRAT6ozm42WERZoh5eyaKDlLowkFfD9OILET4+g0P6jLh2P/3CntYWQgIN093ToVUNHVILCq6/Q7HEYxDCNZuFJF5Scm3vsHmV+Q/Oaazvp9EBjjrMW6zAHrkwLQ==; 24:gqa5efGyjWiTAbibKITA24lopLBAYQmAveug6dLiCZR3Wp10n9d2spn2QVaSW4TgOSL97e3q0szvKngPLU6GEyae9sPwSifeobm6VY1RrN8=; 20:CNmL7Ks9OthJCWElrEBNFudGCyZf/f+XNwVtnpAr7VGW4P/gmhETuqMK5kpidP5gcmjdU9PRk6ruFeqShmXz6uC7idUqyGuD4kp1wc6gZ4HBF2rgBUHAzy1WKYG6RFbyxjseQcNPAczUZ/yXerRPnnHVNXEyiFMwech7j4Ey4lplbDSY+Nw+WzfMI5Nmq3Je5LKoB80rfdXcoTI5aoKrP3/xf37uNW6Md7lyl5GlRbXiM6M3DB0B45UKTy7ouqyN X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2015 11:10:39.4607 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR12MB0133 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, T_TVD_MIME_EPI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hi Bjorn, reading up on the mailing list this is probably not the first time this topic comes up, but this time it is not just for FPGAs but for really end user hardware. Alex and I are the maintainers of the Radeon and Amdgpu kernel drivers for the AMD graphics hardware. Now the newest hardware generation of those devices started to support resizeable PCI BARs as specified here https://pcisig.com/sites/default/files/specification_documents/ECN_Resizable-BAR_24Apr2008.pdf. That feature is rather important for graphics hardware, because the PCI BARs are usually limited to 256MB while on modern cards you can easily find 4GB or more VRAM. The end result is that only a fraction of that VRAM is CPU accessible, causing a whole bunch of workarounds in the driver stack for that hardware. We of course want to avoid that hassle and so I've attached two patches created to test the functionality. The first one is just infrastructure for handling the extension and the second is a hack to resize all BARs to their maximum before probing them by the PCI subsystem. That resizing works astonishing fine, the only problem is that on practical no system I've tested this feature actually works. The reason is that the PCI root bridge doesn't get enough address space assigned by the BIOS for this and the PCI subsystem in the kernel isn't able to reprogram it. Any clever idea how to get this working without waiting for system BIOS providers to pick up that extension? Would adding support for reprogramming the root bridges be possible? Best regards, Christian. From b7201b6af1b3f5452549351b018a27806ee5a723 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20K=C3=B6nig?= Date: Wed, 25 Nov 2015 12:51:24 +0100 Subject: [PATCH 2/2] PCI: WIP hack to resize all BARs to their maximum MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Christian König --- drivers/pci/probe.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 8361d27..1d8fc20 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -318,6 +318,17 @@ static void pci_read_bases(struct pci_dev *dev, unsigned int howmany, int rom) for (pos = 0; pos < howmany; pos++) { struct resource *res = &dev->resource[pos]; + u32 sizes = pci_rbar_get_sizes(dev, pos); + + if (sizes) { + /* Just resize to the maximum for now */ + int size = fls(sizes) - 1; + + if (pci_rbar_set_size(dev, pos, size)) + dev_info(&dev->dev, "Resized BAR %d to %dMB\n", + pos, 1 << size); + } + reg = PCI_BASE_ADDRESS_0 + (pos << 2); pos += __pci_read_base(dev, pci_bar_unknown, res, reg); } -- 2.5.0