From patchwork Fri Jan 13 18:39:17 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Stabellini X-Patchwork-Id: 9516211 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 67756601E5 for ; Fri, 13 Jan 2017 18:41:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 589BB2875D for ; Fri, 13 Jan 2017 18:41:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4B1DE28766; Fri, 13 Jan 2017 18:41:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C6DC12875D for ; Fri, 13 Jan 2017 18:41:46 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cS6l4-0007r4-UZ; Fri, 13 Jan 2017 18:39:26 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cS6l3-0007qv-VW for xen-devel@lists.xen.org; Fri, 13 Jan 2017 18:39:26 +0000 Received: from [85.158.139.211] by server-5.bemta-5.messagelabs.com id 89/7A-04025-DDE19785; Fri, 13 Jan 2017 18:39:25 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrIIsWRWlGSWpSXmKPExsVybKJsh+5tuco Ig1VdUhZLPi5mcWD0OLr7N1MAYxRrZl5SfkUCa8bPs+oFK8Qr3sx5z9jA+Feoi5GLQ0hgKqPE pEn7GLsYOYGc2UwS/477gtgsAtoS8/7vYwWx2QQMJf4+2cTWxcjBIQFkL/nMARIWETCTuNP8n BlkDrPADxaJvQceMYMkhAViJGY83soEYnMKOEg8WHCRBaSXV8BbYsvsRIi9y5glZjbOB5svKq ArcejfHzYQm1dAUOLkzCcsIDazgJbE8unbwGwJgQyJeT1zWCFsL4lFNy5B2WoSV89tYp7AKDg LSfssJO0LGJlWMWoUpxaVpRbpGlnoJRVlpmeU5CZm5ugaGpjq5aYWFyemp+YkJhXrJefnbmIE Bmc9AwPjDsa+VX6HGCU5mJREeb+rVkQI8SXlp1RmJBZnxBeV5qQWH2KU4eBQkuCdJlsZISRYl JqeWpGWmQOME5i0BAePkggvMzBWhHiLCxJzizPTIVKnGHU5Tt04/ZJJiCUvPy9VSpx3O8gMAZ CijNI8uBGwmL3EKCslzMvIwMAgxFOQWpSbWYIq/4pRnINRSZiXBWQVT2ZeCdymV0BHMAEdcdG mHOSIkkSElFQDY0efBmPvutmrtq3urDO/4H83Z5XKQoegfz9ZNt7SMzgmtcnuE1OK6PG0t1eO z9DNbP2REH798YROi1kfwy0ffb6z6YO4vuT0O0q9m85PyZbezNzy6nrh8y3HrgQnfztd9Shid nIhx1VDxso18bbB+hZvrTPkpG/9zhffbb6dcc27RVKJDgq6n5RYijMSDbWYi4oTAU6xm1LUAg AA X-Env-Sender: sstabellini@kernel.org X-Msg-Ref: server-7.tower-206.messagelabs.com!1484332762!79690282!1 X-Originating-IP: [198.145.29.136] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 9.1.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 65532 invoked from network); 13 Jan 2017 18:39:23 -0000 Received: from mail.kernel.org (HELO mail.kernel.org) (198.145.29.136) by server-7.tower-206.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 13 Jan 2017 18:39:23 -0000 Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 7AE1520123; Fri, 13 Jan 2017 18:39:20 +0000 (UTC) Received: from [10.1.10.56] (96-82-76-110-static.hfc.comcastbusiness.net [96.82.76.110]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E012A200D9; Fri, 13 Jan 2017 18:39:18 +0000 (UTC) Date: Fri, 13 Jan 2017 10:39:17 -0800 (PST) From: Stefano Stabellini X-X-Sender: sstabellini@sstabellini-ThinkPad-X260 To: "Pooya.Keshavarzi" In-Reply-To: <40fc7a67-3f23-daa9-44a3-4c2566f76d8f@de.bosch.com> Message-ID: References: <5f7e4c02-3312-3726-7c9b-576b7cc00694@epam.com> <79a88682-1d58-7e68-2385-ae3a82f519bb@epam.com> <2a5febf2-31c2-9002-55c9-b39e809f01f0@de.bosch.com> <40fc7a67-3f23-daa9-44a3-4c2566f76d8f@de.bosch.com> User-Agent: Alpine 2.10 (DEB 1266 2009-07-14) MIME-Version: 1.0 X-Virus-Scanned: ClamAV using ClamSMTP Cc: Artem Mygaiev , "Edgar E. Iglesias" , Dirk Behme , Andrii Anisov , Lars Kurth , Dario Faggioli , Andrii Anisov , Alex_Agizim@epam.com, stewart.hildebrand@dornerworks.com, Julien Grall , Stefano Stabellini , Meng Xu , anastassios.nanos@onapp.com, "Edgar E. Iglesias" , Xen Devel Subject: Re: [Xen-devel] Xen ARM community call - meeting minutes and date for the next one X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP On Fri, 13 Jan 2017, Pooya.Keshavarzi wrote: > On 01/12/2017 07:50 PM, Stefano Stabellini wrote: > > On Thu, 12 Jan 2017, Pooya.Keshavarzi wrote: > >> > >> Firstly sorry for the late reply on this. > >> > >> Regarding the problem with swiotlb-xen here are some more details: > >> > >> If we limit Dom0's memory such that only low-memory (up to 32-bit addressable memory) is available to Dom0, then swiotlb-xen does not have to use bounce buffers and the devices (e.g. USB, ethernet) would work. > >> > >> But when there is some high memory also available to Dom0, the followings happen: > >> - If the the device address happens to be in the device's DMA window (see xen_swiotlb_map_page()), then the device would work. > >> - Otherwise if it has to allocate and map a bounce buffer, then the device would not work. > > > > From what you wrote it looks like the xen_swiotlb_map_page path: > > > > if (dma_capable(dev, dev_addr, size) && > > !range_straddles_page_boundary(phys, size) && > > !xen_arch_need_swiotlb(dev, phys, dev_addr) && > > !swiotlb_force) { > > /* we are not interested in the dma_addr returned by > > * xen_dma_map_page, only in the potential cache flushes executed > > * by the function. */ > > xen_dma_map_page(dev, page, dev_addr, offset, size, dir, attrs); > > return dev_addr; > > } > > > > works, but the other does not. Does it match your understanding? Have > > you done any digging to find the reason why the bounce buffer code path > > is broken on your platform? > > Yes, The above path works but the other one doesn't. > I did some digging but failed to find out what's the problem. The returned address from swiotlb_tbl_map_single() is within the memory range allocated earlier for Xen software IO TLB and is dma capable, so it seem to be OK. > > What's your suggestion for further digging? Is the device DMA coherent? I take that it fails even without running any guests, correct? A few things come to mind: - In xen_dma_map_page, does it take the local path or the foreign path (if(local)...) when it fails? - Check that xen_swiotlb_init initializes the swiotlb memory region appriopriately and that it falls in a memory range supported by the device. - Check that xen_phys_to_bus(map) returns the right dev_addr. As a reference, I know that on some arm32 platforms it wouldn't return the right value. - Does the following patch fixes the issue for you? diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c index 87e6035..17c65fa 100644 --- a/drivers/xen/swiotlb-xen.c +++ b/drivers/xen/swiotlb-xen.c @@ -409,9 +409,9 @@ dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page, if (map == SWIOTLB_MAP_ERROR) return DMA_ERROR_CODE; + dev_addr = xen_phys_to_bus(map); xen_dma_map_page(dev, pfn_to_page(map >> PAGE_SHIFT), dev_addr, map & ~PAGE_MASK, size, dir, attrs); - dev_addr = xen_phys_to_bus(map); /* * Ensure that the address returned is DMA'ble