From patchwork Thu Mar 26 14:58:15 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Hounschell X-Patchwork-Id: 6099881 Return-Path: X-Original-To: patchwork-linux-scsi@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 02CFD9F350 for ; Thu, 26 Mar 2015 14:59:21 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 15D792041B for ; Thu, 26 Mar 2015 14:59:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1559D2041A for ; Thu, 26 Mar 2015 14:59:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753191AbbCZO6T (ORCPT ); Thu, 26 Mar 2015 10:58:19 -0400 Received: from smtp94.ord1c.emailsrvr.com ([108.166.43.94]:57899 "EHLO smtp94.ord1c.emailsrvr.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752914AbbCZO6R (ORCPT ); Thu, 26 Mar 2015 10:58:17 -0400 Received: from smtp4.relay.ord1c.emailsrvr.com (localhost.localdomain [127.0.0.1]) by smtp4.relay.ord1c.emailsrvr.com (SMTP Server) with ESMTP id AC6FC18069A; Thu, 26 Mar 2015 10:58:16 -0400 (EDT) X-SMTPDoctor-Processed: csmtpprox beta Received: from smtp4.relay.ord1c.emailsrvr.com (localhost.localdomain [127.0.0.1]) by smtp4.relay.ord1c.emailsrvr.com (SMTP Server) with ESMTP id 96C34180597; Thu, 26 Mar 2015 10:58:16 -0400 (EDT) Received: by smtp4.relay.ord1c.emailsrvr.com (Authenticated sender: markh-AT-compro.net) with ESMTPSA id 3E32B18069A; Thu, 26 Mar 2015 10:58:16 -0400 (EDT) X-Sender-Id: markh@compro.net Received: from [10.15.15.121] ([UNAVAILABLE]. [12.226.191.242]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA) by 0.0.0.0:465 (trex/5.4.2); Thu, 26 Mar 2015 14:58:16 GMT Message-ID: <55141E87.8040506@compro.net> Date: Thu, 26 Mar 2015 10:58:15 -0400 From: Mark Hounschell Reply-To: markh@compro.net Organization: Compro Computer Svcs. User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Joerg Roedel CC: iommu@lists.linux-foundation.org, Hannes Reinecke , "linux-scsi@vger.kernel.org >> SCSI development list" , linux-kernel@vger.kernel.org Subject: Re: BUG: SCSI aic7xxx driver and AMD IOMMU References: <54E1FFFA.1060403@compro.net> <54F60D33.8090401@compro.net> <20150323150304.GQ4441@8bytes.org> <5511513D.6090106@compro.net> <20150325135937.GR4441@8bytes.org> <20150325151332.GT4441@8bytes.org> <5512E412.9040807@compro.net> <20150326124559.GV4441@8bytes.org> In-Reply-To: <20150326124559.GV4441@8bytes.org> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hi Joerg, On 03/26/2015 08:45 AM, Joerg Roedel wrote: > On Wed, Mar 25, 2015 at 12:36:34PM -0400, Mark Hounschell wrote: >> BTW, so far the first 2 patches are working well. I was going to >> wait until the end of the day to report but so far I have been >> unable to produce the problems I was seeing. And I am in the middle >> of some driver work so lots of unloading/loading going on. > > Great, thanks. Please let me know when you have test results for the > other patches too. > > > Joerg Sorry but CMA was still badly broken. I have a patch below that works. I've tested it with small (no CMA) and large (with CMA) dma's using dma_alloc_coherent. The patch below is just the "git diff" from your cloned tree piped to a file then copied into this email. If you require an official patch I can send one. Just let me know. Also, in my opinion, this CMA thing is clearly a BUG not a feature request. The AMD iommu clearly breaks CMA. I feel what ever fix you are happy with should be back ported to stable. Regards Mark --- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c index a0197d0..5ea4fed 100644 --- a/drivers/iommu/amd_iommu.c +++ b/drivers/iommu/amd_iommu.c @@ -2917,28 +2917,36 @@ static void *alloc_coherent(struct device *dev, size_t size, u64 dma_mask = dev->coherent_dma_mask; struct protection_domain *domain; unsigned long flags; - struct page *page; + struct page *page = 0; + int order; + unsigned int count; + + size = PAGE_ALIGN(size); + order = get_order(size); + count = size >> PAGE_SHIFT; INC_STATS_COUNTER(cnt_alloc_coherent); domain = get_domain(dev); if (PTR_ERR(domain) == -EINVAL) { - page = alloc_pages(flag, get_order(size)); + page = alloc_pages(flag, order); *dma_addr = page_to_phys(page); return page_address(page); } else if (IS_ERR(domain)) return NULL; - dma_mask = dev->coherent_dma_mask; flag &= ~(__GFP_DMA | __GFP_HIGHMEM | __GFP_DMA32); + flag |= __GFP_ZERO; - page = alloc_pages(flag, get_order(size)); + if (flag & __GFP_WAIT) { + page = dma_alloc_from_contiguous(dev, count, order); + if (page && page_to_phys(page) + size > dma_mask) { + dma_release_from_contiguous(dev, page, count); + page = NULL; + } + } if (!page) { - if (!(flag & __GFP_WAIT)) - return NULL; - - page = dma_alloc_from_contiguous(dev, size >> PAGE_SHIFT, - get_order(size)); + page = alloc_pages(flag, order); if (!page) return NULL; } @@ -2951,7 +2959,7 @@ static void *alloc_coherent(struct device *dev, size_t size, *dma_addr = __map_single(dev, domain->priv, page_to_phys(page), size, DMA_BIDIRECTIONAL, true, dma_mask); - if (*dma_addr == DMA_ERROR_CODE) { + if (!dma_addr || (*dma_addr == DMA_ERROR_CODE)) { spin_unlock_irqrestore(&domain->lock, flags); goto out_free; } @@ -2965,7 +2973,7 @@ static void *alloc_coherent(struct device *dev, size_t size, out_free: if (!dma_release_from_contiguous(dev, page, size >> PAGE_SHIFT)) - __free_pages(page, get_order(size)); + __free_pages(page, order); return NULL; } @@ -2979,6 +2987,11 @@ static void free_coherent(struct device *dev, size_t size, { unsigned long flags; struct protection_domain *domain; + int order; + struct page *page = virt_to_page(virt_addr); + + size = PAGE_ALIGN(size); + order = get_order(size); INC_STATS_COUNTER(cnt_free_coherent); @@ -2995,7 +3008,9 @@ static void free_coherent(struct device *dev, size_t size, spin_unlock_irqrestore(&domain->lock, flags); free_mem: - free_pages((unsigned long)virt_addr, get_order(size)); + if (!dma_release_from_contiguous(dev, page, size >> PAGE_SHIFT)) + __free_pages(page, order); + } /*