From patchwork Wed May 24 21:42:19 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Logan Gunthorpe X-Patchwork-Id: 9747123 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D72C4601C2 for ; Wed, 24 May 2017 21:48:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CAA8E26E16 for ; Wed, 24 May 2017 21:48:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BF71D26E4D; Wed, 24 May 2017 21:48:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 27FF026E16 for ; Wed, 24 May 2017 21:48:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1032819AbdEXVro (ORCPT ); Wed, 24 May 2017 17:47:44 -0400 Received: from ale.deltatee.com ([207.54.116.67]:36995 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1033727AbdEXVnV (ORCPT ); Wed, 24 May 2017 17:43:21 -0400 Received: from cgy1-donard.priv.deltatee.com ([172.16.1.31]) by ale.deltatee.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1dDe3j-0007Gp-6r; Wed, 24 May 2017 15:43:15 -0600 Received: from gunthorp by cgy1-donard.priv.deltatee.com with local (Exim 4.84_2) (envelope-from ) id 1dDe3e-0004mK-O3; Wed, 24 May 2017 15:43:06 -0600 From: Logan Gunthorpe To: linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-nvdimm@lists.01.org, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org Cc: Christoph Hellwig , Jason Gunthorpe , Jens Axboe , Dan Williams , Ross Zwisler , Matthew Wilcox , Andrew Morton , Ming Lei , Johannes Berg , Stephen Bates , Sagi Grimberg , Keith Busch , =?UTF-8?q?Christian=20K=C3=B6nig?= , Benjamin Herrenschmidt , Jerome Glisse , Logan Gunthorpe Date: Wed, 24 May 2017 15:42:19 -0600 Message-Id: <1495662147-18277-9-git-send-email-logang@deltatee.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1495662147-18277-1-git-send-email-logang@deltatee.com> References: <1495662147-18277-1-git-send-email-logang@deltatee.com> X-SA-Exim-Connect-IP: 172.16.1.31 X-SA-Exim-Rcpt-To: linux-nvdimm@lists.01.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-rdma@vger.kernel.org, hch@lst.de, jgunthorpe@obsidianresearch.com, axboe@kernel.dk, ross.zwisler@linux.intel.com, mawilcox@microsoft.com, akpm@linux-foundation.org, ming.lei@canonical.com, sbates@raithlin.com, sagi@grimberg.me, dan.j.williams@intel.com, johannes.berg@intel.com, keith.busch@intel.com, christian.koenig@amd.com, benh@kernel.crashing.org, jglisse@redhat.com, logang@deltatee.com X-SA-Exim-Mail-From: gunthorp@deltatee.com Subject: [RFC PATCH 08/16] scatterlist: add iomem support to sg_miter and sg_copy_* X-SA-Exim-Version: 4.2.1 (built Mon, 26 Dec 2011 16:24:06 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP IO memory support can be indicated by an iteratee with the SG_MITRE_SUPPORTS_IOMEM flag. If an unmappable sgl gets into a miter without this support, the kernel will BUGON. For supported cases, the sg_miter code will set the ioaddr pointer and leave addr NULL. sg_copy_buffer is changed to support IO memory by simply calling the appropriate memcpy when required. Signed-off-by: Logan Gunthorpe Signed-off-by: Stephen Bates --- include/linux/scatterlist.h | 3 +++ lib/scatterlist.c | 35 +++++++++++++++++++++++++++++------ 2 files changed, 32 insertions(+), 6 deletions(-) diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h index 8cb662c..2c54c6c 100644 --- a/include/linux/scatterlist.h +++ b/include/linux/scatterlist.h @@ -447,11 +447,14 @@ static inline dma_addr_t sg_page_iter_dma_address(struct sg_page_iter *piter) #define SG_MITER_ATOMIC (1 << 0) /* use kmap_atomic */ #define SG_MITER_TO_SG (1 << 1) /* flush back to phys on unmap */ #define SG_MITER_FROM_SG (1 << 2) /* nop */ +#define SG_MITRE_SUPPORTS_IOMEM (1 << 3) /* io memory is supported by + * the iteratee */ struct sg_mapping_iter { /* the following three fields can be accessed directly */ struct page *page; /* currently mapped page */ void *addr; /* pointer to the mapped area */ + void __iomem *ioaddr; /* address to io memory */ size_t length; /* length of the mapped area */ size_t consumed; /* number of consumed bytes */ struct sg_page_iter piter; /* page iterator */ diff --git a/lib/scatterlist.c b/lib/scatterlist.c index 6bc33e0..1ec768e 100644 --- a/lib/scatterlist.c +++ b/lib/scatterlist.c @@ -601,6 +601,8 @@ EXPORT_SYMBOL(sg_miter_skip); */ bool sg_miter_next(struct sg_mapping_iter *miter) { + pfn_t pfn = miter->piter.sg->__pfn; + sg_miter_stop(miter); /* @@ -610,9 +612,20 @@ bool sg_miter_next(struct sg_mapping_iter *miter) if (!sg_miter_get_next_page(miter)) return false; - miter->page = sg_page_iter_page(&miter->piter); miter->consumed = miter->length = miter->__remaining; + if (!(miter->__flags & SG_MITRE_SUPPORTS_IOMEM)) { + BUG_ON(!pfn_t_is_always_mappable(pfn)); + } else if (pfn_t_is_iomem(pfn)) { + miter->ioaddr = ioremap(pfn_t_to_phys(pfn), PAGE_SIZE); + miter->addr = NULL; + return true; + } + + miter->ioaddr = NULL; + miter->page = pfn_t_to_page(pfn); + BUG_ON(!miter->page); + if (miter->__flags & SG_MITER_ATOMIC) miter->addr = kmap_atomic(miter->page) + miter->__offset; else @@ -660,6 +673,9 @@ void sg_miter_stop(struct sg_mapping_iter *miter) miter->length = 0; miter->consumed = 0; } + + if (miter->ioaddr) + iounmap(miter->ioaddr); } EXPORT_SYMBOL(sg_miter_stop); @@ -681,7 +697,7 @@ size_t sg_copy_buffer(struct scatterlist *sgl, unsigned int nents, void *buf, { unsigned int offset = 0; struct sg_mapping_iter miter; - unsigned int sg_flags = SG_MITER_ATOMIC; + unsigned int sg_flags = SG_MITER_ATOMIC | SG_MITRE_SUPPORTS_IOMEM; if (to_buffer) sg_flags |= SG_MITER_FROM_SG; @@ -698,10 +714,17 @@ size_t sg_copy_buffer(struct scatterlist *sgl, unsigned int nents, void *buf, len = min(miter.length, buflen - offset); - if (to_buffer) - memcpy(buf + offset, miter.addr, len); - else - memcpy(miter.addr, buf + offset, len); + if (miter.addr) { + if (to_buffer) + memcpy(buf + offset, miter.addr, len); + else + memcpy(miter.addr, buf + offset, len); + } else if (miter.ioaddr) { + if (to_buffer) + memcpy_fromio(buf + offset, miter.addr, len); + else + memcpy_toio(miter.addr, buf + offset, len); + } offset += len; }