From patchwork Sat Sep 16 06:11:23 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Harsh Jain X-Patchwork-Id: 9954133 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 58777601D5 for ; Sat, 16 Sep 2017 06:14:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 43D7A293BA for ; Sat, 16 Sep 2017 06:14:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 38B0829518; Sat, 16 Sep 2017 06:14:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B226E293BA for ; Sat, 16 Sep 2017 06:14:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751211AbdIPGNX (ORCPT ); Sat, 16 Sep 2017 02:13:23 -0400 Received: from stargate.chelsio.com ([12.32.117.8]:40491 "EHLO l.chelsio.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751203AbdIPGNW (ORCPT ); Sat, 16 Sep 2017 02:13:22 -0400 Received: from [10.193.190.252] (harsh-pc1.asicdesigners.com [10.193.190.252]) by l.chelsio.com (8.13.8/8.13.8) with ESMTP id v8G6BOp7022278; Fri, 15 Sep 2017 23:11:25 -0700 To: dwmw2@infradead.org, joro@8bytes.org, linux-crypto@vger.kernel.org, iommu@lists.linux-foundation.org Cc: linux-kernel@vger.kernel.org, leedom@chelsio.com From: Harsh Jain Subject: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU Message-ID: Date: Sat, 16 Sep 2017 11:41:23 +0530 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 Content-Language: en-US Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hi, While debugging DMA mapping error in chelsio crypto driver we observed that when scatter/gather list received by driver has some entry with page->offset > 4096 (PAGE_SIZE). It starts giving DMA error.  Without IOMMU it works fine. Before reaching to chelsio crypto driver(driver/crypto/chelsio) following entities change the sg' 1) IN esp_output() "__skb_to_sgvec()" convert skb frags to scatter gather list. At that moment sg->offset was 4094. 2) From esp_output control reaches to "crypto_authenc_encrypt()". Here in "scatterwalk_ffwd()" sg->offset become 4110. 3) Same sg list received by chelsio crypto driver(chcr). When chcr try to do DMA mapping it starts giving DMA errors. Following error observed. first two prints are added for debugging in chcr. Kernel version used to reproduce is 4.9.28 on x86_64. Sep 15 12:40:52 heptagon kernel: process_cipher req src ffff8803cb41f0a8 Sep 15 12:40:52 heptagon kernel: ========= issue hit offset:4110 ======= dma_addr f24b000e ==> DMA mapped address returned by dma_map_sg() Sep 15 12:40:52 heptagon kernel: DMAR: DRHD: handling fault status reg 2 Sep 15 12:40:52 heptagon kernel: DMAR: [DMA Write] Request device [02:00.4] fault addr f24b0000 [fault reason 05] PTE Write access is not set  By applying following hack in kernel. Things start working. 1) We are not expecting issue in "scatterwalk_ffwd" because it is not the only place where kernel updates src->offset without checking page boundary. similar logic used in "__skb_to_sgvec". 2) It cannot be driver's responsibilty to update received sg entries to adjust offset and page because we are not the only one who directly uses received sg list. 3) Since Without IOMMU every thing works fine. We are expecting IOMMU bugs. Regards Harsh Jain diff --git a/crypto/scatterwalk.c b/crypto/scatterwalk.c index c16c94f8..1d75a3a 100644 --- a/crypto/scatterwalk.c +++ b/crypto/scatterwalk.c @@ -78,6 +78,8 @@ struct scatterlist *scatterwalk_ffwd(struct scatterlist dst[2] struct scatterlist *src, unsigned int len) { + unsigned int mod_page_offset; + for (;;) { if (!len) return src; @@ -90,7 +92,9 @@ struct scatterlist *scatterwalk_ffwd(struct scatterlist dst[2] } sg_init_table(dst, 2); - sg_set_page(dst, sg_page(src), src->length - len, src->offset + len); + mod_page_offset = (src->offset + len) / PAGE_SIZE; + sg_set_page(dst, sg_page(src) + mod_page_offset, src->length - len, + (src->offset + len) - (mod_page_offset * PAGE_SIZE)); scatterwalk_crypto_chain(dst, sg_next(src), 0, 2);