From patchwork Fri Aug 12 14:40:17 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 9277205 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 537DB60752 for ; Fri, 12 Aug 2016 14:42:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4482228A3A for ; Fri, 12 Aug 2016 14:42:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3921628A3D; Fri, 12 Aug 2016 14:42:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CA01628A3A for ; Fri, 12 Aug 2016 14:42:05 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.85_2 #1 (Red Hat Linux)) id 1bYDdd-0006Jf-3p; Fri, 12 Aug 2016 14:40:45 +0000 Received: from mail-yw0-x241.google.com ([2607:f8b0:4002:c05::241]) by bombadil.infradead.org with esmtps (Exim 4.85_2 #1 (Red Hat Linux)) id 1bYDdX-0005cJ-KD for linux-arm-kernel@lists.infradead.org; Fri, 12 Aug 2016 14:40:40 +0000 Received: by mail-yw0-x241.google.com with SMTP id r9so1198064ywg.2 for ; Fri, 12 Aug 2016 07:40:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=AAnjGgqNz9Wa9MehrnTLmddnQ2jWpMfamlL/f8CmWY0=; b=d5vCv3X4Jxb0sm2lY+FPZDKxRhCzFl+Rl6ByNU7O6AJvsWrlyMpYZIvJdV3DnwzwC2 q0Da9fXwAiJH+6JwkBN5nxZR0dq2W3NznAhzYFg4Dz4zrC1BwzYWM/L1I/0+FLTRPldt kY7ljBQ6yg3Ns7zsvahAO13OMGRIbdmhqYAd5FLdWDqtNXI182482sOyFsJdIBVVTbYS wjVYmiZbPjZ57Q3QSEY/jegG0e+6tUDDPVkjtr+w/isTT8mkZHFSY1QuRacJ1H9nsxCj sbbW1LfT9XRorv9x6tjWp+vKgCQCi/9CIh4e5nMXPaL2YlaAnC7c0QLS9dCw8KE97QUS IpjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=AAnjGgqNz9Wa9MehrnTLmddnQ2jWpMfamlL/f8CmWY0=; b=ce6yvG8WnSvY+ci/bLv6j+WxNuI+VnnLh5ZcbY/b1lJZiE0pgkwkePt25/j7M1O/vE GPTUyaUW/yoXbr7TAPLSvbsj1hQ8yvW3V0PhnMmhkwqo8ThvWS1+4JgraJAH44/rg1il UmG/OkdgmtCw+zSkAkp4gZdrde9PdmUJ9jED3xWEWEIjFJixShEUDmEELS6uxc+JCFM7 CGbjEeV7Rttb+Ll6C7waDnVwtPhabzZ0FU4xWWpT7cK6DAA4f6tt6pLJwqPnMz7lwo00 1P+VuxcIeG1NXcA3JrEMTOZPmEyzrGasly4Rs0psATlvm5APQz0P/V7QVMkDS9NMxatH v6jg== X-Gm-Message-State: AEkoouvKonj8aDaMWapHt3LJ7lK2f65nsFtWtw2A8rgTrxGe164HMD21Md14gvliP5gbWRpP93JcvXvZfJNR3g== X-Received: by 10.129.82.75 with SMTP id g72mr10699039ywb.3.1471012817511; Fri, 12 Aug 2016 07:40:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.13.205.2 with HTTP; Fri, 12 Aug 2016 07:40:17 -0700 (PDT) In-Reply-To: References: <1465817767-9856-1-git-send-email-sricharan@codeaurora.org> <003a01d1f467$2b180ba0$814822e0$@codeaurora.org> <000001d1f499$fe2caf50$fa860df0$@codeaurora.org> <000501d1f4a0$325f0dd0$971d2970$@codeaurora.org> From: Rob Clark Date: Fri, 12 Aug 2016 10:40:17 -0400 Message-ID: Subject: Re: [PATCH V6 0/6] iommu/msm: Add DT adaptation and generic bindings support To: Sricharan X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20160812_074039_867910_716C8C37 X-CRM114-Status: GOOD ( 21.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "devicetree@vger.kernel.org" , Archit Taneja , Arnd Bergmann , linux-arm-msm , Joerg Roedel , "iommu@lists.linux-foundation.org" , Srinivas Kandagatla , Laurent Pinchart , Thierry Reding , Robin Murphy , "linux-arm-kernel@lists.infradead.org" , stepanm@codeaurora.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP On Fri, Aug 12, 2016 at 10:32 AM, Rob Clark wrote: > On Fri, Aug 12, 2016 at 9:48 AM, Sricharan wrote: >> Hi, >> >>>>>>>> btw, the current state, at least on linaro integration branch, fault >>>>>>>> handling doesn't work so well (ie. device never gets resumed).. which >>>>>>>> is a bit unfortunate for a gpu (and results in a *lot* of rebooting on >>>>>>>> my part when debugging userspace). I haven't had time yet to compare >>>>>>>> to the ancient downstream driver, but not sure if you have any ideas? >>>>>>>> >>>>>>>> I guess probably disabling stall on fault would help. But I'm not >>>>>>>> even getting the "Fault occurred in context.." prints. Seeing the >>>>>>>> fault iova is pretty useful since that plus gpu cmdstream trace helps >>>>>>>> me figure out which texture/etc is being accessed out of bounds. >>>>>>> >>>>>>>fyi, it looks like it is not getting any fault irq.. it's *possible* >>>>>>>that I screwed up the irq #'s when translating from downstream, so you >>>>>>>might want to double check that. I thought I had it right, I assume I >>>>>>>would have noticed during piglit runs if fault recovery wasn't working >>>>>>>(since the result is that *everything* after the faulting test would >>>>>>>have failed since gpu is wedged with no access to memory), but it was >>>>>>>long enough ago that I can't claim that definitively. >>>>>>> >>>>>>>If you need an easy way to trigger a gpu fault, msmtest is a good way, >>>>>>>change this line: >>>>>>> >>>>>>> https://github.com/freedreno/msmtest/blob/master/msmtest.c#L247 >>>>>>> >>>>>>>from OUT_RELOC() to OUT_RING(ring, 0x00000000) will trigger a fault. >>>>>>> >>>>>> So for the irq to be triggered, 'non-secure' irq line has to be >>>>>> populated in DT. There is a 'secure'and 'non-secure' irq lines for these iommus >>>>>> and non-secure irq number is secure + 1. I tested this by having a 'return 0' >>>>>> from the msm_iommu_map (no mapping), and the faults were getting triggered. >>>>>> >>>>>> Can you share me your dts data ? >>>>>> >>>>> >>>>>I think this is what you want: >>>>> >>>>>https://github.com/freedreno/kernel-msm/blob/integration-linux-qcomlt/arch/arm/boot/dts/qcom-apq8064.dtsi#L2008 >>>>> >>>>>I haven't tested a display fault, so I suppose it is possible that >>>>>irq's are working for some iommu instances but not others? >>>> >>>> So in your DT, for gfx3d, the non-secure line is '70' and not '69' (This is secure) . >>>> Infact only '70' should be populated. The driver sets the irq line based on resource 0. >>>> This applies for all iommu nodes in your DT. (only the second irq line is needed). >>> >>>ahh, that would explain. >>> >>>Is it better to remove the extra entry, or should I just swap them >>>all? Ie. might there be some point in the future where the driver >>>would want both? >> I feel better to have one. Not sure why the secure irq was added in first >> place in the downstream data and it would setup/handled by the TZ > > Ok, getting further.. still seems like the gpu is not getting resumed, > but at least we are getting a fault interrupt.. ok, seems like we need: ------- ------- with that it seems like it kinda/mostly recovers, although there is still a bit of strangeness.. BR, -R diff --git a/drivers/iommu/msm_iommu.c b/drivers/iommu/msm_iommu.c index b09692b..f6f596f 100644 --- a/drivers/iommu/msm_iommu.c +++ b/drivers/iommu/msm_iommu.c @@ -628,6 +628,7 @@ irqreturn_t msm_iommu_fault_handler(int irq, void *dev_id) pr_err("Interesting registers:\n"); print_ctx_regs(iommu->base, i); SET_FSR(iommu->base, i, 0x4000000F); + SET_RESUME(iommu->base, i, 1); } } __disable_clocks(iommu);