From patchwork Mon Dec 16 10:53:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sascha Hauer X-Patchwork-Id: 11293775 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 62F29138D for ; Mon, 16 Dec 2019 10:53:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3F7B520725 for ; Mon, 16 Dec 2019 10:53:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727482AbfLPKx7 (ORCPT ); Mon, 16 Dec 2019 05:53:59 -0500 Received: from metis.ext.pengutronix.de ([85.220.165.71]:40647 "EHLO metis.ext.pengutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727403AbfLPKx6 (ORCPT ); Mon, 16 Dec 2019 05:53:58 -0500 Received: from dude.hi.pengutronix.de ([2001:67c:670:100:1d::7]) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1igo0L-0001dX-9V; Mon, 16 Dec 2019 11:53:33 +0100 Received: from sha by dude.hi.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1igo0I-0005t4-Am; Mon, 16 Dec 2019 11:53:30 +0100 From: Sascha Hauer To: dmaengine@vger.kernel.org Cc: Vinod Koul , Peter Ujfalusi , Florian Fainelli , Maxime Ripard , Green Wan , Kukjin Kim , Krzysztof Kozlowski , =?utf-8?q?Andreas_F=C3=A4rber?= , Sean Wang , Eugeniy Paltsev , Pengutronix Kernel Team , NXP Linux Team , Robert Jarzmik , Sascha Hauer Subject: [PATCH 1/9] dmaengine: bcm2835: do not call vchan_vdesc_fini() with lock held Date: Mon, 16 Dec 2019 11:53:20 +0100 Message-Id: <20191216105328.15198-2-s.hauer@pengutronix.de> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191216105328.15198-1-s.hauer@pengutronix.de> References: <20191216105328.15198-1-s.hauer@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2001:67c:670:100:1d::7 X-SA-Exim-Mail-From: sha@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: dmaengine@vger.kernel.org Sender: dmaengine-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org vchan_vdesc_fini() can't be called locked. Instead, call vchan_terminate_vdesc() which delays the freeing of the descriptor to vchan_synchronize(). Signed-off-by: Sascha Hauer --- drivers/dma/bcm2835-dma.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/drivers/dma/bcm2835-dma.c b/drivers/dma/bcm2835-dma.c index e4c593f48575..4768ef26013b 100644 --- a/drivers/dma/bcm2835-dma.c +++ b/drivers/dma/bcm2835-dma.c @@ -797,10 +797,7 @@ static int bcm2835_dma_terminate_all(struct dma_chan *chan) /* stop DMA activity */ if (c->desc) { - if (c->desc->vd.tx.flags & DMA_PREP_INTERRUPT) - vchan_terminate_vdesc(&c->desc->vd); - else - vchan_vdesc_fini(&c->desc->vd); + vchan_terminate_vdesc(&c->desc->vd); c->desc = NULL; bcm2835_dma_abort(c); } From patchwork Mon Dec 16 10:53:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sascha Hauer X-Patchwork-Id: 11293773 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6C398138D for ; Mon, 16 Dec 2019 10:53:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4834120684 for ; Mon, 16 Dec 2019 10:53:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727536AbfLPKxv (ORCPT ); Mon, 16 Dec 2019 05:53:51 -0500 Received: from metis.ext.pengutronix.de ([85.220.165.71]:59057 "EHLO metis.ext.pengutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727482AbfLPKxu (ORCPT ); Mon, 16 Dec 2019 05:53:50 -0500 Received: from dude.hi.pengutronix.de ([2001:67c:670:100:1d::7]) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1igo0L-0001dY-9U; Mon, 16 Dec 2019 11:53:33 +0100 Received: from sha by dude.hi.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1igo0I-0005t7-BW; Mon, 16 Dec 2019 11:53:30 +0100 From: Sascha Hauer To: dmaengine@vger.kernel.org Cc: Vinod Koul , Peter Ujfalusi , Florian Fainelli , Maxime Ripard , Green Wan , Kukjin Kim , Krzysztof Kozlowski , =?utf-8?q?Andreas_F=C3=A4rber?= , Sean Wang , Eugeniy Paltsev , Pengutronix Kernel Team , NXP Linux Team , Robert Jarzmik , Sascha Hauer Subject: [PATCH 2/9] dmaengine: virt-dma: Add missing locking Date: Mon, 16 Dec 2019 11:53:21 +0100 Message-Id: <20191216105328.15198-3-s.hauer@pengutronix.de> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191216105328.15198-1-s.hauer@pengutronix.de> References: <20191216105328.15198-1-s.hauer@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2001:67c:670:100:1d::7 X-SA-Exim-Mail-From: sha@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: dmaengine@vger.kernel.org Sender: dmaengine-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org Originally freeing descriptors was split into a locked and an unlocked part. The locked part in vchan_get_all_descriptors() collected all descriptors on a separate list_head. This was done to allow iterating over that new list in vchan_dma_desc_free_list() without a lock held. This became broken in 13bb26ae8850 ("dmaengine: virt-dma: don't always free descriptor upon completion"). With this commit vchan_dma_desc_free_list() no longer exclusively operates on the separate list, but starts to put descriptors which can be reused back on &vc->desc_allocated. This list operation should have been locked, but wasn't. In the mean time drivers started to call vchan_dma_desc_free_list() with their lock held so that we now have the situation that vchan_dma_desc_free_list() is called locked from some drivers and unlocked from others. To clean this up we have to do two things: 1. Add missing locking in vchan_dma_desc_free_list() 2. Make sure drivers call vchan_dma_desc_free_list() unlocked This needs to be done atomically, so in this patch the locking is added and all drivers are fixed. Signed-off-by: Sascha Hauer --- .../dma/dw-axi-dmac/dw-axi-dmac-platform.c | 8 ++----- drivers/dma/mediatek/mtk-uart-apdma.c | 3 ++- drivers/dma/owl-dma.c | 3 ++- drivers/dma/s3c24xx-dma.c | 22 +++++++++---------- drivers/dma/sf-pdma/sf-pdma.c | 4 ++-- drivers/dma/sun4i-dma.c | 3 ++- drivers/dma/virt-dma.c | 4 ++++ 7 files changed, 25 insertions(+), 22 deletions(-) diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c index a1ce307c502f..14c1ac26f866 100644 --- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c +++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c @@ -636,14 +636,10 @@ static int dma_chan_terminate_all(struct dma_chan *dchan) vchan_get_all_descriptors(&chan->vc, &head); - /* - * As vchan_dma_desc_free_list can access to desc_allocated list - * we need to call it in vc.lock context. - */ - vchan_dma_desc_free_list(&chan->vc, &head); - spin_unlock_irqrestore(&chan->vc.lock, flags); + vchan_dma_desc_free_list(&chan->vc, &head); + dev_vdbg(dchan2dev(dchan), "terminated: %s\n", axi_chan_name(chan)); return 0; diff --git a/drivers/dma/mediatek/mtk-uart-apdma.c b/drivers/dma/mediatek/mtk-uart-apdma.c index c20e6bd4e298..29f1223b285a 100644 --- a/drivers/dma/mediatek/mtk-uart-apdma.c +++ b/drivers/dma/mediatek/mtk-uart-apdma.c @@ -430,9 +430,10 @@ static int mtk_uart_apdma_terminate_all(struct dma_chan *chan) spin_lock_irqsave(&c->vc.lock, flags); vchan_get_all_descriptors(&c->vc, &head); - vchan_dma_desc_free_list(&c->vc, &head); spin_unlock_irqrestore(&c->vc.lock, flags); + vchan_dma_desc_free_list(&c->vc, &head); + return 0; } diff --git a/drivers/dma/owl-dma.c b/drivers/dma/owl-dma.c index 023f951189a7..c683051257fd 100644 --- a/drivers/dma/owl-dma.c +++ b/drivers/dma/owl-dma.c @@ -674,10 +674,11 @@ static int owl_dma_terminate_all(struct dma_chan *chan) } vchan_get_all_descriptors(&vchan->vc, &head); - vchan_dma_desc_free_list(&vchan->vc, &head); spin_unlock_irqrestore(&vchan->vc.lock, flags); + vchan_dma_desc_free_list(&vchan->vc, &head); + return 0; } diff --git a/drivers/dma/s3c24xx-dma.c b/drivers/dma/s3c24xx-dma.c index 43da8eeb18ef..1ed5dc1f597c 100644 --- a/drivers/dma/s3c24xx-dma.c +++ b/drivers/dma/s3c24xx-dma.c @@ -519,15 +519,6 @@ static void s3c24xx_dma_start_next_txd(struct s3c24xx_dma_chan *s3cchan) s3c24xx_dma_start_next_sg(s3cchan, txd); } -static void s3c24xx_dma_free_txd_list(struct s3c24xx_dma_engine *s3cdma, - struct s3c24xx_dma_chan *s3cchan) -{ - LIST_HEAD(head); - - vchan_get_all_descriptors(&s3cchan->vc, &head); - vchan_dma_desc_free_list(&s3cchan->vc, &head); -} - /* * Try to allocate a physical channel. When successful, assign it to * this virtual channel, and initiate the next descriptor. The @@ -709,8 +700,9 @@ static int s3c24xx_dma_terminate_all(struct dma_chan *chan) { struct s3c24xx_dma_chan *s3cchan = to_s3c24xx_dma_chan(chan); struct s3c24xx_dma_engine *s3cdma = s3cchan->host; + LIST_HEAD(head); unsigned long flags; - int ret = 0; + int ret; spin_lock_irqsave(&s3cchan->vc.lock, flags); @@ -734,7 +726,15 @@ static int s3c24xx_dma_terminate_all(struct dma_chan *chan) } /* Dequeue jobs not yet fired as well */ - s3c24xx_dma_free_txd_list(s3cdma, s3cchan); + + vchan_get_all_descriptors(&s3cchan->vc, &head); + + spin_unlock_irqrestore(&s3cchan->vc.lock, flags); + + vchan_dma_desc_free_list(&s3cchan->vc, &head); + + return 0; + unlock: spin_unlock_irqrestore(&s3cchan->vc.lock, flags); diff --git a/drivers/dma/sf-pdma/sf-pdma.c b/drivers/dma/sf-pdma/sf-pdma.c index 465256fe8b1f..6d0bec947636 100644 --- a/drivers/dma/sf-pdma/sf-pdma.c +++ b/drivers/dma/sf-pdma/sf-pdma.c @@ -155,9 +155,9 @@ static void sf_pdma_free_chan_resources(struct dma_chan *dchan) kfree(chan->desc); chan->desc = NULL; vchan_get_all_descriptors(&chan->vchan, &head); - vchan_dma_desc_free_list(&chan->vchan, &head); sf_pdma_disclaim_chan(chan); spin_unlock_irqrestore(&chan->vchan.lock, flags); + vchan_dma_desc_free_list(&chan->vchan, &head); } static size_t sf_pdma_desc_residue(struct sf_pdma_chan *chan, @@ -220,8 +220,8 @@ static int sf_pdma_terminate_all(struct dma_chan *dchan) chan->desc = NULL; chan->xfer_err = false; vchan_get_all_descriptors(&chan->vchan, &head); - vchan_dma_desc_free_list(&chan->vchan, &head); spin_unlock_irqrestore(&chan->vchan.lock, flags); + vchan_dma_desc_free_list(&chan->vchan, &head); return 0; } diff --git a/drivers/dma/sun4i-dma.c b/drivers/dma/sun4i-dma.c index e397a50058c8..4e1575e731d8 100644 --- a/drivers/dma/sun4i-dma.c +++ b/drivers/dma/sun4i-dma.c @@ -885,12 +885,13 @@ static int sun4i_dma_terminate_all(struct dma_chan *chan) } spin_lock_irqsave(&vchan->vc.lock, flags); - vchan_dma_desc_free_list(&vchan->vc, &head); /* Clear these so the vchan is usable again */ vchan->processing = NULL; vchan->pchan = NULL; spin_unlock_irqrestore(&vchan->vc.lock, flags); + vchan_dma_desc_free_list(&vchan->vc, &head); + return 0; } diff --git a/drivers/dma/virt-dma.c b/drivers/dma/virt-dma.c index ec4adf4260a0..660267ca5e42 100644 --- a/drivers/dma/virt-dma.c +++ b/drivers/dma/virt-dma.c @@ -116,7 +116,11 @@ void vchan_dma_desc_free_list(struct virt_dma_chan *vc, struct list_head *head) list_for_each_entry_safe(vd, _vd, head, node) { if (dmaengine_desc_test_reuse(&vd->tx)) { + unsigned long flags; + + spin_lock_irqsave(&vc->lock, flags); list_move_tail(&vd->node, &vc->desc_allocated); + spin_unlock_irqrestore(&vc->lock, flags); } else { dev_dbg(vc->chan.device->dev, "txd %p: freeing\n", vd); list_del(&vd->node); From patchwork Mon Dec 16 10:53:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sascha Hauer X-Patchwork-Id: 11293783 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 09B1F139A for ; Mon, 16 Dec 2019 10:54:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DB9A32072D for ; Mon, 16 Dec 2019 10:54:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727569AbfLPKyX (ORCPT ); Mon, 16 Dec 2019 05:54:23 -0500 Received: from metis.ext.pengutronix.de ([85.220.165.71]:48451 "EHLO metis.ext.pengutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727436AbfLPKyX (ORCPT ); Mon, 16 Dec 2019 05:54:23 -0500 Received: from dude.hi.pengutronix.de ([2001:67c:670:100:1d::7]) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1igo0L-0001dZ-9W; Mon, 16 Dec 2019 11:53:33 +0100 Received: from sha by dude.hi.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1igo0I-0005tA-CD; Mon, 16 Dec 2019 11:53:30 +0100 From: Sascha Hauer To: dmaengine@vger.kernel.org Cc: Vinod Koul , Peter Ujfalusi , Florian Fainelli , Maxime Ripard , Green Wan , Kukjin Kim , Krzysztof Kozlowski , =?utf-8?q?Andreas_F=C3=A4rber?= , Sean Wang , Eugeniy Paltsev , Pengutronix Kernel Team , NXP Linux Team , Robert Jarzmik , Sascha Hauer Subject: [PATCH 3/9] dmaengine: virt-dma: remove debug message Date: Mon, 16 Dec 2019 11:53:22 +0100 Message-Id: <20191216105328.15198-4-s.hauer@pengutronix.de> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191216105328.15198-1-s.hauer@pengutronix.de> References: <20191216105328.15198-1-s.hauer@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2001:67c:670:100:1d::7 X-SA-Exim-Mail-From: sha@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: dmaengine@vger.kernel.org Sender: dmaengine-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org vchan_dma_desc_free_list() basically open codes vchan_vdesc_fini() in the loop body. One difference is an additional debug message. As this isn't overly useful remove it. Signed-off-by: Sascha Hauer --- drivers/dma/virt-dma.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/dma/virt-dma.c b/drivers/dma/virt-dma.c index 660267ca5e42..7ba712888ac7 100644 --- a/drivers/dma/virt-dma.c +++ b/drivers/dma/virt-dma.c @@ -122,7 +122,6 @@ void vchan_dma_desc_free_list(struct virt_dma_chan *vc, struct list_head *head) list_move_tail(&vd->node, &vc->desc_allocated); spin_unlock_irqrestore(&vc->lock, flags); } else { - dev_dbg(vc->chan.device->dev, "txd %p: freeing\n", vd); list_del(&vd->node); vc->desc_free(vd); } From patchwork Mon Dec 16 10:53:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sascha Hauer X-Patchwork-Id: 11293781 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F1AEF138D for ; Mon, 16 Dec 2019 10:54:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D709220684 for ; Mon, 16 Dec 2019 10:54:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727560AbfLPKyP (ORCPT ); Mon, 16 Dec 2019 05:54:15 -0500 Received: from metis.ext.pengutronix.de ([85.220.165.71]:43365 "EHLO metis.ext.pengutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727436AbfLPKyO (ORCPT ); Mon, 16 Dec 2019 05:54:14 -0500 Received: from dude.hi.pengutronix.de ([2001:67c:670:100:1d::7]) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1igo0L-0001da-9N; Mon, 16 Dec 2019 11:53:33 +0100 Received: from sha by dude.hi.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1igo0I-0005tD-Ct; Mon, 16 Dec 2019 11:53:30 +0100 From: Sascha Hauer To: dmaengine@vger.kernel.org Cc: Vinod Koul , Peter Ujfalusi , Florian Fainelli , Maxime Ripard , Green Wan , Kukjin Kim , Krzysztof Kozlowski , =?utf-8?q?Andreas_F=C3=A4rber?= , Sean Wang , Eugeniy Paltsev , Pengutronix Kernel Team , NXP Linux Team , Robert Jarzmik , Sascha Hauer Subject: [PATCH 4/9] dmaengine: virt-dma: Do not call desc_free() under a spin_lock Date: Mon, 16 Dec 2019 11:53:23 +0100 Message-Id: <20191216105328.15198-5-s.hauer@pengutronix.de> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191216105328.15198-1-s.hauer@pengutronix.de> References: <20191216105328.15198-1-s.hauer@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2001:67c:670:100:1d::7 X-SA-Exim-Mail-From: sha@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: dmaengine@vger.kernel.org Sender: dmaengine-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org vchan_vdesc_fini() shouldn't be called under a spin_lock. This is done in two places, once in vchan_terminate_vdesc() and once in vchan_synchronize(). Instead of freeing the vdesc right away, collect the aborted vdescs on a separate list and free them along with the other vdescs. The terminated descs are also freed in vchan_synchronize as done before this patch. Signed-off-by: Sascha Hauer --- drivers/dma/virt-dma.c | 1 + drivers/dma/virt-dma.h | 18 +++++++++--------- 2 files changed, 10 insertions(+), 9 deletions(-) diff --git a/drivers/dma/virt-dma.c b/drivers/dma/virt-dma.c index 7ba712888ac7..26e08c7a7465 100644 --- a/drivers/dma/virt-dma.c +++ b/drivers/dma/virt-dma.c @@ -138,6 +138,7 @@ void vchan_init(struct virt_dma_chan *vc, struct dma_device *dmadev) INIT_LIST_HEAD(&vc->desc_submitted); INIT_LIST_HEAD(&vc->desc_issued); INIT_LIST_HEAD(&vc->desc_completed); + INIT_LIST_HEAD(&vc->desc_terminated); tasklet_init(&vc->task, vchan_complete, (unsigned long)vc); diff --git a/drivers/dma/virt-dma.h b/drivers/dma/virt-dma.h index ab158bac03a7..e213137b6bc1 100644 --- a/drivers/dma/virt-dma.h +++ b/drivers/dma/virt-dma.h @@ -31,9 +31,9 @@ struct virt_dma_chan { struct list_head desc_submitted; struct list_head desc_issued; struct list_head desc_completed; + struct list_head desc_terminated; struct virt_dma_desc *cyclic; - struct virt_dma_desc *vd_terminated; }; static inline struct virt_dma_chan *to_virt_chan(struct dma_chan *chan) @@ -141,11 +141,8 @@ static inline void vchan_terminate_vdesc(struct virt_dma_desc *vd) { struct virt_dma_chan *vc = to_virt_chan(vd->tx.chan); - /* free up stuck descriptor */ - if (vc->vd_terminated) - vchan_vdesc_fini(vc->vd_terminated); + list_add_tail(&vd->node, &vc->desc_terminated); - vc->vd_terminated = vd; if (vc->cyclic == vd) vc->cyclic = NULL; } @@ -179,6 +176,7 @@ static inline void vchan_get_all_descriptors(struct virt_dma_chan *vc, list_splice_tail_init(&vc->desc_submitted, head); list_splice_tail_init(&vc->desc_issued, head); list_splice_tail_init(&vc->desc_completed, head); + list_splice_tail_init(&vc->desc_terminated, head); } static inline void vchan_free_chan_resources(struct virt_dma_chan *vc) @@ -207,16 +205,18 @@ static inline void vchan_free_chan_resources(struct virt_dma_chan *vc) */ static inline void vchan_synchronize(struct virt_dma_chan *vc) { + LIST_HEAD(head); unsigned long flags; tasklet_kill(&vc->task); spin_lock_irqsave(&vc->lock, flags); - if (vc->vd_terminated) { - vchan_vdesc_fini(vc->vd_terminated); - vc->vd_terminated = NULL; - } + + list_splice_tail_init(&vc->desc_terminated, &head); + spin_unlock_irqrestore(&vc->lock, flags); + + vchan_dma_desc_free_list(vc, &head); } #endif From patchwork Mon Dec 16 10:53:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sascha Hauer X-Patchwork-Id: 11293785 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ECCA6138D for ; Mon, 16 Dec 2019 10:54:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CB865207FF for ; Mon, 16 Dec 2019 10:54:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727571AbfLPKyb (ORCPT ); Mon, 16 Dec 2019 05:54:31 -0500 Received: from metis.ext.pengutronix.de ([85.220.165.71]:58385 "EHLO metis.ext.pengutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727570AbfLPKyb (ORCPT ); Mon, 16 Dec 2019 05:54:31 -0500 Received: from dude.hi.pengutronix.de ([2001:67c:670:100:1d::7]) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1igo0L-0001db-9e; Mon, 16 Dec 2019 11:53:33 +0100 Received: from sha by dude.hi.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1igo0I-0005tG-De; Mon, 16 Dec 2019 11:53:30 +0100 From: Sascha Hauer To: dmaengine@vger.kernel.org Cc: Vinod Koul , Peter Ujfalusi , Florian Fainelli , Maxime Ripard , Green Wan , Kukjin Kim , Krzysztof Kozlowski , =?utf-8?q?Andreas_F=C3=A4rber?= , Sean Wang , Eugeniy Paltsev , Pengutronix Kernel Team , NXP Linux Team , Robert Jarzmik , Sascha Hauer Subject: [PATCH 5/9] dmaengine: virt-dma: Add missing locking around list operations Date: Mon, 16 Dec 2019 11:53:24 +0100 Message-Id: <20191216105328.15198-6-s.hauer@pengutronix.de> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191216105328.15198-1-s.hauer@pengutronix.de> References: <20191216105328.15198-1-s.hauer@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2001:67c:670:100:1d::7 X-SA-Exim-Mail-From: sha@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: dmaengine@vger.kernel.org Sender: dmaengine-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org All list operations are protected by &vc->lock. As vchan_vdesc_fini() is called unlocked add the missing locking around the list operations. Signed-off-by: Sascha Hauer --- drivers/dma/virt-dma.h | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/drivers/dma/virt-dma.h b/drivers/dma/virt-dma.h index e213137b6bc1..e9f5250fbe4d 100644 --- a/drivers/dma/virt-dma.h +++ b/drivers/dma/virt-dma.h @@ -113,10 +113,15 @@ static inline void vchan_vdesc_fini(struct virt_dma_desc *vd) { struct virt_dma_chan *vc = to_virt_chan(vd->tx.chan); - if (dmaengine_desc_test_reuse(&vd->tx)) + if (dmaengine_desc_test_reuse(&vd->tx)) { + unsigned long flags; + + spin_lock_irqsave(&vc->lock, flags); list_add(&vd->node, &vc->desc_allocated); - else + spin_unlock_irqrestore(&vc->lock, flags); + } else { vc->desc_free(vd); + } } /** From patchwork Mon Dec 16 10:53:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sascha Hauer X-Patchwork-Id: 11293769 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3774B1593 for ; Mon, 16 Dec 2019 10:53:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1254420725 for ; Mon, 16 Dec 2019 10:53:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727229AbfLPKxf (ORCPT ); Mon, 16 Dec 2019 05:53:35 -0500 Received: from metis.ext.pengutronix.de ([85.220.165.71]:60969 "EHLO metis.ext.pengutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727099AbfLPKxf (ORCPT ); Mon, 16 Dec 2019 05:53:35 -0500 Received: from dude.hi.pengutronix.de ([2001:67c:670:100:1d::7]) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1igo0L-0001dc-9O; Mon, 16 Dec 2019 11:53:33 +0100 Received: from sha by dude.hi.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1igo0I-0005tJ-EH; Mon, 16 Dec 2019 11:53:30 +0100 From: Sascha Hauer To: dmaengine@vger.kernel.org Cc: Vinod Koul , Peter Ujfalusi , Florian Fainelli , Maxime Ripard , Green Wan , Kukjin Kim , Krzysztof Kozlowski , =?utf-8?q?Andreas_F=C3=A4rber?= , Sean Wang , Eugeniy Paltsev , Pengutronix Kernel Team , NXP Linux Team , Robert Jarzmik , Sascha Hauer Subject: [PATCH 6/9] dmaengine: virt-dma: use vchan_vdesc_fini() to free descriptors Date: Mon, 16 Dec 2019 11:53:25 +0100 Message-Id: <20191216105328.15198-7-s.hauer@pengutronix.de> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191216105328.15198-1-s.hauer@pengutronix.de> References: <20191216105328.15198-1-s.hauer@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2001:67c:670:100:1d::7 X-SA-Exim-Mail-From: sha@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: dmaengine@vger.kernel.org Sender: dmaengine-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org vchan_dma_desc_free_list() basically open codes vchan_vdesc_fini() in its loop body. Call it directly rather than duplicating the code. Signed-off-by: Sascha Hauer --- drivers/dma/virt-dma.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/drivers/dma/virt-dma.c b/drivers/dma/virt-dma.c index 26e08c7a7465..95dfe431777e 100644 --- a/drivers/dma/virt-dma.c +++ b/drivers/dma/virt-dma.c @@ -115,16 +115,8 @@ void vchan_dma_desc_free_list(struct virt_dma_chan *vc, struct list_head *head) struct virt_dma_desc *vd, *_vd; list_for_each_entry_safe(vd, _vd, head, node) { - if (dmaengine_desc_test_reuse(&vd->tx)) { - unsigned long flags; - - spin_lock_irqsave(&vc->lock, flags); - list_move_tail(&vd->node, &vc->desc_allocated); - spin_unlock_irqrestore(&vc->lock, flags); - } else { - list_del(&vd->node); - vc->desc_free(vd); - } + list_del(&vd->node); + vchan_vdesc_fini(vd); } } EXPORT_SYMBOL_GPL(vchan_dma_desc_free_list); From patchwork Mon Dec 16 10:53:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sascha Hauer X-Patchwork-Id: 11293777 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 30CBF138D for ; Mon, 16 Dec 2019 10:54:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 179C720725 for ; Mon, 16 Dec 2019 10:54:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727422AbfLPKyG (ORCPT ); Mon, 16 Dec 2019 05:54:06 -0500 Received: from metis.ext.pengutronix.de ([85.220.165.71]:43691 "EHLO metis.ext.pengutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727403AbfLPKyG (ORCPT ); Mon, 16 Dec 2019 05:54:06 -0500 Received: from dude.hi.pengutronix.de ([2001:67c:670:100:1d::7]) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1igo0L-0001dd-9S; Mon, 16 Dec 2019 11:53:33 +0100 Received: from sha by dude.hi.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1igo0I-0005tN-Ew; Mon, 16 Dec 2019 11:53:30 +0100 From: Sascha Hauer To: dmaengine@vger.kernel.org Cc: Vinod Koul , Peter Ujfalusi , Florian Fainelli , Maxime Ripard , Green Wan , Kukjin Kim , Krzysztof Kozlowski , =?utf-8?q?Andreas_F=C3=A4rber?= , Sean Wang , Eugeniy Paltsev , Pengutronix Kernel Team , NXP Linux Team , Robert Jarzmik , Sascha Hauer Subject: [PATCH 7/9] dmaengine: imx-sdma: rename function Date: Mon, 16 Dec 2019 11:53:26 +0100 Message-Id: <20191216105328.15198-8-s.hauer@pengutronix.de> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191216105328.15198-1-s.hauer@pengutronix.de> References: <20191216105328.15198-1-s.hauer@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2001:67c:670:100:1d::7 X-SA-Exim-Mail-From: sha@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: dmaengine@vger.kernel.org Sender: dmaengine-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org Rename sdma_disable_channel_async() after the hook it implements, like done for all other functions in the SDMA driver. Signed-off-by: Sascha Hauer --- drivers/dma/imx-sdma.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/dma/imx-sdma.c b/drivers/dma/imx-sdma.c index c27e206a764c..527f8a81f50b 100644 --- a/drivers/dma/imx-sdma.c +++ b/drivers/dma/imx-sdma.c @@ -1077,7 +1077,7 @@ static void sdma_channel_terminate_work(struct work_struct *work) sdmac->context_loaded = false; } -static int sdma_disable_channel_async(struct dma_chan *chan) +static int sdma_terminate_all(struct dma_chan *chan) { struct sdma_channel *sdmac = to_sdma_chan(chan); @@ -1324,7 +1324,7 @@ static void sdma_free_chan_resources(struct dma_chan *chan) struct sdma_channel *sdmac = to_sdma_chan(chan); struct sdma_engine *sdma = sdmac->sdma; - sdma_disable_channel_async(chan); + sdma_terminate_all(chan); sdma_channel_synchronize(chan); @@ -2103,7 +2103,7 @@ static int sdma_probe(struct platform_device *pdev) sdma->dma_device.device_prep_slave_sg = sdma_prep_slave_sg; sdma->dma_device.device_prep_dma_cyclic = sdma_prep_dma_cyclic; sdma->dma_device.device_config = sdma_config; - sdma->dma_device.device_terminate_all = sdma_disable_channel_async; + sdma->dma_device.device_terminate_all = sdma_terminate_all; sdma->dma_device.device_synchronize = sdma_channel_synchronize; sdma->dma_device.src_addr_widths = SDMA_DMA_BUSWIDTHS; sdma->dma_device.dst_addr_widths = SDMA_DMA_BUSWIDTHS; From patchwork Mon Dec 16 10:53:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sascha Hauer X-Patchwork-Id: 11293787 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DAD6A1593 for ; Mon, 16 Dec 2019 10:54:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C07AC206D7 for ; Mon, 16 Dec 2019 10:54:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727570AbfLPKyj (ORCPT ); Mon, 16 Dec 2019 05:54:39 -0500 Received: from metis.ext.pengutronix.de ([85.220.165.71]:36733 "EHLO metis.ext.pengutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727436AbfLPKyj (ORCPT ); Mon, 16 Dec 2019 05:54:39 -0500 Received: from dude.hi.pengutronix.de ([2001:67c:670:100:1d::7]) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1igo0L-0001de-9d; Mon, 16 Dec 2019 11:53:33 +0100 Received: from sha by dude.hi.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1igo0I-0005tT-Fo; Mon, 16 Dec 2019 11:53:30 +0100 From: Sascha Hauer To: dmaengine@vger.kernel.org Cc: Vinod Koul , Peter Ujfalusi , Florian Fainelli , Maxime Ripard , Green Wan , Kukjin Kim , Krzysztof Kozlowski , =?utf-8?q?Andreas_F=C3=A4rber?= , Sean Wang , Eugeniy Paltsev , Pengutronix Kernel Team , NXP Linux Team , Robert Jarzmik , Sascha Hauer Subject: [PATCH 8/9] dmaengine: imx-sdma: find desc first in sdma_tx_status Date: Mon, 16 Dec 2019 11:53:27 +0100 Message-Id: <20191216105328.15198-9-s.hauer@pengutronix.de> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191216105328.15198-1-s.hauer@pengutronix.de> References: <20191216105328.15198-1-s.hauer@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2001:67c:670:100:1d::7 X-SA-Exim-Mail-From: sha@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: dmaengine@vger.kernel.org Sender: dmaengine-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org In sdma_tx_status() we must first find the current sdma_desc. In cyclic mode we assume that this can always be found with vchan_find_desc(). This is true because do not remove the current descriptor from the desc_issued list: /* * Do not delete the node in desc_issued list in cyclic mode, otherwise * the desc allocated will never be freed in vchan_dma_desc_free_list */ if (!(sdmac->flags & IMX_DMA_SG_LOOP)) list_del(&vd->node); We will change this in the next step, so check if the current descriptor is the desired one also for the cyclic case. Signed-off-by: Sascha Hauer --- drivers/dma/imx-sdma.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/drivers/dma/imx-sdma.c b/drivers/dma/imx-sdma.c index 527f8a81f50b..99dbfd9039cf 100644 --- a/drivers/dma/imx-sdma.c +++ b/drivers/dma/imx-sdma.c @@ -1648,7 +1648,7 @@ static enum dma_status sdma_tx_status(struct dma_chan *chan, struct dma_tx_state *txstate) { struct sdma_channel *sdmac = to_sdma_chan(chan); - struct sdma_desc *desc; + struct sdma_desc *desc = NULL; u32 residue; struct virt_dma_desc *vd; enum dma_status ret; @@ -1659,19 +1659,23 @@ static enum dma_status sdma_tx_status(struct dma_chan *chan, return ret; spin_lock_irqsave(&sdmac->vc.lock, flags); + vd = vchan_find_desc(&sdmac->vc, cookie); - if (vd) { + if (vd) desc = to_sdma_desc(&vd->tx); + else if (sdmac->desc && sdmac->desc->vd.tx.cookie == cookie) + desc = sdmac->desc; + + if (desc) { if (sdmac->flags & IMX_DMA_SG_LOOP) residue = (desc->num_bd - desc->buf_ptail) * desc->period_len - desc->chn_real_count; else residue = desc->chn_count - desc->chn_real_count; - } else if (sdmac->desc && sdmac->desc->vd.tx.cookie == cookie) { - residue = sdmac->desc->chn_count - sdmac->desc->chn_real_count; } else { residue = 0; } + spin_unlock_irqrestore(&sdmac->vc.lock, flags); dma_set_tx_state(txstate, chan->completed_cookie, chan->cookie, From patchwork Mon Dec 16 10:53:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sascha Hauer X-Patchwork-Id: 11293767 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 09537138D for ; Mon, 16 Dec 2019 10:53:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E172120684 for ; Mon, 16 Dec 2019 10:53:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727453AbfLPKxf (ORCPT ); Mon, 16 Dec 2019 05:53:35 -0500 Received: from metis.ext.pengutronix.de ([85.220.165.71]:41925 "EHLO metis.ext.pengutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727229AbfLPKxf (ORCPT ); Mon, 16 Dec 2019 05:53:35 -0500 Received: from dude.hi.pengutronix.de ([2001:67c:670:100:1d::7]) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1igo0L-0001df-9P; Mon, 16 Dec 2019 11:53:33 +0100 Received: from sha by dude.hi.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1igo0I-0005tW-GS; Mon, 16 Dec 2019 11:53:30 +0100 From: Sascha Hauer To: dmaengine@vger.kernel.org Cc: Vinod Koul , Peter Ujfalusi , Florian Fainelli , Maxime Ripard , Green Wan , Kukjin Kim , Krzysztof Kozlowski , =?utf-8?q?Andreas_F=C3=A4rber?= , Sean Wang , Eugeniy Paltsev , Pengutronix Kernel Team , NXP Linux Team , Robert Jarzmik , Sascha Hauer Subject: [PATCH 9/9] dmaengine: imx-sdma: Fix memory leak Date: Mon, 16 Dec 2019 11:53:28 +0100 Message-Id: <20191216105328.15198-10-s.hauer@pengutronix.de> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191216105328.15198-1-s.hauer@pengutronix.de> References: <20191216105328.15198-1-s.hauer@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2001:67c:670:100:1d::7 X-SA-Exim-Mail-From: sha@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: dmaengine@vger.kernel.org Sender: dmaengine-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org The current descriptor is not on any list of the virtual DMA channel. Once sdma_terminate_all() is called when a descriptor is currently in flight then this one is forgotten to be freed. We have to call vchan_terminate_vdesc() on this descriptor to re-add it to the lists. Now that we also free the currently running descriptor we can (and actually have to) remove the current descriptor from its list also for the cyclic case. Signed-off-by: Sascha Hauer Reviewed-by: Robin Gong Tested-by: Robin Gong --- drivers/dma/imx-sdma.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/drivers/dma/imx-sdma.c b/drivers/dma/imx-sdma.c index 99dbfd9039cf..066b21a32232 100644 --- a/drivers/dma/imx-sdma.c +++ b/drivers/dma/imx-sdma.c @@ -760,12 +760,8 @@ static void sdma_start_desc(struct sdma_channel *sdmac) return; } sdmac->desc = desc = to_sdma_desc(&vd->tx); - /* - * Do not delete the node in desc_issued list in cyclic mode, otherwise - * the desc allocated will never be freed in vchan_dma_desc_free_list - */ - if (!(sdmac->flags & IMX_DMA_SG_LOOP)) - list_del(&vd->node); + + list_del(&vd->node); sdma->channel_control[channel].base_bd_ptr = desc->bd_phys; sdma->channel_control[channel].current_bd_ptr = desc->bd_phys; @@ -1071,7 +1067,6 @@ static void sdma_channel_terminate_work(struct work_struct *work) spin_lock_irqsave(&sdmac->vc.lock, flags); vchan_get_all_descriptors(&sdmac->vc, &head); - sdmac->desc = NULL; spin_unlock_irqrestore(&sdmac->vc.lock, flags); vchan_dma_desc_free_list(&sdmac->vc, &head); sdmac->context_loaded = false; @@ -1080,11 +1075,19 @@ static void sdma_channel_terminate_work(struct work_struct *work) static int sdma_terminate_all(struct dma_chan *chan) { struct sdma_channel *sdmac = to_sdma_chan(chan); + unsigned long flags; + + spin_lock_irqsave(&sdmac->vc.lock, flags); sdma_disable_channel(chan); - if (sdmac->desc) + if (sdmac->desc) { + vchan_terminate_vdesc(&sdmac->desc->vd); + sdmac->desc = NULL; schedule_work(&sdmac->terminate_worker); + } + + spin_unlock_irqrestore(&sdmac->vc.lock, flags); return 0; }