Message ID | 20240929134354.20735-1-laurent.pinchart@ideasonboard.com (mailing list archive) |
---|---|
Headers | show |
Series | media: imx8mq-mipi-csi2: Simplify power management handling | expand |
Hi Laurent, > -----Original Message----- > From: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > Sent: Sunday, September 29, 2024 9:44 PM > To: rmfrfs@gmail.com; martink@posteo.de; kernel@puri.sm; > mchehab@kernel.org; shawnguo@kernel.org; s.hauer@pengutronix.de; > kernel@pengutronix.de; festevam@gmail.com > Cc: G.N. Zhou (OSS) <guoniu.zhou@oss.nxp.com>; imx@lists.linux.dev; linux- > media@vger.kernel.org; linux-arm-kernel@lists.infradead.org; linux- > kernel@vger.kernel.org > Subject: [PATCH 0/3] media: imx8mq-mipi-csi2: Simplify power management > handling > > Hello, > > This small patch series is a reaction to "[PATCH] media: nxp: > imx8mq-mipi-csi2: Fix CSI clocks always enabled issue" ([1]). Instead of making > the PM handling more complex, I think it can be greatly simplified. > > I have only compile-tested the patches. Guoniu, could you give this a try ? After applying the patches and test both on iMX8ULP. For iMX8ULP, it will cause kernel dump when access CSI registers and system hang during do suspend/resume while streaming Need to add system suspend/resume handlers and call pm_runtime_force_suspend/resume in the handlers. I tried to debug this issue and found pm runtime callback won't be called when system resume. The state of power domain won't enabled. > > [1] https://lore.kernel.org/r/20240929101635.1648234-1- > guoniu.zhou@oss.nxp.com > > Laurent Pinchart (3): > media: imx8mq-mipi-csi2: Drop stream stop/restart at suspend/resume > time > media: imx8mq-mipi-csi2: Drop ST_SUSPENDED guard > media: imx8mq-mipi-csi2: Drop system suspend/resume handlers > > drivers/media/platform/nxp/imx8mq-mipi-csi2.c | 113 ++---------------- > 1 file changed, 10 insertions(+), 103 deletions(-) > > > base-commit: 81ee62e8d09ee3c7107d11c8bbfd64073ab601ad > -- > Regards, > > Laurent Pinchart
On Mon, Sep 30, 2024 at 07:08:09AM +0000, G.N. Zhou (OSS) wrote: > On Sunday, September 29, 2024 9:44 PM, Laurent Pinchart wrote: > > > > Hello, > > > > This small patch series is a reaction to "[PATCH] media: nxp: > > imx8mq-mipi-csi2: Fix CSI clocks always enabled issue" ([1]). Instead of making > > the PM handling more complex, I think it can be greatly simplified. > > > > I have only compile-tested the patches. Guoniu, could you give this a try ? > > After applying the patches and test both on iMX8ULP. > > For iMX8ULP, it will cause kernel dump when access CSI registers and > system hang during do suspend/resume while streaming > Need to add system suspend/resume handlers and call > pm_runtime_force_suspend/resume in the handlers. > > I tried to debug this issue and found pm runtime callback won't be > called when system resume. The state of power domain won't enabled. Thank you for testing. I wonder if this could be caused by the CSI bridge being resumed from system sleep before the CSI-2 receiver. Could you check if that's the case ? If so, does the following change fix the issue ? diff --git a/drivers/media/platform/nxp/imx7-media-csi.c b/drivers/media/platform/nxp/imx7-media-csi.c index 9566ff738818..c66b0621e395 100644 --- a/drivers/media/platform/nxp/imx7-media-csi.c +++ b/drivers/media/platform/nxp/imx7-media-csi.c @@ -2057,9 +2057,22 @@ static int imx7_csi_notify_bound(struct v4l2_async_notifier *notifier, { struct imx7_csi *csi = imx7_csi_notifier_to_dev(notifier); struct media_pad *sink = &csi->sd.entity.pads[IMX7_CSI_PAD_SINK]; + struct device_link *link; csi->src_sd = sd; + /* + * Enforce suspend/resume ordering between the source (supplier) and + * the CSI (consumer). The source will be suspended before and resume + * after the CSI. + */ + link = device_link_add(csi->dev, sd->dev, DL_FLAG_STATELESS); + if (!link) { + dev_err(csi->dev, + "Failed to create device link to source %s\n", sd->name); + return -EINVAL; + } + return v4l2_create_fwnode_links_to_pad(sd, sink, MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE); } > > [1] https://lore.kernel.org/r/20240929101635.1648234-1-guoniu.zhou@oss.nxp.com > > > > Laurent Pinchart (3): > > media: imx8mq-mipi-csi2: Drop stream stop/restart at suspend/resume > > time > > media: imx8mq-mipi-csi2: Drop ST_SUSPENDED guard > > media: imx8mq-mipi-csi2: Drop system suspend/resume handlers > > > > drivers/media/platform/nxp/imx8mq-mipi-csi2.c | 113 ++---------------- > > 1 file changed, 10 insertions(+), 103 deletions(-) > > > > > > base-commit: 81ee62e8d09ee3c7107d11c8bbfd64073ab601ad
Hi Laurent, > -----Original Message----- > From: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > Sent: Monday, September 30, 2024 3:22 PM > To: G.N. Zhou (OSS) <guoniu.zhou@oss.nxp.com> > Cc: rmfrfs@gmail.com; martink@posteo.de; kernel@puri.sm; > mchehab@kernel.org; shawnguo@kernel.org; s.hauer@pengutronix.de; > kernel@pengutronix.de; festevam@gmail.com; imx@lists.linux.dev; linux- > media@vger.kernel.org; linux-arm-kernel@lists.infradead.org; linux- > kernel@vger.kernel.org > Subject: Re: [PATCH 0/3] media: imx8mq-mipi-csi2: Simplify power management > handling > > On Mon, Sep 30, 2024 at 07:08:09AM +0000, G.N. Zhou (OSS) wrote: > > On Sunday, September 29, 2024 9:44 PM, Laurent Pinchart wrote: > > > > > > Hello, > > > > > > This small patch series is a reaction to "[PATCH] media: nxp: > > > imx8mq-mipi-csi2: Fix CSI clocks always enabled issue" ([1]). > > > Instead of making the PM handling more complex, I think it can be greatly > simplified. > > > > > > I have only compile-tested the patches. Guoniu, could you give this a try ? > > > > After applying the patches and test both on iMX8ULP. > > > > For iMX8ULP, it will cause kernel dump when access CSI registers and > > system hang during do suspend/resume while streaming Need to add > > system suspend/resume handlers and call > > pm_runtime_force_suspend/resume in the handlers. > > > > I tried to debug this issue and found pm runtime callback won't be > > called when system resume. The state of power domain won't enabled. > > Thank you for testing. > > I wonder if this could be caused by the CSI bridge being resumed from system > sleep before the CSI-2 receiver. Could you check if that's the case ? If so, does the > following change fix the issue ? I tested on iMX8ULP which don't use CSI bridge but ISI, not iMX8MQ. In ISI driver, I notice that it already handler the device relationship when subdev bound like bellow: link = device_link_add(isi->dev, sd->dev, DL_FLAG_STATELESS); if (!link) { dev_err(isi->dev, "Failed to create device link to source %s\n", sd->name); return -EINVAL; } For iMX8MQ, I'm trying to enable it, but meet some problems, so can't give you the results in short time. > > diff --git a/drivers/media/platform/nxp/imx7-media-csi.c > b/drivers/media/platform/nxp/imx7-media-csi.c > index 9566ff738818..c66b0621e395 100644 > --- a/drivers/media/platform/nxp/imx7-media-csi.c > +++ b/drivers/media/platform/nxp/imx7-media-csi.c > @@ -2057,9 +2057,22 @@ static int imx7_csi_notify_bound(struct > v4l2_async_notifier *notifier, { > struct imx7_csi *csi = imx7_csi_notifier_to_dev(notifier); > struct media_pad *sink = &csi->sd.entity.pads[IMX7_CSI_PAD_SINK]; > + struct device_link *link; > > csi->src_sd = sd; > > + /* > + * Enforce suspend/resume ordering between the source (supplier) and > + * the CSI (consumer). The source will be suspended before and resume > + * after the CSI. > + */ > + link = device_link_add(csi->dev, sd->dev, DL_FLAG_STATELESS); > + if (!link) { > + dev_err(csi->dev, > + "Failed to create device link to source %s\n", sd->name); > + return -EINVAL; > + } > + > return v4l2_create_fwnode_links_to_pad(sd, sink, > MEDIA_LNK_FL_ENABLED | > MEDIA_LNK_FL_IMMUTABLE); > } > > > > [1] > > > https://lore.kernel.org/r/20240929101635.1648234-1-guoniu.zhou@oss.n > > > xp.com > > > > > > Laurent Pinchart (3): > > > media: imx8mq-mipi-csi2: Drop stream stop/restart at suspend/resume > > > time > > > media: imx8mq-mipi-csi2: Drop ST_SUSPENDED guard > > > media: imx8mq-mipi-csi2: Drop system suspend/resume handlers > > > > > > drivers/media/platform/nxp/imx8mq-mipi-csi2.c | 113 > > > ++---------------- > > > 1 file changed, 10 insertions(+), 103 deletions(-) > > > > > > > > > base-commit: 81ee62e8d09ee3c7107d11c8bbfd64073ab601ad > > -- > Regards, > > Laurent Pinchart
Am Montag, dem 30.09.2024 um 10:21 +0300 schrieb Laurent Pinchart: > On Mon, Sep 30, 2024 at 07:08:09AM +0000, G.N. Zhou (OSS) wrote: > > On Sunday, September 29, 2024 9:44 PM, Laurent Pinchart wrote: > > > > > > Hello, > > > > > > This small patch series is a reaction to "[PATCH] media: nxp: > > > imx8mq-mipi-csi2: Fix CSI clocks always enabled issue" ([1]). > > > Instead of making > > > the PM handling more complex, I think it can be greatly > > > simplified. > > > > > > I have only compile-tested the patches. Guoniu, could you give > > > this a try ? > > > > After applying the patches and test both on iMX8ULP. > > > > For iMX8ULP, it will cause kernel dump when access CSI registers > > and > > system hang during do suspend/resume while streaming > > Need to add system suspend/resume handlers and call > > pm_runtime_force_suspend/resume in the handlers. > > > > I tried to debug this issue and found pm runtime callback won't be > > called when system resume. The state of power domain won't enabled. > > Thank you for testing. > > I wonder if this could be caused by the CSI bridge being resumed from > system sleep before the CSI-2 receiver. Could you check if that's the > case ? If so, does the following change fix the issue ? > > diff --git a/drivers/media/platform/nxp/imx7-media-csi.c > b/drivers/media/platform/nxp/imx7-media-csi.c > index 9566ff738818..c66b0621e395 100644 > --- a/drivers/media/platform/nxp/imx7-media-csi.c > +++ b/drivers/media/platform/nxp/imx7-media-csi.c > @@ -2057,9 +2057,22 @@ static int imx7_csi_notify_bound(struct > v4l2_async_notifier *notifier, > { > struct imx7_csi *csi = imx7_csi_notifier_to_dev(notifier); > struct media_pad *sink = &csi- > >sd.entity.pads[IMX7_CSI_PAD_SINK]; > + struct device_link *link; > > csi->src_sd = sd; > > + /* > + * Enforce suspend/resume ordering between the source > (supplier) and > + * the CSI (consumer). The source will be suspended before > and resume > + * after the CSI. > + */ > + link = device_link_add(csi->dev, sd->dev, DL_FLAG_STATELESS); > + if (!link) { > + dev_err(csi->dev, > + "Failed to create device link to source > %s\n", sd->name); > + return -EINVAL; > + } > + > return v4l2_create_fwnode_links_to_pad(sd, sink, > MEDIA_LNK_FL_ENABLED | > > MEDIA_LNK_FL_IMMUTABLE); > } hi Laurent, I now tested your 3 patches, initially including this extra change, on the imx8mq Librem 5 and indeed streaming continues after system resume now. It works without this extra change too, even though it seems to make sense. so for the 3-patch series at least: Tested-by: Martin Kepplinger-Novaković <martink@posteo.de> thanks. I seem to be able to drop a few patches now, but if anyone else can test it too, please wait for that as well, martin
Hi Guo, On Mon, Sep 30, 2024 at 07:51:45AM +0000, G.N. Zhou (OSS) wrote: > On Monday, September 30, 2024 3:22 PM, Laurent Pinchart wrote: > > On Mon, Sep 30, 2024 at 07:08:09AM +0000, G.N. Zhou (OSS) wrote: > > > On Sunday, September 29, 2024 9:44 PM, Laurent Pinchart wrote: > > > > > > > > Hello, > > > > > > > > This small patch series is a reaction to "[PATCH] media: nxp: > > > > imx8mq-mipi-csi2: Fix CSI clocks always enabled issue" ([1]). > > > > Instead of making the PM handling more complex, I think it can be greatly simplified. > > > > > > > > I have only compile-tested the patches. Guoniu, could you give this a try ? > > > > > > After applying the patches and test both on iMX8ULP. > > > > > > For iMX8ULP, it will cause kernel dump when access CSI registers and > > > system hang during do suspend/resume while streaming Need to add > > > system suspend/resume handlers and call > > > pm_runtime_force_suspend/resume in the handlers. > > > > > > I tried to debug this issue and found pm runtime callback won't be > > > called when system resume. The state of power domain won't enabled. > > > > Thank you for testing. > > > > I wonder if this could be caused by the CSI bridge being resumed from system > > sleep before the CSI-2 receiver. Could you check if that's the case ? If so, does the > > following change fix the issue ? > > I tested on iMX8ULP which don't use CSI bridge but ISI, not iMX8MQ. In ISI driver, I notice that > it already handler the device relationship when subdev bound like bellow: > > link = device_link_add(isi->dev, sd->dev, DL_FLAG_STATELESS); > if (!link) { > dev_err(isi->dev, > "Failed to create device link to source %s\n", sd->name); > return -EINVAL; > } Ah yes indeed with the ISI it should already be handled. I can't test this on hardware now as I'm travelling. Is the system hang happening at suspend or resume time ? What is the order of the suspend/resume handlers calls for the imx8-isi driver and the imx8mq-mipi-csi2 driver ? > For iMX8MQ, I'm trying to enable it, but meet some problems, so can't > give you the results in short time. > > > diff --git a/drivers/media/platform/nxp/imx7-media-csi.c > > b/drivers/media/platform/nxp/imx7-media-csi.c > > index 9566ff738818..c66b0621e395 100644 > > --- a/drivers/media/platform/nxp/imx7-media-csi.c > > +++ b/drivers/media/platform/nxp/imx7-media-csi.c > > @@ -2057,9 +2057,22 @@ static int imx7_csi_notify_bound(struct > > v4l2_async_notifier *notifier, { > > struct imx7_csi *csi = imx7_csi_notifier_to_dev(notifier); > > struct media_pad *sink = &csi->sd.entity.pads[IMX7_CSI_PAD_SINK]; > > + struct device_link *link; > > > > csi->src_sd = sd; > > > > + /* > > + * Enforce suspend/resume ordering between the source (supplier) and > > + * the CSI (consumer). The source will be suspended before and resume > > + * after the CSI. > > + */ > > + link = device_link_add(csi->dev, sd->dev, DL_FLAG_STATELESS); > > + if (!link) { > > + dev_err(csi->dev, > > + "Failed to create device link to source %s\n", sd->name); > > + return -EINVAL; > > + } > > + > > return v4l2_create_fwnode_links_to_pad(sd, sink, MEDIA_LNK_FL_ENABLED | > > MEDIA_LNK_FL_IMMUTABLE); > > } > > > > > > [1] https://lore.kernel.org/r/20240929101635.1648234-1-guoniu.zhou@oss.nxp.com > > > > > > > > Laurent Pinchart (3): > > > > media: imx8mq-mipi-csi2: Drop stream stop/restart at suspend/resume time > > > > media: imx8mq-mipi-csi2: Drop ST_SUSPENDED guard > > > > media: imx8mq-mipi-csi2: Drop system suspend/resume handlers > > > > > > > > drivers/media/platform/nxp/imx8mq-mipi-csi2.c | 113 ++---------------- > > > > 1 file changed, 10 insertions(+), 103 deletions(-) > > > > > > > > > > > > base-commit: 81ee62e8d09ee3c7107d11c8bbfd64073ab601ad
Hi Laruent, > -----Original Message----- > From: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > Sent: Tuesday, October 8, 2024 5:44 AM > To: G.N. Zhou (OSS) <guoniu.zhou@oss.nxp.com> > Cc: rmfrfs@gmail.com; martink@posteo.de; kernel@puri.sm; > mchehab@kernel.org; shawnguo@kernel.org; s.hauer@pengutronix.de; > kernel@pengutronix.de; festevam@gmail.com; imx@lists.linux.dev; linux- > media@vger.kernel.org; linux-arm-kernel@lists.infradead.org; linux- > kernel@vger.kernel.org > Subject: Re: [PATCH 0/3] media: imx8mq-mipi-csi2: Simplify power > management handling > > Hi Guo, > > On Mon, Sep 30, 2024 at 07:51:45AM +0000, G.N. Zhou (OSS) wrote: > > On Monday, September 30, 2024 3:22 PM, Laurent Pinchart wrote: > > > On Mon, Sep 30, 2024 at 07:08:09AM +0000, G.N. Zhou (OSS) wrote: > > > > On Sunday, September 29, 2024 9:44 PM, Laurent Pinchart wrote: > > > > > > > > > > Hello, > > > > > > > > > > This small patch series is a reaction to "[PATCH] media: nxp: > > > > > imx8mq-mipi-csi2: Fix CSI clocks always enabled issue" ([1]). > > > > > Instead of making the PM handling more complex, I think it can be > greatly simplified. > > > > > > > > > > I have only compile-tested the patches. Guoniu, could you give this a > try ? > > > > > > > > After applying the patches and test both on iMX8ULP. > > > > > > > > For iMX8ULP, it will cause kernel dump when access CSI registers > > > > and system hang during do suspend/resume while streaming Need to > > > > add system suspend/resume handlers and call > > > > pm_runtime_force_suspend/resume in the handlers. > > > > > > > > I tried to debug this issue and found pm runtime callback won't be > > > > called when system resume. The state of power domain won't enabled. > > > > > > Thank you for testing. > > > > > > I wonder if this could be caused by the CSI bridge being resumed > > > from system sleep before the CSI-2 receiver. Could you check if > > > that's the case ? If so, does the following change fix the issue ? > > > > I tested on iMX8ULP which don't use CSI bridge but ISI, not iMX8MQ. In > > ISI driver, I notice that it already handler the device relationship when subdev > bound like bellow: > > > > link = device_link_add(isi->dev, sd->dev, DL_FLAG_STATELESS); if > > (!link) { > > dev_err(isi->dev, > > "Failed to create device link to source %s\n", sd->name); > > return -EINVAL; > > } > > Ah yes indeed with the ISI it should already be handled. > > I can't test this on hardware now as I'm travelling. Is the system hang happening > at suspend or resume time ? What is the order of the suspend/resume handlers > calls for the imx8-isi driver and the > imx8mq-mipi-csi2 driver ? #1) Is the system hang happening at suspend or resume time ? After applying your patches, system hang will happened at resume time. Adding log and enable no_console_suspend, I can know the the point of causing issue is accessing CSI registers when top-level(ISI) device try to enable stream from CSI in ISI resume callback like bellow: mxc_isi_pm_resume(dev) mxc_isi_video_resume(pipe) mxc_isi_pipe_enable(pipe) v4l2_subdev_enable_streams(...) mx8mq_mipi_csi_s_stream() imx8mq_mipi_csi_start_stream() imx8mq_mipi_csi_set_params(state) imx8mq_mipi_csi_write(state, CSI2RX_CFG_NUM_LANES, lanes - 1); <--- causing kernel panic and system don't have any response #2) What is the order of the suspend/resume handlers calls for the imx8-isi driver and the imx8mq-mipi-csi2 driver ? Suspend: ISI -> CSI Resume: CSI -> ISI One more thing when I debugging this issue that the value of CSI device "power.usage_count" in two cases Case 1: Don't do system suspend/resume and only run camera test. Top-level device(ISI) can call CSI s_stream when start/stop capture and CSI runtime_suspend/resume will be called in this case since the value of CSI device power.usage_count is 1 Case 2: Run camera test first and keep it run in backend, then run system suspend/resume. Top-level device(ISI) will CSI s_stream() to start capture when run camera test and then, ISI system suspend callback will disable stream from CSI by calling s_stream(), the pm_runtime_put(dev) doesn't call CSI runtime suspend callback since the value of CSI device power.usage_count is 2. I'm not sure if it's helpfull, just for your reference. > > > For iMX8MQ, I'm trying to enable it, but meet some problems, so can't > > give you the results in short time. > > > > > diff --git a/drivers/media/platform/nxp/imx7-media-csi.c > > > b/drivers/media/platform/nxp/imx7-media-csi.c > > > index 9566ff738818..c66b0621e395 100644 > > > --- a/drivers/media/platform/nxp/imx7-media-csi.c > > > +++ b/drivers/media/platform/nxp/imx7-media-csi.c > > > @@ -2057,9 +2057,22 @@ static int imx7_csi_notify_bound(struct > > > v4l2_async_notifier *notifier, { > > > struct imx7_csi *csi = imx7_csi_notifier_to_dev(notifier); > > > struct media_pad *sink = &csi->sd.entity.pads[IMX7_CSI_PAD_SINK]; > > > + struct device_link *link; > > > > > > csi->src_sd = sd; > > > > > > + /* > > > + * Enforce suspend/resume ordering between the source (supplier) and > > > + * the CSI (consumer). The source will be suspended before and resume > > > + * after the CSI. > > > + */ > > > + link = device_link_add(csi->dev, sd->dev, DL_FLAG_STATELESS); > > > + if (!link) { > > > + dev_err(csi->dev, > > > + "Failed to create device link to source %s\n", sd- > >name); > > > + return -EINVAL; > > > + } > > > + > > > return v4l2_create_fwnode_links_to_pad(sd, sink, > MEDIA_LNK_FL_ENABLED | > > > MEDIA_LNK_FL_IMMUTABLE); } > > > > > > > > [1] > > > > > https://lore.kernel.org/r/20240929101635.1648234-1-guoniu.zhou@o > > > > > ss.nxp.com > > > > > > > > > > Laurent Pinchart (3): > > > > > media: imx8mq-mipi-csi2: Drop stream stop/restart at suspend/resume > time > > > > > media: imx8mq-mipi-csi2: Drop ST_SUSPENDED guard > > > > > media: imx8mq-mipi-csi2: Drop system suspend/resume handlers > > > > > > > > > > drivers/media/platform/nxp/imx8mq-mipi-csi2.c | 113 > > > > > ++---------------- > > > > > 1 file changed, 10 insertions(+), 103 deletions(-) > > > > > > > > > > > > > > > base-commit: 81ee62e8d09ee3c7107d11c8bbfd64073ab601ad > > -- > Regards, > > Laurent Pinchart