| Message ID | 241a6fbe-f2d2-3d11-ba45-afc5b02de9c2@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Mon, Nov 20, 2017 at 10:00:27AM +0800, Joseph Qi wrote: > From: Joseph Qi <qijiang.qj@alibaba-inc.com> > > dm device set QUEUE_FLAG_NONROT in resume, which is after register > queue. That is to mean, the previous initialization in > blk_throtl_register_queue is wrong in this case. > Fix it by checking and then updating the info during root tg > initialization as we don't have a better choice. > > Signed-off-by: Joseph Qi <qijiang.qj@alibaba-inc.com> > --- > block/blk-throttle.c | 17 +++++++++++++++++ > 1 file changed, 17 insertions(+) > > diff --git a/block/blk-throttle.c b/block/blk-throttle.c > index bf52035..6d6b220 100644 > --- a/block/blk-throttle.c > +++ b/block/blk-throttle.c > @@ -541,6 +541,23 @@ static void throtl_pd_init(struct blkg_policy_data *pd) > if (cgroup_subsys_on_dfl(io_cgrp_subsys) && blkg->parent) > sq->parent_sq = &blkg_to_tg(blkg->parent)->service_queue; > tg->td = td; > + > + /* > + * dm device set QUEUE_FLAG_NONROT in resume, which is after resister > + * queue, so the previous initialization is wrong in this case. Check > + * and update it here. > + */ > + if (blk_queue_nonrot(blkg->q) && > + td->filtered_latency != LATENCY_FILTERED_SSD) { > + int i; > + > + td->throtl_slice = DFL_THROTL_SLICE_SSD; if CONFIG_BLK_DEV_THROTTLING_LOW isn't not set, we use old slice, can you do the same thing here? Otherwise, Reviewed-by: Shaohua Li <shli@kernel.org> > + td->filtered_latency = LATENCY_FILTERED_SSD; > + for (i = 0; i < LATENCY_BUCKET_SIZE; i++) { > + td->avg_buckets[READ][i].latency = 0; > + td->avg_buckets[WRITE][i].latency = 0; > + } > + } > } > > /* > -- > 1.9.4
On Sun, Nov 19, 2017 at 9:00 PM, Joseph Qi <jiangqi903@gmail.com> wrote: > From: Joseph Qi <qijiang.qj@alibaba-inc.com> > > dm device set QUEUE_FLAG_NONROT in resume, which is after register > queue. That is to mean, the previous initialization in > blk_throtl_register_queue is wrong in this case. > Fix it by checking and then updating the info during root tg > initialization as we don't have a better choice. Given DM motivated this change, curious why you didn't send this to dm-devel? In any case, not sure why you need to reference "resume". Very few people will appreciate that detail. Better to just say: DM device sets QUEUE_FLAG_NONROT after the queue is registered. > Signed-off-by: Joseph Qi <qijiang.qj@alibaba-inc.com> > --- > block/blk-throttle.c | 17 +++++++++++++++++ > 1 file changed, 17 insertions(+) > > diff --git a/block/blk-throttle.c b/block/blk-throttle.c > index bf52035..6d6b220 100644 > --- a/block/blk-throttle.c > +++ b/block/blk-throttle.c > @@ -541,6 +541,23 @@ static void throtl_pd_init(struct blkg_policy_data *pd) > if (cgroup_subsys_on_dfl(io_cgrp_subsys) && blkg->parent) > sq->parent_sq = &blkg_to_tg(blkg->parent)->service_queue; > tg->td = td; > + > + /* > + * dm device set QUEUE_FLAG_NONROT in resume, which is after resister > + * queue, so the previous initialization is wrong in this case. Check > + * and update it here. > + */ typo: s/resister/register/ But again, best to say: DM device sets QUEUE_FLAG_NONROT after the queue is registered, ... Also, an alternative to your patch would be to have DM's dm_table_set_restrictions() call a blk-throttle function to setup blk-throttle after it is done setting queue flags? Saves all other drivers from having to run this 2 stage initialization code.. just a thought. Mike
On Mon, Nov 20, 2017 at 04:02:03PM -0500, Mike Snitzer wrote: > On Sun, Nov 19, 2017 at 9:00 PM, Joseph Qi <jiangqi903@gmail.com> wrote: > > From: Joseph Qi <qijiang.qj@alibaba-inc.com> > > > > dm device set QUEUE_FLAG_NONROT in resume, which is after register > > queue. That is to mean, the previous initialization in > > blk_throtl_register_queue is wrong in this case. > > Fix it by checking and then updating the info during root tg > > initialization as we don't have a better choice. > > Given DM motivated this change, curious why you didn't send this to dm-devel? > > In any case, not sure why you need to reference "resume". Very few > people will appreciate that detail. > > Better to just say: DM device sets QUEUE_FLAG_NONROT after the queue > is registered. > > > Signed-off-by: Joseph Qi <qijiang.qj@alibaba-inc.com> > > --- > > block/blk-throttle.c | 17 +++++++++++++++++ > > 1 file changed, 17 insertions(+) > > > > diff --git a/block/blk-throttle.c b/block/blk-throttle.c > > index bf52035..6d6b220 100644 > > --- a/block/blk-throttle.c > > +++ b/block/blk-throttle.c > > @@ -541,6 +541,23 @@ static void throtl_pd_init(struct blkg_policy_data *pd) > > if (cgroup_subsys_on_dfl(io_cgrp_subsys) && blkg->parent) > > sq->parent_sq = &blkg_to_tg(blkg->parent)->service_queue; > > tg->td = td; > > + > > + /* > > + * dm device set QUEUE_FLAG_NONROT in resume, which is after resister > > + * queue, so the previous initialization is wrong in this case. Check > > + * and update it here. > > + */ > > typo: s/resister/register/ > But again, best to say: > DM device sets QUEUE_FLAG_NONROT after the queue is registered, ... > > Also, an alternative to your patch would be to have DM's > dm_table_set_restrictions() call a blk-throttle function to setup > blk-throttle after it is done setting queue flags? > Saves all other drivers from having to run this 2 stage initialization > code.. just a thought. Though the patch is a workaround, I'd prefer limiting the code into blk-throttle itself. Populating it to drivers is a layer violation to me. The initialization only runs once, so I think it's ok. Thanks, Shaohua
Hi Mike, On 17/11/21 05:02, Mike Snitzer wrote: > On Sun, Nov 19, 2017 at 9:00 PM, Joseph Qi <jiangqi903@gmail.com> wrote: >> From: Joseph Qi <qijiang.qj@alibaba-inc.com> >> >> dm device set QUEUE_FLAG_NONROT in resume, which is after register >> queue. That is to mean, the previous initialization in >> blk_throtl_register_queue is wrong in this case. >> Fix it by checking and then updating the info during root tg >> initialization as we don't have a better choice. > > Given DM motivated this change, curious why you didn't send this to dm-devel? > Since this is io.low specifically, I don't want the change in device mapper layer. But I do agree with you that I have to cc dm-devel as well. Sorry for missing that. > In any case, not sure why you need to reference "resume". Very few > people will appreciate that detail. > > Better to just say: DM device sets QUEUE_FLAG_NONROT after the queue > is registered. > Thanks for your advice. I will update and resend it. Thanks, Joseph >> Signed-off-by: Joseph Qi <qijiang.qj@alibaba-inc.com> >> --- >> block/blk-throttle.c | 17 +++++++++++++++++ >> 1 file changed, 17 insertions(+) >> >> diff --git a/block/blk-throttle.c b/block/blk-throttle.c >> index bf52035..6d6b220 100644 >> --- a/block/blk-throttle.c >> +++ b/block/blk-throttle.c >> @@ -541,6 +541,23 @@ static void throtl_pd_init(struct blkg_policy_data *pd) >> if (cgroup_subsys_on_dfl(io_cgrp_subsys) && blkg->parent) >> sq->parent_sq = &blkg_to_tg(blkg->parent)->service_queue; >> tg->td = td; >> + >> + /* >> + * dm device set QUEUE_FLAG_NONROT in resume, which is after resister >> + * queue, so the previous initialization is wrong in this case. Check >> + * and update it here. >> + */ > > typo: s/resister/register/ > But again, best to say: > DM device sets QUEUE_FLAG_NONROT after the queue is registered, ... > > Also, an alternative to your patch would be to have DM's > dm_table_set_restrictions() call a blk-throttle function to setup > blk-throttle after it is done setting queue flags? > Saves all other drivers from having to run this 2 stage initialization > code.. just a thought. > > Mike >
On 17/11/21 01:15, Shaohua Li wrote: > On Mon, Nov 20, 2017 at 10:00:27AM +0800, Joseph Qi wrote: >> From: Joseph Qi <qijiang.qj@alibaba-inc.com> >> >> dm device set QUEUE_FLAG_NONROT in resume, which is after register >> queue. That is to mean, the previous initialization in >> blk_throtl_register_queue is wrong in this case. >> Fix it by checking and then updating the info during root tg >> initialization as we don't have a better choice. >> >> Signed-off-by: Joseph Qi <qijiang.qj@alibaba-inc.com> >> --- >> block/blk-throttle.c | 17 +++++++++++++++++ >> 1 file changed, 17 insertions(+) >> >> diff --git a/block/blk-throttle.c b/block/blk-throttle.c >> index bf52035..6d6b220 100644 >> --- a/block/blk-throttle.c >> +++ b/block/blk-throttle.c >> @@ -541,6 +541,23 @@ static void throtl_pd_init(struct blkg_policy_data *pd) >> if (cgroup_subsys_on_dfl(io_cgrp_subsys) && blkg->parent) >> sq->parent_sq = &blkg_to_tg(blkg->parent)->service_queue; >> tg->td = td; >> + >> + /* >> + * dm device set QUEUE_FLAG_NONROT in resume, which is after resister >> + * queue, so the previous initialization is wrong in this case. Check >> + * and update it here. >> + */ >> + if (blk_queue_nonrot(blkg->q) && >> + td->filtered_latency != LATENCY_FILTERED_SSD) { >> + int i; >> + >> + td->throtl_slice = DFL_THROTL_SLICE_SSD; > > if CONFIG_BLK_DEV_THROTTLING_LOW isn't not set, we use old slice, can you do > the same thing here? Otherwise, > Reviewed-by: Shaohua Li <shli@kernel.org> > Sure, I will update and resend it. Thanks for pointing this out. Thanks, Joseph >> + td->filtered_latency = LATENCY_FILTERED_SSD; >> + for (i = 0; i < LATENCY_BUCKET_SIZE; i++) { >> + td->avg_buckets[READ][i].latency = 0; >> + td->avg_buckets[WRITE][i].latency = 0; >> + } >> + } >> } >> >> /* >> -- >> 1.9.4
On Mon, Nov 20 2017 at 4:23pm -0500, Shaohua Li <shli@kernel.org> wrote: > On Mon, Nov 20, 2017 at 04:02:03PM -0500, Mike Snitzer wrote: > > On Sun, Nov 19, 2017 at 9:00 PM, Joseph Qi <jiangqi903@gmail.com> wrote: > > > From: Joseph Qi <qijiang.qj@alibaba-inc.com> > > > > > > dm device set QUEUE_FLAG_NONROT in resume, which is after register > > > queue. That is to mean, the previous initialization in > > > blk_throtl_register_queue is wrong in this case. > > > Fix it by checking and then updating the info during root tg > > > initialization as we don't have a better choice. > > > > Given DM motivated this change, curious why you didn't send this to dm-devel? > > > > In any case, not sure why you need to reference "resume". Very few > > people will appreciate that detail. > > > > Better to just say: DM device sets QUEUE_FLAG_NONROT after the queue > > is registered. > > > > > Signed-off-by: Joseph Qi <qijiang.qj@alibaba-inc.com> > > > --- > > > block/blk-throttle.c | 17 +++++++++++++++++ > > > 1 file changed, 17 insertions(+) > > > > > > diff --git a/block/blk-throttle.c b/block/blk-throttle.c > > > index bf52035..6d6b220 100644 > > > --- a/block/blk-throttle.c > > > +++ b/block/blk-throttle.c > > > @@ -541,6 +541,23 @@ static void throtl_pd_init(struct blkg_policy_data *pd) > > > if (cgroup_subsys_on_dfl(io_cgrp_subsys) && blkg->parent) > > > sq->parent_sq = &blkg_to_tg(blkg->parent)->service_queue; > > > tg->td = td; > > > + > > > + /* > > > + * dm device set QUEUE_FLAG_NONROT in resume, which is after resister > > > + * queue, so the previous initialization is wrong in this case. Check > > > + * and update it here. > > > + */ > > > > typo: s/resister/register/ > > But again, best to say: > > DM device sets QUEUE_FLAG_NONROT after the queue is registered, ... > > > > Also, an alternative to your patch would be to have DM's > > dm_table_set_restrictions() call a blk-throttle function to setup > > blk-throttle after it is done setting queue flags? > > Saves all other drivers from having to run this 2 stage initialization > > code.. just a thought. > > Though the patch is a workaround, I'd prefer limiting the code into > blk-throttle itself. Populating it to drivers is a layer violation to me. The > initialization only runs once, so I think it's ok. I fail to see how this is a "workaround". Whatever you call it, the blk-throttle code needs to cope with the fact that DM's request_queue setup is different than other block devices. If having DM call a function that blk-throttle provides (code that is in block/blk-throttle.c) I fail to see the layer violation. No more a layer violation than any of the other block core code that block drivers call to setup the request_queue's capabilities. But I'm not well-versed on blk-throttle code (hence DM coming up lacking relative to it). SO I'll defer to you guys to sort it out how you think best. Thanks, Mike
diff --git a/block/blk-throttle.c b/block/blk-throttle.c index bf52035..6d6b220 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -541,6 +541,23 @@ static void throtl_pd_init(struct blkg_policy_data *pd) if (cgroup_subsys_on_dfl(io_cgrp_subsys) && blkg->parent) sq->parent_sq = &blkg_to_tg(blkg->parent)->service_queue; tg->td = td; + + /* + * dm device set QUEUE_FLAG_NONROT in resume, which is after resister + * queue, so the previous initialization is wrong in this case. Check + * and update it here. + */ + if (blk_queue_nonrot(blkg->q) && + td->filtered_latency != LATENCY_FILTERED_SSD) { + int i; + + td->throtl_slice = DFL_THROTL_SLICE_SSD; + td->filtered_latency = LATENCY_FILTERED_SSD; + for (i = 0; i < LATENCY_BUCKET_SIZE; i++) { + td->avg_buckets[READ][i].latency = 0; + td->avg_buckets[WRITE][i].latency = 0; + } + } } /*