[v2,0/3] blk-mq: Optimise blk_mq_queue_tag_busy_iter() for shared tags

Message

John Garry Dec. 6, 2021, 12:49 p.m. UTC

In [0] Kashyap reports high CPU usage for blk_mq_queue_tag_busy_iter()
and callees for shared tags.

Indeed blk_mq_queue_tag_busy_iter() would be less optimum for moving to
shared tags, but it was not optimum previously.

This series optimises by having only a single iter (per regular and resv
tags) for the shared tags, instead of an iter per HW queue.

[0] https://lore.kernel.org/linux-block/[e-mail]/

Differences to v1:
- Add tested-by and reviewed-by tags
- Reformat 3/3 a bit to keep <= 80 char lines
	- I kept the RB tags, so please check and let me know if not ok
	  with the changes

John Garry (3):
  blk-mq: Drop busy_iter_fn blk_mq_hw_ctx argument
  blk-mq: Delete busy_iter_fn
  blk-mq: Optimise blk_mq_queue_tag_busy_iter() for shared tags

 block/blk-mq-tag.c     | 65 ++++++++++++++++++++++++++++--------------
 block/blk-mq-tag.h     |  2 +-
 block/blk-mq.c         | 17 ++++++-----
 include/linux/blk-mq.h |  2 --
 4 files changed, 53 insertions(+), 33 deletions(-)

Comments

Jens Axboe Dec. 6, 2021, 7:07 p.m. UTC | #1

On 12/6/21 5:49 AM, John Garry wrote:
> In [0] Kashyap reports high CPU usage for blk_mq_queue_tag_busy_iter()
> and callees for shared tags.
> 
> Indeed blk_mq_queue_tag_busy_iter() would be less optimum for moving to
> shared tags, but it was not optimum previously.
> 
> This series optimises by having only a single iter (per regular and resv
> tags) for the shared tags, instead of an iter per HW queue.
> 
> [0] https://lore.kernel.org/linux-block/[e-mail]/

The patch(es) are missing Fixes tags.

John Garry Dec. 6, 2021, 7:34 p.m. UTC | #2

On 06/12/2021 19:07, Jens Axboe wrote:
> On 12/6/21 5:49 AM, John Garry wrote:
>> In [0] Kashyap reports high CPU usage for blk_mq_queue_tag_busy_iter()
>> and callees for shared tags.
>>
>> Indeed blk_mq_queue_tag_busy_iter() would be less optimum for moving to
>> shared tags, but it was not optimum previously.
>>
>> This series optimises by having only a single iter (per regular and resv
>> tags) for the shared tags, instead of an iter per HW queue.
>>
>> [0]https://lore.kernel.org/linux-block/[e-mail]/

Hi Jens,

> The patch(es) are missing Fixes tags.

The first two patches aren't fixes, but are general dev. As for the 
last, it prob should go as a fix for 5.16, but I was not sure how you 
would feel about that - it's not a trivial change, we're late in the 
cycle, and Kashyap was happy for 5.17 .

Let me know if the last could be accepted as a fix and I'll re-send 
separately with a fixes tag.

Thanks,
John

Jens Axboe Dec. 6, 2021, 7:48 p.m. UTC | #3

On 12/6/21 12:34 PM, John Garry wrote:
> On 06/12/2021 19:07, Jens Axboe wrote:
>> On 12/6/21 5:49 AM, John Garry wrote:
>>> In [0] Kashyap reports high CPU usage for blk_mq_queue_tag_busy_iter()
>>> and callees for shared tags.
>>>
>>> Indeed blk_mq_queue_tag_busy_iter() would be less optimum for moving to
>>> shared tags, but it was not optimum previously.
>>>
>>> This series optimises by having only a single iter (per regular and resv
>>> tags) for the shared tags, instead of an iter per HW queue.
>>>
>>> [0]https://lore.kernel.org/linux-block/[e-mail]/
> 
> Hi Jens,
> 
>> The patch(es) are missing Fixes tags.
> 
> The first two patches aren't fixes, but are general dev. As for the 
> last, it prob should go as a fix for 5.16, but I was not sure how you 
> would feel about that - it's not a trivial change, we're late in the 
> cycle, and Kashyap was happy for 5.17 .
> 
> Let me know if the last could be accepted as a fix and I'll re-send 
> separately with a fixes tag.

Regardless of whether it's going into 5.16 or 5.17 it should have a
fixes tag.

John Garry Dec. 6, 2021, 7:59 p.m. UTC | #4

On 06/12/2021 19:48, Jens Axboe wrote:
>> The first two patches aren't fixes, but are general dev. As for the
>> last, it prob should go as a fix for 5.16, but I was not sure how you
>> would feel about that - it's not a trivial change, we're late in the
>> cycle, and Kashyap was happy for 5.17 .
>>

Hi Jens,

>> Let me know if the last could be accepted as a fix and I'll re-send
>> separately with a fixes tag.
> Regardless of whether it's going into 5.16 or 5.17 it should have a
> fixes tag.

ok, so patch 3/3 would fix commit e155b0c238b2 ("blk-mq: Use shared tags 
for shared sbitmap support")

I'll reply to that one with a fixes tag and I think that b4 would pick 
it up in case.

Cheers,
John

Jens Axboe Dec. 7, 2021, 4:18 p.m. UTC | #5

On Mon, 6 Dec 2021 20:49:47 +0800, John Garry wrote:
> In [0] Kashyap reports high CPU usage for blk_mq_queue_tag_busy_iter()
> and callees for shared tags.
> 
> Indeed blk_mq_queue_tag_busy_iter() would be less optimum for moving to
> shared tags, but it was not optimum previously.
> 
> This series optimises by having only a single iter (per regular and resv
> tags) for the shared tags, instead of an iter per HW queue.
> 
> [...]

Applied, thanks!

[1/3] blk-mq: Drop busy_iter_fn blk_mq_hw_ctx argument
      (no commit info)
[2/3] blk-mq: Delete busy_iter_fn
      (no commit info)
[3/3] blk-mq: Optimise blk_mq_queue_tag_busy_iter() for shared tags
      (no commit info)

Best regards,