diff mbox series

[v6,6/6] drm/drm-file: Show finer-grained BO sizes in drm_show_memory_stats

Message ID 20230919233556.1458793-7-adrian.larumbe@collabora.com (mailing list archive)
State Superseded
Headers show
Series Add fdinfo support to Panfrost | expand

Commit Message

Adrián Larumbe Sept. 19, 2023, 11:34 p.m. UTC 
The current implementation will try to pick the highest available size
display unit as soon as the BO size exceeds that of the previous
multiplier. That can lead to loss of precision in contexts of low memory
usage.

The new selection criteria try to preserve precision, whilst also
increasing the display unit selection threshold to render more accurate
values.

Signed-off-by: Adrián Larumbe <adrian.larumbe@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Steven Price <steven.price@arm.com>
---
 drivers/gpu/drm/drm_file.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

Comments

Tvrtko Ursulin Sept. 20, 2023, 3:32 p.m. UTC | #1 
On 20/09/2023 00:34, Adrián Larumbe wrote:
> The current implementation will try to pick the highest available size
> display unit as soon as the BO size exceeds that of the previous
> multiplier. That can lead to loss of precision in contexts of low memory
> usage.
> 
> The new selection criteria try to preserve precision, whilst also
> increasing the display unit selection threshold to render more accurate
> values.
> 
> Signed-off-by: Adrián Larumbe <adrian.larumbe@collabora.com>
> Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
> Reviewed-by: Steven Price <steven.price@arm.com>
> ---
>   drivers/gpu/drm/drm_file.c | 5 ++++-
>   1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/drm_file.c b/drivers/gpu/drm/drm_file.c
> index 762965e3d503..34cfa128ffe5 100644
> --- a/drivers/gpu/drm/drm_file.c
> +++ b/drivers/gpu/drm/drm_file.c
> @@ -872,6 +872,8 @@ void drm_send_event(struct drm_device *dev, struct drm_pending_event *e)
>   }
>   EXPORT_SYMBOL(drm_send_event);
>   
> +#define UPPER_UNIT_THRESHOLD 100
> +
>   static void print_size(struct drm_printer *p, const char *stat,
>   		       const char *region, u64 sz)
>   {
> @@ -879,7 +881,8 @@ static void print_size(struct drm_printer *p, const char *stat,
>   	unsigned u;
>   
>   	for (u = 0; u < ARRAY_SIZE(units) - 1; u++) {
> -		if (sz < SZ_1K)
> +		if ((sz & (SZ_1K - 1)) &&

IS_ALIGNED worth it at all?

> +		    sz < UPPER_UNIT_THRESHOLD * SZ_1K)
>   			break;

Excuse me for a late comment (I was away). I did not get what what is 
special about a ~10% threshold? Sounds to me just going with the lower 
unit, when size is not aligned to the higher one, would be better than 
sometimes precision-sometimes-not.

Regards,

Tvrtko

>   		sz = div_u64(sz, SZ_1K);
>   	}
Tvrtko Ursulin Sept. 21, 2023, 10:14 a.m. UTC | #2 
On 20/09/2023 16:32, Tvrtko Ursulin wrote:
> 
> On 20/09/2023 00:34, Adrián Larumbe wrote:
>> The current implementation will try to pick the highest available size
>> display unit as soon as the BO size exceeds that of the previous
>> multiplier. That can lead to loss of precision in contexts of low memory
>> usage.
>>
>> The new selection criteria try to preserve precision, whilst also
>> increasing the display unit selection threshold to render more accurate
>> values.
>>
>> Signed-off-by: Adrián Larumbe <adrian.larumbe@collabora.com>
>> Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
>> Reviewed-by: Steven Price <steven.price@arm.com>
>> ---
>>   drivers/gpu/drm/drm_file.c | 5 ++++-
>>   1 file changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/gpu/drm/drm_file.c b/drivers/gpu/drm/drm_file.c
>> index 762965e3d503..34cfa128ffe5 100644
>> --- a/drivers/gpu/drm/drm_file.c
>> +++ b/drivers/gpu/drm/drm_file.c
>> @@ -872,6 +872,8 @@ void drm_send_event(struct drm_device *dev, struct 
>> drm_pending_event *e)
>>   }
>>   EXPORT_SYMBOL(drm_send_event);
>> +#define UPPER_UNIT_THRESHOLD 100
>> +
>>   static void print_size(struct drm_printer *p, const char *stat,
>>                  const char *region, u64 sz)
>>   {
>> @@ -879,7 +881,8 @@ static void print_size(struct drm_printer *p, 
>> const char *stat,
>>       unsigned u;
>>       for (u = 0; u < ARRAY_SIZE(units) - 1; u++) {
>> -        if (sz < SZ_1K)
>> +        if ((sz & (SZ_1K - 1)) &&
> 
> IS_ALIGNED worth it at all?
> 
>> +            sz < UPPER_UNIT_THRESHOLD * SZ_1K)
>>               break;
> 
> Excuse me for a late comment (I was away). I did not get what what is 
> special about a ~10% threshold? Sounds to me just going with the lower 
> unit, when size is not aligned to the higher one, would be better than 
> sometimes precision-sometimes-not.

FWIW both current and the threshold option make testing the feature very 
annoying.

So I'd really propose we simply use smaller unit when unaligned.

Regards,

Tvrtko
Adrián Larumbe Sept. 22, 2023, 11:01 a.m. UTC | #3 
On 20.09.2023 16:32, Tvrtko Ursulin wrote:
>
>On 20/09/2023 00:34, Adrián Larumbe wrote:
>> The current implementation will try to pick the highest available size
>> display unit as soon as the BO size exceeds that of the previous
>> multiplier. That can lead to loss of precision in contexts of low memory
>> usage.
>> 
>> The new selection criteria try to preserve precision, whilst also
>> increasing the display unit selection threshold to render more accurate
>> values.
>> 
>> Signed-off-by: Adrián Larumbe <adrian.larumbe@collabora.com>
>> Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
>> Reviewed-by: Steven Price <steven.price@arm.com>
>> ---
>>   drivers/gpu/drm/drm_file.c | 5 ++++-
>>   1 file changed, 4 insertions(+), 1 deletion(-)
>> 
>> diff --git a/drivers/gpu/drm/drm_file.c b/drivers/gpu/drm/drm_file.c
>> index 762965e3d503..34cfa128ffe5 100644
>> --- a/drivers/gpu/drm/drm_file.c
>> +++ b/drivers/gpu/drm/drm_file.c
>> @@ -872,6 +872,8 @@ void drm_send_event(struct drm_device *dev, struct drm_pending_event *e)
>>   }
>>   EXPORT_SYMBOL(drm_send_event);
>> +#define UPPER_UNIT_THRESHOLD 100
>> +
>>   static void print_size(struct drm_printer *p, const char *stat,
>>   		       const char *region, u64 sz)
>>   {
>> @@ -879,7 +881,8 @@ static void print_size(struct drm_printer *p, const char *stat,
>>   	unsigned u;
>>   	for (u = 0; u < ARRAY_SIZE(units) - 1; u++) {
>> -		if (sz < SZ_1K)
>> +		if ((sz & (SZ_1K - 1)) &&
>
>IS_ALIGNED worth it at all?

This could look better, yeah.

>> +		    sz < UPPER_UNIT_THRESHOLD * SZ_1K)
>>   			break;
>
>Excuse me for a late comment (I was away). I did not get what what is special
>about a ~10% threshold? Sounds to me just going with the lower unit, when size
>is not aligned to the higher one, would be better than sometimes
>precision-sometimes-not.

We had a bit of a debate over this in previous revisions of the patch. It all began
when a Panfrost user complained that for relatively small BOs, they were losing
precision in the fdinfo file because the sum of the sizes of all BOs for a drm file
was in the order of MiBs, but not big enough to warrant losing accuracy when
plotting them on nvtop or gputop.

At first I thought of letting drivers pick their own preferred unit, but this would
lead to inconsistency in the units presented in the fdinfo file across different
DRM devices. Rob then suggested imposing a unit multiple threshold, while Boris
made the suggestion of checking for unit size alignment to lessen precision loss.

In the end Rob thought that minding both constraints was a good solution of compromise.

The unit threshold was picked sort of arbitrarily, and suggested by Rob himself. The
point of having it is avoiding huge number representations for BO size tallies that
aren't aligned to the next unit, and also because BO size sums are scaled when
plotting them on a Y axis, so complete accuracy isn't a requirement.

>Regards,
>
>Tvrtko
>
>>   		sz = div_u64(sz, SZ_1K);
>>   	}

Adrian Larumbe
Adrián Larumbe Sept. 22, 2023, 11:03 a.m. UTC | #4 
On 21.09.2023 11:14, Tvrtko Ursulin wrote:
>
>On 20/09/2023 16:32, Tvrtko Ursulin wrote:
>> 
>> On 20/09/2023 00:34, Adrián Larumbe wrote:
>> > The current implementation will try to pick the highest available size
>> > display unit as soon as the BO size exceeds that of the previous
>> > multiplier. That can lead to loss of precision in contexts of low memory
>> > usage.
>> > 
>> > The new selection criteria try to preserve precision, whilst also
>> > increasing the display unit selection threshold to render more accurate
>> > values.
>> > 
>> > Signed-off-by: Adrián Larumbe <adrian.larumbe@collabora.com>
>> > Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
>> > Reviewed-by: Steven Price <steven.price@arm.com>
>> > ---
>> >   drivers/gpu/drm/drm_file.c | 5 ++++-
>> >   1 file changed, 4 insertions(+), 1 deletion(-)
>> > 
>> > diff --git a/drivers/gpu/drm/drm_file.c b/drivers/gpu/drm/drm_file.c
>> > index 762965e3d503..34cfa128ffe5 100644
>> > --- a/drivers/gpu/drm/drm_file.c
>> > +++ b/drivers/gpu/drm/drm_file.c
>> > @@ -872,6 +872,8 @@ void drm_send_event(struct drm_device *dev, struct
>> > drm_pending_event *e)
>> >   }
>> >   EXPORT_SYMBOL(drm_send_event);
>> > +#define UPPER_UNIT_THRESHOLD 100
>> > +
>> >   static void print_size(struct drm_printer *p, const char *stat,
>> >                  const char *region, u64 sz)
>> >   {
>> > @@ -879,7 +881,8 @@ static void print_size(struct drm_printer *p,
>> > const char *stat,
>> >       unsigned u;
>> >       for (u = 0; u < ARRAY_SIZE(units) - 1; u++) {
>> > -        if (sz < SZ_1K)
>> > +        if ((sz & (SZ_1K - 1)) &&
>> 
>> IS_ALIGNED worth it at all?
>> 
>> > +            sz < UPPER_UNIT_THRESHOLD * SZ_1K)
>> >               break;
>> 
>> Excuse me for a late comment (I was away). I did not get what what is
>> special about a ~10% threshold? Sounds to me just going with the lower
>> unit, when size is not aligned to the higher one, would be better than
>> sometimes precision-sometimes-not.
>
>FWIW both current and the threshold option make testing the feature very
>annoying.

How so?

>So I'd really propose we simply use smaller unit when unaligned.

Like I said in the previous reply, for drm files whose overall BO size sum is enormous
but not a multiple of a MiB, this would render huge number representations in KiB.
I don't find this particularly comfortable to read, and then this extra precision
would mean nothing to nvtop or gputop, which would have to scale the size to their
available screen dimensions when plotting them.

>Regards,
>
>Tvrtko
Tvrtko Ursulin Sept. 22, 2023, 2:02 p.m. UTC | #5 
On 22/09/2023 12:03, Adrián Larumbe wrote:
> On 21.09.2023 11:14, Tvrtko Ursulin wrote:
>>
>> On 20/09/2023 16:32, Tvrtko Ursulin wrote:
>>>
>>> On 20/09/2023 00:34, Adrián Larumbe wrote:
>>>> The current implementation will try to pick the highest available size
>>>> display unit as soon as the BO size exceeds that of the previous
>>>> multiplier. That can lead to loss of precision in contexts of low memory
>>>> usage.
>>>>
>>>> The new selection criteria try to preserve precision, whilst also
>>>> increasing the display unit selection threshold to render more accurate
>>>> values.
>>>>
>>>> Signed-off-by: Adrián Larumbe <adrian.larumbe@collabora.com>
>>>> Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
>>>> Reviewed-by: Steven Price <steven.price@arm.com>
>>>> ---
>>>>    drivers/gpu/drm/drm_file.c | 5 ++++-
>>>>    1 file changed, 4 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/drm_file.c b/drivers/gpu/drm/drm_file.c
>>>> index 762965e3d503..34cfa128ffe5 100644
>>>> --- a/drivers/gpu/drm/drm_file.c
>>>> +++ b/drivers/gpu/drm/drm_file.c
>>>> @@ -872,6 +872,8 @@ void drm_send_event(struct drm_device *dev, struct
>>>> drm_pending_event *e)
>>>>    }
>>>>    EXPORT_SYMBOL(drm_send_event);
>>>> +#define UPPER_UNIT_THRESHOLD 100
>>>> +
>>>>    static void print_size(struct drm_printer *p, const char *stat,
>>>>                   const char *region, u64 sz)
>>>>    {
>>>> @@ -879,7 +881,8 @@ static void print_size(struct drm_printer *p,
>>>> const char *stat,
>>>>        unsigned u;
>>>>        for (u = 0; u < ARRAY_SIZE(units) - 1; u++) {
>>>> -        if (sz < SZ_1K)
>>>> +        if ((sz & (SZ_1K - 1)) &&
>>>
>>> IS_ALIGNED worth it at all?
>>>
>>>> +            sz < UPPER_UNIT_THRESHOLD * SZ_1K)
>>>>                break;
>>>
>>> Excuse me for a late comment (I was away). I did not get what what is
>>> special about a ~10% threshold? Sounds to me just going with the lower
>>> unit, when size is not aligned to the higher one, would be better than
>>> sometimes precision-sometimes-not.
>>
>> FWIW both current and the threshold option make testing the feature very
>> annoying.
> 
> How so?

I have to build in the knowledge of implementation details of 
print_size() into my IGT in order to use the right size BOs, so test is 
able to verify stats move as expected. It just feels wrong.

>> So I'd really propose we simply use smaller unit when unaligned.
> 
> Like I said in the previous reply, for drm files whose overall BO size sum is enormous
> but not a multiple of a MiB, this would render huge number representations in KiB.
> I don't find this particularly comfortable to read, and then this extra precision
> would mean nothing to nvtop or gputop, which would have to scale the size to their
> available screen dimensions when plotting them.

I don't think numbers in KiB are so huge.

And I don't think people will end up reading them manually a lot anyway, 
since you have to hunt the pid, and fd, etc.. It is much more realistic 
that some tool like gputop will be used.

And I don't think consistency of units across drivers or whatever 
matters. Even better to keep userspace parser on their toes and make 
then follow drm-usage-stats.rst and not any implementations, at some 
point in time.

Regards,

Tvrtko
diff mbox series

Patch

diff --git a/drivers/gpu/drm/drm_file.c b/drivers/gpu/drm/drm_file.c
index 762965e3d503..34cfa128ffe5 100644
--- a/drivers/gpu/drm/drm_file.c
+++ b/drivers/gpu/drm/drm_file.c
@@ -872,6 +872,8 @@  void drm_send_event(struct drm_device *dev, struct drm_pending_event *e)
 }
 EXPORT_SYMBOL(drm_send_event);
 
+#define UPPER_UNIT_THRESHOLD 100
+
 static void print_size(struct drm_printer *p, const char *stat,
 		       const char *region, u64 sz)
 {
@@ -879,7 +881,8 @@  static void print_size(struct drm_printer *p, const char *stat,
 	unsigned u;
 
 	for (u = 0; u < ARRAY_SIZE(units) - 1; u++) {
-		if (sz < SZ_1K)
+		if ((sz & (SZ_1K - 1)) &&
+		    sz < UPPER_UNIT_THRESHOLD * SZ_1K)
 			break;
 		sz = div_u64(sz, SZ_1K);
 	}