[v4,10/19] selftests/resctrl: Express span internally in bytes

Commit Message

Ilpo Järvinen July 13, 2023, 1:19 p.m. UTC

MBA and MBM tests to use megabytes to represent span. CMT test uses
bytes. The difference requires run_benchmark() to size the buffer
differently based on the test name, which in turn requires passing the
test name into run_benchmark().

Convert MBA and MBM tests to use internally bytes like CMT test to
remove the internal inconsistency between the tests. Remove the test
dependent buffer sizing from run_benchmark().

This change eliminates one of the reasons why the test name has to be
passed around but there are still other users too so the test name
passing cannot yet be removed.

Co-developed-by: Fenghua Yu <[e-mail]>
Signed-off-by: Fenghua Yu <[e-mail]>
Signed-off-by: Ilpo Järvinen <[e-mail]>
---
 tools/testing/selftests/resctrl/mbm_test.c      | 2 +-
 tools/testing/selftests/resctrl/resctrl_tests.c | 2 +-
 tools/testing/selftests/resctrl/resctrlfs.c     | 9 ++-------
 3 files changed, 4 insertions(+), 9 deletions(-)

Comments

Reinette Chatre July 13, 2023, 11 p.m. UTC | #1

Hi Ilpo,

On 7/13/2023 6:19 AM, Ilpo Järvinen wrote:
> MBA and MBM tests to use megabytes to represent span. CMT test uses
> bytes. The difference requires run_benchmark() to size the buffer
> differently based on the test name, which in turn requires passing the
> test name into run_benchmark().
> 
> Convert MBA and MBM tests to use internally bytes like CMT test to
> remove the internal inconsistency between the tests. Remove the test
> dependent buffer sizing from run_benchmark().

If I understand correctly the intention is to always use bytes internally
and only convert to megabytes when displayed to user space. The above
implies that this takes care of the conversion but there still seems
to be places that that do not follow my understanding. For example,
resctrl_val.c:measure_vals() converts to megabytes before proceeding.

While MBA, MBM, and CMT tests use resctrl_val() for testing it seems
as though the function still exits with the MBA/MBM data recorded in
megabytes with the CMT data recorded in bytes. That seems to be why
show_mba_info() needs no conversion when displaying the data.

Reinette

Maciej Wieczor-Retman July 14, 2023, 6:43 a.m. UTC | #2

Hi Reinette!

On 14.07.2023 01:00, Reinette Chatre wrote:
> Hi Ilpo,
> 
> On 7/13/2023 6:19 AM, Ilpo Järvinen wrote:
>> MBA and MBM tests to use megabytes to represent span. CMT test uses
>> bytes. The difference requires run_benchmark() to size the buffer
>> differently based on the test name, which in turn requires passing the
>> test name into run_benchmark().
>>
>> Convert MBA and MBM tests to use internally bytes like CMT test to
>> remove the internal inconsistency between the tests. Remove the test
>> dependent buffer sizing from run_benchmark().
> 
> If I understand correctly the intention is to always use bytes internally
> and only convert to megabytes when displayed to user space. The above
> implies that this takes care of the conversion but there still seems
> to be places that that do not follow my understanding. For example,
> resctrl_val.c:measure_vals() converts to megabytes before proceeding.

Doesn't the use case inside resctrl_val.c:measure_vals() satisfy
the idea of only displaying data to the user space? From my
understanding it reads the number of bytes and only converts to
MB when printing the value. Or did I miss some detail there?

> 
> While MBA, MBM, and CMT tests use resctrl_val() for testing it seems
> as though the function still exits with the MBA/MBM data recorded in
> megabytes with the CMT data recorded in bytes. That seems to be why
> show_mba_info() needs no conversion when displaying the data.
> 
> Reinette

Kind regards
Maciej Wieczór-Retman

---------------------------------------------------------------------
Intel Technology Poland sp. z o.o.
ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN.
Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych.

Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione.
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.

Ilpo Järvinen July 14, 2023, 10:22 a.m. UTC | #3

On Fri, 14 Jul 2023, Wieczor-Retman, Maciej wrote:

> Hi Reinette!
> 
> On 14.07.2023 01:00, Reinette Chatre wrote:
> > Hi Ilpo,
> > 
> > On 7/13/2023 6:19 AM, Ilpo Järvinen wrote:
> >> MBA and MBM tests to use megabytes to represent span. CMT test uses
> >> bytes. The difference requires run_benchmark() to size the buffer
> >> differently based on the test name, which in turn requires passing the
> >> test name into run_benchmark().
> >>
> >> Convert MBA and MBM tests to use internally bytes like CMT test to
> >> remove the internal inconsistency between the tests. Remove the test
> >> dependent buffer sizing from run_benchmark().
> > 
> > If I understand correctly the intention is to always use bytes internally
> > and only convert to megabytes when displayed to user space. The above
> > implies that this takes care of the conversion but there still seems
> > to be places that that do not follow my understanding. For example,
> > resctrl_val.c:measure_vals() converts to megabytes before proceeding.
> 
> Doesn't the use case inside resctrl_val.c:measure_vals() satisfy
> the idea of only displaying data to the user space? From my
> understanding it reads the number of bytes and only converts to
> MB when printing the value. Or did I miss some detail there?

It's for printing there yes.

But it's not about span in the first place so I'm not sure why it is 
related.

Reinette Chatre July 14, 2023, 5:38 p.m. UTC | #4

Hi Maciej,

On 7/13/2023 11:43 PM, Wieczor-Retman, Maciej wrote:
> Hi Reinette!
> 
> On 14.07.2023 01:00, Reinette Chatre wrote:
>> Hi Ilpo,
>>
>> On 7/13/2023 6:19 AM, Ilpo Järvinen wrote:
>>> MBA and MBM tests to use megabytes to represent span. CMT test uses
>>> bytes. The difference requires run_benchmark() to size the buffer
>>> differently based on the test name, which in turn requires passing the
>>> test name into run_benchmark().
>>>
>>> Convert MBA and MBM tests to use internally bytes like CMT test to
>>> remove the internal inconsistency between the tests. Remove the test
>>> dependent buffer sizing from run_benchmark().
>>
>> If I understand correctly the intention is to always use bytes internally
>> and only convert to megabytes when displayed to user space. The above
>> implies that this takes care of the conversion but there still seems
>> to be places that that do not follow my understanding. For example,
>> resctrl_val.c:measure_vals() converts to megabytes before proceeding.
> 
> Doesn't the use case inside resctrl_val.c:measure_vals() satisfy
> the idea of only displaying data to the user space? From my
> understanding it reads the number of bytes and only converts to
> MB when printing the value. Or did I miss some detail there?

For MBM and MBA tests resctrl_val.c:measure_vals() converts data
to MB before it writes the raw data to the file. Compare to CMT
test that uses cache.c:measure_cache_vals() to write raw data
in bytes to file.

On a high level I see the MBA, MBM, and CMT tests in three stages:
(1) A "benchmark" is run in background, 
(2) raw data is collected at intervals and written to a file,
(3) finally, data from file is processed and results displayed to user.

From what I can tell MBA and MBM tests continue to write data to file
in MB while CMT writes data in bytes.

The changelog claims "Convert MBA and MBM tests to use internally
bytes like CMT test to remove the internal inconsistency between
the tests.". This claim is not clear to me because from what I can tell
the three tests continue to handle data inconsistently after this change.

Reinette

Reinette Chatre July 14, 2023, 5:38 p.m. UTC | #5

Hi Ilpo,

On 7/14/2023 3:22 AM, Ilpo Järvinen wrote:
> On Fri, 14 Jul 2023, Wieczor-Retman, Maciej wrote:
>> On 14.07.2023 01:00, Reinette Chatre wrote:
>>> Hi Ilpo,
>>>
>>> On 7/13/2023 6:19 AM, Ilpo Järvinen wrote:
>>>> MBA and MBM tests to use megabytes to represent span. CMT test uses
>>>> bytes. The difference requires run_benchmark() to size the buffer
>>>> differently based on the test name, which in turn requires passing the
>>>> test name into run_benchmark().
>>>>
>>>> Convert MBA and MBM tests to use internally bytes like CMT test to
>>>> remove the internal inconsistency between the tests. Remove the test
>>>> dependent buffer sizing from run_benchmark().
>>>
>>> If I understand correctly the intention is to always use bytes internally
>>> and only convert to megabytes when displayed to user space. The above
>>> implies that this takes care of the conversion but there still seems
>>> to be places that that do not follow my understanding. For example,
>>> resctrl_val.c:measure_vals() converts to megabytes before proceeding.
>>
>> Doesn't the use case inside resctrl_val.c:measure_vals() satisfy
>> the idea of only displaying data to the user space? From my
>> understanding it reads the number of bytes and only converts to
>> MB when printing the value. Or did I miss some detail there?
> 
> It's for printing there yes.
> 
> But it's not about span in the first place so I'm not sure why it is 
> related.
> 

If this change is just about how "span" is interpreted by the different
tests then the changelog could be more specific to not create expectation
that with this change there are no longer "bytes vs megabytes" internal
inconsistency between MBA, MBM, and CMT tests.

Reinette

Ilpo Järvinen July 17, 2023, 12:30 p.m. UTC | #6

On Fri, 14 Jul 2023, Reinette Chatre wrote:
> On 7/14/2023 3:22 AM, Ilpo Järvinen wrote:
> > On Fri, 14 Jul 2023, Wieczor-Retman, Maciej wrote:
> >> On 14.07.2023 01:00, Reinette Chatre wrote:
> >>> Hi Ilpo,
> >>>
> >>> On 7/13/2023 6:19 AM, Ilpo Järvinen wrote:
> >>>> MBA and MBM tests to use megabytes to represent span. CMT test uses
> >>>> bytes. The difference requires run_benchmark() to size the buffer
> >>>> differently based on the test name, which in turn requires passing the
> >>>> test name into run_benchmark().
> >>>>
> >>>> Convert MBA and MBM tests to use internally bytes like CMT test to
> >>>> remove the internal inconsistency between the tests. Remove the test
> >>>> dependent buffer sizing from run_benchmark().
> >>>
> >>> If I understand correctly the intention is to always use bytes internally
> >>> and only convert to megabytes when displayed to user space. The above
> >>> implies that this takes care of the conversion but there still seems
> >>> to be places that that do not follow my understanding. For example,
> >>> resctrl_val.c:measure_vals() converts to megabytes before proceeding.
> >>
> >> Doesn't the use case inside resctrl_val.c:measure_vals() satisfy
> >> the idea of only displaying data to the user space? From my
> >> understanding it reads the number of bytes and only converts to
> >> MB when printing the value. Or did I miss some detail there?
> > 
> > It's for printing there yes.
> > 
> > But it's not about span in the first place so I'm not sure why it is 
> > related.
> > 
> 
> If this change is just about how "span" is interpreted by the different
> tests then the changelog could be more specific to not create expectation
> that with this change there are no longer "bytes vs megabytes" internal
> inconsistency between MBA, MBM, and CMT tests.

The shortlog and changelog are already pretty specific in mentioning 
"span" a few times :-). I added yet another "span" into the changelog's 
2nd paragraph.

Your general observation about the other MB/bytes inconsistency is still
a good one so I added it also to my todo list, it just doesn't belong to 
this patch (IMHO).

Reinette Chatre July 17, 2023, 4:10 p.m. UTC | #7

Hi Ilpo,

On 7/17/2023 5:30 AM, Ilpo Järvinen wrote:
> On Fri, 14 Jul 2023, Reinette Chatre wrote:
>> On 7/14/2023 3:22 AM, Ilpo Järvinen wrote:
>>> On Fri, 14 Jul 2023, Wieczor-Retman, Maciej wrote:
>>>> On 14.07.2023 01:00, Reinette Chatre wrote:
>>>>> Hi Ilpo,
>>>>>
>>>>> On 7/13/2023 6:19 AM, Ilpo Järvinen wrote:
>>>>>> MBA and MBM tests to use megabytes to represent span. CMT test uses
>>>>>> bytes. The difference requires run_benchmark() to size the buffer
>>>>>> differently based on the test name, which in turn requires passing the
>>>>>> test name into run_benchmark().
>>>>>>
>>>>>> Convert MBA and MBM tests to use internally bytes like CMT test to
>>>>>> remove the internal inconsistency between the tests. Remove the test
>>>>>> dependent buffer sizing from run_benchmark().
>>>>>
>>>>> If I understand correctly the intention is to always use bytes internally
>>>>> and only convert to megabytes when displayed to user space. The above
>>>>> implies that this takes care of the conversion but there still seems
>>>>> to be places that that do not follow my understanding. For example,
>>>>> resctrl_val.c:measure_vals() converts to megabytes before proceeding.
>>>>
>>>> Doesn't the use case inside resctrl_val.c:measure_vals() satisfy
>>>> the idea of only displaying data to the user space? From my
>>>> understanding it reads the number of bytes and only converts to
>>>> MB when printing the value. Or did I miss some detail there?
>>>
>>> It's for printing there yes.
>>>
>>> But it's not about span in the first place so I'm not sure why it is 
>>> related.
>>>
>>
>> If this change is just about how "span" is interpreted by the different
>> tests then the changelog could be more specific to not create expectation
>> that with this change there are no longer "bytes vs megabytes" internal
>> inconsistency between MBA, MBM, and CMT tests.
> 
> The shortlog and changelog are already pretty specific in mentioning 
> "span" a few times :-). I added yet another "span" into the changelog's 
> 2nd paragraph.

There are many things to consider when reviewing a patch. There is the code
changes in the patch itself that can be reviewed for correctness but there is
also the review of the changelog's solution statement to review if the code
changes in the patch accomplishes the stated solution. In one sense the review
considers the code in the patch and in another the review considers what
code may be missing from the patch.

I looked at the v5 changelog and I am still left with the same impression:
The changelog claims that this change removes internal consistency between
the tests, which it does not. Since you posted the next version before this
discussion completed (which is not ideal) I'll respond further to that patch.
In the end it may just be better to drop the "remove the internal inconsistency
between the tests" claim.

> Your general observation about the other MB/bytes inconsistency is still
> a good one so I added it also to my todo list, it just doesn't belong to 
> this patch (IMHO).

ok

Reinette

Ilpo Järvinen July 18, 2023, 10:10 a.m. UTC | #8

On Mon, 17 Jul 2023, Reinette Chatre wrote:
> On 7/17/2023 5:30 AM, Ilpo Järvinen wrote:
> > On Fri, 14 Jul 2023, Reinette Chatre wrote:
> >> On 7/14/2023 3:22 AM, Ilpo Järvinen wrote:
> >>> On Fri, 14 Jul 2023, Wieczor-Retman, Maciej wrote:
> >>>> On 14.07.2023 01:00, Reinette Chatre wrote:
> >>>>> Hi Ilpo,
> >>>>>
> >>>>> On 7/13/2023 6:19 AM, Ilpo Järvinen wrote:
> >>>>>> MBA and MBM tests to use megabytes to represent span. CMT test uses
> >>>>>> bytes. The difference requires run_benchmark() to size the buffer
> >>>>>> differently based on the test name, which in turn requires passing the
> >>>>>> test name into run_benchmark().
> >>>>>>
> >>>>>> Convert MBA and MBM tests to use internally bytes like CMT test to
> >>>>>> remove the internal inconsistency between the tests. Remove the test
> >>>>>> dependent buffer sizing from run_benchmark().
> >>>>>
> >>>>> If I understand correctly the intention is to always use bytes internally
> >>>>> and only convert to megabytes when displayed to user space. The above
> >>>>> implies that this takes care of the conversion but there still seems
> >>>>> to be places that that do not follow my understanding. For example,
> >>>>> resctrl_val.c:measure_vals() converts to megabytes before proceeding.
> >>>>
> >>>> Doesn't the use case inside resctrl_val.c:measure_vals() satisfy
> >>>> the idea of only displaying data to the user space? From my
> >>>> understanding it reads the number of bytes and only converts to
> >>>> MB when printing the value. Or did I miss some detail there?
> >>>
> >>> It's for printing there yes.
> >>>
> >>> But it's not about span in the first place so I'm not sure why it is 
> >>> related.
> >>>
> >>
> >> If this change is just about how "span" is interpreted by the different
> >> tests then the changelog could be more specific to not create expectation
> >> that with this change there are no longer "bytes vs megabytes" internal
> >> inconsistency between MBA, MBM, and CMT tests.
> > 
> > The shortlog and changelog are already pretty specific in mentioning 
> > "span" a few times :-). I added yet another "span" into the changelog's 
> > 2nd paragraph.
> 
> There are many things to consider when reviewing a patch. There is the code
> changes in the patch itself that can be reviewed for correctness but there is
> also the review of the changelog's solution statement to review if the code
> changes in the patch accomplishes the stated solution. In one sense the review
> considers the code in the patch and in another the review considers what
> code may be missing from the patch.

Sure review can enlarge the scope, however, if I add some other things 
unrelated to span into this patch, it no longer is in the minimal form.

> I looked at the v5 changelog and I am still left with the same impression:
> The changelog claims that this change removes internal consistency between
> the tests, which it does not. Since you posted the next version before this
> discussion completed (which is not ideal) I'll respond further to that patch.
> In the end it may just be better to drop the "remove the internal inconsistency
> between the tests" claim.

I'm sorry, it was unintentional to send it like that. ...I honestly 
thought I had addressed all the concerns you had by adding yet another 
"span" (even if the scope was already pretty clear even w/o it, IMHO).

I see nowhere in the changelog mentioned that this change removes all 
internal inconsistencies between the test types. But I'm not attached to 
any particular wording, if the text is good enough for you w/o the 
internal inconsistency wording, I can cut that out.

Patch

diff --git a/tools/testing/selftests/resctrl/mbm_test.c b/tools/testing/selftests/resctrl/mbm_test.c
index fd116158d008..2d58d4b8a918 100644
--- a/tools/testing/selftests/resctrl/mbm_test.c
+++ b/tools/testing/selftests/resctrl/mbm_test.c
@@ -40,7 +40,7 @@  show_bw_info(unsigned long *bw_imc, unsigned long *bw_resc, size_t span)
 	ksft_print_msg("%s Check MBM diff within %d%%\n",
 		       ret ? "Fail:" : "Pass:", MAX_DIFF_PERCENT);
 	ksft_print_msg("avg_diff_per: %d%%\n", avg_diff_per);
-	ksft_print_msg("Span (MB): %zu\n", span);
+	ksft_print_msg("Span (MB): %zu\n", span / MB);
 	ksft_print_msg("avg_bw_imc: %lu\n", avg_bw_imc);
 	ksft_print_msg("avg_bw_resc: %lu\n", avg_bw_resc);
 
diff --git a/tools/testing/selftests/resctrl/resctrl_tests.c b/tools/testing/selftests/resctrl/resctrl_tests.c
index 6b2ca1b4744f..5205a13ed250 100644
--- a/tools/testing/selftests/resctrl/resctrl_tests.c
+++ b/tools/testing/selftests/resctrl/resctrl_tests.c
@@ -185,8 +185,8 @@  int main(int argc, char **argv)
 	char benchmark_cmd_area[BENCHMARK_ARGS][BENCHMARK_ARG_SIZE];
 	int c, cpu_no = 1, argc_new = argc, i, no_of_bits = 0;
 	int ben_ind, ben_count, tests = 0;
+	size_t span = 250 * MB;
 	bool cat_test = true;
-	size_t span = 250;
 
 	for (i = 0; i < argc; i++) {
 		if (strcmp(argv[i], "-b") == 0) {
diff --git a/tools/testing/selftests/resctrl/resctrlfs.c b/tools/testing/selftests/resctrl/resctrlfs.c
index 8be5b745226d..847e0a80c6cd 100644
--- a/tools/testing/selftests/resctrl/resctrlfs.c
+++ b/tools/testing/selftests/resctrl/resctrlfs.c
@@ -298,9 +298,9 @@  int taskset_benchmark(pid_t bm_pid, int cpu_no)
 void run_benchmark(int signum, siginfo_t *info, void *ucontext)
 {
 	int operation, ret, malloc_and_init_memory, memflush;
-	size_t span, buffer_span;
 	char **benchmark_cmd;
 	char resctrl_val[64];
+	size_t span;
 	FILE *fp;
 
 	benchmark_cmd = info->si_ptr;
@@ -321,12 +321,7 @@  void run_benchmark(int signum, siginfo_t *info, void *ucontext)
 		operation = atoi(benchmark_cmd[4]);
 		sprintf(resctrl_val, "%s", benchmark_cmd[5]);
 
-		if (strncmp(resctrl_val, CMT_STR, sizeof(CMT_STR)))
-			buffer_span = span * MB;
-		else
-			buffer_span = span;
-
-		if (run_fill_buf(buffer_span, malloc_and_init_memory, memflush,
+		if (run_fill_buf(span, malloc_and_init_memory, memflush,
 				 operation, resctrl_val))
 			fprintf(stderr, "Error in running fill buffer\n");
 	} else {