| Message ID | 1550607835-11755-1-git-send-email-chaitanya.kulkarni@wdc.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | nvme: fix output formats | expand |
On Tue, Feb 19, 2019 at 12:23:53PM -0800, Chaitanya Kulkarni wrote: > Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com> > --- > tests/nvme/002.out | 4996 ++++++++++++++++++++++++++-------------------------- > 1 file changed, 2498 insertions(+), 2498 deletions(-) > > diff --git a/tests/nvme/002.out b/tests/nvme/002.out > index aa71d8f..abecda5 100644 > --- a/tests/nvme/002.out > +++ b/tests/nvme/002.out > @@ -1,11 +1,11 @@ > Running nvme/002 > > -Discovery Log Number of Records 1000, Generation counter 1000 > +Discovery Log Number of Records 1000, Generation counter 2000 > =====Discovery Log Entry 0====== > trtype: loop > adrfam: pci > subtype: nvme subsystem > -treq: not specified > +treq: not specified, sq flow control disable supported > portid: X > trsvcid: > subnqn: blktests-subsystem-0 [snip] Why did this output change? An nvme-cli change, or a kernel change, or something else? And will this now fail with older versions of nvme-cli or the kernel instead?
On 02/20/2019 10:04 AM, Omar Sandoval wrote: > On Tue, Feb 19, 2019 at 12:23:53PM -0800, Chaitanya Kulkarni wrote: >> Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com> >> --- >> tests/nvme/002.out | 4996 ++++++++++++++++++++++++++-------------------------- >> 1 file changed, 2498 insertions(+), 2498 deletions(-) >> >> diff --git a/tests/nvme/002.out b/tests/nvme/002.out >> index aa71d8f..abecda5 100644 >> --- a/tests/nvme/002.out >> +++ b/tests/nvme/002.out >> @@ -1,11 +1,11 @@ >> Running nvme/002 >> >> -Discovery Log Number of Records 1000, Generation counter 1000 >> +Discovery Log Number of Records 1000, Generation counter 2000 >> =====Discovery Log Entry 0====== >> trtype: loop >> adrfam: pci >> subtype: nvme subsystem >> -treq: not specified >> +treq: not specified, sq flow control disable supported >> portid: X >> trsvcid: >> subnqn: blktests-subsystem-0 > > [snip] > > Why did this output change? An nvme-cli change, or a kernel change, or > something else? And will this now fail with older versions of nvme-cli > or the kernel instead? > This change is specific to nvme-cli only. It should fail on older cli versions, we want latest cli/tools to be used when running these testcases so that all the latest tools are also been tested for the compliance, else we might be running tests with any bugs with older version(s) which are fixed already. Actually I'd like to add quick nvme-cli version check at the start of each test that will certainly help/force user to have a latest version.
On Wed, Feb 20, 2019 at 06:28:50PM +0000, Chaitanya Kulkarni wrote: > On 02/20/2019 10:04 AM, Omar Sandoval wrote: > > On Tue, Feb 19, 2019 at 12:23:53PM -0800, Chaitanya Kulkarni wrote: > >> Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com> > >> --- > >> tests/nvme/002.out | 4996 ++++++++++++++++++++++++++-------------------------- > >> 1 file changed, 2498 insertions(+), 2498 deletions(-) > >> > >> diff --git a/tests/nvme/002.out b/tests/nvme/002.out > >> index aa71d8f..abecda5 100644 > >> --- a/tests/nvme/002.out > >> +++ b/tests/nvme/002.out > >> @@ -1,11 +1,11 @@ > >> Running nvme/002 > >> > >> -Discovery Log Number of Records 1000, Generation counter 1000 > >> +Discovery Log Number of Records 1000, Generation counter 2000 > >> =====Discovery Log Entry 0====== > >> trtype: loop > >> adrfam: pci > >> subtype: nvme subsystem > >> -treq: not specified > >> +treq: not specified, sq flow control disable supported > >> portid: X > >> trsvcid: > >> subnqn: blktests-subsystem-0 > > > > [snip] > > > > Why did this output change? An nvme-cli change, or a kernel change, or > > something else? And will this now fail with older versions of nvme-cli > > or the kernel instead? > > > > This change is specific to nvme-cli only. > > It should fail on older cli versions, we want latest cli/tools to be > used when running these testcases so that all the latest tools are > also been tested for the compliance, else we might be running tests > with any bugs with older version(s) which are fixed already. > > Actually I'd like to add quick nvme-cli version check at the start of > each test that will certainly help/force user to have a latest version. Perfect, I was going to suggest adding a check for the minimum version needed for each test.
On Wed, Feb 20, 2019 at 11:35:00AM -0800, Omar Sandoval wrote: > > It should fail on older cli versions, we want latest cli/tools to be > > used when running these testcases so that all the latest tools are > > also been tested for the compliance, else we might be running tests > > with any bugs with older version(s) which are fixed already. > > > > Actually I'd like to add quick nvme-cli version check at the start of > > each test that will certainly help/force user to have a latest version. > > Perfect, I was going to suggest adding a check for the minimum version > needed for each test. It would be great if we could document what is (a) the minimum version for things to work at all, and (b) the ideal version to use so that tests aren't being skipped due to version constraints and so we are testing the kernel as much as possible. Ideally, tests should either be designed primarily to test userspace, or to test the kernel, so that people who want to use blktests aren't having to build the latest versions of userspace. (As an aside, Bart has told me that the failures I'm seeing with srp/002 and srp/011 are due to the fact that I'm using a version of multipathd which is too old. Apparently both the version of multipathd in Debian stable *and* Debian unstable are still too old, since they are still failing. Next stop, building multipathd from the git repo.... Grrr......) - Ted
On 02/21/2019 07:04 AM, Theodore Y. Ts'o wrote: > On Wed, Feb 20, 2019 at 11:35:00AM -0800, Omar Sandoval wrote: >>> It should fail on older cli versions, we want latest cli/tools to be >>> used when running these testcases so that all the latest tools are >>> also been tested for the compliance, else we might be running tests >>> with any bugs with older version(s) which are fixed already. >>> >>> Actually I'd like to add quick nvme-cli version check at the start of >>> each test that will certainly help/force user to have a latest version. >> >> Perfect, I was going to suggest adding a check for the minimum version >> needed for each test. > > It would be great if we could document what is (a) the minimum version > for things to work at all, Actually the latest version or the HEAD of the git repo should work all the time if it is breaking test or the tool or driver we need to add a fix respectively. and (b) the ideal version to use so that > tests aren't being skipped due to version constraints and so we are > testing the kernel as much as possible. > Agree. > Ideally, tests should either be designed primarily to test userspace, > or to test the kernel, so that people who want to use blktests aren't > having to build the latest versions of userspace. > At least with the current NVMe infrastructure we need latest tools (git HEAD) to be compliant all the time with the tests, since it guarantees the tools, driver and kernel compatibility. We can a little install script to build and install the cli if desired version is not present. > (As an aside, Bart has told me that the failures I'm seeing with > srp/002 and srp/011 are due to the fact that I'm using a version of > multipathd which is too old. Apparently both the version of > multipathd in Debian stable *and* Debian unstable are still too old, > since they are still failing. Next stop, building multipathd from > the git repo.... Grrr......) This is exact reason why tools needs to be built and check with latest HEAD. > > - Ted >
On Fri, Feb 22, 2019 at 12:00:43AM +0000, Chaitanya Kulkarni wrote: > On 02/21/2019 07:04 AM, Theodore Y. Ts'o wrote: > > On Wed, Feb 20, 2019 at 11:35:00AM -0800, Omar Sandoval wrote: > >>> It should fail on older cli versions, we want latest cli/tools to be > >>> used when running these testcases so that all the latest tools are > >>> also been tested for the compliance, else we might be running tests > >>> with any bugs with older version(s) which are fixed already. > >>> > >>> Actually I'd like to add quick nvme-cli version check at the start of > >>> each test that will certainly help/force user to have a latest version. > >> > >> Perfect, I was going to suggest adding a check for the minimum version > >> needed for each test. > > > > It would be great if we could document what is (a) the minimum version > > for things to work at all, > Actually the latest version or the HEAD of the git repo should work all > the time if it is breaking test or the tool or driver we need to add > a fix respectively. > > and (b) the ideal version to use so that > > tests aren't being skipped due to version constraints and so we are > > testing the kernel as much as possible. > > > Agree. > > Ideally, tests should either be designed primarily to test userspace, > > or to test the kernel, so that people who want to use blktests aren't > > having to build the latest versions of userspace. > > > At least with the current NVMe infrastructure we need latest tools > (git HEAD) to be compliant all the time with the tests, since it > guarantees the tools, driver and kernel compatibility. > > We can a little install script to build and install the > cli if desired version is not present. > > > (As an aside, Bart has told me that the failures I'm seeing with > > srp/002 and srp/011 are due to the fact that I'm using a version of > > multipathd which is too old. Apparently both the version of > > multipathd in Debian stable *and* Debian unstable are still too old, > > since they are still failing. Next stop, building multipathd from > > the git repo.... Grrr......) > This is exact reason why tools needs to be built and check with latest > HEAD. I don't think we should be _requiring_ that the tests are run with the latest tools from git. That just adds another annoying step to someone trying to get the tests up and running. Like I said, each test should check for the minimum version it needs, and we should avoid making the golden output too dependent on version. However, I do think it's a reasonable expectation that if you want maximum test coverage, you should build the tools from git.