| Message ID | 20250228195708.48035-1-nirsof@gmail.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | [v2] iotest: Unbreak 302 with python 3.13 | expand |
On Fri, Feb 28, 2025 at 09:57:08PM +0200, Nir Soffer wrote: > This test depends on TarFile.addfile() to add tar member header without > writing the member data, which we write ourself using qemu-nbd. Python > 3.13 changed the function in a backward incompatible way[1] to require a > file object for tarinfo with non-zero size, breaking the test: > > -[{"name": "vm.ovf", "offset": 512, "size": 6}, {"name": "disk", "offset": 1536, "size": 393216}] > +Traceback (most recent call last): > + File "/home/stefanha/qemu/tests/qemu-iotests/302", line 118, in <module> > + tar.addfile(disk) > + ~~~~~~~~~~~^^^^^^ > + File "/usr/lib64/python3.13/tarfile.py", line 2262, in addfile > + raise ValueError("fileobj not provided for non zero-size regular file") > +ValueError: fileobj not provided for non zero-size regular file > > The new behavior makes sense for most users, but breaks our unusual > usage. Fix the test to add the member header directly using public but > undocumented attributes. This is more fragile but the test works again. > > This also fixes a bug in the previous code - when calling addfile() > without a fileobject, tar.offset points to the start of the member data > instead of the end. > > [1] https://github.com/python/cpython/pull/117988 > > Signed-off-by: Nir Soffer <nirsof@gmail.com> Reviewed-by: Eric Blake <eblake@redhat.com> > --- > tests/qemu-iotests/302 | 19 ++++++++++++++----- > 1 file changed, 14 insertions(+), 5 deletions(-) > > diff --git a/tests/qemu-iotests/302 b/tests/qemu-iotests/302 > index a6d79e727b..e980ec513f 100755 > --- a/tests/qemu-iotests/302 > +++ b/tests/qemu-iotests/302 > @@ -115,13 +115,22 @@ with tarfile.open(tar_file, "w") as tar: > > disk = tarfile.TarInfo("disk") > disk.size = actual_size > - tar.addfile(disk) > > - # 6. Shrink the tar to the actual size, aligned to 512 bytes. > + # Since python 3.13 we cannot use addfile() to create the member header. > + # Add the tarinfo directly using public but undocumented attributes. > > - tar_size = offset + (disk.size + 511) & ~511 > - tar.fileobj.seek(tar_size) > - tar.fileobj.truncate(tar_size) > + buf = disk.tobuf(tar.format, tar.encoding, tar.errors) > + tar.fileobj.write(buf) > + tar.members.append(disk) > + > + # Update the offset and position to the location of the next member. > + > + tar.offset = offset + (disk.size + 511) & ~511 > + tar.fileobj.seek(tar.offset) > + > + # 6. Shrink the tar to the actual size. > + > + tar.fileobj.truncate(tar.offset) > > with tarfile.open(tar_file) as tar: > members = [{"name": m.name, "size": m.size, "offset": m.offset_data} > -- > 2.39.5 (Apple Git-154) > >
On Fri, Feb 28, 2025 at 09:57:08PM +0200, Nir Soffer wrote: > This test depends on TarFile.addfile() to add tar member header without > writing the member data, which we write ourself using qemu-nbd. Python > 3.13 changed the function in a backward incompatible way[1] to require a > file object for tarinfo with non-zero size, breaking the test: > > -[{"name": "vm.ovf", "offset": 512, "size": 6}, {"name": "disk", "offset": 1536, "size": 393216}] > +Traceback (most recent call last): > + File "/home/stefanha/qemu/tests/qemu-iotests/302", line 118, in <module> > + tar.addfile(disk) > + ~~~~~~~~~~~^^^^^^ > + File "/usr/lib64/python3.13/tarfile.py", line 2262, in addfile > + raise ValueError("fileobj not provided for non zero-size regular file") > +ValueError: fileobj not provided for non zero-size regular file > > The new behavior makes sense for most users, but breaks our unusual > usage. Fix the test to add the member header directly using public but > undocumented attributes. This is more fragile but the test works again. > > This also fixes a bug in the previous code - when calling addfile() > without a fileobject, tar.offset points to the start of the member data > instead of the end. > > [1] https://github.com/python/cpython/pull/117988 > > Signed-off-by: Nir Soffer <nirsof@gmail.com> > --- > tests/qemu-iotests/302 | 19 ++++++++++++++----- > 1 file changed, 14 insertions(+), 5 deletions(-) Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
diff --git a/tests/qemu-iotests/302 b/tests/qemu-iotests/302 index a6d79e727b..e980ec513f 100755 --- a/tests/qemu-iotests/302 +++ b/tests/qemu-iotests/302 @@ -115,13 +115,22 @@ with tarfile.open(tar_file, "w") as tar: disk = tarfile.TarInfo("disk") disk.size = actual_size - tar.addfile(disk) - # 6. Shrink the tar to the actual size, aligned to 512 bytes. + # Since python 3.13 we cannot use addfile() to create the member header. + # Add the tarinfo directly using public but undocumented attributes. - tar_size = offset + (disk.size + 511) & ~511 - tar.fileobj.seek(tar_size) - tar.fileobj.truncate(tar_size) + buf = disk.tobuf(tar.format, tar.encoding, tar.errors) + tar.fileobj.write(buf) + tar.members.append(disk) + + # Update the offset and position to the location of the next member. + + tar.offset = offset + (disk.size + 511) & ~511 + tar.fileobj.seek(tar.offset) + + # 6. Shrink the tar to the actual size. + + tar.fileobj.truncate(tar.offset) with tarfile.open(tar_file) as tar: members = [{"name": m.name, "size": m.size, "offset": m.offset_data}
This test depends on TarFile.addfile() to add tar member header without writing the member data, which we write ourself using qemu-nbd. Python 3.13 changed the function in a backward incompatible way[1] to require a file object for tarinfo with non-zero size, breaking the test: -[{"name": "vm.ovf", "offset": 512, "size": 6}, {"name": "disk", "offset": 1536, "size": 393216}] +Traceback (most recent call last): + File "/home/stefanha/qemu/tests/qemu-iotests/302", line 118, in <module> + tar.addfile(disk) + ~~~~~~~~~~~^^^^^^ + File "/usr/lib64/python3.13/tarfile.py", line 2262, in addfile + raise ValueError("fileobj not provided for non zero-size regular file") +ValueError: fileobj not provided for non zero-size regular file The new behavior makes sense for most users, but breaks our unusual usage. Fix the test to add the member header directly using public but undocumented attributes. This is more fragile but the test works again. This also fixes a bug in the previous code - when calling addfile() without a fileobject, tar.offset points to the start of the member data instead of the end. [1] https://github.com/python/cpython/pull/117988 Signed-off-by: Nir Soffer <nirsof@gmail.com> --- tests/qemu-iotests/302 | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-)