mbox series

[0/2] Increase amount of data for monitor to read

Message ID 1604666522-545580-1-git-send-email-andrey.shinkevich@virtuozzo.com (mailing list archive)
Headers show
Series Increase amount of data for monitor to read | expand

Message

Andrey Shinkevich Nov. 6, 2020, 12:42 p.m. UTC
The subject was discussed here:
https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg00206.html

This series is a solution for the issue with QMP monitor buffered input.
A little parser is introduced to throttle JSON commands read from the
buffer so that QMP requests do not overwhelm the monitor input queue.
A side effect raised in the test #247 was managed in the first patch.
It may be considered as a workaround. Any sane fix suggested will be
appreciated.

Note:
This series goes after the Vladimir's one:
'[PATCH v3 00/25] backup performance: block_status + async"'
To make the test #129 passed, the following patch should be applied first:
'[PATCH v3 01/25] iotests: 129 don't check backup "busy"'.

Andrey Shinkevich (2):
  iotests: add another bash sleep command to 247
  monitor: increase amount of data for monitor to read

 chardev/char-fd.c          | 64 +++++++++++++++++++++++++++++++++++++++++++++-
 chardev/char-socket.c      | 54 +++++++++++++++++++++++++++-----------
 chardev/char.c             | 40 +++++++++++++++++++++++++++++
 include/chardev/char.h     | 15 +++++++++++
 monitor/monitor.c          |  2 +-
 tests/qemu-iotests/247     |  2 ++
 tests/qemu-iotests/247.out |  1 +
 7 files changed, 161 insertions(+), 17 deletions(-)

Comments

Andrey Shinkevich Nov. 6, 2020, 3:31 p.m. UTC | #1
Please exclude this address when reply:

jcody@redhat.com

Andrey
Vladimir Sementsov-Ogievskiy Nov. 9, 2020, 8:50 a.m. UTC | #2
06.11.2020 15:42, Andrey Shinkevich wrote:
> The subject was discussed here:
> https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg00206.html
> 
> This series is a solution for the issue with QMP monitor buffered input.
> A little parser is introduced to throttle JSON commands read from the
> buffer so that QMP requests do not overwhelm the monitor input queue.
> A side effect raised in the test #247 was managed in the first patch.
> It may be considered as a workaround. Any sane fix suggested will be
> appreciated.
> 
> Note:
> This series goes after the Vladimir's one:
> '[PATCH v3 00/25] backup performance: block_status + async"'
> To make the test #129 passed, the following patch should be applied first:
> '[PATCH v3 01/25] iotests: 129 don't check backup "busy"'.
> 

Hi!

I tried the following test:

start qemu:

   ./qemu-system-x86_64 -qmp stdio

type the following in one line:

   { 'execute': 'qmp_capabilities' }{ 'execute': 'quit' }

press Enter.

Without your patches, the qemu quits immediately, printing double "{"return": {}}".
With your patches applied qemu prints "{"return": {}}" only once and doesn't quit, until I press Enter the second time.


> Andrey Shinkevich (2):
>    iotests: add another bash sleep command to 247
>    monitor: increase amount of data for monitor to read
> 
>   chardev/char-fd.c          | 64 +++++++++++++++++++++++++++++++++++++++++++++-
>   chardev/char-socket.c      | 54 +++++++++++++++++++++++++++-----------
>   chardev/char.c             | 40 +++++++++++++++++++++++++++++
>   include/chardev/char.h     | 15 +++++++++++
>   monitor/monitor.c          |  2 +-
>   tests/qemu-iotests/247     |  2 ++
>   tests/qemu-iotests/247.out |  1 +
>   7 files changed, 161 insertions(+), 17 deletions(-)
>
Vladimir Sementsov-Ogievskiy Nov. 9, 2020, 10:04 a.m. UTC | #3
09.11.2020 11:50, Vladimir Sementsov-Ogievskiy wrote:
> 06.11.2020 15:42, Andrey Shinkevich wrote:
>> The subject was discussed here:
>> https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg00206.html
>>
>> This series is a solution for the issue with QMP monitor buffered input.
>> A little parser is introduced to throttle JSON commands read from the
>> buffer so that QMP requests do not overwhelm the monitor input queue.
>> A side effect raised in the test #247 was managed in the first patch.
>> It may be considered as a workaround. Any sane fix suggested will be
>> appreciated.
>>
>> Note:
>> This series goes after the Vladimir's one:
>> '[PATCH v3 00/25] backup performance: block_status + async"'
>> To make the test #129 passed, the following patch should be applied first:
>> '[PATCH v3 01/25] iotests: 129 don't check backup "busy"'.
>>
> 
> Hi!
> 
> I tried the following test:
> 
> start qemu:
> 
>    ./qemu-system-x86_64 -qmp stdio
> 
> type the following in one line:
> 
>    { 'execute': 'qmp_capabilities' }{ 'execute': 'quit' }
> 
> press Enter.
> 
> Without your patches, the qemu quits immediately, printing double "{"return": {}}".
> With your patches applied qemu prints "{"return": {}}" only once and doesn't quit, until I press Enter the second time.


Positive thing: the patches do increase performance:

for me, the following command:

(echo "{ 'execute': 'qmp_capabilities' }"; for i in {1..10000}; do echo "{ 'execute': 'query-block-jobs' }"; done; echo "{ 'execute': 'quit' }" ) | time ./qemu-system-x86_64 -qmp stdio > /dev/null

shows 2.4s on master and 0.6s after patches


> 
> 
>> Andrey Shinkevich (2):
>>    iotests: add another bash sleep command to 247
>>    monitor: increase amount of data for monitor to read
>>
>>   chardev/char-fd.c          | 64 +++++++++++++++++++++++++++++++++++++++++++++-
>>   chardev/char-socket.c      | 54 +++++++++++++++++++++++++++-----------
>>   chardev/char.c             | 40 +++++++++++++++++++++++++++++
>>   include/chardev/char.h     | 15 +++++++++++
>>   monitor/monitor.c          |  2 +-
>>   tests/qemu-iotests/247     |  2 ++
>>   tests/qemu-iotests/247.out |  1 +
>>   7 files changed, 161 insertions(+), 17 deletions(-)
>>
> 
>
Andrey Shinkevich Nov. 9, 2020, 5:08 p.m. UTC | #4
On 09.11.2020 13:04, Vladimir Sementsov-Ogievskiy wrote:
> 09.11.2020 11:50, Vladimir Sementsov-Ogievskiy wrote:
>> 06.11.2020 15:42, Andrey Shinkevich wrote:
>>> The subject was discussed here:
>>> https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg00206.html
>>>
>>> This series is a solution for the issue with QMP monitor buffered input.
>>> A little parser is introduced to throttle JSON commands read from the
>>> buffer so that QMP requests do not overwhelm the monitor input queue.
>>> A side effect raised in the test #247 was managed in the first patch.
>>> It may be considered as a workaround. Any sane fix suggested will be
>>> appreciated.
>>>
>>> Note:
>>> This series goes after the Vladimir's one:
>>> '[PATCH v3 00/25] backup performance: block_status + async"'
>>> To make the test #129 passed, the following patch should be applied 
>>> first:
>>> '[PATCH v3 01/25] iotests: 129 don't check backup "busy"'.
>>>

[...]

> 
> 
> Positive thing: the patches do increase performance:
> 
> for me, the following command:
> 
> (echo "{ 'execute': 'qmp_capabilities' }"; for i in {1..10000}; do echo 
> "{ 'execute': 'query-block-jobs' }"; done; echo "{ 'execute': 'quit' }" 
> ) | time ./qemu-system-x86_64 -qmp stdio > /dev/null
> 
> shows 2.4s on master and 0.6s after patches
> 
> 

Thank you for testing it. I'd like to include the result to the patch 
description with "Tested-by: ..."

Andrey