[AUTOTEST,RFC] KVM test: keep record of supported qemu options

Commit Message

Michael Goldish July 29, 2009, 7:18 a.m. UTC

I'm not entirely sure this is needed:
- How often do tests fail due to unsupported flags? I thought the -mem-path
problem was only due to a temporary typo (I'm not sure how temporary).
Once it's fixed, will we ever encounter that misspelling again?
- A typo is a bug too, so maybe we shouldn't make an effort to support it.
- Maybe it's not so bad to ask users to change the config file when they
want the system to work with a known typo. That's why we put stuff in the
config file rather than hardcode it into Autotest -- the config file can
be modified more easily.

Also, it _might_ not work:
- "-help" is sometimes outdated as far as I know. Right now it seems up to
date, but not long ago it didn't mention -incoming and I think -rtc-td-hack
was missing too, so I'm not sure relying on it is safe.
- If -mem-path is a typo, what appears in the output of -help? The typo or
the correct spelling? If the correct spelling appears, then -help is of no
use to us.

Otherwise, the implementation looks nice. I'm not sure I'm right about the
things listed above so please tell me what you think.

Thanks,
Michael

----- Original Message -----
From: "Lucas Meneghel Rodrigues" <[e-mail]>
To: [e-mail]
Cc: [e-mail], [e-mail], [e-mail], [e-mail], "Lucas Meneghel Rodrigues" <[e-mail]>
Sent: Wednesday, July 29, 2009 6:40:29 AM (GMT+0200) Auto-Detected
Subject: [KVM AUTOTEST PATCH] [RFC] KVM test: keep record of supported qemu options

In order to make it easier to figure out problems and
also to avoid aborting tests prematurely due to
incompatible qemu options, keep record of supported
qemu options, and if extra options are passed to qemu,
verify if they are amongst the supported options. Also,
try to replace known misspelings on options in case
something goes wrong, and be generous logging any problems.

This first version of the patch gets supported flags from
the output of qemu --help. I thought this would be good
enough for a first start. I am asking for input on whether
this is needed, and if yes, if the approach looks good.

Signed-off-by: Lucas Meneghel Rodrigues <[e-mail]>
---
 client/tests/kvm/kvm_vm.py |   79 ++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 77 insertions(+), 2 deletions(-)

Comments

Lucas Meneghel Rodrigues July 29, 2009, 12:10 p.m. UTC | #1

On Wed, Jul 29, 2009 at 4:18 AM, Michael Goldish<[e-mail]> wrote:
> I'm not entirely sure this is needed:

I wasn't sure either... I guess I ended up spending time to fix a
problem that didn't need to be fixed...

> - How often do tests fail due to unsupported flags? I thought the -mem-path
> problem was only due to a temporary typo (I'm not sure how temporary).
> Once it's fixed, will we ever encounter that misspelling again?
> - A typo is a bug too, so maybe we shouldn't make an effort to support it.
> - Maybe it's not so bad to ask users to change the config file when they
> want the system to work with a known typo. That's why we put stuff in the
> config file rather than hardcode it into Autotest -- the config file can
> be modified more easily.
>
> Also, it _might_ not work:
> - "-help" is sometimes outdated as far as I know. Right now it seems up to
> date, but not long ago it didn't mention -incoming and I think -rtc-td-hack
> was missing too, so I'm not sure relying on it is safe.
> - If -mem-path is a typo, what appears in the output of -help? The typo or
> the correct spelling? If the correct spelling appears, then -help is of no
> use to us.

Yes, I was talking about this earlier on IRC, and to be rigorous, I
got the suggestion that we could inspect qemu under GDB and verify the
actual options. Comparing the actual flags found with GDB and what we
get from help could be a nice test to spot out this sort of bug in the
future though.

That said, your comment and Lukáš's already convinced me that this
patch should be disregarded. Thank you folks!

> Otherwise, the implementation looks nice. I'm not sure I'm right about the
> things listed above so please tell me what you think.
>
> Thanks,
> Michael
>
> ----- Original Message -----
> From: "Lucas Meneghel Rodrigues" <[e-mail]>
> To: [e-mail]
> Cc: [e-mail], [e-mail], [e-mail], [e-mail], "Lucas Meneghel Rodrigues" <[e-mail]>
> Sent: Wednesday, July 29, 2009 6:40:29 AM (GMT+0200) Auto-Detected
> Subject: [KVM AUTOTEST PATCH] [RFC] KVM test: keep record of supported qemu options
>
> In order to make it easier to figure out problems and
> also to avoid aborting tests prematurely due to
> incompatible qemu options, keep record of supported
> qemu options, and if extra options are passed to qemu,
> verify if they are amongst the supported options. Also,
> try to replace known misspelings on options in case
> something goes wrong, and be generous logging any problems.
>
> This first version of the patch gets supported flags from
> the output of qemu --help. I thought this would be good
> enough for a first start. I am asking for input on whether
> this is needed, and if yes, if the approach looks good.
>
> Signed-off-by: Lucas Meneghel Rodrigues <[e-mail]>
> ---
>  client/tests/kvm/kvm_vm.py |   79 ++++++++++++++++++++++++++++++++++++++++++-
>  1 files changed, 77 insertions(+), 2 deletions(-)
>
> diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
> index eba9b84..0dd34c2 100644
> --- a/client/tests/kvm/kvm_vm.py
> +++ b/client/tests/kvm/kvm_vm.py
> @@ -121,6 +121,7 @@ class VM:
>         self.qemu_path = qemu_path
>         self.image_dir = image_dir
>         self.iso_dir = iso_dir
> +        self.qemu_supported_flags = self.get_qemu_supported_flags()
>
>
>         # Find available monitor filename
> @@ -258,7 +259,7 @@ class VM:
>
>         extra_params = params.get("extra_params")
>         if extra_params:
> -            qemu_cmd += " %s" % extra_params
> +            qemu_cmd += " %s" % self.process_qemu_extra_params(extra_params)
>
>         for redir_name in kvm_utils.get_sub_dict_names(params, "redirs"):
>             redir_params = kvm_utils.get_sub_dict(params, redir_name)
> @@ -751,7 +752,7 @@ class VM:
>             else:
>                 self.send_key(char)
>
> -
> +
>     def get_uuid(self):
>         """
>         Catch UUID of the VM.
> @@ -762,3 +763,77 @@ class VM:
>             return self.uuid
>         else:
>             return self.params.get("uuid", None)
> +
> +
> +    def get_qemu_supported_flags(self):
> +        """
> +        Gets all supported qemu options from qemu-help. This is a useful
> +        procedure to quickly spot problems with incompatible qemu flags.
> +        """
> +        cmd = self.qemu_path + ' --help'
> +        (status, output) = kvm_subprocess.run_fg(cmd)
> +        supported_flags = []
> +
> +        if status:
> +            logging.error('Process qemu --help ended with exit code !=0. '
> +                          'No supported qemu flags will be recorded.')
> +            return supported_flags
> +
> +        for line in output.split('\n'):
> +            if line and line.startswith('-'):
> +                flag = line.split()[0]
> +                if flag not in supported_flags:
> +                    supported_flags.append(flag)
> +
> +        return supported_flags
> +
> +
> +    def process_qemu_extra_params(self, extra_params):
> +        """
> +        Verifies an extra param passed to qemu to see if it's supported by the
> +        current qemu version. If it's not supported, try to find an appropriate
> +        replacement on a list of known option misspellings.
> +
> +        @param extra_params: String with a qemu command line option.
> +        """
> +        flag = extra_params.split()[0]
> +
> +        if flag not in self.qemu_supported_flags:
> +            logging.error("Flag %s does not seem to be supported by the "
> +                          "current qemu version. Looking for a replacement...",
> +                          flag)
> +            supported_flag = self.get_qemu_flag_replacement(flag)
> +            if supported_flag:
> +                logging.debug("Replacing flag %s with %s", flag,
> +                              supported_flag)
> +                extra_params = extra_params.replace(flag, supported_flag)
> +            else:
> +                logging.error("No valid replacement was found for flag %s.",
> +                              flag)
> +
> +        return extra_params
> +
> +
> +    def get_qemu_flag_replacement(self, option):
> +        """
> +        Searches on a list of known misspellings for qemu options and returns
> +        a replacement. If no replacement can be found, return None.
> +
> +        @param option: String representing qemu option (such as -mem).
> +
> +        @return: Option replacement, or None, if none found.
> +        """
> +        list_mispellings = [['-mem-path', '-mempath'],]
> +        replacement = None
> +
> +        for mispellings in list_mispellings:
> +            if option in mispellings:
> +                option_position = mispellings.index(option)
> +                replacement = mispellings[1 - option_position]
> +
> +        if replacement not in self.qemu_supported_flags:
> +            logging.error("Replacement %s also does not seem to be a valid "
> +                          "qemu flag, aborting replacement.", replacement)
> +            return None
> +
> +        return replacement
> --
> 1.6.2.5
>
> _______________________________________________
> Autotest mailing list
> [e-mail]
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>

Patch

diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
index eba9b84..0dd34c2 100644
--- a/client/tests/kvm/kvm_vm.py
+++ b/client/tests/kvm/kvm_vm.py
@@ -121,6 +121,7 @@  class VM:
         self.qemu_path = qemu_path
         self.image_dir = image_dir
         self.iso_dir = iso_dir
+        self.qemu_supported_flags = self.get_qemu_supported_flags()
 
 
         # Find available monitor filename
@@ -258,7 +259,7 @@  class VM:
 
         extra_params = params.get("extra_params")
         if extra_params:
-            qemu_cmd += " %s" % extra_params
+            qemu_cmd += " %s" % self.process_qemu_extra_params(extra_params)
 
         for redir_name in kvm_utils.get_sub_dict_names(params, "redirs"):
             redir_params = kvm_utils.get_sub_dict(params, redir_name)
@@ -751,7 +752,7 @@  class VM:
             else:
                 self.send_key(char)
 
-    
+
     def get_uuid(self):
         """
         Catch UUID of the VM.
@@ -762,3 +763,77 @@  class VM:
             return self.uuid
         else:
             return self.params.get("uuid", None)
+
+
+    def get_qemu_supported_flags(self):
+        """
+        Gets all supported qemu options from qemu-help. This is a useful
+        procedure to quickly spot problems with incompatible qemu flags.
+        """
+        cmd = self.qemu_path + ' --help'
+        (status, output) = kvm_subprocess.run_fg(cmd)
+        supported_flags = []
+
+        if status:
+            logging.error('Process qemu --help ended with exit code !=0. '
+                          'No supported qemu flags will be recorded.')
+            return supported_flags
+
+        for line in output.split('\n'):
+            if line and line.startswith('-'):
+                flag = line.split()[0]
+                if flag not in supported_flags:
+                    supported_flags.append(flag)
+
+        return supported_flags
+
+
+    def process_qemu_extra_params(self, extra_params):
+        """
+        Verifies an extra param passed to qemu to see if it's supported by the
+        current qemu version. If it's not supported, try to find an appropriate
+        replacement on a list of known option misspellings.
+
+        @param extra_params: String with a qemu command line option.
+        """
+        flag = extra_params.split()[0]
+
+        if flag not in self.qemu_supported_flags:
+            logging.error("Flag %s does not seem to be supported by the "
+                          "current qemu version. Looking for a replacement...",
+                          flag)
+            supported_flag = self.get_qemu_flag_replacement(flag)
+            if supported_flag:
+                logging.debug("Replacing flag %s with %s", flag,
+                              supported_flag)
+                extra_params = extra_params.replace(flag, supported_flag)
+            else:
+                logging.error("No valid replacement was found for flag %s.",
+                              flag)
+
+        return extra_params
+
+
+    def get_qemu_flag_replacement(self, option):
+        """
+        Searches on a list of known misspellings for qemu options and returns
+        a replacement. If no replacement can be found, return None.
+
+        @param option: String representing qemu option (such as -mem).
+
+        @return: Option replacement, or None, if none found.
+        """
+        list_mispellings = [['-mem-path', '-mempath'],]
+        replacement = None
+
+        for mispellings in list_mispellings:
+            if option in mispellings:
+                option_position = mispellings.index(option)
+                replacement = mispellings[1 - option_position]
+
+        if replacement not in self.qemu_supported_flags:
+            logging.error("Replacement %s also does not seem to be a valid "
+                          "qemu flag, aborting replacement.", replacement)
+            return None
+
+        return replacement