From patchwork Fri Feb 14 01:53:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yong Huang X-Patchwork-Id: 13974277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7BAA2C021A4 for ; Fri, 14 Feb 2025 01:55:40 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tikun-0006bF-5u; Thu, 13 Feb 2025 20:54:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tikul-0006aX-00 for qemu-devel@nongnu.org; Thu, 13 Feb 2025 20:54:47 -0500 Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tikuj-0004NF-00 for qemu-devel@nongnu.org; Thu, 13 Feb 2025 20:54:46 -0500 Received: by mail-pl1-x629.google.com with SMTP id d9443c01a7336-220d398bea9so21093085ad.3 for ; Thu, 13 Feb 2025 17:54:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1739498083; x=1740102883; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NdqCgSxKf8rhz7PsW2KR1K7z/q/nMu6KnVnCzC9I9Hc=; b=kzPXP/K1+CdZlIC6hoGTRXeE7zW/j5t1zU+eACV7LmhUuMxkCgWSyvJdaqMumZfz7b IwRJy1JzJdJfBv5nbgj+ax2m0Cs+9RudAF/KSwvwE50HJ0+DBbsbI1jNsArmXISVZleF UXkF3gXHxCgRgvkSoyEhmXNdtdNEDMGHCc5i6PM1e+GCvaQ2LYleuAkvz9IE7kXVQdVU OIzMGTyr9LSd7vnpMuPZ8g184Ow/P//UwCpTcWuuoLCwk4E/hnnlfrG48Bh3U1cJQTru ITq1ZWYDp1UiCYZJWpj4Cy8UKUqupQ/JptsvinS7gHtwsZ1rfE5niqKHu1AOVJ3nYdPg m8Lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739498083; x=1740102883; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NdqCgSxKf8rhz7PsW2KR1K7z/q/nMu6KnVnCzC9I9Hc=; b=rLBPWY9QvwlrHx2yq18qeJNdVzMDUJkWtn/lzBxApuM61M1QP0c4il8rQbAZPUcdgR LgI4+OifungEzZvHDSEodXaHLzI7xd9wFWR6MHSP2H3YntGBQ4F2fSijW/CNzJkLbmlJ mR0dJ2oRVxtiVvIwS3vmd2RI1HOk0c/4H1ePloyfdt+Lj4qCRp3npDSqBXjXA6MLpB/l PipWWd/cryf9QYiuiHAUedrzV64Jo/D10x3KmzqQgz6bESRRTCxhHl3agpkMgU+nxe7F G1ake0ySzqQZBnhlE76jkYdvcA5/X4jMnki70qb40QbTuFlu//CmO+XLwffQMZAivtYV oYug== X-Gm-Message-State: AOJu0YyQh/JHZikghF51NsbkoJb/mqK2Qc52GHPfop6YJDsdiVxcpcQD sR12K6Mu2u3Iw65pyUuWUFbVZx8dop0BukD3XghP73StmN0yRvpd/9mW07q0y7BGM9JEeQ4GW6b AJUzpN2hq X-Gm-Gg: ASbGnctgH/2toUdpPf+5knM9+T5aRsf1/Z+O+DuK/umVA8o42vBe3DrD7Ctt9j7EMIl ryGaSj8telQDzUzUGsNHiAg8QivI6y5nvlCZ5egTYhWDRr0RvfmMMrBawTiJb5kKbo9r7vhhdT8 1ArsK4sUUbMmHl3JeYNwl/5jBmgZvgiCGe+ayrthDeC0Z1xdJOGhVSrjMbe4xGa/1T2rLNFsaQj ktfjEFSv7dKfk+6KIGxIkaMSEiUsPedJDsQD6BQm/YvlYKrw1e9FgwSxFH+zEgPS65MAXI8icza iLprOnqpYiv+aTQPLE9ill7g+3MnfFEc3+gfnZn6 X-Google-Smtp-Source: AGHT+IHV8wdPMGuQjFRK0boX5C7OVrhUUpz62b/tGTU0/evOQeUUbE0pzPKqderg0FdTnK91IJk64Q== X-Received: by 2002:a17:903:18e:b0:220:ea90:191e with SMTP id d9443c01a7336-220ea901a6emr29415115ad.4.1739498082595; Thu, 13 Feb 2025 17:54:42 -0800 (PST) Received: from Hyman-Dev-Euler.zelin.local ([154.64.226.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d545d67fsm18831515ad.143.2025.02.13.17.54.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Feb 2025 17:54:42 -0800 (PST) From: yong.huang@smartx.com To: qemu-devel Cc: Fabiano Rosas , Peter Xu , Hyman Huang , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [PATCH v4 1/4] guestperf: Support deferred migration for multifd Date: Fri, 14 Feb 2025 09:53:33 +0800 Message-Id: <8874e170f890ce0bc6f25cb0d9b9ae307ce2e070.1739497707.git.yong.huang@smartx.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::629; envelope-from=yong.huang@smartx.com; helo=mail-pl1-x629.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hyman Huang The way to enable multifd migration has been changed by commit, 82137e6c8c (migration: enforce multifd and postcopy preempt to be set before incoming), and guestperf has not made the necessary changes. If multifd migration had been enabled in the previous manner, the following error would have occurred: Multifd must be set before incoming starts Supporting deferred migration will fix it. Signed-off-by: Hyman Huang Reviewed-by: Daniel P. Berrangé --- tests/migration-stress/guestperf/engine.py | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/tests/migration-stress/guestperf/engine.py b/tests/migration-stress/guestperf/engine.py index 608d7270f6..4b15322e8d 100644 --- a/tests/migration-stress/guestperf/engine.py +++ b/tests/migration-stress/guestperf/engine.py @@ -106,7 +106,8 @@ def _migrate_progress(self, vm): info.get("dirty-limit-ring-full-time", 0), ) - def _migrate(self, hardware, scenario, src, dst, connect_uri): + def _migrate(self, hardware, scenario, src, + dst, connect_uri, defer_migrate): src_qemu_time = [] src_vcpu_time = [] src_pid = src.get_pid() @@ -220,6 +221,8 @@ def _migrate(self, hardware, scenario, src, dst, connect_uri): resp = src.cmd("migrate-set-parameters", vcpu_dirty_limit=scenario._vcpu_dirty_limit) + if defer_migrate: + resp = dst.cmd("migrate-incoming", uri=connect_uri) resp = src.cmd("migrate", uri=connect_uri) post_copy = False @@ -373,11 +376,14 @@ def _get_common_args(self, hardware, tunnelled=False): def _get_src_args(self, hardware): return self._get_common_args(hardware) - def _get_dst_args(self, hardware, uri): + def _get_dst_args(self, hardware, uri, defer_migrate): tunnelled = False if self._dst_host != "localhost": tunnelled = True argv = self._get_common_args(hardware, tunnelled) + + if defer_migrate: + return argv + ["-incoming", "defer"] return argv + ["-incoming", uri] @staticmethod @@ -424,6 +430,7 @@ def _get_timings(self, vm): def run(self, hardware, scenario, result_dir=os.getcwd()): abs_result_dir = os.path.join(result_dir, scenario._name) + defer_migrate = False if self._transport == "tcp": uri = "tcp:%s:9000" % self._dst_host @@ -439,6 +446,9 @@ def run(self, hardware, scenario, result_dir=os.getcwd()): except: pass + if scenario._multifd: + defer_migrate = True + if self._dst_host != "localhost": dstmonaddr = ("localhost", 9001) else: @@ -452,7 +462,7 @@ def run(self, hardware, scenario, result_dir=os.getcwd()): monitor_address=srcmonaddr) dst = QEMUMachine(self._binary, - args=self._get_dst_args(hardware, uri), + args=self._get_dst_args(hardware, uri, defer_migrate), wrapper=self._get_dst_wrapper(hardware), name="qemu-dst-%d" % os.getpid(), monitor_address=dstmonaddr) @@ -461,7 +471,8 @@ def run(self, hardware, scenario, result_dir=os.getcwd()): src.launch() dst.launch() - ret = self._migrate(hardware, scenario, src, dst, uri) + ret = self._migrate(hardware, scenario, src, + dst, uri, defer_migrate) progress_history = ret[0] qemu_timings = ret[1] vcpu_timings = ret[2] From patchwork Fri Feb 14 01:53:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yong Huang X-Patchwork-Id: 13974279 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 54669C021A0 for ; Fri, 14 Feb 2025 01:55:46 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tikup-0006bT-4U; Thu, 13 Feb 2025 20:54:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tikun-0006bH-2W for qemu-devel@nongnu.org; Thu, 13 Feb 2025 20:54:49 -0500 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tikul-0004NW-Am for qemu-devel@nongnu.org; Thu, 13 Feb 2025 20:54:48 -0500 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-220e989edb6so14517545ad.1 for ; Thu, 13 Feb 2025 17:54:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1739498085; x=1740102885; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=a9RC6MJMkMFddmaETe/J83cE8JOgsz6X2WCSfCIYCZQ=; b=PIAzLM4WLO0dG7s4KOPz8L6Qucw827qdZyYW9WlfNZ4jUyT7EzOyJ/Shuuo6m/k/8N vIFM5aac3J8OCnzMC5M7oMY56Bs079NZfL4gGV/eB6c4zemKttC4bdsYlp33gAm3JQAj ghUuICpspPnoFf7MEl6VUW319+KD1WcqfIYzT6oQYz2wI6YXRRahJpiGh24Lvb5gycT9 jYV5vWmqS1Zo+bvCLbxbZfTe/5BZkodc9J8U5qOkgInJ6ueDaKsrmuaLoIBa1GnurLXE ZVzr9UYFu40jRs66kI8zXx8S7RLi5Sf0pmKaguzzLsfojIeOjF0wxeNUy1vw3PDjbvMz flEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739498085; x=1740102885; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=a9RC6MJMkMFddmaETe/J83cE8JOgsz6X2WCSfCIYCZQ=; b=AG93knN83ASl2X8S2d5DgJMDhMhhYSJHb3VuoQ93dhN4g3S07wAMKeP9hKnTUSvjEg uw34GYVfzGRCb9HYQU8QQG2iUJ6PsDMQFDa5ja71OK1plX87CGcuF0mxRGJ8vOnFzOA3 Bi0VCzC9Cx4/L+1vyqNoVpZbzO7xuHpNV3vD3NvdXH1WZnIOt98Sn7PxAFUGBfGVG1+1 iPZtv1E3cPm7l0ODpnP8dzTDCxv/P3tgSH6zDfp5TQ75ggzY6GZV1fxlEwAtKyeOJJ6g LZOe0DqyfhYtWE/iTaNWrX+BDZNAx9CtP6NNzc69xCeZ1hhwPGm1Wk55HaL7Ftb7JBIT 8ocQ== X-Gm-Message-State: AOJu0Ywf72wQi1YC1sf3PmK/WVEtdzEFQnSS2LRPJI4L622OG4rvM4bj W1WFzO2UkEB2/aPRZmd2/bud4FyySlV7ckV36BPrzETu51Z//MRyGusQYj5HqlCmSd4PYLXkqmE BetQnPStF X-Gm-Gg: ASbGncvjNHwpQP+bJ5HCJPd+y68BrIn6gwe9YjXtNjqNworzLkl/DtHhJRb3OQf0zsA yPaOdy3xO8tABRzwbZaE/HgDIsANtGZv8Hgl/gLhxrSH/96fCMrDEOwwo5xDDjFNpqyoxKPfHR9 MFEnHN7ksxlwLLPbjLLeXasqp2bYs5TcMd5IMT/WYlazldPy8vnxE7fxvUnjG39il94udyFXxcw BehNS/Z9reYtuVP/kpH7gGNxC6PDoUrCmf3VSJE2E5/tdVntt3kLVqhzj/ArB8aXWoQh/Ev1oS1 F2E0pgNfB4oGr8/Z0M4KdZEZFLG0qBDDAtXu71Q2 X-Google-Smtp-Source: AGHT+IGDuncs+s7Lnxw3U/Ztn+70I3oPVHtrCX7o9uswgNFjSa8XyUvknA4tgckxkiAdU2cx1I09TA== X-Received: by 2002:a17:902:c952:b0:21f:85ee:f2df with SMTP id d9443c01a7336-220d1ed85famr97543505ad.15.1739498085351; Thu, 13 Feb 2025 17:54:45 -0800 (PST) Received: from Hyman-Dev-Euler.zelin.local ([154.64.226.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d545d67fsm18831515ad.143.2025.02.13.17.54.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Feb 2025 17:54:45 -0800 (PST) From: yong.huang@smartx.com To: qemu-devel Cc: Fabiano Rosas , Peter Xu , Hyman Huang , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [PATCH v4 2/4] guestperf: Nitpick the inconsistent parameters Date: Fri, 14 Feb 2025 09:53:34 +0800 Message-Id: X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=yong.huang@smartx.com; helo=mail-pl1-x62d.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hyman Huang Signed-off-by: Hyman Huang Reviewed-by: Fabiano Rosas Reviewed-by: Daniel P. Berrangé --- tests/migration-stress/guestperf/comparison.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/migration-stress/guestperf/comparison.py b/tests/migration-stress/guestperf/comparison.py index 42cc0372d1..40e9d2eb1d 100644 --- a/tests/migration-stress/guestperf/comparison.py +++ b/tests/migration-stress/guestperf/comparison.py @@ -127,7 +127,7 @@ def __init__(self, name, scenarios): # varying numbers of channels Comparison("compr-multifd", scenarios = [ Scenario("compr-multifd-channels-4", - multifd=True, multifd_channels=2), + multifd=True, multifd_channels=4), Scenario("compr-multifd-channels-8", multifd=True, multifd_channels=8), Scenario("compr-multifd-channels-32", From patchwork Fri Feb 14 01:53:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yong Huang X-Patchwork-Id: 13974280 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C65A6C021A0 for ; Fri, 14 Feb 2025 01:55:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tikur-0006bx-A5; Thu, 13 Feb 2025 20:54:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tikup-0006bg-OW for qemu-devel@nongnu.org; Thu, 13 Feb 2025 20:54:51 -0500 Received: from mail-pl1-x62f.google.com ([2607:f8b0:4864:20::62f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tikun-0004O6-T8 for qemu-devel@nongnu.org; Thu, 13 Feb 2025 20:54:51 -0500 Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-220d132f16dso22431955ad.0 for ; Thu, 13 Feb 2025 17:54:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1739498088; x=1740102888; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vjFJmABBgiPrIkJaTy2ViAXPV7FmihtNfTlnacdhu6U=; b=Tut34dnT1dWMMyPKpxR95eGL/GgIs1htKckf+MIJL3V6PQ80hq5Hxe3b0CWhAeYQJT 2OZMK8fk4rAZ374EcJ5dd948lZWLCdyhMb2BY32SiKGk9XVpcrRkRniHa7qnD5Ihf9Qp 1Ilg9QJ9t8PSO5SwwRPdO52Sz7NSrGHovqXr77IEmsYUgnRK3O133Di+/o08drK+Gkqd dgBokBNdQkuJ6BLXvWiEQ2iKVbIZ+LS9FBz/YA4kCAq5MAgy6HZv7SQ6FcA2GXH2KcR8 S0j0ilJk0IHF8z35ytyrrTj8j7hEHxiXbqTAvC84gvuquhKKfAkKzYloutd92Uk1x+7N mBmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739498088; x=1740102888; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vjFJmABBgiPrIkJaTy2ViAXPV7FmihtNfTlnacdhu6U=; b=vkQ0oQIBZluCZewMJSxXaUaIAPaNv2p9CHv3BxwiRtlkXSmQLqIWYPYKcJ2OyU9WGV nbpVSYyScfVYNuCkYDEe2FowazCBHscHp3crX+mwSdZ1L9HLSKqM7oCcLeUv/RdXGsjD 6s92o95Rqb6e4VnqoqXktzuCupTBbO2cm4oH4/2vYnXmxu9DEj7IqV/kIu4ERaIKlKSN IfYJ9qW1CrnSxTa0F7XoJibqwVralW6RT0H7ld62Ky7tXxtZN5uoi6yjqFz2Lk6olonc nACL+xu1rSNoviMXTkf/7pdP7kEx5JirW9y2fx00SW4LoEuQYeWt24SJON6vFNzWa3Kf S2vw== X-Gm-Message-State: AOJu0YxBW+o4PV6SxYKEYot7w/qpSQua5UwmFueGBrJlAo9yInnTcZbS THTtqdjEcZ9Fuv59u8aHaPW8PCAE3pqDw+H8Ygc2ic0ZAYGNTcx+YYLek+ygGpVfZnJVmvIPRXU 0IS08amzh X-Gm-Gg: ASbGncsWpdfFXJn7ajjRywpTuGI7dtkP+ZsYe72t2Ew/JXNilOqcg25pIOWvjSbaodD e7meDhQPjK9IP/9vp9g5ImEZmXA3vmk1qg3NB4O4VAcBGcHg6m9AplpRe8C2uqlNFAbjItg6r5l da5JESVDI2QSp3s7INKFCxDmlqefQZyXwa2o/ghbR5f719s6minhuSD7orOwzutQW2JyDCUUmq2 cqEPVPXcp6vs6sjJ4znH7HHjaSO4rOk2Cn1uQME+1WVRkgJieVHBt5C0Dg1GcUgzDiIoFdq5gll 0E5crkLOwtFead4LuLFUGhT4pyRL5cE6f+kagCAD X-Google-Smtp-Source: AGHT+IG2wg4XgCJDXA1ppMwa9yxXz0MOFZMZx4A+4qieefAamZxr2fX9TBkPR/y2nRQHwT+jQVWIPQ== X-Received: by 2002:a17:902:d2c9:b0:220:ee5e:6bb with SMTP id d9443c01a7336-220ee5e072fmr4595275ad.20.1739498088226; Thu, 13 Feb 2025 17:54:48 -0800 (PST) Received: from Hyman-Dev-Euler.zelin.local ([154.64.226.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d545d67fsm18831515ad.143.2025.02.13.17.54.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Feb 2025 17:54:47 -0800 (PST) From: yong.huang@smartx.com To: qemu-devel Cc: Fabiano Rosas , Peter Xu , Hyman Huang , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [PATCH v4 3/4] guestperf: Introduce multifd compression option Date: Fri, 14 Feb 2025 09:53:35 +0800 Message-Id: X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62f; envelope-from=yong.huang@smartx.com; helo=mail-pl1-x62f.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hyman Huang Guestperf tool does not cover the multifd compression option currently, it is worth supporting so that developers can analysis the migration performance with different compression algorithms. Multifd support 4 compression algorithms currently: zlib, zstd, qpl, uadk To request that multifd with the specified compression algorithm such as zlib: $ ./tests/migration-stress/guestperf.py \ --multifd --multifd-channels 4 --multifd-compression zlib \ --output output.json To run the entire standardized set of multifd compression comparisons, with unix migration: $ ./tests/migration-stress/guestperf-batch.py \ --dst-host localhost --transport unix \ --filter compr-multifd-compression* --output outputdir Signed-off-by: Hyman Huang Reviewed-by: Daniel P. Berrangé --- tests/migration-stress/guestperf/comparison.py | 13 +++++++++++++ tests/migration-stress/guestperf/engine.py | 14 ++++++++++++++ tests/migration-stress/guestperf/scenario.py | 7 +++++-- tests/migration-stress/guestperf/shell.py | 3 +++ 4 files changed, 35 insertions(+), 2 deletions(-) diff --git a/tests/migration-stress/guestperf/comparison.py b/tests/migration-stress/guestperf/comparison.py index 40e9d2eb1d..dee3ac25e4 100644 --- a/tests/migration-stress/guestperf/comparison.py +++ b/tests/migration-stress/guestperf/comparison.py @@ -158,4 +158,17 @@ def __init__(self, name, scenarios): Scenario("compr-dirty-limit-50MB", dirty_limit=True, vcpu_dirty_limit=50), ]), + + # Looking at effect of multifd with + # different compression algorithms + Comparison("compr-multifd-compression", scenarios = [ + Scenario("compr-multifd-compression-zlib", + multifd=True, multifd_channels=2, multifd_compression="zlib"), + Scenario("compr-multifd-compression-zstd", + multifd=True, multifd_channels=2, multifd_compression="zstd"), + Scenario("compr-multifd-compression-qpl", + multifd=True, multifd_channels=2, multifd_compression="qpl"), + Scenario("compr-multifd-compression-uadk", + multifd=True, multifd_channels=2, multifd_compression="uadk"), + ]), ] diff --git a/tests/migration-stress/guestperf/engine.py b/tests/migration-stress/guestperf/engine.py index 4b15322e8d..e11f6a8496 100644 --- a/tests/migration-stress/guestperf/engine.py +++ b/tests/migration-stress/guestperf/engine.py @@ -31,6 +31,8 @@ '..', '..', '..', 'python')) from qemu.machine import QEMUMachine +# multifd supported compression algorithms +MULTIFD_CMP_ALGS = ("zlib", "zstd", "qpl", "uadk") class Engine(object): @@ -191,6 +193,12 @@ def _migrate(self, hardware, scenario, src, scenario._compression_xbzrle_cache)) if scenario._multifd: + if (scenario._multifd_compression and + (scenario._multifd_compression not in MULTIFD_CMP_ALGS)): + raise Exception("unsupported multifd compression " + "algorithm: %s" % + scenario._multifd_compression) + resp = src.cmd("migrate-set-capabilities", capabilities = [ { "capability": "multifd", @@ -206,6 +214,12 @@ def _migrate(self, hardware, scenario, src, resp = dst.cmd("migrate-set-parameters", multifd_channels=scenario._multifd_channels) + if scenario._multifd_compression: + resp = src.cmd("migrate-set-parameters", + multifd_compression=scenario._multifd_compression) + resp = dst.cmd("migrate-set-parameters", + multifd_compression=scenario._multifd_compression) + if scenario._dirty_limit: if not hardware._dirty_ring_size: raise Exception("dirty ring size must be configured when " diff --git a/tests/migration-stress/guestperf/scenario.py b/tests/migration-stress/guestperf/scenario.py index 154c4f5d5f..4be7fafebf 100644 --- a/tests/migration-stress/guestperf/scenario.py +++ b/tests/migration-stress/guestperf/scenario.py @@ -30,7 +30,7 @@ def __init__(self, name, auto_converge=False, auto_converge_step=10, compression_mt=False, compression_mt_threads=1, compression_xbzrle=False, compression_xbzrle_cache=10, - multifd=False, multifd_channels=2, + multifd=False, multifd_channels=2, multifd_compression="", dirty_limit=False, x_vcpu_dirty_limit_period=500, vcpu_dirty_limit=1): @@ -61,6 +61,7 @@ def __init__(self, name, self._multifd = multifd self._multifd_channels = multifd_channels + self._multifd_compression = multifd_compression self._dirty_limit = dirty_limit self._x_vcpu_dirty_limit_period = x_vcpu_dirty_limit_period @@ -85,6 +86,7 @@ def serialize(self): "compression_xbzrle_cache": self._compression_xbzrle_cache, "multifd": self._multifd, "multifd_channels": self._multifd_channels, + "multifd_compression": self._multifd_compression, "dirty_limit": self._dirty_limit, "x_vcpu_dirty_limit_period": self._x_vcpu_dirty_limit_period, "vcpu_dirty_limit": self._vcpu_dirty_limit, @@ -109,4 +111,5 @@ def deserialize(cls, data): data["compression_xbzrle"], data["compression_xbzrle_cache"], data["multifd"], - data["multifd_channels"]) + data["multifd_channels"], + data["multifd_compression"]) diff --git a/tests/migration-stress/guestperf/shell.py b/tests/migration-stress/guestperf/shell.py index 046afeb84e..63bbe3226c 100644 --- a/tests/migration-stress/guestperf/shell.py +++ b/tests/migration-stress/guestperf/shell.py @@ -131,6 +131,8 @@ def __init__(self): action="store_true") parser.add_argument("--multifd-channels", dest="multifd_channels", default=2, type=int) + parser.add_argument("--multifd-compression", dest="multifd_compression", + default="") parser.add_argument("--dirty-limit", dest="dirty_limit", default=False, action="store_true") @@ -167,6 +169,7 @@ def get_scenario(self, args): multifd=args.multifd, multifd_channels=args.multifd_channels, + multifd_compression=args.multifd_compression, dirty_limit=args.dirty_limit, x_vcpu_dirty_limit_period=\ From patchwork Fri Feb 14 01:53:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong Huang X-Patchwork-Id: 13974281 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 08D25C021A4 for ; Fri, 14 Feb 2025 01:56:06 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tikuu-0006cL-9w; Thu, 13 Feb 2025 20:54:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tikus-0006cD-1r for qemu-devel@nongnu.org; Thu, 13 Feb 2025 20:54:54 -0500 Received: from mail-pl1-x62a.google.com ([2607:f8b0:4864:20::62a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tikuq-0004OI-Ca for qemu-devel@nongnu.org; Thu, 13 Feb 2025 20:54:53 -0500 Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-21f3c119fe6so36764755ad.0 for ; Thu, 13 Feb 2025 17:54:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1739498091; x=1740102891; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZVFXlsrJ6uqFDQzwPn+Dorbteo5aZgEE94Bb7QudTvo=; b=LMvgGEIoGABrSlGSr8VXsIrEch8UBxBpcVxt4Wbg8bjKTtCBTeM8IcT3bdC8mmaAiG FwpGVt3ZK1wreJTGgAgReyrItFnhtIjhZJRygHvhLx+K4XuNsV+/AlY2iKIPLE5moqBM IS9N1zaBy+qUpyLp5q3IL0MkholIXQJGZb+B7OqTyruHeI76Tz44/dwj8KsagQbbwmQY rSz8aLkrOsd6uOWjfeEJa7/cxUwOUug3j94u5xnmrkTHvXkyeqvucRuetHnD9+9JuXPz yoRFaAUPyWZTtga9MVmby4qXMgzW3D3FxENYceP3ER9hr6hE6jK82knTfDnvyw0s5bT/ T8xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739498091; x=1740102891; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZVFXlsrJ6uqFDQzwPn+Dorbteo5aZgEE94Bb7QudTvo=; b=XzjEJcAvJGe0Bjvf3/CQ8llRJf0f95IgApwmtNayV6cUmgF9amykg4F3se5Co6E8fF Ke5QpaNj9Ymjx4NKo9/1yglsZzoC6QcTLgbSpc0xZ0O+9w9/+vS7ybcI8uncKkkq/9aL sYZRrQTHVNG/CrrGxGU4gL31pGZihVJn5+wq4BLTNMzUZYtlUOL2nxlzKuhp3u/gi5u9 WnjNqlpYYTi2wj68DxoBczDJf6gMKVHAirqeV1iP+mBUWanrvgcymxfcAXJwgxTUx3VE +Z4pXNhGMnkZ+/Jqm3nLxrMytuxn5m2DQNNOUuNC6sxzKmzZ+YC2k69MEaJTSVMfdq6c qwrw== X-Gm-Message-State: AOJu0YyppK4WEsw76IrAMPFf+lUef7qbHQjisUqQ1nRC8OTrpU6vr7i2 0xi33m71UPNQyyKMCzRYhpUrgR30ob0XwMHTZmThfYBpq8ELxtHNBubr/r/rhuCrNPhwt5PPmkV QNey9Hak+ X-Gm-Gg: ASbGncsARCtrET1yf4kZjKLf9Ci8ZyuX7/p6Q7WW79omwDVfdYeI3/7ONw5lItu6TQE Gq+/bOzeg6z7QPA+ChNk5dc17vapXtQBzghrVIXvGjLnvC94ryWrXWh9neNbpXRiUVEQR59M3Hi 74l5ZRp7ArozdlggVtCsFflK7l65pkRJvbRSW1Lml/JKEcnFLiFX5Q/ZruHy42hJMHppLiqrsoz IIXMmCE32VaNeFzds8f1KTkjuFgtxGqZv++tdK9CyUABw181vcxTMPHGKSa7m/o97QJF1rmq6S4 Gzg5XAY5LpZxofT39i0pOK/sxZcS9vY2EsO6Hf87 X-Google-Smtp-Source: AGHT+IEMr8r1D2mBahIy+3Wi+rk8IKg9kZb7JYZS+mYYyKZBn8MzYAcHSk3duSVqre/rcDQLzIG0jg== X-Received: by 2002:a17:902:d511:b0:21a:8d70:3865 with SMTP id d9443c01a7336-220bbb21947mr165166025ad.14.1739498090721; Thu, 13 Feb 2025 17:54:50 -0800 (PST) Received: from Hyman-Dev-Euler.zelin.local ([154.64.226.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d545d67fsm18831515ad.143.2025.02.13.17.54.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Feb 2025 17:54:50 -0800 (PST) From: yong.huang@smartx.com To: qemu-devel Cc: Fabiano Rosas , Peter Xu , Hyman Huang Subject: [PATCH v4 4/4] guestperf: Add test result data into report Date: Fri, 14 Feb 2025 09:53:36 +0800 Message-Id: <6303400c2983ffe5647f07caa6406f00ceae4581.1739497707.git.yong.huang@smartx.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62a; envelope-from=yong.huang@smartx.com; helo=mail-pl1-x62a.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hyman Huang The migration result data is not included in the guestperf report information; include the result as a report entry so the developer can check whether the migration was successful after running guestperf. Signed-off-by: Hyman Huang --- tests/migration-stress/guestperf/engine.py | 10 ++++++++-- tests/migration-stress/guestperf/report.py | 20 ++++++++++++++++++++ 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/tests/migration-stress/guestperf/engine.py b/tests/migration-stress/guestperf/engine.py index e11f6a8496..d8462db765 100644 --- a/tests/migration-stress/guestperf/engine.py +++ b/tests/migration-stress/guestperf/engine.py @@ -24,7 +24,7 @@ import time from guestperf.progress import Progress, ProgressStats -from guestperf.report import Report +from guestperf.report import Report, ReportResult from guestperf.timings import TimingRecord, Timings sys.path.append(os.path.join(os.path.dirname(__file__), @@ -276,7 +276,11 @@ def _migrate(self, hardware, scenario, src, src_vcpu_time.extend(self._vcpu_timing(src_pid, src_threads)) sleep_secs -= 1 - return [progress_history, src_qemu_time, src_vcpu_time] + result = ReportResult() + if progress._status == "completed" and not paused: + result = ReportResult(True) + + return [progress_history, src_qemu_time, src_vcpu_time, result] if self._verbose and (loop % 20) == 0: print("Iter %d: remain %5dMB of %5dMB (total %5dMB @ %5dMb/sec)" % ( @@ -490,6 +494,7 @@ def run(self, hardware, scenario, result_dir=os.getcwd()): progress_history = ret[0] qemu_timings = ret[1] vcpu_timings = ret[2] + result = ret[3] if uri[0:5] == "unix:" and os.path.exists(uri[5:]): os.remove(uri[5:]) @@ -509,6 +514,7 @@ def run(self, hardware, scenario, result_dir=os.getcwd()): Timings(self._get_timings(src) + self._get_timings(dst)), Timings(qemu_timings), Timings(vcpu_timings), + result, self._binary, self._dst_host, self._kernel, self._initrd, self._transport, self._sleep) except Exception as e: diff --git a/tests/migration-stress/guestperf/report.py b/tests/migration-stress/guestperf/report.py index 1efd40c868..e135e01be6 100644 --- a/tests/migration-stress/guestperf/report.py +++ b/tests/migration-stress/guestperf/report.py @@ -24,6 +24,22 @@ from guestperf.progress import Progress from guestperf.timings import Timings +class ReportResult(object): + + def __init__(self, success=False): + self._success = success + + def serialize(self): + return { + "success": self._success, + } + + @classmethod + def deserialize(cls, data): + return cls( + data["success"]) + + class Report(object): def __init__(self, @@ -33,6 +49,7 @@ def __init__(self, guest_timings, qemu_timings, vcpu_timings, + result, binary, dst_host, kernel, @@ -46,6 +63,7 @@ def __init__(self, self._guest_timings = guest_timings self._qemu_timings = qemu_timings self._vcpu_timings = vcpu_timings + self._result = result self._binary = binary self._dst_host = dst_host self._kernel = kernel @@ -61,6 +79,7 @@ def serialize(self): "guest_timings": self._guest_timings.serialize(), "qemu_timings": self._qemu_timings.serialize(), "vcpu_timings": self._vcpu_timings.serialize(), + "result": self._result.serialize(), "binary": self._binary, "dst_host": self._dst_host, "kernel": self._kernel, @@ -78,6 +97,7 @@ def deserialize(cls, data): Timings.deserialize(data["guest_timings"]), Timings.deserialize(data["qemu_timings"]), Timings.deserialize(data["vcpu_timings"]), + ReportResult.deserialize(data["result"]), data["binary"], data["dst_host"], data["kernel"],