From patchwork Sun Apr 2 17:06:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Straub X-Patchwork-Id: 13197499 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 03083C761AF for ; Sun, 2 Apr 2023 17:07:28 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pj1A5-0002Su-EO; Sun, 02 Apr 2023 13:06:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pj1A3-0002Si-F9 for qemu-devel@nongnu.org; Sun, 02 Apr 2023 13:06:35 -0400 Received: from mout.web.de ([212.227.15.14]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pj1A1-0004Y2-LL for qemu-devel@nongnu.org; Sun, 02 Apr 2023 13:06:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=s29768273; t=1680455188; i=lukasstraub2@web.de; bh=/apVL7JItbe3d/TDj/ythtFw7lFlzFcl2W9T16lnNQc=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject; b=YuRsxJspDfpz/WJ9aSJmuayZeJNJ3+UvU9BEDJ1Rt9pOdl4QVUIT4vwRLl/eVTazj INA4QJ7+XySQO5Iy/pfToXj4CyQQRrMWDu8KCyO0s42UgPWDfu0plhjpnw49rcwc3s 0jaVuJMqFyzXuY9Ec8K7Z8BifPDecRVUnx7/VMzNP58yHVtZ/iOenkUuDl13B+7IF/ qh8sghbEnp8chCquw5rdFDtOJGLKk/r9KAzuKOM9Gfj1f5obTz/qCWpgrLXtbswW// DPiEwtfC+TS7dq053XLWI7a71d1wDBOrG+DGiUMto63pRL5aJaX/dNAY+4qTo8z/mJ PC6NuuPYTaDrA== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from gecko.fritz.box ([82.207.254.111]) by smtp.web.de (mrweb006 [213.165.67.108]) with ESMTPSA (Nemesis) id 1Mav2l-1qGA3T2KcN-00bxzT; Sun, 02 Apr 2023 19:06:28 +0200 Date: Sun, 2 Apr 2023 17:06:18 +0000 From: Lukas Straub To: qemu-devel Cc: "Dr. David Alan Gilbert" , Juan Quintela , Peter Xu , Thomas Huth , Laurent Vivier , Paolo Bonzini Subject: [PATCH 1/2] qtest/migration-test.c: Add test with compress enabled Message-ID: <20230402170618.6b0edb87@gecko.fritz.box> MIME-Version: 1.0 X-Provags-ID: V03:K1:2RsVYc+uy2IkJvPiACqaaJ8yZjBVOZIdedoDoZ3tXtQPVpGWMxG VvYPOxdXFgvCST/JiSYJMFNps3ofoySxAjVz3xIASt3X2vCjIN9Gw7c1HMWf/udXpX7z066 LdwD3S9EVTR8jlZ5JqeaccnH9HTKDCvaEDKRIKiQoHGhelQUytqkbndC+N9XvxyZvsbtMJY asvV4PmxLrYGR1qBWDLdQ== UI-OutboundReport: notjunk:1;M01:P0:1/CuZu20+m4=;z5uogJhkulcLHA7aIvFvqQ9IWlz 5hDj1mxtBB1yu9FnFMA0EKisd8oTy9mQRHqk7+SXiBoJtwL8/ey23ylcLIoEcDW11dEVnwkzK xo7OHzZ+BO0/9Zlg8klY53EoIdQCPkU5LsbTyxXybu6zsXKpsgw54+m+f5c3byny1xvA6nOok JZ70EjltyZuGniv2O1LVT3wESjsIXVSDOoIEaGxi8gS4qye9Fb75KcdRwvJtKWZLzAlFXnekE pNo4n0OGNLZJK22ZW8IHUfrSGJRRDPv2C7d2N/bSOzoQW1Qq83UJw0H9b24ZSu+PLhTNAgHcI gKOuUUGbKNp/Kh7I2DG82uaKJN8mbKpE2SqJdK4bzxlIV96SyE5XRrdAA8Xe0L9ZrjQ/BcABh 4r7ucT6v8/8j7onlp1M2lqtH4XwDR9NRcA7qRLBDe2WE9eb5vKF1nj7/lOx/fKi/CK9gfUWXz +uZpkLpLZ68hRASl1oj8ACp2uy9pGhXdGbOhYACpQCti+CDK/3pXyj44HdCBX1+m+cNh6ZBna 19jiJAj4aoM4yPY7IQ2w9J6ApnpiFEIfw4BNrKT4m3ZE5NdHADCqqValnaCB6XfbLWaJki42t R4wZiUEEid6NPVCRJdn2AsoBux2r27nVzMAfly35BItvInYUPQcCK7yla1KPpI1iUYEik5XDS 34UiDPpFqsssIm4GPXuMcDc5pF5J22sM/1MbEPmTxZoEXbD9VR7cVu76eoORrYXvpP1rz7ZQI T0TnPYSB7Q+2I+TpvKjT/EHbjmKqDoHEhO7+lNrUmY/d1zkcVcb88YocQolMGdNOE+lbRZUHd iR5NAHuQKxKvmW3MFyoUro9vZe+S+Wj4DZQrB73/PuhbhJV6H2aXc6/EaOLD2FcIZonurKcHc AoGU++kwDahaV1ini/Ux/JDQJnrMXuTzDFSEh/2BjzGYc800HpIDe9oONQLEzRAl7O8wNGHGZ o62Cjg== Received-SPF: pass client-ip=212.227.15.14; envelope-from=lukasstraub2@web.de; helo=mout.web.de X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org There has never been a test for migration with compress enabled. Add a suitable test, testing with compress-wait-thread = false too. iterations = 2 is intentional, so it also tests that no invalid thread state is left over from the previous iteration. Signed-off-by: Lukas Straub --- tests/qtest/migration-test.c | 67 ++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 3b615b0da9..dbcab2e8ae 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -406,6 +406,41 @@ static void migrate_set_parameter_str(QTestState *who, const char *parameter, migrate_check_parameter_str(who, parameter, value); } +static long long migrate_get_parameter_bool(QTestState *who, + const char *parameter) +{ + QDict *rsp; + int result; + + rsp = wait_command(who, "{ 'execute': 'query-migrate-parameters' }"); + result = qdict_get_bool(rsp, parameter); + qobject_unref(rsp); + return !!result; +} + +static void migrate_check_parameter_bool(QTestState *who, const char *parameter, + int value) +{ + int result; + + result = migrate_get_parameter_bool(who, parameter); + g_assert_cmpint(result, ==, value); +} + +static void migrate_set_parameter_bool(QTestState *who, const char *parameter, + int value) +{ + QDict *rsp; + + rsp = qtest_qmp(who, + "{ 'execute': 'migrate-set-parameters'," + "'arguments': { %s: %i } }", + parameter, value); + g_assert(qdict_haskey(rsp, "return")); + qobject_unref(rsp); + migrate_check_parameter_bool(who, parameter, value); +} + static void migrate_ensure_non_converge(QTestState *who) { /* Can't converge with 1ms downtime + 3 mbs bandwidth limit */ @@ -1524,6 +1559,36 @@ static void test_precopy_unix_xbzrle(void) test_precopy_common(&args); } +static void * +test_migrate_compress_start(QTestState *from, + QTestState *to) +{ + migrate_set_parameter_int(from, "compress-level", 9); + migrate_set_parameter_int(from, "compress-threads", 1); + migrate_set_parameter_bool(from, "compress-wait-thread", false); + migrate_set_parameter_int(to, "decompress-threads", 1); + + migrate_set_capability(from, "compress", true); + migrate_set_capability(to, "compress", true); + + return NULL; +} + +static void test_precopy_unix_compress(void) +{ + g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs); + MigrateCommon args = { + .connect_uri = uri, + .listen_uri = uri, + + .start_hook = test_migrate_compress_start, + + .iterations = 2, + }; + + test_precopy_common(&args); +} + static void test_precopy_tcp_plain(void) { MigrateCommon args = { @@ -2515,6 +2580,8 @@ int main(int argc, char **argv) qtest_add_func("/migration/bad_dest", test_baddest); qtest_add_func("/migration/precopy/unix/plain", test_precopy_unix_plain); qtest_add_func("/migration/precopy/unix/xbzrle", test_precopy_unix_xbzrle); + qtest_add_func("/migration/precopy/unix/compress", + test_precopy_unix_compress); #ifdef CONFIG_GNUTLS qtest_add_func("/migration/precopy/unix/tls/psk", test_precopy_unix_tls_psk); From patchwork Sun Apr 2 17:06:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Straub X-Patchwork-Id: 13197498 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0304BC7619A for ; Sun, 2 Apr 2023 17:07:28 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pj1AC-0002TZ-3v; Sun, 02 Apr 2023 13:06:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pj1A8-0002TE-QQ for qemu-devel@nongnu.org; Sun, 02 Apr 2023 13:06:41 -0400 Received: from mout.web.de ([217.72.192.78]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pj1A7-0004YJ-5l for qemu-devel@nongnu.org; Sun, 02 Apr 2023 13:06:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=s29768273; t=1680455194; i=lukasstraub2@web.de; bh=Y9sy+Zvc8l399K1gf8uq6kzHYrDDTQ6yMy1kVT/nRNg=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject; b=Db2+7nI/qH/Oi7t2IHwtvSK9TJj7zCPZZbqyz6Sz60dlp3N4zXQuaw38jU1NU6826 vCjGhHndEY0FSsgMNuNCmVVHdtsN3Q22AjCPVycc6SRxsEGyKao0X+iDiznuswBaC2 Qqck51UR/WqgqG1O2aHiBVr7KDagWYOvXcm0tGsY5yf3PZXcfHuAR0oeUa6r+2WY7A wpI/NeVGV0z3CSuQiFw43rwI1/AIC508ogdXerGaAx+U9SghkVEG5IZiiyiFpjfx72 9OZzyoSqTurfc7QobDNoeYWtiqvR8a7pOOOdmtcMb5w3AdkmUeq63ZIJkqvvaJuML6 oYOMR8EQCSAQw== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from gecko.fritz.box ([82.207.254.111]) by smtp.web.de (mrweb105 [213.165.67.124]) with ESMTPSA (Nemesis) id 1M1rP8-1pgpIf40D2-002eGF; Sun, 02 Apr 2023 19:06:34 +0200 Date: Sun, 2 Apr 2023 17:06:32 +0000 From: Lukas Straub To: qemu-devel Cc: "Dr. David Alan Gilbert" , Juan Quintela , Peter Xu , Thomas Huth , Laurent Vivier , Paolo Bonzini Subject: [PATCH 2/2] migration/ram.c: Fix migration with compress enabled Message-ID: <20230402170632.44f2a749@gecko.fritz.box> MIME-Version: 1.0 X-Provags-ID: V03:K1:wwxaY+2X4tH0Z7vDND6u7hxb4O8Ww3qpnzurseRgZqnvKtf+oPm FTaxK1dn3WREffa3fpkNP5oV1KjUS0UbbCB8YYPUQ7U0roNZ27A1xso4MFoxTA3Q4G420S2 ogNgwdglgjGg8NvfpfCrq+vt3yAr2jUyi9mWZ26n2elh60vlHwq0tzUhcwhiqWGF2v4Mh5I rD32e2fFGMAHCZIFTdMww== UI-OutboundReport: notjunk:1;M01:P0:Mez4drqg3Q4=;tn/ItfBGJAoYH7fQRCHBwf5MZFd n8SMyiZ2V0aY3ldixIoHMcaGwxLo8cw1T2qCyb1y1yp3BqgT1Pet1q+IAUnpocGcDluh8h8l5 6IOIdR8lH0cpEkLNEQPEnL6PStYSgJJf3LJpZuX807jXgq7fkd8VQiWP7/gyevXf4dldSqkrc vkspB7QltLwf/nMiD83a3t6FC6hzUSJP1yeZ8FClueUhEVuAw1vaAqGMaieRifCBp2zVv19NH 6D9TOHt7RgKSAk6LhUMiDTnrmt73BRUw3gMzY4pRXReK2VsIqbFvpBKfDjjIL54xTIzcgc0eu kloTwnWS4vxh2WIy4J7xEOv+a9FHK2eBpMKvBfDS4X1ZV90r2mlgSeh4hyILQAK4N07YY9mBF yuGVctSygsrk9fMMFvBSd/dd91XBFbfvB4yNyg+qvyuQceBRUdzf1fI84LJnbWeUKZ5MdxrWK 4A5shysPuLvyyYuRnRj59YNUuEmGsQ7iemLqYswmwl4OrtiV6Clm8QnoiDhuh7v6043E5X+sT O2iB0YxJePeTjaNZNuExTbbju4ZSg8E06PGNiNjDrKQpdjJQAEsfiA75/e+Uv6rBO0KOayybo VAkcv89eQhvMj90rIIK+M0vPgZ7P+SJihYBC8dAj3DXt2/KBA7M2/cbOuWNaJen7vNYaKbm4Z 0fljgdOqldN9WNbxHTEiqkgeNnKvGBoIaYGDE3Ib79Xj4/mXI+9x4TITz1+G7WUYDLsv6g4xo 3MW5K47lYBotrO4HYMjXw6hoREe/uu/5YyfrED2oILb1q9c2irXhdHGE7TyG9Jva0L1d1WkuY qIL1KSLv5Qoclkt/f0lTRGWyWRszs7ZJp/72q+0B9OL1IQzs5fdyqQg4AIHJtz0aC674IHDh/ p7E3lATyvmZu/BqvT5zzMtQIekDcf/fcrRItuKlZhQ1JMgJ4cQdG/7pn3CcnGYOSK8xc99gnf DASUm/Vj081Q7y90deA2N2jxxiY= Received-SPF: pass client-ip=217.72.192.78; envelope-from=lukasstraub2@web.de; helo=mout.web.de X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Since ec6f3ab9, migration with compress enabled was broken, because the compress threads use a dummy QEMUFile which just acts as a buffer and that commit accidentally changed it to use the outgoing migration channel instead. Fix this by using the dummy file again in the compress threads. Signed-off-by: Lukas Straub Reviewed-by: Juan Quintela --- migration/ram.c | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 96e8a19a58..9d1817ab7b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -688,12 +688,11 @@ exit: * @offset: offset inside the block for the page * in the lower bits, it contains flags */ -static size_t save_page_header(PageSearchStatus *pss, RAMBlock *block, - ram_addr_t offset) +static size_t save_page_header(PageSearchStatus *pss, QEMUFile *f, + RAMBlock *block, ram_addr_t offset) { size_t size, len; bool same_block = (block == pss->last_sent_block); - QEMUFile *f = pss->pss_channel; if (same_block) { offset |= RAM_SAVE_FLAG_CONTINUE; @@ -867,7 +866,7 @@ static int save_xbzrle_page(RAMState *rs, PageSearchStatus *pss, } /* Send XBZRLE based compressed page */ - bytes_xbzrle = save_page_header(pss, block, + bytes_xbzrle = save_page_header(pss, pss->pss_channel, block, offset | RAM_SAVE_FLAG_XBZRLE); qemu_put_byte(file, ENCODING_FLAG_XBZRLE); qemu_put_be16(file, encoded_len); @@ -1302,15 +1301,14 @@ void ram_release_page(const char *rbname, uint64_t offset) * @block: block that contains the page we want to send * @offset: offset inside the block for the page */ -static int save_zero_page_to_file(PageSearchStatus *pss, +static int save_zero_page_to_file(PageSearchStatus *pss, QEMUFile *file, RAMBlock *block, ram_addr_t offset) { uint8_t *p = block->host + offset; - QEMUFile *file = pss->pss_channel; int len = 0; if (buffer_is_zero(p, TARGET_PAGE_SIZE)) { - len += save_page_header(pss, block, offset | RAM_SAVE_FLAG_ZERO); + len += save_page_header(pss, file, block, offset | RAM_SAVE_FLAG_ZERO); qemu_put_byte(file, 0); len += 1; ram_release_page(block->idstr, offset); @@ -1327,10 +1325,10 @@ static int save_zero_page_to_file(PageSearchStatus *pss, * @block: block that contains the page we want to send * @offset: offset inside the block for the page */ -static int save_zero_page(PageSearchStatus *pss, RAMBlock *block, +static int save_zero_page(PageSearchStatus *pss, QEMUFile *f, RAMBlock *block, ram_addr_t offset) { - int len = save_zero_page_to_file(pss, block, offset); + int len = save_zero_page_to_file(pss, f, block, offset); if (len) { stat64_add(&ram_atomic_counters.duplicate, 1); @@ -1394,7 +1392,7 @@ static int save_normal_page(PageSearchStatus *pss, RAMBlock *block, { QEMUFile *file = pss->pss_channel; - ram_transferred_add(save_page_header(pss, block, + ram_transferred_add(save_page_header(pss, pss->pss_channel, block, offset | RAM_SAVE_FLAG_PAGE)); if (async) { qemu_put_buffer_async(file, buf, TARGET_PAGE_SIZE, @@ -1473,11 +1471,11 @@ static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, uint8_t *p = block->host + offset; int ret; - if (save_zero_page_to_file(pss, block, offset)) { + if (save_zero_page_to_file(pss, f, block, offset)) { return true; } - save_page_header(pss, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); + save_page_header(pss, f, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); /* * copy it to a internal buffer to avoid it being modified by VM @@ -2355,7 +2353,7 @@ static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss) return 1; } - res = save_zero_page(pss, block, offset); + res = save_zero_page(pss, pss->pss_channel, block, offset); if (res > 0) { /* Must let xbzrle know, otherwise a previous (now 0'd) cached * page would be stale