From patchwork Thu Nov 22 07:20:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 10693589 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0054515A7 for ; Thu, 22 Nov 2018 07:20:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C60632CBBB for ; Thu, 22 Nov 2018 07:20:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B39A72CBD5; Thu, 22 Nov 2018 07:20:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3C8092CBBB for ; Thu, 22 Nov 2018 07:20:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392544AbeKVR6p (ORCPT ); Thu, 22 Nov 2018 12:58:45 -0500 Received: from mail-pf1-f196.google.com ([209.85.210.196]:41780 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392528AbeKVR6p (ORCPT ); Thu, 22 Nov 2018 12:58:45 -0500 Received: by mail-pf1-f196.google.com with SMTP id b7so1494079pfi.8 for ; Wed, 21 Nov 2018 23:20:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=/kYC6HFF3VRChbJ95X4GaAaMwwZ2vS0aZhIwXhtjcKI=; b=Us9aeLgv+HcA6bqswIp0+RIAu10tYOaNgwneIebpdncfyB8UiP2TRtQt+afwUfHN68 rP0L+UaRBDpuTH/Jl3m2rt/o9fEnLJbKDXWUTijyCMKAO2cwZM8RJCj5lFK+2HAQx6Nd +vYML/P/lYDEJJjpclOJPoD7hP1UsafAWkWzd5gct00rqISIPJ8+g+qNQwNWWZ25GXFx i/oP9ErBEr/jYf50CT6VX/wQ32D1zy2RbmCqxU2RcuSa0nMhx+D7f8c8aeDpp4RuLV25 cg1+C32h2zOLD7W5uxb2nv7tjPn4VGz/I4+0Bxuj7lC49855wM0YtDZbXO3ZGjQicb3Y A8DQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=/kYC6HFF3VRChbJ95X4GaAaMwwZ2vS0aZhIwXhtjcKI=; b=Y1G6PRtEk2T5gDbi33irMjdVOMTFL8uJElUtBLo3lFO5tss9SwAzaUWIbsyQC77qMW pfMMw/uPCSrfp9zgYDSnwACT7R+4LDonMoV/y5M0aIQ0fcVxs7yENZr1PiC6LhnQIr09 6uj6nHHe3+gbDV3xzKe8+oh9i/f+tIW0yFnjwq+TpmhkPCMYUfQjBSjuD97Ymhf0UfKr RSDwULwKT5TqxV+th6/zzCnQMEIpDOL6y48At570uaINksXKyt9wOPbL6/cpe6GWav9U vXMMKxnGJiVgveCu8nPcoRnmP41o9KCPqsz1+IHe+As9pqz4R7BoKXE3dKnf9/W0B9ux zVhQ== X-Gm-Message-State: AA+aEWblESo/Ds7kR1cpsLvkh5NBcyWYPY0gAOkY3Sh5ZbpHHa4N2oJR qfVA+Q1LH4idmdM1F2OPW5U= X-Google-Smtp-Source: AFSGD/Vk7jZ9jBaDbZJK+FAIU3BIwlb10fxTalaprqkiN/GfuIAKvJv7w3St8WnWa9Vxhs6Xrswxrw== X-Received: by 2002:a62:4e83:: with SMTP id c125mr8397848pfb.101.1542871236636; Wed, 21 Nov 2018 23:20:36 -0800 (PST) Received: from localhost.localdomain ([203.205.141.36]) by smtp.gmail.com with ESMTPSA id 19sm52731312pfs.108.2018.11.21.23.20.32 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 21 Nov 2018 23:20:36 -0800 (PST) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org, dgilbert@redhat.com, peterx@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn, eblake@redhat.com, quintela@redhat.com, cota@braap.org, Xiao Guangrong Subject: [PATCH v3 0/5] migration: improve multithreads Date: Thu, 22 Nov 2018 15:20:23 +0800 Message-Id: <20181122072028.22819-1-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.5 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Xiao Guangrong Changelog in v3: Thanks to Emilio's comments and his example code, the changes in this version are: 1. move @requests from the shared data struct to each single thread 2. move completion ev from the shared data struct to each single thread 3. move bitmaps from the shared data struct to each single thread 4. limit the number of request that each thread need handle to 64, then use uint64_t instead of bitmap pointer. The performance is measured by using the benchmark we introduced in this pachset: ./tests/threaded-workqueue-bench -c 20 -m 16 -t N the data is as followings: The baseline of v2: Thread #, Throughput 1, 0.428024 4, 1.668876 8, 3.501940 12, 5.026403 16, 1.912374 20, 1.174771 24, 1.074085 28, 0.747920 32, 0.651409 36, 0.533240 40, 0.517421 44, 0.482153 48, 0.525176 52, 0.492677 56, 0.798679 60, 0.733868 64, 0.751396 After this patchset: Thread #, Throughput 1, 0.449192 4, 1.849271 8, 3.644339 12, 4.809391 16, 4.709095 20, 4.942153 24, 5.116967 28, 4.921542 32, 5.008816 36, 5.408070 40, 5.166064 44, 4.994953 48, 4.853351 52, 4.797540 56, 4.815153 60, 4.793704 64, 4.913544 To see more detailed compression by each step, please refer to https://ibb.co/hq7u5V Xiao Guangrong (5): bitops: introduce change_bit_atomic util: introduce threaded workqueue migration: use threaded workqueue for compression migration: use threaded workqueue for decompression tests: add threaded-workqueue-bench include/qemu/bitops.h | 13 + include/qemu/threaded-workqueue.h | 106 ++++++++ migration/ram.c | 530 ++++++++++++++------------------------ tests/Makefile.include | 5 +- tests/threaded-workqueue-bench.c | 255 ++++++++++++++++++ util/Makefile.objs | 1 + util/threaded-workqueue.c | 463 +++++++++++++++++++++++++++++++++ 7 files changed, 1029 insertions(+), 344 deletions(-) create mode 100644 include/qemu/threaded-workqueue.h create mode 100644 tests/threaded-workqueue-bench.c create mode 100644 util/threaded-workqueue.c