From patchwork Wed Oct 9 23:45:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13829384 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0E9ADCED628 for ; Wed, 9 Oct 2024 23:50:12 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sygPA-0002zw-VJ; Wed, 09 Oct 2024 19:47:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sygP6-0002yw-Uz for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:41 -0400 Received: from mail-qv1-xf2b.google.com ([2607:f8b0:4864:20::f2b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sygP4-0006ba-VT for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:40 -0400 Received: by mail-qv1-xf2b.google.com with SMTP id 6a1803df08f44-6cbceb48613so2465476d6.2 for ; Wed, 09 Oct 2024 16:47:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1728517654; x=1729122454; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6OLRasB0CC5bovyz2wxcoxtO2uT5qwr3ejcqOwE8FJE=; b=Gmz0Zcm10/Gd6Ei3dwkFi8UWXBfJRM0NEUa6y9QHs67FC7qLiLHZ+Mz0Sxqal+0+Qs EhQxh0K4AwqdeaxmFannjpNa0sjphZaD7JIhs7NGAV9UT7trIhG9RuTQN/JJsx+39vZR wmzgW9/aH5BSVFjeSYzUF65oInaPQNogqve+XcUSmQfwJrVOMse/opz1pre2vQPhuUYn B5kZB6jOfMeFU9ifJlw4jZQd2RmjHaMPKKfRd+2XNbp/Ry3RR51Na3xmGzu8J2lrz4L5 iaS1jkrg2wlcdkgM+ebtWdNQp1yzNFjV0j2Qm3foLEgzYGRXN8aM0ejPE5XjYH9V6nx+ c/BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728517654; x=1729122454; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6OLRasB0CC5bovyz2wxcoxtO2uT5qwr3ejcqOwE8FJE=; b=jgYQLJNIr9NiV74t200tZtHKYymY+KmnsC2qFNKNPhEq+MULZWjY3AFOAD19qrKvwR UU4aKrFsOLGF1CPMCU9ep3PNsaTHL23HHpG59qJZ6bcTaUd34yLQjQjmXPJNEbn5zatU RaReUEZj67Mdu0Lp5boDiDdNfWSkjNuh4BCOUkL27tE6UbaOK/Be7SOTBvyEKmDihNOA FS0QID/a2jaoVHJWVLna0lJ8U6HGaz4aOrc/VkmkUl3CSuc5OXfB94Jpxxuhbz4IMTZD CLT6RZqtcfTxYSUtPsvv0f1DmxmYgAcgwN5hu/j7L/+yMoZ9S3Br7FLJtoZa9Rrkaz69 SY+Q== X-Forwarded-Encrypted: i=1; AJvYcCXNi33B8pjC5H9A/pSg+fCd6c2qNZf0HM7bbOz8ENpj91p/6cmXzyQmE0o24ekJHzaxCVjRpEtTP+5k@nongnu.org X-Gm-Message-State: AOJu0YxlVwNkf/+n3L87UzUup8yUBHrcVWMFfbACRUqog5i/6yAoHzOV atNS2H3ArtT5EwqKSuKFPTK8DArTPs7XcRZcrlIRKF7Z0Z+z7V5DGwyY4a+jIws= X-Google-Smtp-Source: AGHT+IE97e9pilUf2xOw+kfKMNGuAbQW9PAJKWk1g1U2+F99rdMyWaBk+y6DKVGmyRbTzDLO16dSPA== X-Received: by 2002:a05:6214:3b87:b0:6cb:a355:10a9 with SMTP id 6a1803df08f44-6cbe4b05ff6mr25716966d6.34.1728517654104; Wed, 09 Oct 2024 16:47:34 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([2605:a7c0:0:301::44]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbe85d856fsm264386d6.72.2024.10.09.16.47.32 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 09 Oct 2024 16:47:33 -0700 (PDT) From: Yichen Wang To: "Dr. David Alan Gilbert" , Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?Phil?= =?utf-8?q?ippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" Subject: [PATCH v6 01/12] meson: Introduce new instruction set enqcmd to the build system. Date: Wed, 9 Oct 2024 16:45:59 -0700 Message-Id: <20241009234610.27039-2-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20241009234610.27039-1-yichen.wang@bytedance.com> References: <20241009234610.27039-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::f2b; envelope-from=yichen.wang@bytedance.com; helo=mail-qv1-xf2b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang Enable instruction set enqcmd in build. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- meson.build | 14 ++++++++++++++ meson_options.txt | 2 ++ scripts/meson-buildoptions.sh | 3 +++ 3 files changed, 19 insertions(+) diff --git a/meson.build b/meson.build index 4ea1984fc5..a7b545cd88 100644 --- a/meson.build +++ b/meson.build @@ -2954,6 +2954,20 @@ config_host_data.set('CONFIG_AVX512BW_OPT', get_option('avx512bw') \ int main(int argc, char *argv[]) { return bar(argv[0]); } '''), error_message: 'AVX512BW not available').allowed()) +config_host_data.set('CONFIG_DSA_OPT', get_option('enqcmd') \ + .require(have_cpuid_h, error_message: 'cpuid.h not available, cannot enable ENQCMD') \ + .require(cc.links(''' + #include + #include + #include + static int __attribute__((target("enqcmd"))) bar(void *a) { + uint64_t dst[8] = { 0 }; + uint64_t src[8] = { 0 }; + return _enqcmd(dst, src); + } + int main(int argc, char *argv[]) { return bar(argv[argc - 1]); } + '''), error_message: 'ENQCMD not available').allowed()) + # For both AArch64 and AArch32, detect if builtins are available. config_host_data.set('CONFIG_ARM_AES_BUILTIN', cc.compiles(''' #include diff --git a/meson_options.txt b/meson_options.txt index 5ee1d95c9c..2c02c654f0 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -123,6 +123,8 @@ option('avx2', type: 'feature', value: 'auto', description: 'AVX2 optimizations') option('avx512bw', type: 'feature', value: 'auto', description: 'AVX512BW optimizations') +option('enqcmd', type: 'feature', value: 'disabled', + description: 'ENQCMD optimizations') option('keyring', type: 'feature', value: 'auto', description: 'Linux keyring support') option('libkeyutils', type: 'feature', value: 'auto', diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index 3bee1c56df..f6c7f1251e 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -96,6 +96,7 @@ meson_options_help() { printf "%s\n" ' auth-pam PAM access control' printf "%s\n" ' avx2 AVX2 optimizations' printf "%s\n" ' avx512bw AVX512BW optimizations' + printf "%s\n" ' enqcmd ENQCMD optimizations' printf "%s\n" ' blkio libblkio block device driver' printf "%s\n" ' bochs bochs image format support' printf "%s\n" ' bpf eBPF support' @@ -241,6 +242,8 @@ _meson_option_parse() { --disable-avx2) printf "%s" -Davx2=disabled ;; --enable-avx512bw) printf "%s" -Davx512bw=enabled ;; --disable-avx512bw) printf "%s" -Davx512bw=disabled ;; + --enable-enqcmd) printf "%s" -Denqcmd=enabled ;; + --disable-enqcmd) printf "%s" -Denqcmd=disabled ;; --enable-gcov) printf "%s" -Db_coverage=true ;; --disable-gcov) printf "%s" -Db_coverage=false ;; --enable-lto) printf "%s" -Db_lto=true ;; From patchwork Wed Oct 9 23:46:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13829375 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A18C9CED628 for ; Wed, 9 Oct 2024 23:48:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sygP8-0002yy-G2; Wed, 09 Oct 2024 19:47:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sygP5-0002yb-Ko for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:39 -0400 Received: from mail-qv1-xf29.google.com ([2607:f8b0:4864:20::f29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sygP3-0006bo-Qs for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:39 -0400 Received: by mail-qv1-xf29.google.com with SMTP id 6a1803df08f44-6cbce9e4598so2573796d6.2 for ; Wed, 09 Oct 2024 16:47:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1728517657; x=1729122457; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FOMeFKnXZZxuQzQ4Ug4ts6vhOgDMsElUKC3GROOqZJI=; b=CUb8RL9rSL5j/g3WIgUEd4+TIhwzo23pz310arH9EXBc2numM4MbY+baBgxRMJXSYS EToQVB4YaXP8sSXf2FoKcrDq91WnjdWC2qbnNkfngcoe6/3VgQ7g5MJPTBkHdfCE21ch LmKVleOB+Jg5h02+7FwfLyRYo7NK2+M17CTf0EhhFuBni4xILvGQGsJqY1uq/MsAjFJA emPFEHmsjWM1pae31MJfNONhAIyc2rnCe6KmoaxwIkScXxjca7SPs23kcDUS5IIsBkl7 bzviXytdyIdm8tAxAPyrlYjKUA0BR4RYQkdsS4ctCeXauKu29aEatkPciM/tngu5weUz Qo9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728517657; x=1729122457; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FOMeFKnXZZxuQzQ4Ug4ts6vhOgDMsElUKC3GROOqZJI=; b=m3cij7t4YwwX5Ic5WpTEcJaLcwpgNsUh3v4kGe/7DbYa97SdEkPOOD1oZ3FyWqmeB1 KUv7AuCLGgxFAc7e1YQPscek1mEUb8/dHDP/V/CKOaA0pFisn6OOkPkRoRJqPjDMy78+ o04UMAD7Fs1nXIcImDFzW1A8pZHH9JdsA7pMueae5WaSY4X5WwjAoeuLQrmW5los1csa gx18DTldEuPDb7YKUuffcDPaEaI+V5EUWzL6GmLPMZHdoHFMt91bO07dPE8RvhxEYkNo AZy2kaaN8SejxMeFIwOpFNFHlh1CRmBlnov2yWdXDbXFot13XifjaxVumr6432aoLO76 VKmw== X-Forwarded-Encrypted: i=1; AJvYcCUCxLXZ+NwswUacyPQDo3ON8fOXRIcEe2wk7rwfsSI0KWaeA7tmZ1GPMfRZ37X8asB6InspERAHosoQ@nongnu.org X-Gm-Message-State: AOJu0YyOQXYL14En/qz2I2Rafa93bg/ALnjMaiKFebuv3NN/rbv2BE7r qnCti2qrN3g6Y7pfeGSPGuZDl4Z767YFkEoyBSIEjsAOFIjvSC1OPdIxn536fyY= X-Google-Smtp-Source: AGHT+IH7EmEF2zU6XQj9WxH8lIjFWc3CL249lzDa3fXLfwEVPokRkKk1cKlM3Z3iStjdfE0ps4c3Ng== X-Received: by 2002:a05:6214:490a:b0:6cb:807b:eaec with SMTP id 6a1803df08f44-6cbc954d8e0mr73875046d6.21.1728517656667; Wed, 09 Oct 2024 16:47:36 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([2605:a7c0:0:301::44]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbe85d856fsm264386d6.72.2024.10.09.16.47.34 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 09 Oct 2024 16:47:36 -0700 (PDT) From: Yichen Wang To: "Dr. David Alan Gilbert" , Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?Phil?= =?utf-8?q?ippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" Subject: [PATCH v6 02/12] util/dsa: Add idxd into linux header copy list. Date: Wed, 9 Oct 2024 16:46:00 -0700 Message-Id: <20241009234610.27039-3-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20241009234610.27039-1-yichen.wang@bytedance.com> References: <20241009234610.27039-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::f29; envelope-from=yichen.wang@bytedance.com; helo=mail-qv1-xf29.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Yichen Wang --- scripts/update-linux-headers.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/update-linux-headers.sh b/scripts/update-linux-headers.sh index c34ac6454e..5aba95d9cb 100755 --- a/scripts/update-linux-headers.sh +++ b/scripts/update-linux-headers.sh @@ -193,7 +193,7 @@ rm -rf "$output/linux-headers/linux" mkdir -p "$output/linux-headers/linux" for header in const.h stddef.h kvm.h vfio.h vfio_ccw.h vfio_zdev.h vhost.h \ psci.h psp-sev.h userfaultfd.h memfd.h mman.h nvme_ioctl.h \ - vduse.h iommufd.h bits.h; do + vduse.h iommufd.h bits.h idxd.h; do cp "$hdrdir/include/linux/$header" "$output/linux-headers/linux" done From patchwork Wed Oct 9 23:46:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13829377 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3298CCED628 for ; Wed, 9 Oct 2024 23:48:37 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sygPG-00030I-Hv; Wed, 09 Oct 2024 19:47:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sygP8-0002ze-L4 for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:42 -0400 Received: from mail-qv1-xf2f.google.com ([2607:f8b0:4864:20::f2f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sygP6-0006cD-EV for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:42 -0400 Received: by mail-qv1-xf2f.google.com with SMTP id 6a1803df08f44-6cbc28f8e1bso2741256d6.0 for ; Wed, 09 Oct 2024 16:47:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1728517659; x=1729122459; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nOKR97XKPQsX0XTIJnmVl1Wv6IjTAPxbl7mcNCJ+f7A=; b=PKUpChBhmcXtY2nIOmOA3Q+aC1ClhLD8AZPEmP6Q7+UhKT54/yEaa8s3aSpCK+/iH/ Der/DEqunNJugN1IDQwGdTRhpcsndL4CzkLmIlf9HQPiG/7zsBJ6F8/xU4G05pSRamiq a8cKhsBn+7Ig8MdAyUbVKUAQFYjqCjLOAgGaER/CgfviwPLIF0ngvlaDE00CMHT+PIEi bEz36JhYjTsLi2DIsh2gzxXK9LMgXY4u3Cx0kiGHoB8rvNnjruEYjNfLj3F8uFD9EkZy UEJfezgI6JMXUQOp3OHzZMt4wkk7JKbyfnmVSUYf/Jo5wopg0yrhL2gGDOX/EoEBh+T/ JZtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728517659; x=1729122459; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nOKR97XKPQsX0XTIJnmVl1Wv6IjTAPxbl7mcNCJ+f7A=; b=vddfoOZiUqgZeLoW8sw8BGVKreUN34wQrcP5rMFV1PqBvgWrqiTH01B3DL1KYMhLCm 420gZavUnnceK5L+yAxXInswOHXfzYnepSUpS7lxi3npB+AY16QY0EHeKM76mJh9jPW+ w92rcNNk+7YdmOo64YAQ6UOgQD7FBQblwRR7pc/UKfpcjKTijtZn+MMKSsU47Fv74zAJ 95vydPZm1vkJMploW7wLJT4Hsu43Dtw9M4P/pmA369HEoFs56BvqnU8kndULZK1M3LpE MPwP0qJB0vTzFzDY3QvAt13p/4X+z7Sigj8iOU+HvBAiF8tQwPHZcLZnnJcClKk0WAym FGdg== X-Forwarded-Encrypted: i=1; AJvYcCVeYx5zbQAU1Ie3ViwjMUtsw+nQg84YSWMBswW07rgsxTvy51R6+GfDGfTBXYHKZC/UMMkHu0Drb7zn@nongnu.org X-Gm-Message-State: AOJu0YyahAV/kxQ4xIYtnPcZUZ+/ZWz/h7GtrO4i7zBS7RnfVuG51i+Q 71LHkJn2RUkig+/dVHxevYCCxEv96u962WBhaGjnu1VjTvlBEZRojPvYEv37KKY= X-Google-Smtp-Source: AGHT+IFoTa3ZlzIvCCJmkT8jpuiP/CyEw8bMDc+2y6BKzzcUcnGbUpfs3AwnJjDzpS8A3qmLviiGQw== X-Received: by 2002:a05:6214:3d9e:b0:6cb:48c4:a557 with SMTP id 6a1803df08f44-6cbc955a98amr92780446d6.26.1728517659284; Wed, 09 Oct 2024 16:47:39 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([2605:a7c0:0:301::44]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbe85d856fsm264386d6.72.2024.10.09.16.47.37 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 09 Oct 2024 16:47:38 -0700 (PDT) From: Yichen Wang To: "Dr. David Alan Gilbert" , Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?Phil?= =?utf-8?q?ippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" , Bryan Zhang Subject: [PATCH v6 03/12] util/dsa: Implement DSA device start and stop logic. Date: Wed, 9 Oct 2024 16:46:01 -0700 Message-Id: <20241009234610.27039-4-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20241009234610.27039-1-yichen.wang@bytedance.com> References: <20241009234610.27039-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::f2f; envelope-from=yichen.wang@bytedance.com; helo=mail-qv1-xf2f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang * DSA device open and close. * DSA group contains multiple DSA devices. * DSA group configure/start/stop/clean. Signed-off-by: Hao Xiang Signed-off-by: Bryan Zhang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 103 +++++++++++++++++ util/dsa.c | 282 +++++++++++++++++++++++++++++++++++++++++++++ util/meson.build | 3 + 3 files changed, 388 insertions(+) create mode 100644 include/qemu/dsa.h create mode 100644 util/dsa.c diff --git a/include/qemu/dsa.h b/include/qemu/dsa.h new file mode 100644 index 0000000000..501bb8c70d --- /dev/null +++ b/include/qemu/dsa.h @@ -0,0 +1,103 @@ +/* + * Interface for using Intel Data Streaming Accelerator to offload certain + * background operations. + * + * Copyright (C) Bytedance Ltd. + * + * Authors: + * Hao Xiang + * Yichen Wang + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef QEMU_DSA_H +#define QEMU_DSA_H + +#include "qemu/error-report.h" +#include "qemu/thread.h" +#include "qemu/queue.h" + +#ifdef CONFIG_DSA_OPT + +#pragma GCC push_options +#pragma GCC target("enqcmd") + +#include +#include "x86intrin.h" + +typedef struct { + void *work_queue; +} QemuDsaDevice; + +typedef QSIMPLEQ_HEAD(QemuDsaTaskQueue, QemuDsaBatchTask) QemuDsaTaskQueue; + +typedef struct { + QemuDsaDevice *dsa_devices; + int num_dsa_devices; + /* The index of the next DSA device to be used. */ + uint32_t device_allocator_index; + bool running; + QemuMutex task_queue_lock; + QemuCond task_queue_cond; + QemuDsaTaskQueue task_queue; +} QemuDsaDeviceGroup; + +/** + * @brief Initializes DSA devices. + * + * @param dsa_parameter A list of DSA device path from migration parameter. + * + * @return int Zero if successful, otherwise non zero. + */ +int qemu_dsa_init(const strList *dsa_parameter, Error **errp); + +/** + * @brief Start logic to enable using DSA. + */ +void qemu_dsa_start(void); + +/** + * @brief Stop the device group and the completion thread. + */ +void qemu_dsa_stop(void); + +/** + * @brief Clean up system resources created for DSA offloading. + */ +void qemu_dsa_cleanup(void); + +/** + * @brief Check if DSA is running. + * + * @return True if DSA is running, otherwise false. + */ +bool qemu_dsa_is_running(void); + +#else + +static inline bool qemu_dsa_is_running(void) +{ + return false; +} + +static inline int qemu_dsa_init(const strList *dsa_parameter, Error **errp) +{ + if (dsa_parameter != NULL && strlen(dsa_parameter) != 0) { + error_setg(errp, "DSA is not supported."); + return -1; + } + + return 0; +} + +static inline void qemu_dsa_start(void) {} + +static inline void qemu_dsa_stop(void) {} + +static inline void qemu_dsa_cleanup(void) {} + +#endif + +#endif diff --git a/util/dsa.c b/util/dsa.c new file mode 100644 index 0000000000..54d0e20c29 --- /dev/null +++ b/util/dsa.c @@ -0,0 +1,282 @@ +/* + * Use Intel Data Streaming Accelerator to offload certain background + * operations. + * + * Copyright (C) Bytedance Ltd. + * + * Authors: + * Hao Xiang + * Bryan Zhang + * Yichen Wang + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu/queue.h" +#include "qemu/memalign.h" +#include "qemu/lockable.h" +#include "qemu/cutils.h" +#include "qemu/dsa.h" +#include "qemu/bswap.h" +#include "qemu/error-report.h" +#include "qemu/rcu.h" + +#pragma GCC push_options +#pragma GCC target("enqcmd") + +#include +#include "x86intrin.h" + +#define DSA_WQ_PORTAL_SIZE 4096 +#define MAX_DSA_DEVICES 16 + +uint32_t max_retry_count; +static QemuDsaDeviceGroup dsa_group; + + +/** + * @brief This function opens a DSA device's work queue and + * maps the DSA device memory into the current process. + * + * @param dsa_wq_path A pointer to the DSA device work queue's file path. + * @return A pointer to the mapped memory, or MAP_FAILED on failure. + */ +static void * +map_dsa_device(const char *dsa_wq_path) +{ + void *dsa_device; + int fd; + + fd = open(dsa_wq_path, O_RDWR); + if (fd < 0) { + error_report("Open %s failed with errno = %d.", + dsa_wq_path, errno); + return MAP_FAILED; + } + dsa_device = mmap(NULL, DSA_WQ_PORTAL_SIZE, PROT_WRITE, + MAP_SHARED | MAP_POPULATE, fd, 0); + close(fd); + if (dsa_device == MAP_FAILED) { + error_report("mmap failed with errno = %d.", errno); + return MAP_FAILED; + } + return dsa_device; +} + +/** + * @brief Initializes a DSA device structure. + * + * @param instance A pointer to the DSA device. + * @param work_queue A pointer to the DSA work queue. + */ +static void +dsa_device_init(QemuDsaDevice *instance, + void *dsa_work_queue) +{ + instance->work_queue = dsa_work_queue; +} + +/** + * @brief Cleans up a DSA device structure. + * + * @param instance A pointer to the DSA device to cleanup. + */ +static void +dsa_device_cleanup(QemuDsaDevice *instance) +{ + if (instance->work_queue != MAP_FAILED) { + munmap(instance->work_queue, DSA_WQ_PORTAL_SIZE); + } +} + +/** + * @brief Initializes a DSA device group. + * + * @param group A pointer to the DSA device group. + * @param dsa_parameter A list of DSA device path from are separated by space + * character migration parameter. Multiple DSA device path. + * + * @return Zero if successful, non-zero otherwise. + */ +static int +dsa_device_group_init(QemuDsaDeviceGroup *group, + const strList *dsa_parameter, + Error **errp) +{ + if (dsa_parameter == NULL) { + error_setg(errp, "dsa device path is not supplied."); + return -1; + } + + int ret = 0; + const char *dsa_path[MAX_DSA_DEVICES]; + int num_dsa_devices = 0; + + while (dsa_parameter) { + dsa_path[num_dsa_devices++] = dsa_parameter->value; + if (num_dsa_devices == MAX_DSA_DEVICES) { + break; + } + dsa_parameter = dsa_parameter->next; + } + + group->dsa_devices = + g_new0(QemuDsaDevice, num_dsa_devices); + group->num_dsa_devices = num_dsa_devices; + group->device_allocator_index = 0; + + group->running = false; + qemu_mutex_init(&group->task_queue_lock); + qemu_cond_init(&group->task_queue_cond); + QSIMPLEQ_INIT(&group->task_queue); + + void *dsa_wq = MAP_FAILED; + for (int i = 0; i < num_dsa_devices; i++) { + dsa_wq = map_dsa_device(dsa_path[i]); + if (dsa_wq == MAP_FAILED) { + error_setg(errp, "map_dsa_device failed MAP_FAILED."); + ret = -1; + goto exit; + } + dsa_device_init(&dsa_group.dsa_devices[i], dsa_wq); + } + +exit: + return ret; +} + +/** + * @brief Starts a DSA device group. + * + * @param group A pointer to the DSA device group. + */ +static void +dsa_device_group_start(QemuDsaDeviceGroup *group) +{ + group->running = true; +} + +/** + * @brief Stops a DSA device group. + * + * @param group A pointer to the DSA device group. + */ +__attribute__((unused)) +static void +dsa_device_group_stop(QemuDsaDeviceGroup *group) +{ + group->running = false; +} + +/** + * @brief Cleans up a DSA device group. + * + * @param group A pointer to the DSA device group. + */ +static void +dsa_device_group_cleanup(QemuDsaDeviceGroup *group) +{ + if (!group->dsa_devices) { + return; + } + for (int i = 0; i < group->num_dsa_devices; i++) { + dsa_device_cleanup(&group->dsa_devices[i]); + } + g_free(group->dsa_devices); + group->dsa_devices = NULL; + + qemu_mutex_destroy(&group->task_queue_lock); + qemu_cond_destroy(&group->task_queue_cond); +} + +/** + * @brief Returns the next available DSA device in the group. + * + * @param group A pointer to the DSA device group. + * + * @return struct QemuDsaDevice* A pointer to the next available DSA device + * in the group. + */ +__attribute__((unused)) +static QemuDsaDevice * +dsa_device_group_get_next_device(QemuDsaDeviceGroup *group) +{ + if (group->num_dsa_devices == 0) { + return NULL; + } + uint32_t current = qatomic_fetch_inc(&group->device_allocator_index); + current %= group->num_dsa_devices; + return &group->dsa_devices[current]; +} + +/** + * @brief Check if DSA is running. + * + * @return True if DSA is running, otherwise false. + */ +bool qemu_dsa_is_running(void) +{ + return false; +} + +static void +dsa_globals_init(void) +{ + max_retry_count = UINT32_MAX; +} + +/** + * @brief Initializes DSA devices. + * + * @param dsa_parameter A list of DSA device path from migration parameter. + * + * @return int Zero if successful, otherwise non zero. + */ +int qemu_dsa_init(const strList *dsa_parameter, Error **errp) +{ + dsa_globals_init(); + + return dsa_device_group_init(&dsa_group, dsa_parameter, errp); +} + +/** + * @brief Start logic to enable using DSA. + * + */ +void qemu_dsa_start(void) +{ + if (dsa_group.num_dsa_devices == 0) { + return; + } + if (dsa_group.running) { + return; + } + dsa_device_group_start(&dsa_group); +} + +/** + * @brief Stop the device group and the completion thread. + * + */ +void qemu_dsa_stop(void) +{ + QemuDsaDeviceGroup *group = &dsa_group; + + if (!group->running) { + return; + } +} + +/** + * @brief Clean up system resources created for DSA offloading. + * + */ +void qemu_dsa_cleanup(void) +{ + qemu_dsa_stop(); + dsa_device_group_cleanup(&dsa_group); +} + diff --git a/util/meson.build b/util/meson.build index 5d8bef9891..3360f62923 100644 --- a/util/meson.build +++ b/util/meson.build @@ -88,6 +88,9 @@ if have_block or have_ga endif if have_block util_ss.add(files('aio-wait.c')) + if config_host_data.get('CONFIG_DSA_OPT') + util_ss.add(files('dsa.c')) + endif util_ss.add(files('buffer.c')) util_ss.add(files('bufferiszero.c')) util_ss.add(files('hbitmap.c')) From patchwork Wed Oct 9 23:46:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13829385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7FFFECED611 for ; Wed, 9 Oct 2024 23:51:31 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sygPI-00032t-UR; Wed, 09 Oct 2024 19:47:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sygPA-000303-W9 for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:45 -0400 Received: from mail-qv1-xf2d.google.com ([2607:f8b0:4864:20::f2d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sygP9-0006cl-28 for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:44 -0400 Received: by mail-qv1-xf2d.google.com with SMTP id 6a1803df08f44-6cbcc2bd7fcso2594876d6.1 for ; Wed, 09 Oct 2024 16:47:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1728517662; x=1729122462; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7HsvJpMyVdEOQrXuc/l7m9MbsUuBZHEJ4MSbE8s9wc4=; b=QRgte4uBRhEyRmGCDSJuuQ3eVPoA0D3DIt8Bz6nyuTt6L/oTndEPlg4FIepIFVAKFk 9SC7SA02y3y9DQTv8Ypgx0+rhgZ+sYD450UXNs7lFRzETreNUGYs+qvyP8fLvuC9IrGp C6HijKoy7Q6LMFm27PM/wcKIWfx+U08xKmJXV8rfzndVDervSMCRM1Me96uZ11dE2qhI WOhT8QKHAXa0Xx8ZBkbgtOm2+f6OVaHbwJnL0Y5p6w7JuKdc5S3edkWHYxysxbiIuNQP LCqGmbZQAKYnPj768n7omrLjOo6gJWPqctTTPiIGZZaBnXrYyUPzKQhZaYcDEl/6PCh9 Ormw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728517662; x=1729122462; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7HsvJpMyVdEOQrXuc/l7m9MbsUuBZHEJ4MSbE8s9wc4=; b=kvFM/XKnpX9SoJcXG01JtK0Hl2uOxAmiMAzPCQ2Z/vTGz+K8nUXFyZlxAGG9Mavo38 vrXY5INCDilplM0usSvjaEM3FjfhqFuYgWijC3EYtfiUJKY8vQTGSTW/rgzff36SQLNs P3LHksqZ7G+hxpwiEjTa5jawmXbyiLcuhha5omAuUBQHYLbkEpBXXgRsLBC00vjFmzdZ HdOPnN4ySPvDFlmA/uyqCY3LHkny4iF45Gu/sK12TRg0d6+D1VdC6eTDLtk9fdrShCfn pMKUGfqrRZ0WECm2QBBIB+4ah9U5v+2Ni66NfcK069Oy4IL7/WuIsqhShJxSVuh4j3CY r67A== X-Forwarded-Encrypted: i=1; AJvYcCX/1sUBgSz/PjorvIosUTxF//ygV6DwcOEwICmMUhjlFNT1xZ4tCJkLHkQg0UP1WLTRDum8uVDliK5o@nongnu.org X-Gm-Message-State: AOJu0Yy117gO6zIci+1x3XvHq3FQQML0v0lampCgPCjbQp3O8ReldP61 jUlRzp4LSomJO3FV8katscuIKQk4WOb/Q4ka7n5MQLuZHGm57okMljvQNMRWbho= X-Google-Smtp-Source: AGHT+IGTeZCM7Cuj3OmtShASwOdF4PbHEsC8pbAnm8Eu6/F5vBra0Py4rYHiTByCud19h7XUH4yS2w== X-Received: by 2002:a0c:f410:0:b0:6cb:20f4:30eb with SMTP id 6a1803df08f44-6cbc95476a1mr53124876d6.36.1728517661927; Wed, 09 Oct 2024 16:47:41 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([2605:a7c0:0:301::44]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbe85d856fsm264386d6.72.2024.10.09.16.47.39 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 09 Oct 2024 16:47:41 -0700 (PDT) From: Yichen Wang To: "Dr. David Alan Gilbert" , Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?Phil?= =?utf-8?q?ippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" Subject: [PATCH v6 04/12] util/dsa: Implement DSA task enqueue and dequeue. Date: Wed, 9 Oct 2024 16:46:02 -0700 Message-Id: <20241009234610.27039-5-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20241009234610.27039-1-yichen.wang@bytedance.com> References: <20241009234610.27039-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::f2d; envelope-from=yichen.wang@bytedance.com; helo=mail-qv1-xf2d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang * Use a safe thread queue for DSA task enqueue/dequeue. * Implement DSA task submission. * Implement DSA batch task submission. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 29 +++++++ util/dsa.c | 202 ++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 230 insertions(+), 1 deletion(-) diff --git a/include/qemu/dsa.h b/include/qemu/dsa.h index 501bb8c70d..f39533f4ac 100644 --- a/include/qemu/dsa.h +++ b/include/qemu/dsa.h @@ -27,6 +27,17 @@ #include #include "x86intrin.h" +typedef enum QemuDsaTaskType { + QEMU_DSA_TASK = 0, + QEMU_DSA_BATCH_TASK +} QemuDsaTaskType; + +typedef enum QemuDsaTaskStatus { + QEMU_DSA_TASK_READY = 0, + QEMU_DSA_TASK_PROCESSING, + QEMU_DSA_TASK_COMPLETION +} QemuDsaTaskStatus; + typedef struct { void *work_queue; } QemuDsaDevice; @@ -44,6 +55,24 @@ typedef struct { QemuDsaTaskQueue task_queue; } QemuDsaDeviceGroup; +typedef void (*qemu_dsa_completion_fn)(void *); + +typedef struct QemuDsaBatchTask { + struct dsa_hw_desc batch_descriptor; + struct dsa_hw_desc *descriptors; + struct dsa_completion_record batch_completion __attribute__((aligned(32))); + struct dsa_completion_record *completions; + QemuDsaDeviceGroup *group; + QemuDsaDevice *device; + qemu_dsa_completion_fn completion_callback; + QemuSemaphore sem_task_complete; + QemuDsaTaskType task_type; + QemuDsaTaskStatus status; + int batch_size; + QSIMPLEQ_ENTRY(QemuDsaBatchTask) entry; +} QemuDsaBatchTask; + + /** * @brief Initializes DSA devices. * diff --git a/util/dsa.c b/util/dsa.c index 54d0e20c29..79e305cb6e 100644 --- a/util/dsa.c +++ b/util/dsa.c @@ -31,6 +31,7 @@ #include "x86intrin.h" #define DSA_WQ_PORTAL_SIZE 4096 +#define DSA_WQ_DEPTH 128 #define MAX_DSA_DEVICES 16 uint32_t max_retry_count; @@ -212,6 +213,198 @@ dsa_device_group_get_next_device(QemuDsaDeviceGroup *group) return &group->dsa_devices[current]; } +/** + * @brief Empties out the DSA task queue. + * + * @param group A pointer to the DSA device group. + */ +static void +dsa_empty_task_queue(QemuDsaDeviceGroup *group) +{ + qemu_mutex_lock(&group->task_queue_lock); + QemuDsaTaskQueue *task_queue = &group->task_queue; + while (!QSIMPLEQ_EMPTY(task_queue)) { + QSIMPLEQ_REMOVE_HEAD(task_queue, entry); + } + qemu_mutex_unlock(&group->task_queue_lock); +} + +/** + * @brief Adds a task to the DSA task queue. + * + * @param group A pointer to the DSA device group. + * @param task A pointer to the DSA task to enqueue. + * + * @return int Zero if successful, otherwise a proper error code. + */ +static int +dsa_task_enqueue(QemuDsaDeviceGroup *group, + QemuDsaBatchTask *task) +{ + bool notify = false; + + qemu_mutex_lock(&group->task_queue_lock); + + if (!group->running) { + error_report("DSA: Tried to queue task to stopped device queue."); + qemu_mutex_unlock(&group->task_queue_lock); + return -1; + } + + /* The queue is empty. This enqueue operation is a 0->1 transition. */ + if (QSIMPLEQ_EMPTY(&group->task_queue)) { + notify = true; + } + + QSIMPLEQ_INSERT_TAIL(&group->task_queue, task, entry); + + /* We need to notify the waiter for 0->1 transitions. */ + if (notify) { + qemu_cond_signal(&group->task_queue_cond); + } + + qemu_mutex_unlock(&group->task_queue_lock); + + return 0; +} + +/** + * @brief Takes a DSA task out of the task queue. + * + * @param group A pointer to the DSA device group. + * @return QemuDsaBatchTask* The DSA task being dequeued. + */ +__attribute__((unused)) +static QemuDsaBatchTask * +dsa_task_dequeue(QemuDsaDeviceGroup *group) +{ + QemuDsaBatchTask *task = NULL; + + qemu_mutex_lock(&group->task_queue_lock); + + while (true) { + if (!group->running) { + goto exit; + } + task = QSIMPLEQ_FIRST(&group->task_queue); + if (task != NULL) { + break; + } + qemu_cond_wait(&group->task_queue_cond, &group->task_queue_lock); + } + + QSIMPLEQ_REMOVE_HEAD(&group->task_queue, entry); + +exit: + qemu_mutex_unlock(&group->task_queue_lock); + return task; +} + +/** + * @brief Submits a DSA work item to the device work queue. + * + * @param wq A pointer to the DSA work queue's device memory. + * @param descriptor A pointer to the DSA work item descriptor. + * + * @return Zero if successful, non-zero otherwise. + */ +static int +submit_wi_int(void *wq, struct dsa_hw_desc *descriptor) +{ + uint32_t retry = 0; + + _mm_sfence(); + + while (true) { + if (_enqcmd(wq, descriptor) == 0) { + break; + } + retry++; + if (retry > max_retry_count) { + error_report("Submit work retry %u times.", retry); + return -1; + } + } + + return 0; +} + +/** + * @brief Synchronously submits a DSA work item to the + * device work queue. + * + * @param wq A pointer to the DSA work queue's device memory. + * @param descriptor A pointer to the DSA work item descriptor. + * + * @return int Zero if successful, non-zero otherwise. + */ +__attribute__((unused)) +static int +submit_wi(void *wq, struct dsa_hw_desc *descriptor) +{ + return submit_wi_int(wq, descriptor); +} + +/** + * @brief Asynchronously submits a DSA work item to the + * device work queue. + * + * @param task A pointer to the task. + * + * @return int Zero if successful, non-zero otherwise. + */ +__attribute__((unused)) +static int +submit_wi_async(QemuDsaBatchTask *task) +{ + QemuDsaDeviceGroup *device_group = task->group; + QemuDsaDevice *device_instance = task->device; + int ret; + + assert(task->task_type == QEMU_DSA_TASK); + + task->status = QEMU_DSA_TASK_PROCESSING; + + ret = submit_wi_int(device_instance->work_queue, + &task->descriptors[0]); + if (ret != 0) { + return ret; + } + + return dsa_task_enqueue(device_group, task); +} + +/** + * @brief Asynchronously submits a DSA batch work item to the + * device work queue. + * + * @param batch_task A pointer to the batch task. + * + * @return int Zero if successful, non-zero otherwise. + */ +__attribute__((unused)) +static int +submit_batch_wi_async(QemuDsaBatchTask *batch_task) +{ + QemuDsaDeviceGroup *device_group = batch_task->group; + QemuDsaDevice *device_instance = batch_task->device; + int ret; + + assert(batch_task->task_type == QEMU_DSA_BATCH_TASK); + assert(batch_task->batch_descriptor.desc_count <= batch_task->batch_size); + assert(batch_task->status == QEMU_DSA_TASK_READY); + + batch_task->status = QEMU_DSA_TASK_PROCESSING; + + ret = submit_wi_int(device_instance->work_queue, + &batch_task->batch_descriptor); + if (ret != 0) { + return ret; + } + + return dsa_task_enqueue(device_group, batch_task); +} + /** * @brief Check if DSA is running. * @@ -225,7 +418,12 @@ bool qemu_dsa_is_running(void) static void dsa_globals_init(void) { - max_retry_count = UINT32_MAX; + /* + * This value follows a reference example by Intel. The POLL_RETRY_MAX is + * defined to 10000, so here we used the max WQ depth * 100 for the the max + * polling retry count. + */ + max_retry_count = DSA_WQ_DEPTH * 100; } /** @@ -268,6 +466,8 @@ void qemu_dsa_stop(void) if (!group->running) { return; } + + dsa_empty_task_queue(group); } /** From patchwork Wed Oct 9 23:46:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13829387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 65F7BCEDDA3 for ; Wed, 9 Oct 2024 23:51:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sygPK-00033P-Br; Wed, 09 Oct 2024 19:47:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sygPD-00030U-Si for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:50 -0400 Received: from mail-qv1-xf35.google.com ([2607:f8b0:4864:20::f35]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sygPB-0006dC-Vs for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:47 -0400 Received: by mail-qv1-xf35.google.com with SMTP id 6a1803df08f44-6cbbe3f6931so3725956d6.0 for ; Wed, 09 Oct 2024 16:47:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1728517665; x=1729122465; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7WUmwCNj9B4h3+NTsNbKxCDcNXgP/1g/8yeQPXdEzHo=; b=K99W7a9SrTQLeDUDRTiv8pa1UIVWizTkywdVu0H62FyHqm0GAIqf4W8l7jOg7R0wzk aLjmh96ABjBgkeooqRsJwOiMorDAOulGfYenAUdw50md8sYhx36yll9XmcickvtZKT/6 oga1MVyphY4sCEJZnBmBnSDFdvcpArPpHpufaOhfTgfuMt0tCxD0uvdeKqN/2Ndq7azw yli281EzYEVAqRcIjWAqpEDHUBlBlfmVGFK7mpMJ/RroERNI28FMBbh60yi1pK9DoSO6 wkXkr1pUAs+GhBs1KyZdkRPag0YAgMLHztVe6TkCvHihssCibKppcjHv7Jdbe2Uftq0I nVOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728517665; x=1729122465; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7WUmwCNj9B4h3+NTsNbKxCDcNXgP/1g/8yeQPXdEzHo=; b=uM2hb7Ew8PLTY1oal+Q9O9GTdrW4Hqnmj76ctlXxzB0Cgz0hVg37l6Nb/Siv9xWeN8 jWT4uGdZSBN0meykEsUWGJxaMYSRnAymluXK4uZBAJIEIaq1hozMjkqAlhEPSXva/7/W ne/DRQQdVGeddmsE/o/Nv3tlev/nHWVnyMtxU4ViE5leeOC27ldpFobALDaQ7zAbKBhz 5fWXyZdJW6OGapB4XHHPue/RtC1+XnJB0qvcGfB7Lkc41OWWmj8vWAajDXDWd5yhvNdJ 1QaN8vtl542SSFtTmg0gV/PQKLN692vEkNev9MNi3/rXemrVMU00z1XDmxbkuDC6SRwA 1kvg== X-Forwarded-Encrypted: i=1; AJvYcCWBCdKM8/9lcFmkYamOkN9iO5En3PtI061c7+aLketUG7EkjHuBmN94nc09cNccp2QrCtTDFq7BlMcm@nongnu.org X-Gm-Message-State: AOJu0Yz2Z76t8wbbnHi9GHbo2Z5tkbNdjDrBs0yT3jCdxjQHsUsH1FHP 7wTFB0Hmh9ZSBws6ghyuI151c5SVe2Ge09CMDxUsS6NWOX94d9mq/NaOX71B4tU= X-Google-Smtp-Source: AGHT+IFRTNg1/qFrWIsgSZAt7IflR0jKWfWRs05c62hDyD+L3zz0JcGbNWHf/FX1qkV788qx/+RHxw== X-Received: by 2002:a05:6214:5788:b0:6cb:d1a7:aa1e with SMTP id 6a1803df08f44-6cbd1a7af4cmr37410286d6.21.1728517664725; Wed, 09 Oct 2024 16:47:44 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([2605:a7c0:0:301::44]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbe85d856fsm264386d6.72.2024.10.09.16.47.42 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 09 Oct 2024 16:47:44 -0700 (PDT) From: Yichen Wang To: "Dr. David Alan Gilbert" , Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?Phil?= =?utf-8?q?ippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" Subject: [PATCH v6 05/12] util/dsa: Implement DSA task asynchronous completion thread model. Date: Wed, 9 Oct 2024 16:46:03 -0700 Message-Id: <20241009234610.27039-6-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20241009234610.27039-1-yichen.wang@bytedance.com> References: <20241009234610.27039-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::f35; envelope-from=yichen.wang@bytedance.com; helo=mail-qv1-xf35.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang * Create a dedicated thread for DSA task completion. * DSA completion thread runs a loop and poll for completed tasks. * Start and stop DSA completion thread during DSA device start stop. User space application can directly submit task to Intel DSA accelerator by writing to DSA's device memory (mapped in user space). Once a task is submitted, the device starts processing it and write the completion status back to the task. A user space application can poll the task's completion status to check for completion. This change uses a dedicated thread to perform DSA task completion checking. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 1 + util/dsa.c | 274 ++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 274 insertions(+), 1 deletion(-) diff --git a/include/qemu/dsa.h b/include/qemu/dsa.h index f39533f4ac..7b30303791 100644 --- a/include/qemu/dsa.h +++ b/include/qemu/dsa.h @@ -69,6 +69,7 @@ typedef struct QemuDsaBatchTask { QemuDsaTaskType task_type; QemuDsaTaskStatus status; int batch_size; + bool *results; QSIMPLEQ_ENTRY(QemuDsaBatchTask) entry; } QemuDsaBatchTask; diff --git a/util/dsa.c b/util/dsa.c index 79e305cb6e..56828bec21 100644 --- a/util/dsa.c +++ b/util/dsa.c @@ -33,9 +33,20 @@ #define DSA_WQ_PORTAL_SIZE 4096 #define DSA_WQ_DEPTH 128 #define MAX_DSA_DEVICES 16 +#define DSA_COMPLETION_THREAD "qemu_dsa_completion" + +typedef struct { + bool stopping; + bool running; + QemuThread thread; + int thread_id; + QemuSemaphore sem_init_done; + QemuDsaDeviceGroup *group; +} QemuDsaCompletionThread; uint32_t max_retry_count; static QemuDsaDeviceGroup dsa_group; +static QemuDsaCompletionThread completion_thread; /** @@ -405,6 +416,265 @@ submit_batch_wi_async(QemuDsaBatchTask *batch_task) return dsa_task_enqueue(device_group, batch_task); } +/** + * @brief Poll for the DSA work item completion. + * + * @param completion A pointer to the DSA work item completion record. + * @param opcode The DSA opcode. + * + * @return Zero if successful, non-zero otherwise. + */ +static int +poll_completion(struct dsa_completion_record *completion, + enum dsa_opcode opcode) +{ + uint8_t status; + uint64_t retry = 0; + + while (true) { + /* The DSA operation completes successfully or fails. */ + status = completion->status; + if (status == DSA_COMP_SUCCESS || + status == DSA_COMP_PAGE_FAULT_NOBOF || + status == DSA_COMP_BATCH_PAGE_FAULT || + status == DSA_COMP_BATCH_FAIL) { + break; + } else if (status != DSA_COMP_NONE) { + error_report("DSA opcode %d failed with status = %d.", + opcode, status); + return 1; + } + retry++; + if (retry > max_retry_count) { + error_report("DSA wait for completion retry %lu times.", retry); + return 1; + } + _mm_pause(); + } + + return 0; +} + +/** + * @brief Complete a single DSA task in the batch task. + * + * @param task A pointer to the batch task structure. + * + * @return Zero if successful, otherwise non-zero. + */ +static int +poll_task_completion(QemuDsaBatchTask *task) +{ + assert(task->task_type == QEMU_DSA_TASK); + + struct dsa_completion_record *completion = &task->completions[0]; + uint8_t status; + int ret; + + ret = poll_completion(completion, task->descriptors[0].opcode); + if (ret != 0) { + goto exit; + } + + status = completion->status; + if (status == DSA_COMP_SUCCESS) { + task->results[0] = (completion->result == 0); + goto exit; + } + + assert(status == DSA_COMP_PAGE_FAULT_NOBOF); + +exit: + return ret; +} + +/** + * @brief Poll a batch task status until it completes. If DSA task doesn't + * complete properly, use CPU to complete the task. + * + * @param batch_task A pointer to the DSA batch task. + * + * @return Zero if successful, otherwise non-zero. + */ +static int +poll_batch_task_completion(QemuDsaBatchTask *batch_task) +{ + struct dsa_completion_record *batch_completion = + &batch_task->batch_completion; + struct dsa_completion_record *completion; + uint8_t batch_status; + uint8_t status; + bool *results = batch_task->results; + uint32_t count = batch_task->batch_descriptor.desc_count; + int ret; + + ret = poll_completion(batch_completion, + batch_task->batch_descriptor.opcode); + if (ret != 0) { + goto exit; + } + + batch_status = batch_completion->status; + + if (batch_status == DSA_COMP_SUCCESS) { + if (batch_completion->bytes_completed == count) { + /* + * Let's skip checking for each descriptors' completion status + * if the batch descriptor says all succedded. + */ + for (int i = 0; i < count; i++) { + assert(batch_task->completions[i].status == DSA_COMP_SUCCESS); + results[i] = (batch_task->completions[i].result == 0); + } + goto exit; + } + } else { + assert(batch_status == DSA_COMP_BATCH_FAIL || + batch_status == DSA_COMP_BATCH_PAGE_FAULT); + } + + for (int i = 0; i < count; i++) { + + completion = &batch_task->completions[i]; + status = completion->status; + + if (status == DSA_COMP_SUCCESS) { + results[i] = (completion->result == 0); + continue; + } + + assert(status == DSA_COMP_PAGE_FAULT_NOBOF); + + if (status != DSA_COMP_PAGE_FAULT_NOBOF) { + error_report("Unexpected DSA completion status = %u.", status); + ret = 1; + goto exit; + } + } + +exit: + return ret; +} + +/** + * @brief Handles an asynchronous DSA batch task completion. + * + * @param task A pointer to the batch buffer zero task structure. + */ +static void +dsa_batch_task_complete(QemuDsaBatchTask *batch_task) +{ + batch_task->status = QEMU_DSA_TASK_COMPLETION; + batch_task->completion_callback(batch_task); +} + +/** + * @brief The function entry point called by a dedicated DSA + * work item completion thread. + * + * @param opaque A pointer to the thread context. + * + * @return void* Not used. + */ +static void * +dsa_completion_loop(void *opaque) +{ + QemuDsaCompletionThread *thread_context = + (QemuDsaCompletionThread *)opaque; + QemuDsaBatchTask *batch_task; + QemuDsaDeviceGroup *group = thread_context->group; + int ret; + + rcu_register_thread(); + + thread_context->thread_id = qemu_get_thread_id(); + qemu_sem_post(&thread_context->sem_init_done); + + while (thread_context->running) { + batch_task = dsa_task_dequeue(group); + assert(batch_task != NULL || !group->running); + if (!group->running) { + assert(!thread_context->running); + break; + } + if (batch_task->task_type == QEMU_DSA_TASK) { + ret = poll_task_completion(batch_task); + } else { + assert(batch_task->task_type == QEMU_DSA_BATCH_TASK); + ret = poll_batch_task_completion(batch_task); + } + + if (ret != 0) { + goto exit; + } + + dsa_batch_task_complete(batch_task); + } + +exit: + if (ret != 0) { + error_report("DSA completion thread exited due to internal error."); + } + rcu_unregister_thread(); + return NULL; +} + +/** + * @brief Initializes a DSA completion thread. + * + * @param completion_thread A pointer to the completion thread context. + * @param group A pointer to the DSA device group. + */ +static void +dsa_completion_thread_init( + QemuDsaCompletionThread *completion_thread, + QemuDsaDeviceGroup *group) +{ + completion_thread->stopping = false; + completion_thread->running = true; + completion_thread->thread_id = -1; + qemu_sem_init(&completion_thread->sem_init_done, 0); + completion_thread->group = group; + + qemu_thread_create(&completion_thread->thread, + DSA_COMPLETION_THREAD, + dsa_completion_loop, + completion_thread, + QEMU_THREAD_JOINABLE); + + /* Wait for initialization to complete */ + qemu_sem_wait(&completion_thread->sem_init_done); +} + +/** + * @brief Stops the completion thread (and implicitly, the device group). + * + * @param opaque A pointer to the completion thread. + */ +static void dsa_completion_thread_stop(void *opaque) +{ + QemuDsaCompletionThread *thread_context = + (QemuDsaCompletionThread *)opaque; + + QemuDsaDeviceGroup *group = thread_context->group; + + qemu_mutex_lock(&group->task_queue_lock); + + thread_context->stopping = true; + thread_context->running = false; + + /* Prevent the compiler from setting group->running first. */ + barrier(); + dsa_device_group_stop(group); + + qemu_cond_signal(&group->task_queue_cond); + qemu_mutex_unlock(&group->task_queue_lock); + + qemu_thread_join(&thread_context->thread); + + qemu_sem_destroy(&thread_context->sem_init_done); +} + /** * @brief Check if DSA is running. * @@ -412,7 +682,7 @@ submit_batch_wi_async(QemuDsaBatchTask *batch_task) */ bool qemu_dsa_is_running(void) { - return false; + return completion_thread.running; } static void @@ -453,6 +723,7 @@ void qemu_dsa_start(void) return; } dsa_device_group_start(&dsa_group); + dsa_completion_thread_init(&completion_thread, &dsa_group); } /** @@ -467,6 +738,7 @@ void qemu_dsa_stop(void) return; } + dsa_completion_thread_stop(&completion_thread); dsa_empty_task_queue(group); } From patchwork Wed Oct 9 23:46:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13829383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C7754CED628 for ; Wed, 9 Oct 2024 23:50:07 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sygPK-00033q-S5; Wed, 09 Oct 2024 19:47:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sygPI-00032o-L9 for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:52 -0400 Received: from mail-qv1-xf2a.google.com ([2607:f8b0:4864:20::f2a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sygPF-0006di-Tz for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:51 -0400 Received: by mail-qv1-xf2a.google.com with SMTP id 6a1803df08f44-6cbbe3f6931so3726346d6.0 for ; Wed, 09 Oct 2024 16:47:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1728517668; x=1729122468; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vLW7XuH/q4y8CO5N/BxmAjy5sbKpsYaBuYs92f4yGcA=; b=HDm4A0XNUJO9z8+1CECfoSuSzpXgVomIJ2MnOCMPIh1eWQUa9pVSEgsrkEuvizyI91 O81Pt4kMW39hpYr3NCVlosNxVizyhvLFdwPE1JFt3YsL0qkKcbF6rsf1OJ6k7c2MTu06 T2W3dmY2Bthw5j+bP1SX2OVoChBaJc3wD9cM3dLHbe8Fp1FROWDR1YG6o/dFJV/ZFv5O z1cj/dZIH8Oa5Lhv2mLlBsiYSvxbE8VmlwhQhuZe+hJim+iZ9oOfsrvck9MKUAYj3+qv QEuDOInUFY1Yt8jYgtFK+Xjoq7ZFv8LwFXerhKsb91IVoJph41aQE8DECXgcay6afoKY 0vvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728517668; x=1729122468; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vLW7XuH/q4y8CO5N/BxmAjy5sbKpsYaBuYs92f4yGcA=; b=uz7W+IJ6PcWP6y3Wsr88AL1kVQR4fYS/kPwnmtk0MrlZLLE5FqArz94DOs49ODBG0s 7eo1urHpE3H0A5T1XDF1ZN+VJLc2Ih5EVl17H11mnNBnC77B1CDz+zTorFNYeMnK0a22 tCbWYe9j21xeMzOk+QcXhLYQ0L1FkGjYc6GV9NByTrdhtENuQu0FVnxHJg974xoK+rqy KeEINgyTXYHeyvvWfsVkcdoqGhjakDfjUVI1ZOoda4uzcpZHVPCgvEzsIPPzJPbXOll7 DC1vH9kt+wfInKrqNzt9MLdydNdqrhLhZLBb7YxNnav8LX5mnmrCX2u9GgQ9poDTxnWS nj1Q== X-Forwarded-Encrypted: i=1; AJvYcCXZLvWrSkT5DR8WNJdcn7VblOU0KOuWZ1NZtKDZ/MnZixFh1miFyLGYfNAdXfhjb6103ybRsym4T74u@nongnu.org X-Gm-Message-State: AOJu0YyYcD6uNXS4wj7N0VWQI/dDtnMF6N7TG4QQ0dUNyCE4E4jzOzHa NcDoUDoeASBi91soRe62oLTPxnuHwW2yxQ1IBPVljoUlkQOSotw4byhdJAshBrY= X-Google-Smtp-Source: AGHT+IEF5BxxPt9yUwTeYUW/QU4xqlSCXQwWXly+zql4TOom/uSxF/Vukg74JKzJK/O85aGiBUj5FQ== X-Received: by 2002:a05:6214:5690:b0:6cb:e52c:c8dc with SMTP id 6a1803df08f44-6cbe52cc9e6mr25204376d6.16.1728517667827; Wed, 09 Oct 2024 16:47:47 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([2605:a7c0:0:301::44]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbe85d856fsm264386d6.72.2024.10.09.16.47.45 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 09 Oct 2024 16:47:47 -0700 (PDT) From: Yichen Wang To: "Dr. David Alan Gilbert" , Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?Phil?= =?utf-8?q?ippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" , Bryan Zhang Subject: [PATCH v6 06/12] util/dsa: Implement zero page checking in DSA task. Date: Wed, 9 Oct 2024 16:46:04 -0700 Message-Id: <20241009234610.27039-7-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20241009234610.27039-1-yichen.wang@bytedance.com> References: <20241009234610.27039-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::f2a; envelope-from=yichen.wang@bytedance.com; helo=mail-qv1-xf2a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang Create DSA task with operation code DSA_OPCODE_COMPVAL. Here we create two types of DSA tasks, a single DSA task and a batch DSA task. Batch DSA task reduces task submission overhead and hence should be the default option. However, due to the way DSA hardware works, a DSA batch task must contain at least two individual tasks. There are times we need to submit a single task and hence a single DSA task submission is also required. Signed-off-by: Hao Xiang Signed-off-by: Bryan Zhang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 45 ++++++-- util/dsa.c | 254 +++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 270 insertions(+), 29 deletions(-) diff --git a/include/qemu/dsa.h b/include/qemu/dsa.h index 7b30303791..89841a6ffa 100644 --- a/include/qemu/dsa.h +++ b/include/qemu/dsa.h @@ -16,6 +16,7 @@ #define QEMU_DSA_H #include "qemu/error-report.h" +#include "exec/cpu-common.h" #include "qemu/thread.h" #include "qemu/queue.h" @@ -70,10 +71,11 @@ typedef struct QemuDsaBatchTask { QemuDsaTaskStatus status; int batch_size; bool *results; + /* Address of each pages in pages */ + ram_addr_t *addr; QSIMPLEQ_ENTRY(QemuDsaBatchTask) entry; } QemuDsaBatchTask; - /** * @brief Initializes DSA devices. * @@ -105,8 +107,26 @@ void qemu_dsa_cleanup(void); */ bool qemu_dsa_is_running(void); +/** + * @brief Initializes a buffer zero DSA batch task. + * + * @param batch_size The number of zero page checking tasks in the batch. + * @return A pointer to the zero page checking tasks initialized. + */ +QemuDsaBatchTask * +buffer_zero_batch_task_init(int batch_size); + +/** + * @brief Performs the proper cleanup on a DSA batch task. + * + * @param task A pointer to the batch task to cleanup. + */ +void buffer_zero_batch_task_destroy(QemuDsaBatchTask *task); + #else +typedef struct QemuDsaBatchTask {} QemuDsaBatchTask; + static inline bool qemu_dsa_is_running(void) { return false; @@ -114,19 +134,28 @@ static inline bool qemu_dsa_is_running(void) static inline int qemu_dsa_init(const strList *dsa_parameter, Error **errp) { - if (dsa_parameter != NULL && strlen(dsa_parameter) != 0) { - error_setg(errp, "DSA is not supported."); - return -1; - } - - return 0; + error_setg(errp, "DSA accelerator is not enabled."); + return -1; } static inline void qemu_dsa_start(void) {} static inline void qemu_dsa_stop(void) {} -static inline void qemu_dsa_cleanup(void) {} +static inline QemuDsaBatchTask *buffer_zero_batch_task_init(int batch_size) +{ + return NULL; +} + +static inline void buffer_zero_batch_task_destroy(QemuDsaBatchTask *task) {} + +static inline int +buffer_is_zero_dsa_batch_sync(QemuDsaBatchTask *batch_task, + const void **buf, size_t count, size_t len) +{ + error_setg(errp, "DSA accelerator is not enabled."); + return -1; +} #endif diff --git a/util/dsa.c b/util/dsa.c index 56828bec21..d9c066ff00 100644 --- a/util/dsa.c +++ b/util/dsa.c @@ -48,6 +48,7 @@ uint32_t max_retry_count; static QemuDsaDeviceGroup dsa_group; static QemuDsaCompletionThread completion_thread; +static void buffer_zero_dsa_completion(void *context); /** * @brief This function opens a DSA device's work queue and @@ -176,7 +177,6 @@ dsa_device_group_start(QemuDsaDeviceGroup *group) * * @param group A pointer to the DSA device group. */ -__attribute__((unused)) static void dsa_device_group_stop(QemuDsaDeviceGroup *group) { @@ -212,7 +212,6 @@ dsa_device_group_cleanup(QemuDsaDeviceGroup *group) * @return struct QemuDsaDevice* A pointer to the next available DSA device * in the group. */ -__attribute__((unused)) static QemuDsaDevice * dsa_device_group_get_next_device(QemuDsaDeviceGroup *group) { @@ -285,7 +284,6 @@ dsa_task_enqueue(QemuDsaDeviceGroup *group, * @param group A pointer to the DSA device group. * @return QemuDsaBatchTask* The DSA task being dequeued. */ -__attribute__((unused)) static QemuDsaBatchTask * dsa_task_dequeue(QemuDsaDeviceGroup *group) { @@ -340,22 +338,6 @@ submit_wi_int(void *wq, struct dsa_hw_desc *descriptor) return 0; } -/** - * @brief Synchronously submits a DSA work item to the - * device work queue. - * - * @param wq A pointer to the DSA work queue's device memory. - * @param descriptor A pointer to the DSA work item descriptor. - * - * @return int Zero if successful, non-zero otherwise. - */ -__attribute__((unused)) -static int -submit_wi(void *wq, struct dsa_hw_desc *descriptor) -{ - return submit_wi_int(wq, descriptor); -} - /** * @brief Asynchronously submits a DSA work item to the * device work queue. @@ -364,7 +346,6 @@ submit_wi(void *wq, struct dsa_hw_desc *descriptor) * * @return int Zero if successful, non-zero otherwise. */ -__attribute__((unused)) static int submit_wi_async(QemuDsaBatchTask *task) { @@ -393,7 +374,6 @@ submit_wi_async(QemuDsaBatchTask *task) * * @return int Zero if successful, non-zero otherwise. */ -__attribute__((unused)) static int submit_batch_wi_async(QemuDsaBatchTask *batch_task) { @@ -752,3 +732,235 @@ void qemu_dsa_cleanup(void) dsa_device_group_cleanup(&dsa_group); } + +/* Buffer zero comparison DSA task implementations */ +/* =============================================== */ + +/** + * @brief Sets a buffer zero comparison DSA task. + * + * @param descriptor A pointer to the DSA task descriptor. + * @param buf A pointer to the memory buffer. + * @param len The length of the buffer. + */ +static void +buffer_zero_task_set_int(struct dsa_hw_desc *descriptor, + const void *buf, + size_t len) +{ + struct dsa_completion_record *completion = + (struct dsa_completion_record *)descriptor->completion_addr; + + descriptor->xfer_size = len; + descriptor->src_addr = (uintptr_t)buf; + completion->status = 0; + completion->result = 0; +} + +/** + * @brief Resets a buffer zero comparison DSA batch task. + * + * @param task A pointer to the DSA batch task. + */ +static void +buffer_zero_task_reset(QemuDsaBatchTask *task) +{ + task->completions[0].status = DSA_COMP_NONE; + task->task_type = QEMU_DSA_TASK; + task->status = QEMU_DSA_TASK_READY; +} + +/** + * @brief Resets a buffer zero comparison DSA batch task. + * + * @param task A pointer to the batch task. + * @param count The number of DSA tasks this batch task will contain. + */ +static void +buffer_zero_batch_task_reset(QemuDsaBatchTask *task, size_t count) +{ + task->batch_completion.status = DSA_COMP_NONE; + task->batch_descriptor.desc_count = count; + task->task_type = QEMU_DSA_BATCH_TASK; + task->status = QEMU_DSA_TASK_READY; +} + +/** + * @brief Sets a buffer zero comparison DSA task. + * + * @param task A pointer to the DSA task. + * @param buf A pointer to the memory buffer. + * @param len The buffer length. + */ +static void +buffer_zero_task_set(QemuDsaBatchTask *task, + const void *buf, + size_t len) +{ + buffer_zero_task_reset(task); + buffer_zero_task_set_int(&task->descriptors[0], buf, len); +} + +/** + * @brief Sets a buffer zero comparison batch task. + * + * @param batch_task A pointer to the batch task. + * @param buf An array of memory buffers. + * @param count The number of buffers in the array. + * @param len The length of the buffers. + */ +static void +buffer_zero_batch_task_set(QemuDsaBatchTask *batch_task, + const void **buf, size_t count, size_t len) +{ + assert(count > 0); + assert(count <= batch_task->batch_size); + + buffer_zero_batch_task_reset(batch_task, count); + for (int i = 0; i < count; i++) { + buffer_zero_task_set_int(&batch_task->descriptors[i], buf[i], len); + } +} + +/** + * @brief Asychronously perform a buffer zero DSA operation. + * + * @param task A pointer to the batch task structure. + * @param buf A pointer to the memory buffer. + * @param len The length of the memory buffer. + * + * @return int Zero if successful, otherwise an appropriate error code. + */ +__attribute__((unused)) +static int +buffer_zero_dsa_async(QemuDsaBatchTask *task, + const void *buf, size_t len) +{ + buffer_zero_task_set(task, buf, len); + + return submit_wi_async(task); +} + +/** + * @brief Sends a memory comparison batch task to a DSA device and wait + * for completion. + * + * @param batch_task The batch task to be submitted to DSA device. + * @param buf An array of memory buffers to check for zero. + * @param count The number of buffers. + * @param len The buffer length. + */ +__attribute__((unused)) +static int +buffer_zero_dsa_batch_async(QemuDsaBatchTask *batch_task, + const void **buf, size_t count, size_t len) +{ + assert(count <= batch_task->batch_size); + buffer_zero_batch_task_set(batch_task, buf, count, len); + + return submit_batch_wi_async(batch_task); +} + +/** + * @brief The completion callback function for buffer zero + * comparison DSA task completion. + * + * @param context A pointer to the callback context. + */ +static void +buffer_zero_dsa_completion(void *context) +{ + assert(context != NULL); + + QemuDsaBatchTask *task = (QemuDsaBatchTask *)context; + qemu_sem_post(&task->sem_task_complete); +} + +/** + * @brief Wait for the asynchronous DSA task to complete. + * + * @param batch_task A pointer to the buffer zero comparison batch task. + */ +__attribute__((unused)) +static void +buffer_zero_dsa_wait(QemuDsaBatchTask *batch_task) +{ + qemu_sem_wait(&batch_task->sem_task_complete); +} + +/** + * @brief Initializes a buffer zero comparison DSA task. + * + * @param descriptor A pointer to the DSA task descriptor. + * @param completion A pointer to the DSA task completion record. + */ +static void +buffer_zero_task_init_int(struct dsa_hw_desc *descriptor, + struct dsa_completion_record *completion) +{ + descriptor->opcode = DSA_OPCODE_COMPVAL; + descriptor->flags = IDXD_OP_FLAG_RCR | IDXD_OP_FLAG_CRAV; + descriptor->comp_pattern = (uint64_t)0; + descriptor->completion_addr = (uint64_t)completion; +} + +/** + * @brief Initializes a buffer zero DSA batch task. + * + * @param batch_size The number of zero page checking tasks in the batch. + * @return A pointer to the zero page checking tasks initialized. + */ +QemuDsaBatchTask * +buffer_zero_batch_task_init(int batch_size) +{ + QemuDsaBatchTask *task = qemu_memalign(64, sizeof(QemuDsaBatchTask)); + int descriptors_size = sizeof(*task->descriptors) * batch_size; + + memset(task, 0, sizeof(*task)); + task->addr = g_new0(ram_addr_t, batch_size); + task->results = g_new0(bool, batch_size); + task->batch_size = batch_size; + task->descriptors = + (struct dsa_hw_desc *)qemu_memalign(64, descriptors_size); + memset(task->descriptors, 0, descriptors_size); + task->completions = (struct dsa_completion_record *)qemu_memalign( + 32, sizeof(*task->completions) * batch_size); + + task->batch_completion.status = DSA_COMP_NONE; + task->batch_descriptor.completion_addr = (uint64_t)&task->batch_completion; + /* TODO: Ensure that we never send a batch with count <= 1 */ + task->batch_descriptor.desc_count = 0; + task->batch_descriptor.opcode = DSA_OPCODE_BATCH; + task->batch_descriptor.flags = IDXD_OP_FLAG_RCR | IDXD_OP_FLAG_CRAV; + task->batch_descriptor.desc_list_addr = (uintptr_t)task->descriptors; + task->status = QEMU_DSA_TASK_READY; + task->group = &dsa_group; + task->device = dsa_device_group_get_next_device(&dsa_group); + + for (int i = 0; i < task->batch_size; i++) { + buffer_zero_task_init_int(&task->descriptors[i], + &task->completions[i]); + } + + qemu_sem_init(&task->sem_task_complete, 0); + task->completion_callback = buffer_zero_dsa_completion; + + return task; +} + +/** + * @brief Performs the proper cleanup on a DSA batch task. + * + * @param task A pointer to the batch task to cleanup. + */ +void +buffer_zero_batch_task_destroy(QemuDsaBatchTask *task) +{ + g_free(task->addr); + g_free(task->results); + qemu_vfree(task->descriptors); + qemu_vfree(task->completions); + task->results = NULL; + qemu_sem_destroy(&task->sem_task_complete); + qemu_vfree(task); +} From patchwork Wed Oct 9 23:46:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13829376 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BDA1DCEDDA6 for ; Wed, 9 Oct 2024 23:48:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sygPM-000340-1Z; Wed, 09 Oct 2024 19:47:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sygPK-00033H-5H for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:54 -0400 Received: from mail-qt1-x830.google.com ([2607:f8b0:4864:20::830]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sygPI-0006e6-CG for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:53 -0400 Received: by mail-qt1-x830.google.com with SMTP id d75a77b69052e-460414d5250so1071111cf.0 for ; Wed, 09 Oct 2024 16:47:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1728517670; x=1729122470; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=smtARH6HYgdJtrl2FWYeE5W3midpBOU+64GAZEvxYdY=; b=Mw69cmn6ybdHl7v6R4X8fqy8IViaB6bouOopSLJl0hFcn0LMHxwrOQDkSZcEJuxGmr Y1EMBCKkQGtNU9BrGwihb08/+mBYubbnkOe4aUFZGIE+03KjszqOPmDqeoj7cYFQnhQI jKhr9s4N6Vu2Wp98aRxDbRDEk/BeCRvn8UjWf87pHIhBE2Ei/LKkUUiaEjLfldFLFaeC HDBWOTJKYjBDW/wM3edu9oJeaHhDV8DF12j8jrZWboU08P5EGr8aIiyqhccKDHJ0g+Dl hWfPMtkcdfIG/Uu7hHfKxhZSZm9Oy3bgbIEd41/97xUPAPPquCHZ1ijfLXQB7wbp/ta8 uYPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728517670; x=1729122470; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=smtARH6HYgdJtrl2FWYeE5W3midpBOU+64GAZEvxYdY=; b=myQ9dBld9uIfoSqnmBKMs54W3o3aEpKEEvm6qOqACCG+6Kvv6zTGWKDmCcLTjDzkHV XganzjcgpkcdNCB+3bkXDm8tPNcjAD4XbyWaYlx9oB5cYHHiELU0LDTghwDj/HY1Kmpi d/A4XCDXb/7hBpuefhol8jl/0Op+Zfnk3ZyN8DdoYjySan1AOwA3w/L8m4smO4kfZYOm 0SwcpaXNmVzMrpL37mZAYHHYcD7z/qAR8UlLAx4k6cexAeSRxx6P/Cv4S8CTzskWGg5I yM/ak11k+ILumouAoKIftzdZOHABH/iqkcvUjc2aRcqxYyz4RMFLec3863YH7q3aHLxq 7v/g== X-Forwarded-Encrypted: i=1; AJvYcCVDkFfkzV7FvAt8vsY1EXo339iZpM8rHcZFkJHWXj7iO0ZY3rcI8KwPEW6T3uNqYyuaAd4Hr8Oqpa7W@nongnu.org X-Gm-Message-State: AOJu0YxQgT+GVfxo8G864d1JLplPsVL0D01CUFVuVjcf7ExMfX6CRPIP WwQNCu6R5/YhNYVcJsMMbi9/dU7M57Cb3OlTIEYwVcQkB8hD88bEg4mW8DlO7aU= X-Google-Smtp-Source: AGHT+IHkfDL8BsEP2++RGoxorCQCEq++h1PQWkbsHGkQnPI+avOSYCFxCEsB3+HgNU9BvN0VHUx8Kg== X-Received: by 2002:a05:6214:5d8b:b0:6cb:8267:e285 with SMTP id 6a1803df08f44-6cbc91f4d0dmr44132516d6.0.1728517670382; Wed, 09 Oct 2024 16:47:50 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([2605:a7c0:0:301::44]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbe85d856fsm264386d6.72.2024.10.09.16.47.48 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 09 Oct 2024 16:47:50 -0700 (PDT) From: Yichen Wang To: "Dr. David Alan Gilbert" , Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?Phil?= =?utf-8?q?ippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" , Bryan Zhang Subject: [PATCH v6 07/12] util/dsa: Implement DSA task asynchronous submission and wait for completion. Date: Wed, 9 Oct 2024 16:46:05 -0700 Message-Id: <20241009234610.27039-8-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20241009234610.27039-1-yichen.wang@bytedance.com> References: <20241009234610.27039-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::830; envelope-from=yichen.wang@bytedance.com; helo=mail-qt1-x830.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang * Add a DSA task completion callback. * DSA completion thread will call the tasks's completion callback on every task/batch task completion. * DSA submission path to wait for completion. * Implement CPU fallback if DSA is not able to complete the task. Signed-off-by: Hao Xiang Signed-off-by: Bryan Zhang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 14 +++++ util/dsa.c | 125 +++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 135 insertions(+), 4 deletions(-) diff --git a/include/qemu/dsa.h b/include/qemu/dsa.h index 89841a6ffa..a3b502ee41 100644 --- a/include/qemu/dsa.h +++ b/include/qemu/dsa.h @@ -123,6 +123,20 @@ buffer_zero_batch_task_init(int batch_size); */ void buffer_zero_batch_task_destroy(QemuDsaBatchTask *task); +/** + * @brief Performs buffer zero comparison on a DSA batch task synchronously. + * + * @param batch_task A pointer to the batch task. + * @param buf An array of memory buffers. + * @param count The number of buffers in the array. + * @param len The buffer length. + * + * @return Zero if successful, otherwise non-zero. + */ +int +buffer_is_zero_dsa_batch_sync(QemuDsaBatchTask *batch_task, + const void **buf, size_t count, size_t len); + #else typedef struct QemuDsaBatchTask {} QemuDsaBatchTask; diff --git a/util/dsa.c b/util/dsa.c index d9c066ff00..cbaa47c360 100644 --- a/util/dsa.c +++ b/util/dsa.c @@ -435,6 +435,42 @@ poll_completion(struct dsa_completion_record *completion, return 0; } +/** + * @brief Helper function to use CPU to complete a single + * zero page checking task. + * + * @param completion A pointer to a DSA task completion record. + * @param descriptor A pointer to a DSA task descriptor. + * @param result A pointer to the result of a zero page checking. + */ +static void +task_cpu_fallback_int(struct dsa_completion_record *completion, + struct dsa_hw_desc *descriptor, bool *result) +{ + const uint8_t *buf; + size_t len; + + if (completion->status == DSA_COMP_SUCCESS) { + return; + } + + /* + * DSA was able to partially complete the operation. Check the + * result. If we already know this is not a zero page, we can + * return now. + */ + if (completion->bytes_completed != 0 && completion->result != 0) { + *result = false; + return; + } + + /* Let's fallback to use CPU to complete it. */ + buf = (const uint8_t *)descriptor->src_addr; + len = descriptor->xfer_size; + *result = buffer_is_zero(buf + completion->bytes_completed, + len - completion->bytes_completed); +} + /** * @brief Complete a single DSA task in the batch task. * @@ -563,7 +599,7 @@ dsa_completion_loop(void *opaque) (QemuDsaCompletionThread *)opaque; QemuDsaBatchTask *batch_task; QemuDsaDeviceGroup *group = thread_context->group; - int ret; + int ret = 0; rcu_register_thread(); @@ -831,7 +867,6 @@ buffer_zero_batch_task_set(QemuDsaBatchTask *batch_task, * * @return int Zero if successful, otherwise an appropriate error code. */ -__attribute__((unused)) static int buffer_zero_dsa_async(QemuDsaBatchTask *task, const void *buf, size_t len) @@ -850,7 +885,6 @@ buffer_zero_dsa_async(QemuDsaBatchTask *task, * @param count The number of buffers. * @param len The buffer length. */ -__attribute__((unused)) static int buffer_zero_dsa_batch_async(QemuDsaBatchTask *batch_task, const void **buf, size_t count, size_t len) @@ -881,13 +915,61 @@ buffer_zero_dsa_completion(void *context) * * @param batch_task A pointer to the buffer zero comparison batch task. */ -__attribute__((unused)) static void buffer_zero_dsa_wait(QemuDsaBatchTask *batch_task) { qemu_sem_wait(&batch_task->sem_task_complete); } +/** + * @brief Use CPU to complete the zero page checking task if DSA + * is not able to complete it. + * + * @param batch_task A pointer to the batch task. + */ +static void +buffer_zero_cpu_fallback(QemuDsaBatchTask *batch_task) +{ + if (batch_task->task_type == QEMU_DSA_TASK) { + if (batch_task->completions[0].status == DSA_COMP_SUCCESS) { + return; + } + task_cpu_fallback_int(&batch_task->completions[0], + &batch_task->descriptors[0], + &batch_task->results[0]); + } else if (batch_task->task_type == QEMU_DSA_BATCH_TASK) { + struct dsa_completion_record *batch_completion = + &batch_task->batch_completion; + struct dsa_completion_record *completion; + uint8_t status; + bool *results = batch_task->results; + uint32_t count = batch_task->batch_descriptor.desc_count; + + /* DSA is able to complete the entire batch task. */ + if (batch_completion->status == DSA_COMP_SUCCESS) { + assert(count == batch_completion->bytes_completed); + return; + } + + /* + * DSA encounters some error and is not able to complete + * the entire batch task. Use CPU fallback. + */ + for (int i = 0; i < count; i++) { + + completion = &batch_task->completions[i]; + status = completion->status; + + assert(status == DSA_COMP_SUCCESS || + status == DSA_COMP_PAGE_FAULT_NOBOF); + + task_cpu_fallback_int(completion, + &batch_task->descriptors[i], + &results[i]); + } + } +} + /** * @brief Initializes a buffer zero comparison DSA task. * @@ -964,3 +1046,38 @@ buffer_zero_batch_task_destroy(QemuDsaBatchTask *task) qemu_sem_destroy(&task->sem_task_complete); qemu_vfree(task); } + +/** + * @brief Performs buffer zero comparison on a DSA batch task synchronously. + * + * @param batch_task A pointer to the batch task. + * @param buf An array of memory buffers. + * @param count The number of buffers in the array. + * @param len The buffer length. + * + * @return Zero if successful, otherwise non-zero. + */ +int +buffer_is_zero_dsa_batch_sync(QemuDsaBatchTask *batch_task, + const void **buf, size_t count, size_t len) +{ + if (count <= 0 || count > batch_task->batch_size) { + return -1; + } + + assert(batch_task != NULL); + assert(len != 0); + assert(buf != NULL); + + if (count == 1) { + /* DSA doesn't take batch operation with only 1 task. */ + buffer_zero_dsa_async(batch_task, buf[0], len); + } else { + buffer_zero_dsa_batch_async(batch_task, buf, count, len); + } + + buffer_zero_dsa_wait(batch_task); + buffer_zero_cpu_fallback(batch_task); + + return 0; +} From patchwork Wed Oct 9 23:46:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13829378 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F23E8CED628 for ; Wed, 9 Oct 2024 23:48:46 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sygQ2-0003Ks-66; Wed, 09 Oct 2024 19:48:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sygPM-00034p-Bl for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:56 -0400 Received: from mail-qv1-xf2c.google.com ([2607:f8b0:4864:20::f2c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sygPK-0006eQ-3M for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:56 -0400 Received: by mail-qv1-xf2c.google.com with SMTP id 6a1803df08f44-6cbd57cc35bso6901176d6.1 for ; Wed, 09 Oct 2024 16:47:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1728517673; x=1729122473; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6rh4iSF4JcVdFm8rItOAee3jMjOlBonh6C/ENTJRWd4=; b=XEfvbbOZHss6JWlThT/TvcdCCy6eO/fPS3Z+f29cai+qdH0dEuNQ2WDsPd+rZ3Gr4D o1aAQqiMiBzzb3Q+AD+Wi8qrLPm3TJ0bm3o5HR0PCDbacuUPuHa1JDKpeF1kpepoIPor RBLbaEZsqIZqsmaEVGLh3I+aJmxv+0ZV2hywbarJs1XisnhphxJCqS4CIs9Itm5AhsaR Z17n3sdb+1vBcF45tVDDOShXIxdnzac9pL5BVGnlHLI7whss4QaVg2RIYZZ3sMkNZGHi GyfFN4n282/KXtU098rTr+WSU/a8OT9+foxBqV9IZaEE0daTLaBE2JwUgWi+w6weoPoz 6UXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728517673; x=1729122473; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6rh4iSF4JcVdFm8rItOAee3jMjOlBonh6C/ENTJRWd4=; b=ghWjvIrO7Gu8HrcyawpcPE7bm+JvAnkYIOb9wQ89lGzVzxkMe4yvobXZO0nAvZ8WGB AWucE16SjV2r4Snu5B7YE9V7ZimyRM57uYwIJcQ5k+YGWcePtGPMkMVOWankkxANbvKp i/KaVvIjyJ/iQlDKEBqpdWvUGwNZSYQ5PQz5+OO8vhLykBBuwV/Lfap5YGayGXh1jYOg frgA+3jsGS0WHOKqtUqblAoblFBjGKVfw5EUECAHj/hTHbvEu678TVyxpLi2kTA5gvan 7pLWNMPn9lOMEGThAzNzTTVBf1arJQKiuOyxQdqeaiF4yVxc4Ivb4olbK1Yk0rS5ORJ5 /bDA== X-Forwarded-Encrypted: i=1; AJvYcCW65bbESjdQ6xPDMjpd9ZT6OpdfL3EH6oWqJczEU7QuQSU2Z7WAJjQx658869rpEU08+LJkZWFi+NKW@nongnu.org X-Gm-Message-State: AOJu0YxssSXrAzd65LompWbg7Eaesx4mRtUG2aW1y+YXyvYH96rJR4K9 CLKrhWDCBHrVlgUs8eze7lkpq+ehi0654TXXdh28Z+rIrX1sxVYhnmSrj65JeDI= X-Google-Smtp-Source: AGHT+IEKMgMaN1oKXDZ4B5dJ1c1frQ5fvuc7hbHG52TAHGsxjFdHYK8kyaf5doskNvebH6Lll4Mp2g== X-Received: by 2002:a05:6214:2e44:b0:6cb:bf0d:a56d with SMTP id 6a1803df08f44-6cbe521e0a9mr23184856d6.10.1728517673028; Wed, 09 Oct 2024 16:47:53 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([2605:a7c0:0:301::44]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbe85d856fsm264386d6.72.2024.10.09.16.47.50 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 09 Oct 2024 16:47:52 -0700 (PDT) From: Yichen Wang To: "Dr. David Alan Gilbert" , Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?Phil?= =?utf-8?q?ippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" Subject: [PATCH v6 08/12] migration/multifd: Add new migration option for multifd DSA offloading. Date: Wed, 9 Oct 2024 16:46:06 -0700 Message-Id: <20241009234610.27039-9-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20241009234610.27039-1-yichen.wang@bytedance.com> References: <20241009234610.27039-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::f2c; envelope-from=yichen.wang@bytedance.com; helo=mail-qv1-xf2c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang Intel DSA offloading is an optional feature that turns on if proper hardware and software stack is available. To turn on DSA offloading in multifd live migration by setting: zero-page-detection=dsa-accel dsa-accel-path=[dsa_dev_path1] [dsa_dev_path2] ... [dsa_dev_pathX] This feature is turned off by default. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- hmp-commands.hx | 2 +- include/qemu/dsa.h | 13 +++++++++++++ migration/migration-hmp-cmds.c | 19 ++++++++++++++++++- migration/options.c | 30 ++++++++++++++++++++++++++++++ migration/options.h | 1 + qapi/migration.json | 32 ++++++++++++++++++++++++++++---- util/dsa.c | 31 +++++++++++++++++++++++++++++++ 7 files changed, 122 insertions(+), 6 deletions(-) diff --git a/hmp-commands.hx b/hmp-commands.hx index 06746f0afc..0e04eac7c7 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -1009,7 +1009,7 @@ ERST { .name = "migrate_set_parameter", - .args_type = "parameter:s,value:s", + .args_type = "parameter:s,value:S", .params = "parameter value", .help = "Set the parameter for migration", .cmd = hmp_migrate_set_parameter, diff --git a/include/qemu/dsa.h b/include/qemu/dsa.h index a3b502ee41..b1bb6daad2 100644 --- a/include/qemu/dsa.h +++ b/include/qemu/dsa.h @@ -100,6 +100,13 @@ void qemu_dsa_stop(void); */ void qemu_dsa_cleanup(void); +/** + * @brief Check if DSA is supported. + * + * @return True if DSA is supported, otherwise false. + */ +bool qemu_dsa_is_supported(void); + /** * @brief Check if DSA is running. * @@ -141,6 +148,12 @@ buffer_is_zero_dsa_batch_sync(QemuDsaBatchTask *batch_task, typedef struct QemuDsaBatchTask {} QemuDsaBatchTask; +static inline bool qemu_dsa_is_supported(void) +{ + return false; +} + + static inline bool qemu_dsa_is_running(void) { return false; diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 20d1a6e219..983f13b73c 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -312,7 +312,16 @@ void hmp_info_migrate_parameters(Monitor *mon, const QDict *qdict) monitor_printf(mon, "%s: '%s'\n", MigrationParameter_str(MIGRATION_PARAMETER_TLS_AUTHZ), params->tls_authz); - + if (params->has_dsa_accel_path) { + strList *dsa_accel_path = params->dsa_accel_path; + monitor_printf(mon, "%s:", + MigrationParameter_str(MIGRATION_PARAMETER_DSA_ACCEL_PATH)); + while (dsa_accel_path) { + monitor_printf(mon, " '%s'", dsa_accel_path->value); + dsa_accel_path = dsa_accel_path->next; + } + monitor_printf(mon, "\n"); + } if (params->has_block_bitmap_mapping) { const BitmapMigrationNodeAliasList *bmnal; @@ -563,6 +572,14 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) p->has_x_checkpoint_delay = true; visit_type_uint32(v, param, &p->x_checkpoint_delay, &err); break; + case MIGRATION_PARAMETER_DSA_ACCEL_PATH: + p->has_dsa_accel_path = true; + g_autofree char **strv = g_strsplit(valuestr ? : "", " ", -1); + strList **tail = &p->dsa_accel_path; + for (int i = 0; strv[i]; i++) { + QAPI_LIST_APPEND(tail, strv[i]); + } + break; case MIGRATION_PARAMETER_MULTIFD_CHANNELS: p->has_multifd_channels = true; visit_type_uint8(v, param, &p->multifd_channels, &err); diff --git a/migration/options.c b/migration/options.c index 147cd2b8fd..a0b3a7d291 100644 --- a/migration/options.c +++ b/migration/options.c @@ -13,6 +13,7 @@ #include "qemu/osdep.h" #include "qemu/error-report.h" +#include "qemu/dsa.h" #include "exec/target_page.h" #include "qapi/clone-visitor.h" #include "qapi/error.h" @@ -832,6 +833,13 @@ const char *migrate_tls_creds(void) return s->parameters.tls_creds; } +const strList *migrate_dsa_accel_path(void) +{ + MigrationState *s = migrate_get_current(); + + return s->parameters.dsa_accel_path; +} + const char *migrate_tls_hostname(void) { MigrationState *s = migrate_get_current(); @@ -945,6 +953,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp) params->zero_page_detection = s->parameters.zero_page_detection; params->has_direct_io = true; params->direct_io = s->parameters.direct_io; + params->has_dsa_accel_path = true; + params->dsa_accel_path = QAPI_CLONE(strList, s->parameters.dsa_accel_path); return params; } @@ -953,6 +963,7 @@ void migrate_params_init(MigrationParameters *params) { params->tls_hostname = g_strdup(""); params->tls_creds = g_strdup(""); + params->dsa_accel_path = NULL; /* Set has_* up only for parameter checks */ params->has_throttle_trigger_threshold = true; @@ -1165,6 +1176,14 @@ bool migrate_params_check(MigrationParameters *params, Error **errp) return false; } + if (params->has_zero_page_detection && + params->zero_page_detection == ZERO_PAGE_DETECTION_DSA_ACCEL) { + if (!qemu_dsa_is_supported()) { + error_setg(errp, "DSA acceleration is not supported."); + return false; + } + } + return true; } @@ -1278,6 +1297,11 @@ static void migrate_params_test_apply(MigrateSetParameters *params, if (params->has_direct_io) { dest->direct_io = params->direct_io; } + + if (params->has_dsa_accel_path) { + dest->has_dsa_accel_path = true; + dest->dsa_accel_path = params->dsa_accel_path; + } } static void migrate_params_apply(MigrateSetParameters *params, Error **errp) @@ -1410,6 +1434,12 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp) if (params->has_direct_io) { s->parameters.direct_io = params->direct_io; } + if (params->has_dsa_accel_path) { + qapi_free_strList(s->parameters.dsa_accel_path); + s->parameters.has_dsa_accel_path = true; + s->parameters.dsa_accel_path = + QAPI_CLONE(strList, params->dsa_accel_path); + } } void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) diff --git a/migration/options.h b/migration/options.h index a0bd6edc06..8198b220bd 100644 --- a/migration/options.h +++ b/migration/options.h @@ -86,6 +86,7 @@ const char *migrate_tls_creds(void); const char *migrate_tls_hostname(void); uint64_t migrate_xbzrle_cache_size(void); ZeroPageDetection migrate_zero_page_detection(void); +const strList *migrate_dsa_accel_path(void); /* parameters helpers */ diff --git a/qapi/migration.json b/qapi/migration.json index b66cccf107..d8b42ceae6 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -626,10 +626,14 @@ # multifd migration is enabled, else in the main migration thread # as for @legacy. # +# @dsa-accel: Perform zero page checking with the DSA accelerator +# offloading in multifd sender thread if multifd migration is +# enabled, else in the main migration thread as for @legacy. +# # Since: 9.0 ## { 'enum': 'ZeroPageDetection', - 'data': [ 'none', 'legacy', 'multifd' ] } + 'data': [ 'none', 'legacy', 'multifd', 'dsa-accel' ] } ## # @BitmapMigrationBitmapAliasTransform: @@ -837,6 +841,12 @@ # See description in @ZeroPageDetection. Default is 'multifd'. # (since 9.0) # +# @dsa-accel-path: If enabled, use DSA accelerator offloading for +# certain memory operations. Enable DSA accelerator for zero +# page detection offloading by setting the @zero-page-detection +# to dsa-accel. This parameter defines the dsa device path, and +# defaults to an empty list. (Since 9.2) +# # @direct-io: Open migration files with O_DIRECT when possible. This # only has effect if the @mapped-ram capability is enabled. # (Since 9.1) @@ -855,7 +865,7 @@ 'cpu-throttle-initial', 'cpu-throttle-increment', 'cpu-throttle-tailslow', 'tls-creds', 'tls-hostname', 'tls-authz', 'max-bandwidth', - 'avail-switchover-bandwidth', 'downtime-limit', + 'avail-switchover-bandwidth', 'downtime-limit', 'dsa-accel-path', { 'name': 'x-checkpoint-delay', 'features': [ 'unstable' ] }, 'multifd-channels', 'xbzrle-cache-size', 'max-postcopy-bandwidth', @@ -1018,6 +1028,12 @@ # See description in @ZeroPageDetection. Default is 'multifd'. # (since 9.0) # +# @dsa-accel-path: If enabled, use DSA accelerator offloading for +# certain memory operations. Enable DSA accelerator for zero +# page detection offloading by setting the @zero-page-detection +# to dsa-accel. This parameter defines the dsa device path, and +# defaults to an empty list. (Since 9.2) +# # @direct-io: Open migration files with O_DIRECT when possible. This # only has effect if the @mapped-ram capability is enabled. # (Since 9.1) @@ -1063,7 +1079,8 @@ '*vcpu-dirty-limit': 'uint64', '*mode': 'MigMode', '*zero-page-detection': 'ZeroPageDetection', - '*direct-io': 'bool' } } + '*direct-io': 'bool', + '*dsa-accel-path': [ 'str' ] } } ## # @migrate-set-parameters: @@ -1228,6 +1245,12 @@ # See description in @ZeroPageDetection. Default is 'multifd'. # (since 9.0) # +# @dsa-accel-path: If enabled, use DSA accelerator offloading for +# certain memory operations. Enable DSA accelerator for zero +# page detection offloading by setting the @zero-page-detection +# to dsa-accel. This parameter defines the dsa device path, and +# defaults to an empty list. (Since 9.2) +# # @direct-io: Open migration files with O_DIRECT when possible. This # only has effect if the @mapped-ram capability is enabled. # (Since 9.1) @@ -1270,7 +1293,8 @@ '*vcpu-dirty-limit': 'uint64', '*mode': 'MigMode', '*zero-page-detection': 'ZeroPageDetection', - '*direct-io': 'bool' } } + '*direct-io': 'bool', + '*dsa-accel-path': [ 'str' ] } } ## # @query-migrate-parameters: diff --git a/util/dsa.c b/util/dsa.c index cbaa47c360..eeede3c0c7 100644 --- a/util/dsa.c +++ b/util/dsa.c @@ -23,6 +23,7 @@ #include "qemu/bswap.h" #include "qemu/error-report.h" #include "qemu/rcu.h" +#include #pragma GCC push_options #pragma GCC target("enqcmd") @@ -691,6 +692,36 @@ static void dsa_completion_thread_stop(void *opaque) qemu_sem_destroy(&thread_context->sem_init_done); } +/** + * @brief Check if DSA is supported. + * + * @return True if DSA is supported, otherwise false. + */ +bool qemu_dsa_is_supported(void) +{ + /* + * movdir64b is indicated by bit 28 of ecx in CPUID leaf 7, subleaf 0. + * enqcmd is indicated by bit 29 of ecx in CPUID leaf 7, subleaf 0. + * Doc: https://cdrdv2-public.intel.com/819680/architecture-instruction-\ + * set-extensions-programming-reference.pdf + */ + uint32_t eax, ebx, ecx, edx; + bool movedirb_enabled; + bool enqcmd_enabled; + + __get_cpuid_count(7, 0, &eax, &ebx, &ecx, &edx); + movedirb_enabled = (ecx >> 28) & 0x1; + if (!movedirb_enabled) { + return false; + } + enqcmd_enabled = (ecx >> 29) & 0x1; + if (!enqcmd_enabled) { + return false; + } + + return true; +} + /** * @brief Check if DSA is running. * From patchwork Wed Oct 9 23:46:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13829386 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 37436CED611 for ; Wed, 9 Oct 2024 23:51:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sygQ4-0003PB-Ik; Wed, 09 Oct 2024 19:48:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sygPP-00035F-1q for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:48:03 -0400 Received: from mail-qv1-xf31.google.com ([2607:f8b0:4864:20::f31]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sygPN-0006em-0Y for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:47:58 -0400 Received: by mail-qv1-xf31.google.com with SMTP id 6a1803df08f44-6cbd1a4217dso3951606d6.2 for ; Wed, 09 Oct 2024 16:47:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1728517676; x=1729122476; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XlYEPr5EXzua/WZWQ/aIB6wAzegCRf16oOTlmRliNYk=; b=ObGJZHKKYgdrfAdi43cTCGzlqGroxmwFLAB2IXilrv3+88/IaaJPjer+4+khVsqdZa pGndsPEJBLiPIWrEnTKHOxwTGhntlfys/knzJvi/J/kz55L0mKWNFrlshkgS5ep8wxct vHqqX24RnEbnJSwbpBdwR7YiOFK2/CNGk+94OxPeZ7gyoQt7xWk/+7LnKyVRQvA2B21X 5Sza5fD/S7BkC4jz3q41KKxq/voMWHLbxVysxD3lHPufxh+67lnhmYGb0sC+osG9bqoV OnCTc6VFfLe22NJvuoPnQUOsqDv8qHfa4SqgQ9BI91fo5s4qyZh76yXpm9W/9fkrOiPu HjAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728517676; x=1729122476; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XlYEPr5EXzua/WZWQ/aIB6wAzegCRf16oOTlmRliNYk=; b=X8Bk1OI+HdWhze6DrdVtyZ7lVmDL/RVaqr3cIqGXWJcpC8qaGBQXpJK6FUkyeBvX/X Cqnm4U/RfFnReii3fHloMp9LT4glmxGezeL9xVFMeHWM3kajgfsqQQn8IZSvGrtBg68m 4zW8lvksPRjgnb2gIk1VeDS1xgxXaJK12TEiljW5TLyrXjIgFKi3jKbYihf5P1b1wEHF z50xjGOqscQFex21L2mXBoKO0+xY+2T6NBV0nBLkkIkj0Jo+GSGiWvBBlURmumFR7HKU jArtblW0qCho/POr+4zNRZzCSb0nDCpnlpEMxmTjKXuiBKmrMUh/2uC/6WtTVeIYPIww Nhpw== X-Forwarded-Encrypted: i=1; AJvYcCWJWL2+EayD1sRM9V+z14wTy/rgwIwCcQWqoyOQ5teY7D0e2OhzEyGjX3QpkkZcGZcL7DaDS+QN14A/@nongnu.org X-Gm-Message-State: AOJu0YxfKx2G2qio7CM1ZFCRyb8GXzwRVkaNfixRj5Zb2lcLGqRLUBY4 R5Kbm+7oDPCfHJw0kzqz0yPrOtAPyE9m46cVQU1NTCDQisKDhFD5QlDCFyyy9JQ= X-Google-Smtp-Source: AGHT+IGHI2l0RZMfI4/bgExczjxcWjkpzeWqZYTlDzfMuipTKy45fYIU7ZwYDsrDM7i5HsHX363YxA== X-Received: by 2002:a05:6214:5d0c:b0:6cb:e409:8d9a with SMTP id 6a1803df08f44-6cbe4098dc3mr33563426d6.8.1728517675763; Wed, 09 Oct 2024 16:47:55 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([2605:a7c0:0:301::44]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbe85d856fsm264386d6.72.2024.10.09.16.47.53 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 09 Oct 2024 16:47:55 -0700 (PDT) From: Yichen Wang To: "Dr. David Alan Gilbert" , Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?Phil?= =?utf-8?q?ippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" Subject: [PATCH v6 09/12] migration/multifd: Enable DSA offloading in multifd sender path. Date: Wed, 9 Oct 2024 16:46:07 -0700 Message-Id: <20241009234610.27039-10-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20241009234610.27039-1-yichen.wang@bytedance.com> References: <20241009234610.27039-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::f31; envelope-from=yichen.wang@bytedance.com; helo=mail-qv1-xf31.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang Multifd sender path gets an array of pages queued by the migration thread. It performs zero page checking on every page in the array. The pages are classfied as either a zero page or a normal page. This change uses Intel DSA to offload the zero page checking from CPU to the DSA accelerator. The sender thread submits a batch of pages to DSA hardware and waits for the DSA completion thread to signal for work completion. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- migration/multifd-zero-page.c | 133 ++++++++++++++++++++++++++++++---- migration/multifd.c | 19 ++++- migration/multifd.h | 5 ++ 3 files changed, 141 insertions(+), 16 deletions(-) diff --git a/migration/multifd-zero-page.c b/migration/multifd-zero-page.c index f1e988a959..e4bfff23a4 100644 --- a/migration/multifd-zero-page.c +++ b/migration/multifd-zero-page.c @@ -21,7 +21,9 @@ static bool multifd_zero_page_enabled(void) { - return migrate_zero_page_detection() == ZERO_PAGE_DETECTION_MULTIFD; + ZeroPageDetection curMethod = migrate_zero_page_detection(); + return (curMethod == ZERO_PAGE_DETECTION_MULTIFD || + curMethod == ZERO_PAGE_DETECTION_DSA_ACCEL); } static void swap_page_offset(ram_addr_t *pages_offset, int a, int b) @@ -37,26 +39,49 @@ static void swap_page_offset(ram_addr_t *pages_offset, int a, int b) pages_offset[b] = temp; } +#ifdef CONFIG_DSA_OPT + +static void swap_result(bool *results, int a, int b) +{ + bool temp; + + if (a == b) { + return; + } + + temp = results[a]; + results[a] = results[b]; + results[b] = temp; +} + /** - * multifd_send_zero_page_detect: Perform zero page detection on all pages. + * zero_page_detect_dsa: Perform zero page detection using + * Intel Data Streaming Accelerator (DSA). * - * Sorts normal pages before zero pages in p->pages->offset and updates - * p->pages->normal_num. + * Sorts normal pages before zero pages in pages->offset and updates + * pages->normal_num. * * @param p A pointer to the send params. */ -void multifd_send_zero_page_detect(MultiFDSendParams *p) +static void zero_page_detect_dsa(MultiFDSendParams *p) { MultiFDPages_t *pages = &p->data->u.ram; RAMBlock *rb = pages->block; - int i = 0; - int j = pages->num - 1; + bool *results = p->dsa_batch_task->results; - if (!multifd_zero_page_enabled()) { - pages->normal_num = pages->num; - goto out; + for (int i = 0; i < pages->num; i++) { + p->dsa_batch_task->addr[i] = + (ram_addr_t)(rb->host + pages->offset[i]); } + buffer_is_zero_dsa_batch_sync(p->dsa_batch_task, + (const void **)p->dsa_batch_task->addr, + pages->num, + multifd_ram_page_size()); + + int i = 0; + int j = pages->num - 1; + /* * Sort the page offset array by moving all normal pages to * the left and all zero pages to the right of the array. @@ -64,23 +89,39 @@ void multifd_send_zero_page_detect(MultiFDSendParams *p) while (i <= j) { uint64_t offset = pages->offset[i]; - if (!buffer_is_zero(rb->host + offset, multifd_ram_page_size())) { + if (!results[i]) { i++; continue; } + swap_result(results, i, j); swap_page_offset(pages->offset, i, j); ram_release_page(rb->idstr, offset); j--; } pages->normal_num = i; +} -out: - stat64_add(&mig_stats.normal_pages, pages->normal_num); - stat64_add(&mig_stats.zero_pages, pages->num - pages->normal_num); +void multifd_dsa_cleanup(void) +{ + qemu_dsa_cleanup(); +} + +#else + +static void zero_page_detect_dsa(MultiFDSendParams *p) +{ + g_assert_not_reached(); +} + +void multifd_dsa_cleanup(void) +{ + g_assert_not_reached(); } +#endif + void multifd_recv_zero_page_process(MultiFDRecvParams *p) { for (int i = 0; i < p->zero_num; i++) { @@ -92,3 +133,67 @@ void multifd_recv_zero_page_process(MultiFDRecvParams *p) } } } + +/** + * zero_page_detect_cpu: Perform zero page detection using CPU. + * + * Sorts normal pages before zero pages in p->pages->offset and updates + * p->pages->normal_num. + * + * @param p A pointer to the send params. + */ +static void zero_page_detect_cpu(MultiFDSendParams *p) +{ + MultiFDPages_t *pages = &p->data->u.ram; + RAMBlock *rb = pages->block; + int i = 0; + int j = pages->num - 1; + + if (!multifd_zero_page_enabled()) { + pages->normal_num = pages->num; + return; + } + + /* + * Sort the page offset array by moving all normal pages to + * the left and all zero pages to the right of the array. + */ + while (i <= j) { + uint64_t offset = pages->offset[i]; + + if (!buffer_is_zero(rb->host + offset, multifd_ram_page_size())) { + i++; + continue; + } + + swap_page_offset(pages->offset, i, j); + ram_release_page(rb->idstr, offset); + j--; + } + + pages->normal_num = i; +} + +/** + * multifd_send_zero_page_detect: Perform zero page detection on all pages. + * + * @param p A pointer to the send params. + */ +void multifd_send_zero_page_detect(MultiFDSendParams *p) +{ + MultiFDPages_t *pages = &p->data->u.ram; + + if (!multifd_zero_page_enabled()) { + pages->normal_num = pages->num; + return; + } + + if (qemu_dsa_is_running()) { + zero_page_detect_dsa(p); + } else { + zero_page_detect_cpu(p); + } + + stat64_add(&mig_stats.normal_pages, pages->normal_num); + stat64_add(&mig_stats.zero_pages, pages->num - pages->normal_num); +} diff --git a/migration/multifd.c b/migration/multifd.c index 9b200f4ad9..e255ccf0c7 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -13,6 +13,7 @@ #include "qemu/osdep.h" #include "qemu/cutils.h" #include "qemu/rcu.h" +#include "qemu/dsa.h" #include "exec/target_page.h" #include "sysemu/sysemu.h" #include "exec/ramblock.h" @@ -462,6 +463,8 @@ static bool multifd_send_cleanup_channel(MultiFDSendParams *p, Error **errp) p->name = NULL; g_free(p->data); p->data = NULL; + buffer_zero_batch_task_destroy(p->dsa_batch_task); + p->dsa_batch_task = NULL; p->packet_len = 0; g_free(p->packet); p->packet = NULL; @@ -493,6 +496,8 @@ void multifd_send_shutdown(void) multifd_send_terminate_threads(); + multifd_dsa_cleanup(); + for (i = 0; i < migrate_multifd_channels(); i++) { MultiFDSendParams *p = &multifd_send_state->params[i]; Error *local_err = NULL; @@ -814,11 +819,21 @@ bool multifd_send_setup(void) uint32_t page_count = multifd_ram_page_count(); bool use_packets = multifd_use_packets(); uint8_t i; + Error *local_err = NULL; if (!migrate_multifd()) { return true; } + if (s && + s->parameters.zero_page_detection == ZERO_PAGE_DETECTION_DSA_ACCEL) { + const strList *dsa_parameter = migrate_dsa_accel_path(); + if (qemu_dsa_init(dsa_parameter, &local_err)) { + return false; + } + qemu_dsa_start(); + } + thread_count = migrate_multifd_channels(); multifd_send_state = g_malloc0(sizeof(*multifd_send_state)); multifd_send_state->params = g_new0(MultiFDSendParams, thread_count); @@ -829,12 +844,12 @@ bool multifd_send_setup(void) for (i = 0; i < thread_count; i++) { MultiFDSendParams *p = &multifd_send_state->params[i]; - Error *local_err = NULL; qemu_sem_init(&p->sem, 0); qemu_sem_init(&p->sem_sync, 0); p->id = i; p->data = multifd_send_data_alloc(); + p->dsa_batch_task = buffer_zero_batch_task_init(page_count); if (use_packets) { p->packet_len = sizeof(MultiFDPacket_t) @@ -865,7 +880,6 @@ bool multifd_send_setup(void) for (i = 0; i < thread_count; i++) { MultiFDSendParams *p = &multifd_send_state->params[i]; - Error *local_err = NULL; ret = multifd_send_state->ops->send_setup(p, &local_err); if (ret) { @@ -1047,6 +1061,7 @@ void multifd_recv_cleanup(void) qemu_thread_join(&p->thread); } } + multifd_dsa_cleanup(); for (i = 0; i < migrate_multifd_channels(); i++) { multifd_recv_cleanup_channel(&multifd_recv_state->params[i]); } diff --git a/migration/multifd.h b/migration/multifd.h index 50d58c0c9c..e293ddbc1d 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -15,6 +15,7 @@ #include "exec/target_page.h" #include "ram.h" +#include "qemu/dsa.h" typedef struct MultiFDRecvData MultiFDRecvData; typedef struct MultiFDSendData MultiFDSendData; @@ -155,6 +156,9 @@ typedef struct { bool pending_sync; MultiFDSendData *data; + /* Zero page checking batch task */ + QemuDsaBatchTask *dsa_batch_task; + /* thread local variables. No locking required */ /* pointer to the packet */ @@ -313,6 +317,7 @@ void multifd_send_fill_packet(MultiFDSendParams *p); bool multifd_send_prepare_common(MultiFDSendParams *p); void multifd_send_zero_page_detect(MultiFDSendParams *p); void multifd_recv_zero_page_process(MultiFDRecvParams *p); +void multifd_dsa_cleanup(void); static inline void multifd_send_prepare_header(MultiFDSendParams *p) { From patchwork Wed Oct 9 23:46:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13829382 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 39A12CED628 for ; Wed, 9 Oct 2024 23:50:02 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sygQA-0003e6-Bb; Wed, 09 Oct 2024 19:48:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sygPX-000361-IP for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:48:14 -0400 Received: from mail-qv1-xf30.google.com ([2607:f8b0:4864:20::f30]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sygPU-0006fP-GL for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:48:07 -0400 Received: by mail-qv1-xf30.google.com with SMTP id 6a1803df08f44-6cbcd49b833so2802106d6.3 for ; Wed, 09 Oct 2024 16:48:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1728517683; x=1729122483; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9Ig7VZtud++DwJTDivnw0CFSKCXooNbIa4fqqdu6Buo=; b=deYBkI0m6Zey0hUef88RKCPRdKRq38/8SEyFTA8Ex9Ii5CzZEs/VbCqaWPL+nNKtJw /fG1yt73uQZyL80q7gC3R/H7OrX/NuJRzdCSIGJLlWc3fZ4bIJ7Kef6XuH/xhdDkCZxZ yC8kvyK9K8bLJ+Jh65WeZY2He8BK1Gxf0aumlZCgpLmOkBgHZOc1lvJ3WHNp3ZbkimYN Bq5dRdhYxdta9vDwiSBSzXjfHIB4D7V9C9fTLDvCCSLe/LeV5RrtLJVjSs5393jXlw6+ iQwHzZfe3dgPc+9F17lUNYH4JRWQtLEhVYi2gzUMtbeMziDTDtqM7WagHaU0H27RISzA 9YBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728517683; x=1729122483; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9Ig7VZtud++DwJTDivnw0CFSKCXooNbIa4fqqdu6Buo=; b=DSHRia0fKlc3iXrCx003+O7E7A6IQFWhn1M2E5aStEtuan6RwSSVaGTKUjDkStaKP3 SfE4Q2mg6LtaDe+CgMNsbdp+Zd+nhPtZ4enqX9wgsmu+31Kc1jfLTklQSLlurE30FznS aMeYafOFJaGy7yz7Pb4i/KHT4I8ETcpblCMSRLEJ8jGL6vI1J5Emj9zymhydY0QKXK4W C4f9zL9Daxh/RIEH0/WhfDJsvJVPee5p/2K+0bXyS07U0wsreDEw0yrAPhCqkJUKhcNH NvTccYHJR/tyH5PkzbNIMBPHCeWig5rxm9RSUPcb9RKneqweJ8XwDelM7nAhpfXVfY4Z SmAw== X-Forwarded-Encrypted: i=1; AJvYcCUBoYda4YyIodhyjezWbq3ZqVHXqX46TBI7K5PeZ8vI0L+rLYOnhM9UeSILWVOSRwvJYz3IdlxznxoD@nongnu.org X-Gm-Message-State: AOJu0YzFQ5bvNYa6XYyRnzKC6usFPw0UIR+D4eg5SH9vJC8FGBK4qoH1 tMUIqlTKl+WaDQ9YBSEiQjqG/wGGu4Ut69Iznc63HaamcgrR7nUaqjRfy5nm/24= X-Google-Smtp-Source: AGHT+IFlQJUWyvPQBJToiabA9PHYz8GJfsaCBbRMVAZyPSbdhI9yUR8Ew93pBYXklAZjwYn1KbjVhA== X-Received: by 2002:ad4:46d2:0:b0:6cb:cda0:df6 with SMTP id 6a1803df08f44-6cbcda00f67mr45735206d6.3.1728517683378; Wed, 09 Oct 2024 16:48:03 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([2605:a7c0:0:301::44]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbe85d856fsm264386d6.72.2024.10.09.16.48.01 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 09 Oct 2024 16:48:02 -0700 (PDT) From: Yichen Wang To: "Dr. David Alan Gilbert" , Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?Phil?= =?utf-8?q?ippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" Subject: [PATCH v6 10/12] migration/multifd: Add migration option set packet size. Date: Wed, 9 Oct 2024 16:46:08 -0700 Message-Id: <20241009234610.27039-11-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20241009234610.27039-1-yichen.wang@bytedance.com> References: <20241009234610.27039-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::f30; envelope-from=yichen.wang@bytedance.com; helo=mail-qv1-xf30.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang During live migration, if the latency between sender and receiver is high and bandwidth is also high (a long and fat pipe), using a bigger packet size can help reduce migration total time. The current multifd packet size is 128 * 4kb. In addition, Intel DSA offloading performs better with a large batch task. This change adds an option to set the packet size, which is also useful for performance tuning. Both sender and receiver needs to set the same packet size for things to work. Set the option: migrate_set_parameter multifd-packet-size 4190208 Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- migration/migration-hmp-cmds.c | 7 ++++++ migration/multifd-zlib.c | 6 ++++-- migration/multifd-zstd.c | 6 ++++-- migration/options.c | 39 ++++++++++++++++++++++++++++++++++ migration/options.h | 1 + qapi/migration.json | 21 +++++++++++++++--- 6 files changed, 73 insertions(+), 7 deletions(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 983f13b73c..561ed45250 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -292,6 +292,9 @@ void hmp_info_migrate_parameters(Monitor *mon, const QDict *qdict) monitor_printf(mon, "%s: %u ms\n", MigrationParameter_str(MIGRATION_PARAMETER_X_CHECKPOINT_DELAY), params->x_checkpoint_delay); + monitor_printf(mon, "%s: %" PRIu64 "\n", + MigrationParameter_str(MIGRATION_PARAMETER_MULTIFD_PACKET_SIZE), + params->multifd_packet_size); monitor_printf(mon, "%s: %u\n", MigrationParameter_str(MIGRATION_PARAMETER_MULTIFD_CHANNELS), params->multifd_channels); @@ -580,6 +583,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) QAPI_LIST_APPEND(tail, strv[i]); } break; + case MIGRATION_PARAMETER_MULTIFD_PACKET_SIZE: + p->has_multifd_packet_size = true; + visit_type_size(v, param, &p->multifd_packet_size, &err); + break; case MIGRATION_PARAMETER_MULTIFD_CHANNELS: p->has_multifd_channels = true; visit_type_uint8(v, param, &p->multifd_channels, &err); diff --git a/migration/multifd-zlib.c b/migration/multifd-zlib.c index 8cf8a26bb4..58c278533a 100644 --- a/migration/multifd-zlib.c +++ b/migration/multifd-zlib.c @@ -39,6 +39,7 @@ static int multifd_zlib_send_setup(MultiFDSendParams *p, Error **errp) struct zlib_data *z = g_new0(struct zlib_data, 1); z_stream *zs = &z->zs; const char *err_msg; + uint64_t multifd_packet_size = migrate_multifd_packet_size(); zs->zalloc = Z_NULL; zs->zfree = Z_NULL; @@ -48,7 +49,7 @@ static int multifd_zlib_send_setup(MultiFDSendParams *p, Error **errp) goto err_free_z; } /* This is the maximum size of the compressed buffer */ - z->zbuff_len = compressBound(MULTIFD_PACKET_SIZE); + z->zbuff_len = compressBound(multifd_packet_size); z->zbuff = g_try_malloc(z->zbuff_len); if (!z->zbuff) { err_msg = "out of memory for zbuff"; @@ -162,6 +163,7 @@ out: static int multifd_zlib_recv_setup(MultiFDRecvParams *p, Error **errp) { + uint64_t multifd_packet_size = migrate_multifd_packet_size(); struct zlib_data *z = g_new0(struct zlib_data, 1); z_stream *zs = &z->zs; @@ -176,7 +178,7 @@ static int multifd_zlib_recv_setup(MultiFDRecvParams *p, Error **errp) return -1; } /* To be safe, we reserve twice the size of the packet */ - z->zbuff_len = MULTIFD_PACKET_SIZE * 2; + z->zbuff_len = multifd_packet_size * 2; z->zbuff = g_try_malloc(z->zbuff_len); if (!z->zbuff) { inflateEnd(zs); diff --git a/migration/multifd-zstd.c b/migration/multifd-zstd.c index abed140855..1f97a5417c 100644 --- a/migration/multifd-zstd.c +++ b/migration/multifd-zstd.c @@ -39,6 +39,7 @@ struct zstd_data { static int multifd_zstd_send_setup(MultiFDSendParams *p, Error **errp) { + uint64_t multifd_packet_size = migrate_multifd_packet_size(); struct zstd_data *z = g_new0(struct zstd_data, 1); int res; @@ -58,7 +59,7 @@ static int multifd_zstd_send_setup(MultiFDSendParams *p, Error **errp) return -1; } /* This is the maximum size of the compressed buffer */ - z->zbuff_len = ZSTD_compressBound(MULTIFD_PACKET_SIZE); + z->zbuff_len = ZSTD_compressBound(multifd_packet_size); z->zbuff = g_try_malloc(z->zbuff_len); if (!z->zbuff) { ZSTD_freeCStream(z->zcs); @@ -149,6 +150,7 @@ out: static int multifd_zstd_recv_setup(MultiFDRecvParams *p, Error **errp) { + uint64_t multifd_packet_size = migrate_multifd_packet_size(); struct zstd_data *z = g_new0(struct zstd_data, 1); int ret; @@ -170,7 +172,7 @@ static int multifd_zstd_recv_setup(MultiFDRecvParams *p, Error **errp) } /* To be safe, we reserve twice the size of the packet */ - z->zbuff_len = MULTIFD_PACKET_SIZE * 2; + z->zbuff_len = multifd_packet_size * 2; z->zbuff = g_try_malloc(z->zbuff_len); if (!z->zbuff) { ZSTD_freeDStream(z->zds); diff --git a/migration/options.c b/migration/options.c index a0b3a7d291..b1eaf1c095 100644 --- a/migration/options.c +++ b/migration/options.c @@ -80,6 +80,13 @@ #define DEFAULT_MIGRATE_ANNOUNCE_ROUNDS 5 #define DEFAULT_MIGRATE_ANNOUNCE_STEP 100 +/* + * Parameter for multifd packet size. + */ +#define DEFAULT_MIGRATE_MULTIFD_PACKET_SIZE (128 * 4 * 1024) +/* DSA device supports up to 1024 batches, i.e. 1024 * 4K pages */ +#define MAX_MIGRATE_MULTIFD_PACKET_SIZE (1024 * 4 * 1024) + #define DEFINE_PROP_MIG_CAP(name, x) \ DEFINE_PROP_BOOL(name, MigrationState, capabilities[x], false) @@ -173,6 +180,9 @@ Property migration_properties[] = { DEFINE_PROP_ZERO_PAGE_DETECTION("zero-page-detection", MigrationState, parameters.zero_page_detection, ZERO_PAGE_DETECTION_MULTIFD), + DEFINE_PROP_SIZE("multifd-packet-size", MigrationState, + parameters.multifd_packet_size, + DEFAULT_MIGRATE_MULTIFD_PACKET_SIZE), /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), @@ -783,6 +793,13 @@ int migrate_multifd_channels(void) return s->parameters.multifd_channels; } +uint64_t migrate_multifd_packet_size(void) +{ + MigrationState *s = migrate_get_current(); + + return s->parameters.multifd_packet_size; +} + MultiFDCompression migrate_multifd_compression(void) { MigrationState *s = migrate_get_current(); @@ -911,6 +928,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp) params->downtime_limit = s->parameters.downtime_limit; params->has_x_checkpoint_delay = true; params->x_checkpoint_delay = s->parameters.x_checkpoint_delay; + params->has_multifd_packet_size = true; + params->multifd_packet_size = s->parameters.multifd_packet_size; params->has_multifd_channels = true; params->multifd_channels = s->parameters.multifd_channels; params->has_multifd_compression = true; @@ -973,6 +992,7 @@ void migrate_params_init(MigrationParameters *params) params->has_max_bandwidth = true; params->has_downtime_limit = true; params->has_x_checkpoint_delay = true; + params->has_multifd_packet_size = true; params->has_multifd_channels = true; params->has_multifd_compression = true; params->has_multifd_zlib_level = true; @@ -1055,6 +1075,19 @@ bool migrate_params_check(MigrationParameters *params, Error **errp) /* x_checkpoint_delay is now always positive */ + if (params->has_multifd_packet_size && + ((params->multifd_packet_size < DEFAULT_MIGRATE_MULTIFD_PACKET_SIZE) || + (params->multifd_packet_size > MAX_MIGRATE_MULTIFD_PACKET_SIZE) || + (params->multifd_packet_size % qemu_target_page_size() != 0))) { + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, + "multifd_packet_size", + "an integer in the range of " + stringify(DEFAULT_MIGRATE_MULTIFD_PACKET_SIZE) + " to "stringify(MAX_MIGRATE_MULTIFD_PACKET_SIZE)", " + "and must be a multiple of guest VM's page size."); + return false; + } + if (params->has_multifd_channels && (params->multifd_channels < 1)) { error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "multifd_channels", @@ -1236,6 +1269,9 @@ static void migrate_params_test_apply(MigrateSetParameters *params, dest->x_checkpoint_delay = params->x_checkpoint_delay; } + if (params->has_multifd_packet_size) { + dest->multifd_packet_size = params->multifd_packet_size; + } if (params->has_multifd_channels) { dest->multifd_channels = params->multifd_channels; } @@ -1364,6 +1400,9 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp) colo_checkpoint_delay_set(); } + if (params->has_multifd_packet_size) { + s->parameters.multifd_packet_size = params->multifd_packet_size; + } if (params->has_multifd_channels) { s->parameters.multifd_channels = params->multifd_channels; } diff --git a/migration/options.h b/migration/options.h index 8198b220bd..8158d4879d 100644 --- a/migration/options.h +++ b/migration/options.h @@ -87,6 +87,7 @@ const char *migrate_tls_hostname(void); uint64_t migrate_xbzrle_cache_size(void); ZeroPageDetection migrate_zero_page_detection(void); const strList *migrate_dsa_accel_path(void); +uint64_t migrate_multifd_packet_size(void); /* parameters helpers */ diff --git a/qapi/migration.json b/qapi/migration.json index d8b42ceae6..1d14d8e82f 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -851,6 +851,10 @@ # only has effect if the @mapped-ram capability is enabled. # (Since 9.1) # +# @multifd-packet-size: Packet size in bytes used to migrate data. +# The value needs to be a multiple of guest page size. +# The default value is 524288 and max value is 4190208. (Since 9.2) +# # Features: # # @unstable: Members @x-checkpoint-delay and @@ -877,7 +881,8 @@ 'vcpu-dirty-limit', 'mode', 'zero-page-detection', - 'direct-io'] } + 'direct-io', + 'multifd-packet-size'] } ## # @MigrateSetParameters: @@ -1038,6 +1043,10 @@ # only has effect if the @mapped-ram capability is enabled. # (Since 9.1) # +# @multifd-packet-size: Packet size in bytes used to migrate data. +# The value needs to be a multiple of guest page size. +# The default value is 524288 and max value is 4190208. (Since 9.2) +# # Features: # # @unstable: Members @x-checkpoint-delay and @@ -1080,7 +1089,8 @@ '*mode': 'MigMode', '*zero-page-detection': 'ZeroPageDetection', '*direct-io': 'bool', - '*dsa-accel-path': [ 'str' ] } } + '*dsa-accel-path': [ 'str' ], + '*multifd-packet-size' : 'uint64'} } ## # @migrate-set-parameters: @@ -1255,6 +1265,10 @@ # only has effect if the @mapped-ram capability is enabled. # (Since 9.1) # +# @multifd-packet-size: Packet size in bytes used to migrate data. +# The value needs to be a multiple of guest page size. +# The default value is 524288 and max value is 4190208. (Since 9.2) +# # Features: # # @unstable: Members @x-checkpoint-delay and @@ -1294,7 +1308,8 @@ '*mode': 'MigMode', '*zero-page-detection': 'ZeroPageDetection', '*direct-io': 'bool', - '*dsa-accel-path': [ 'str' ] } } + '*dsa-accel-path': [ 'str' ], + '*multifd-packet-size': 'uint64' } } ## # @query-migrate-parameters: From patchwork Wed Oct 9 23:46:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13829379 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9B9C5CED611 for ; Wed, 9 Oct 2024 23:48:58 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sygQB-0003j1-Gh; Wed, 09 Oct 2024 19:48:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sygPa-000369-Pe for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:48:16 -0400 Received: from mail-qv1-xf2e.google.com ([2607:f8b0:4864:20::f2e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sygPY-0006ff-7r for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:48:10 -0400 Received: by mail-qv1-xf2e.google.com with SMTP id 6a1803df08f44-6cbd1a4217dso3952876d6.2 for ; Wed, 09 Oct 2024 16:48:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1728517686; x=1729122486; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QhcJa+cHhe6ao2Bt0/MyVwDuA8e0xe0uuQoaXDZtjb0=; b=AcXmLja/UgFAdH9l4M5RXCloc7now7N+wcIrRqXDWzGgk1mc/7YC1SHeHVswNaytkd lsgmU677bSkPL2T6y0tkMXGFemir7skAoy7ZB280qaBnMilzX6sh3Nf7oGftn+3M8V99 TqP+kjLL4QR2J0wT9HUHJ4ocqCuGITkuKJKYvyGpIk+cm3s/0wZh/AWzX8usqY5gvaLa ZRLjBnwK43mnHvVXPP/zx8H5n/I6yLUgkZ2Xa9TSNpjzwRebwuoNUWi6U/B8kpGjDWoK nYk4PiOz3m/1GOBK9IwpW29TuTp+BLnwLV02O5flg6fP94MJH8PWduoJpjHWQ3G4Rb5C Qa4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728517686; x=1729122486; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QhcJa+cHhe6ao2Bt0/MyVwDuA8e0xe0uuQoaXDZtjb0=; b=NMckti0LSFs9VD/O8Cx8plaHyvNf7csJIYApd969VWMLc1nTyoKeJujeE0Ac+3P5RM x4mVPo/DMBbe2yBhsmXOtWxfxqc5dFKzlt1RRaGSkM9xNODabs3gPcH1UAezfquIrX/+ g2+1cNIPuA2TaX2gq4JqaCarekUsuvA++1VJTGFXiwDYzHfNjDI2TeZHpKoBtCkW362v aMMsLMhLEnn6PpVgtH3Jrh5beCmYgjTr3tojFlDFMoAChP6ISfr4w7VDPmkbZ+mc3bG3 GFphh9UUwig1zOD1ZrYWDb0ovCw++xcFqwNC6kCezqp+9K+Hg01vIe9NCx/aUHht6IIu JoHg== X-Forwarded-Encrypted: i=1; AJvYcCUbUhGt/amXpM6Mq2O4wk5hHkxHTCgXCRMg+cdvxcArAI1/BKB2TSVQ+szhs/2Vl7lTM4cy5bTK69Hi@nongnu.org X-Gm-Message-State: AOJu0Ywz7vVOFCrepEVGTHLcQ8O1bxbtnXE9OcKKBoLsI8V5VR4uUzSV m3jm4k0qk9X0qPhAV7SLP+5uz0ry5UOSrl5nhyvCrCj9+boyrNELU5Kezvya5vw= X-Google-Smtp-Source: AGHT+IFIH2/Ya8lcM6cXjzRu1LhzD2e8MRNHm/qroKhbcuTh/SY1fpOTG6Lsa/jgDXp2DPzcX5uwMg== X-Received: by 2002:a05:6214:5788:b0:6cb:d1a7:aa1e with SMTP id 6a1803df08f44-6cbd1a7af4cmr37422846d6.21.1728517686080; Wed, 09 Oct 2024 16:48:06 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([2605:a7c0:0:301::44]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbe85d856fsm264386d6.72.2024.10.09.16.48.03 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 09 Oct 2024 16:48:05 -0700 (PDT) From: Yichen Wang To: "Dr. David Alan Gilbert" , Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?Phil?= =?utf-8?q?ippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" , Bryan Zhang Subject: [PATCH v6 11/12] util/dsa: Add unit test coverage for Intel DSA task submission and completion. Date: Wed, 9 Oct 2024 16:46:09 -0700 Message-Id: <20241009234610.27039-12-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20241009234610.27039-1-yichen.wang@bytedance.com> References: <20241009234610.27039-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::f2e; envelope-from=yichen.wang@bytedance.com; helo=mail-qv1-xf2e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang * Test DSA start and stop path. * Test DSA configure and cleanup path. * Test DSA task submission and completion path. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- tests/unit/meson.build | 6 + tests/unit/test-dsa.c | 503 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 509 insertions(+) create mode 100644 tests/unit/test-dsa.c diff --git a/tests/unit/meson.build b/tests/unit/meson.build index d5248ae51d..394219e903 100644 --- a/tests/unit/meson.build +++ b/tests/unit/meson.build @@ -50,6 +50,12 @@ tests = { 'test-fifo': [], } +if config_host_data.get('CONFIG_DSA_OPT') + tests += { + 'test-dsa': [], + } +endif + if have_system or have_tools tests += { 'test-qmp-event': [testqapi], diff --git a/tests/unit/test-dsa.c b/tests/unit/test-dsa.c new file mode 100644 index 0000000000..181a547528 --- /dev/null +++ b/tests/unit/test-dsa.c @@ -0,0 +1,503 @@ +/* + * Test DSA functions. + * + * Copyright (C) Bytedance Ltd. + * + * Authors: + * Hao Xiang + * Bryan Zhang + * Yichen Wang + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qemu/host-utils.h" + +#include "qemu/cutils.h" +#include "qemu/memalign.h" +#include "qemu/dsa.h" + +/* + * TODO Communicate that DSA must be configured to support this batch size. + * TODO Alternatively, poke the DSA device to figure out batch size. + */ +#define batch_size 128 +#define page_size 4096 + +#define oversized_batch_size (batch_size + 1) +#define num_devices 2 +#define max_buffer_size (64 * 1024) + +/* TODO Make these not-hardcoded. */ +static const strList path1[] = { + {.value = (char *)"/dev/dsa/wq4.0", .next = NULL} +}; +static const strList path2[] = { + {.value = (char *)"/dev/dsa/wq4.0", .next = (strList*)&path2[1]}, + {.value = (char *)"/dev/dsa/wq4.1", .next = NULL} +}; + +static Error **errp; + +static QemuDsaBatchTask *task; + +/* A helper for running a single task and checking for correctness. */ +static void do_single_task(void) +{ + task = buffer_zero_batch_task_init(batch_size); + char buf[page_size]; + char *ptr = buf; + + buffer_is_zero_dsa_batch_sync(task, + (const void **)&ptr, + 1, + page_size); + g_assert(task->results[0] == buffer_is_zero(buf, page_size)); + + buffer_zero_batch_task_destroy(task); +} + +static void test_single_zero(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + qemu_dsa_start(); + + task = buffer_zero_batch_task_init(batch_size); + + char buf[page_size]; + char *ptr = buf; + + memset(buf, 0x0, page_size); + buffer_is_zero_dsa_batch_sync(task, + (const void **)&ptr, + 1, page_size); + g_assert(task->results[0]); + + buffer_zero_batch_task_destroy(task); + + qemu_dsa_cleanup(); +} + +static void test_single_zero_async(void) +{ + test_single_zero(); +} + +static void test_single_nonzero(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + qemu_dsa_start(); + + task = buffer_zero_batch_task_init(batch_size); + + char buf[page_size]; + char *ptr = buf; + + memset(buf, 0x1, page_size); + buffer_is_zero_dsa_batch_sync(task, + (const void **)&ptr, + 1, page_size); + g_assert(!task->results[0]); + + buffer_zero_batch_task_destroy(task); + + qemu_dsa_cleanup(); +} + +static void test_single_nonzero_async(void) +{ + test_single_nonzero(); +} + +/* count == 0 should return quickly without calling into DSA. */ +static void test_zero_count_async(void) +{ + char buf[page_size]; + buffer_is_zero_dsa_batch_sync(task, + (const void **)&buf, + 0, + page_size); +} + +static void test_null_task_async(void) +{ + if (g_test_subprocess()) { + g_assert(!qemu_dsa_init(path1, errp)); + + char buf[page_size * batch_size]; + char *addrs[batch_size]; + for (int i = 0; i < batch_size; i++) { + addrs[i] = buf + (page_size * i); + } + + buffer_is_zero_dsa_batch_sync(NULL, (const void **)addrs, + batch_size, + page_size); + } else { + g_test_trap_subprocess(NULL, 0, 0); + g_test_trap_assert_failed(); + } +} + +static void test_oversized_batch(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + qemu_dsa_start(); + + task = buffer_zero_batch_task_init(batch_size); + + char buf[page_size * oversized_batch_size]; + char *addrs[batch_size]; + for (int i = 0; i < oversized_batch_size; i++) { + addrs[i] = buf + (page_size * i); + } + + int ret = buffer_is_zero_dsa_batch_sync(task, + (const void **)addrs, + oversized_batch_size, + page_size); + g_assert(ret != 0); + + buffer_zero_batch_task_destroy(task); + + qemu_dsa_cleanup(); +} + +static void test_oversized_batch_async(void) +{ + test_oversized_batch(); +} + +static void test_zero_len_async(void) +{ + if (g_test_subprocess()) { + g_assert(!qemu_dsa_init(path1, errp)); + + task = buffer_zero_batch_task_init(batch_size); + + char buf[page_size]; + + buffer_is_zero_dsa_batch_sync(task, + (const void **)&buf, + 1, + 0); + + buffer_zero_batch_task_destroy(task); + } else { + g_test_trap_subprocess(NULL, 0, 0); + g_test_trap_assert_failed(); + } +} + +static void test_null_buf_async(void) +{ + if (g_test_subprocess()) { + g_assert(!qemu_dsa_init(path1, errp)); + + task = buffer_zero_batch_task_init(batch_size); + + buffer_is_zero_dsa_batch_sync(task, NULL, 1, page_size); + + buffer_zero_batch_task_destroy(task); + } else { + g_test_trap_subprocess(NULL, 0, 0); + g_test_trap_assert_failed(); + } +} + +static void test_batch(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + qemu_dsa_start(); + + task = buffer_zero_batch_task_init(batch_size); + + char buf[page_size * batch_size]; + char *addrs[batch_size]; + for (int i = 0; i < batch_size; i++) { + addrs[i] = buf + (page_size * i); + } + + /* + * Using whatever is on the stack is somewhat random. + * Manually set some pages to zero and some to nonzero. + */ + memset(buf + 0, 0, page_size * 10); + memset(buf + (10 * page_size), 0xff, page_size * 10); + + buffer_is_zero_dsa_batch_sync(task, + (const void **)addrs, + batch_size, + page_size); + + bool is_zero; + for (int i = 0; i < batch_size; i++) { + is_zero = buffer_is_zero((const void *)&buf[page_size * i], page_size); + g_assert(task->results[i] == is_zero); + } + + buffer_zero_batch_task_destroy(task); + + qemu_dsa_cleanup(); +} + +static void test_batch_async(void) +{ + test_batch(); +} + +static void test_page_fault(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + qemu_dsa_start(); + + char *buf[2]; + int prot = PROT_READ | PROT_WRITE; + int flags = MAP_SHARED | MAP_ANON; + buf[0] = (char *)mmap(NULL, page_size * batch_size, prot, flags, -1, 0); + assert(buf[0] != MAP_FAILED); + buf[1] = (char *)malloc(page_size * batch_size); + assert(buf[1] != NULL); + + for (int j = 0; j < 2; j++) { + task = buffer_zero_batch_task_init(batch_size); + + char *addrs[batch_size]; + for (int i = 0; i < batch_size; i++) { + addrs[i] = buf[j] + (page_size * i); + } + + buffer_is_zero_dsa_batch_sync(task, + (const void **)addrs, + batch_size, + page_size); + + bool is_zero; + for (int i = 0; i < batch_size; i++) { + is_zero = buffer_is_zero((const void *)&buf[j][page_size * i], + page_size); + g_assert(task->results[i] == is_zero); + } + buffer_zero_batch_task_destroy(task); + } + + assert(!munmap(buf[0], page_size * batch_size)); + free(buf[1]); + qemu_dsa_cleanup(); +} + +static void test_various_buffer_sizes(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + qemu_dsa_start(); + + char *buf = malloc(max_buffer_size * batch_size); + char *addrs[batch_size]; + + for (int len = 16; len <= max_buffer_size; len *= 2) { + task = buffer_zero_batch_task_init(batch_size); + + for (int i = 0; i < batch_size; i++) { + addrs[i] = buf + (len * i); + } + + buffer_is_zero_dsa_batch_sync(task, + (const void **)addrs, + batch_size, + len); + + bool is_zero; + for (int j = 0; j < batch_size; j++) { + is_zero = buffer_is_zero((const void *)&buf[len * j], len); + g_assert(task->results[j] == is_zero); + } + + buffer_zero_batch_task_destroy(task); + } + + free(buf); + + qemu_dsa_cleanup(); +} + +static void test_various_buffer_sizes_async(void) +{ + test_various_buffer_sizes(); +} + +static void test_double_start_stop(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + /* Double start */ + qemu_dsa_start(); + qemu_dsa_start(); + g_assert(qemu_dsa_is_running()); + do_single_task(); + + /* Double stop */ + qemu_dsa_stop(); + g_assert(!qemu_dsa_is_running()); + qemu_dsa_stop(); + g_assert(!qemu_dsa_is_running()); + + /* Restart */ + qemu_dsa_start(); + g_assert(qemu_dsa_is_running()); + do_single_task(); + qemu_dsa_cleanup(); +} + +static void test_is_running(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + + g_assert(!qemu_dsa_is_running()); + qemu_dsa_start(); + g_assert(qemu_dsa_is_running()); + qemu_dsa_stop(); + g_assert(!qemu_dsa_is_running()); + qemu_dsa_cleanup(); +} + +static void test_multiple_engines(void) +{ + g_assert(!qemu_dsa_init(path2, errp)); + qemu_dsa_start(); + + QemuDsaBatchTask *tasks[num_devices]; + char bufs[num_devices][page_size * batch_size]; + char *addrs[num_devices][batch_size]; + + /* + * This is a somewhat implementation-specific way + * of testing that the tasks have unique engines + * assigned to them. + */ + tasks[0] = buffer_zero_batch_task_init(batch_size); + tasks[1] = buffer_zero_batch_task_init(batch_size); + g_assert(tasks[0]->device != tasks[1]->device); + + for (int i = 0; i < num_devices; i++) { + for (int j = 0; j < batch_size; j++) { + addrs[i][j] = bufs[i] + (page_size * j); + } + + buffer_is_zero_dsa_batch_sync(tasks[i], + (const void **)addrs[i], + batch_size, page_size); + + bool is_zero; + for (int j = 0; j < batch_size; j++) { + is_zero = buffer_is_zero((const void *)&bufs[i][page_size * j], + page_size); + g_assert(tasks[i]->results[j] == is_zero); + } + } + + buffer_zero_batch_task_destroy(tasks[0]); + buffer_zero_batch_task_destroy(tasks[1]); + + qemu_dsa_cleanup(); +} + +static void test_configure_dsa_twice(void) +{ + g_assert(!qemu_dsa_init(path2, errp)); + g_assert(!qemu_dsa_init(path2, errp)); + qemu_dsa_start(); + do_single_task(); + qemu_dsa_cleanup(); +} + +static void test_configure_dsa_bad_path(void) +{ + const strList *bad_path = &(strList) { + .value = (char *)"/not/a/real/path", .next = NULL + }; + g_assert(qemu_dsa_init(bad_path, errp)); +} + +static void test_cleanup_before_configure(void) +{ + qemu_dsa_cleanup(); + g_assert(!qemu_dsa_init(path2, errp)); +} + +static void test_configure_dsa_num_devices(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + qemu_dsa_start(); + + do_single_task(); + qemu_dsa_stop(); + qemu_dsa_cleanup(); +} + +static void test_cleanup_twice(void) +{ + g_assert(!qemu_dsa_init(path2, errp)); + qemu_dsa_cleanup(); + qemu_dsa_cleanup(); + + g_assert(!qemu_dsa_init(path2, errp)); + qemu_dsa_start(); + do_single_task(); + qemu_dsa_cleanup(); +} + +static int check_test_setup(void) +{ + const strList *path[2] = {path1, path2}; + for (int i = 0; i < sizeof(path) / sizeof(strList *); i++) { + if (qemu_dsa_init(path[i], errp)) { + return -1; + } + qemu_dsa_cleanup(); + } + return 0; +} + +int main(int argc, char **argv) +{ + g_test_init(&argc, &argv, NULL); + + if (check_test_setup() != 0) { + /* + * This test requires extra setup. The current + * setup is not correct. Just skip this test + * for now. + */ + exit(0); + } + + if (num_devices > 1) { + g_test_add_func("/dsa/multiple_engines", test_multiple_engines); + } + + g_test_add_func("/dsa/async/batch", test_batch_async); + g_test_add_func("/dsa/async/various_buffer_sizes", + test_various_buffer_sizes_async); + g_test_add_func("/dsa/async/null_buf", test_null_buf_async); + g_test_add_func("/dsa/async/zero_len", test_zero_len_async); + g_test_add_func("/dsa/async/oversized_batch", test_oversized_batch_async); + g_test_add_func("/dsa/async/zero_count", test_zero_count_async); + g_test_add_func("/dsa/async/single_zero", test_single_zero_async); + g_test_add_func("/dsa/async/single_nonzero", test_single_nonzero_async); + g_test_add_func("/dsa/async/null_task", test_null_task_async); + g_test_add_func("/dsa/async/page_fault", test_page_fault); + + g_test_add_func("/dsa/double_start_stop", test_double_start_stop); + g_test_add_func("/dsa/is_running", test_is_running); + + g_test_add_func("/dsa/configure_dsa_twice", test_configure_dsa_twice); + g_test_add_func("/dsa/configure_dsa_bad_path", test_configure_dsa_bad_path); + g_test_add_func("/dsa/cleanup_before_configure", + test_cleanup_before_configure); + g_test_add_func("/dsa/configure_dsa_num_devices", + test_configure_dsa_num_devices); + g_test_add_func("/dsa/cleanup_twice", test_cleanup_twice); + + return g_test_run(); +} From patchwork Wed Oct 9 23:46:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13829380 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 99D6BCED611 for ; Wed, 9 Oct 2024 23:49:12 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sygQ0-0003Hj-8B; Wed, 09 Oct 2024 19:48:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sygPb-00036H-HF for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:48:16 -0400 Received: from mail-qv1-xf31.google.com ([2607:f8b0:4864:20::f31]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sygPZ-0006gD-Vz for qemu-devel@nongnu.org; Wed, 09 Oct 2024 19:48:11 -0400 Received: by mail-qv1-xf31.google.com with SMTP id 6a1803df08f44-6cbceb48613so2467836d6.2 for ; Wed, 09 Oct 2024 16:48:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1728517689; x=1729122489; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Yby7vSJjPVA9a3PElfIZJQ0FPWIMr9xtoFf7KH5DZnk=; b=G5UMqpjjdTPyQNbHBiqk4cp2L02Tlf8WaD0vFK0hpLI3q57NJ/F1BL57QkttCMgZlM WKQmD2kDQJ27wxZCxeYWpW8u9LS9msgGuYDVts4kMUTLVnlLEFlLu0K72DRUmQr9zeqS 7jBsHnoWOPGHFkWc/5rJz38TcQePLNYyf8VC0e8wkzrV9hzKqUU7klERhxfIwNg/1KYc dudpIbPXXsWSLrNd4SR9Mk5GC7hNz5iYd5YChmG/BFjwZVYFuCSna0ANKtQskZgF2D/y UFF+ZyP5q4flxprUYFrW6G+78IiSEeQODYgy+iNvDpHVdlBflkTtV6PbWkShuPWb1pzJ DFnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728517689; x=1729122489; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Yby7vSJjPVA9a3PElfIZJQ0FPWIMr9xtoFf7KH5DZnk=; b=vXs9zxbXFQ3QpOwDA2vNFc1ikklE07xVjZKuIuVSYIVjIfTCT7CZgcUEsJARob5U70 ju/me4wifLNeGss5KPf64MKhReJe1BdCOLILn+8Fv0O/3tz7OabV8szB8IEJKFTo8q6q jxYjqy6ffYauAm7ZxLCDusxqVLys8TZLXF38wGqT4XAssduG0x6mHOafe4p5WWEd+qjK RUmNAoejCBU7Z3ZnwAD8cYx8JAK0btPOjT4gHxjDUnLSLoIO8tEAuOpXYhDmWskh+Z+D ZtagEMvOuCvY4lYGszUp5/Drr7q0n4aTPDsicQFmWvPF21NZcAHYbiHKPT6QcBqEMDSm Mf/w== X-Forwarded-Encrypted: i=1; AJvYcCVUY9IYQIWHIkbYcQcSJeaSruwbwBTwV9byvQ5wfOFpXF4v9uet8unskYdfDPM/VTpoctk5zKJcHHec@nongnu.org X-Gm-Message-State: AOJu0Ywzx7VqUovMtrF8iU8/HlfMS7UECtqoou1eBRyS9eFFb2pya+Mj EtqWOchIkNtoZnU2wfOJ4Tdf5tZVdG+isdcu9LHFjRyF6vtZqF8HKdw404a8jZk= X-Google-Smtp-Source: AGHT+IGXWAF3cktR536cbh8bgJlIcfWGBEX+1dbf84Bnu/XDLJKLYB5eosBs6C2jlEjDrgvAKt3WaA== X-Received: by 2002:a05:6214:5d0c:b0:6cb:8282:d02f with SMTP id 6a1803df08f44-6cbe4a2be52mr28755176d6.6.1728517688982; Wed, 09 Oct 2024 16:48:08 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([2605:a7c0:0:301::44]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbe85d856fsm264386d6.72.2024.10.09.16.48.06 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 09 Oct 2024 16:48:08 -0700 (PDT) From: Yichen Wang To: "Dr. David Alan Gilbert" , Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?Phil?= =?utf-8?q?ippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" , Bryan Zhang Subject: [PATCH v6 12/12] migration/multifd: Add integration tests for multifd with Intel DSA offloading. Date: Wed, 9 Oct 2024 16:46:10 -0700 Message-Id: <20241009234610.27039-13-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20241009234610.27039-1-yichen.wang@bytedance.com> References: <20241009234610.27039-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::f31; envelope-from=yichen.wang@bytedance.com; helo=mail-qv1-xf31.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang * Add test case to start and complete multifd live migration with DSA offloading enabled. * Add test case to start and cancel multifd live migration with DSA offloading enabled. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- tests/qtest/migration-test.c | 80 +++++++++++++++++++++++++++++++++++- 1 file changed, 79 insertions(+), 1 deletion(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 814ec109a6..47a9dd8c50 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -611,6 +611,13 @@ typedef struct { bool suspend_me; } MigrateStart; +/* + * It requires separate steps to configure and enable DSA device. + * This test assumes that the configuration is done already. + */ +static const char *dsa_dev_path_p = "['/dev/dsa/wq4.0']"; +static const char *dsa_dev_path = "/dev/dsa/wq4.0"; + /* * A hook that runs after the src and dst QEMUs have been * created, but before the migration is started. This can @@ -3230,7 +3237,7 @@ static void test_multifd_tcp_tls_x509_reject_anon_client(void) * * And see that it works */ -static void test_multifd_tcp_cancel(void) +static void test_multifd_tcp_cancel_common(bool use_dsa) { MigrateStart args = { .hide_stderr = true, @@ -3250,6 +3257,11 @@ static void test_multifd_tcp_cancel(void) migrate_set_capability(from, "multifd", true); migrate_set_capability(to, "multifd", true); + if (use_dsa) { + migrate_set_parameter_str(from, "zero-page-detection", "dsa-accel"); + migrate_set_parameter_str(from, "dsa-accel-path", dsa_dev_path_p); + } + /* Start incoming migration from the 1st socket */ migrate_incoming_qmp(to, "tcp:127.0.0.1:0", "{}"); @@ -3300,6 +3312,49 @@ static void test_multifd_tcp_cancel(void) test_migrate_end(from, to2, true); } +/* + * This test does: + * source target + * migrate_incoming + * migrate + * migrate_cancel + * launch another target + * migrate + * + * And see that it works + */ +static void test_multifd_tcp_cancel(void) +{ + test_multifd_tcp_cancel_common(false); +} + +#ifdef CONFIG_DSA_OPT + +static void *test_migrate_precopy_tcp_multifd_start_dsa(QTestState *from, + QTestState *to) +{ + migrate_set_parameter_str(from, "zero-page-detection", "dsa-accel"); + migrate_set_parameter_str(from, "dsa-accel-path", dsa_dev_path_p); + return test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); +} + +static void test_multifd_tcp_zero_page_dsa(void) +{ + MigrateCommon args = { + .listen_uri = "defer", + .start_hook = test_migrate_precopy_tcp_multifd_start_dsa, + }; + + test_precopy_common(&args); +} + +static void test_multifd_tcp_cancel_dsa(void) +{ + test_multifd_tcp_cancel_common(true); +} + +#endif + static void calc_dirty_rate(QTestState *who, uint64_t calc_time) { qtest_qmp_assert_success(who, @@ -3727,6 +3782,19 @@ static bool kvm_dirty_ring_supported(void) #endif } +#ifdef CONFIG_DSA_OPT +static int test_dsa_setup(void) +{ + int fd; + fd = open(dsa_dev_path, O_RDWR); + if (fd < 0) { + return -1; + } + close(fd); + return 0; +} +#endif + int main(int argc, char **argv) { bool has_kvm, has_tcg; @@ -3939,6 +4007,16 @@ int main(int argc, char **argv) test_multifd_tcp_zero_page_legacy); migration_test_add("/migration/multifd/tcp/plain/zero-page/none", test_multifd_tcp_no_zero_page); + +#ifdef CONFIG_DSA_OPT + if (g_str_equal(arch, "x86_64") && test_dsa_setup() == 0) { + migration_test_add("/migration/multifd/tcp/plain/zero-page/dsa", + test_multifd_tcp_zero_page_dsa); + migration_test_add("/migration/multifd/tcp/plain/cancel/dsa", + test_multifd_tcp_cancel_dsa); + } +#endif + migration_test_add("/migration/multifd/tcp/plain/cancel", test_multifd_tcp_cancel); migration_test_add("/migration/multifd/tcp/plain/zlib",