From patchwork Tue Sep 24 17:07:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Badarinarayan Joshi X-Patchwork-Id: 13810925 Received: from mailout3.samsung.com (mailout3.samsung.com [203.254.224.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 919A93FE55 for ; Tue, 24 Sep 2024 12:57:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.33 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727182659; cv=none; b=pH9N0cecpKSw8FZyGEKjxbok8dgK9UY3+ma3VW9Wto9UuFu3R78TV2YiK0tbc7gGtK5P30aaZqYUcMvjn0DZHJefSrnd+OJbwdnd3ZWD8cB37nwAAMmKNa4cEPoZ9sycm4jAS+AL5X90qNixIzMUbCWhETXPOnJNAQDYubd8e+Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727182659; c=relaxed/simple; bh=52ggZerXLdw5Z3zUVOIoz1SBN2kCekOPN/k+0O1I5qo=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type: References; b=ND8Y5ZxpuZ0lShG5MKh+8/in0uIJqYju1+QSPHPQ5fIOQA7wOXSwr1cHdyw6mIaSy+z8uflSS75nZCnO9ehxFoiBCtNcJ7gmCQeGb9Ew9g81tlruWfXJNiLOcqUS8L9jr/fVvvcYO5faGsCltBCPIEB1QP3z5Qre9p5m2lTujLA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=IYStrbd6; arc=none smtp.client-ip=203.254.224.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="IYStrbd6" Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20240924125728epoutp039813b918b06504e6a010adafcf2a1abd~4L3e3rb_02995529955epoutp03D for ; Tue, 24 Sep 2024 12:57:28 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20240924125728epoutp039813b918b06504e6a010adafcf2a1abd~4L3e3rb_02995529955epoutp03D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1727182648; bh=gGwGNQ4pCFBuyrltye9kzLy9M0vca6w83jeHsFp5cCM=; h=From:To:Cc:Subject:Date:References:From; b=IYStrbd6Ssw3cKgI1J6CWS3J7bQ0HNwQCAAK8y2rKKbWW/AoJY7NfLFKGP3QKt9ci 5XvL0kdfQRuPCY4Kalv3Awox9DA/UvLvy4UaRmgT0Ab22ozoE9e0xiCWO0frkGN/gK IQr4e7PKTBGo77kii4QClfkVrqCOjgsTpFBJp60w= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20240924125728epcas5p3a0d240dfddd41740126d644b09cab81e~4L3eqgcGj2477024770epcas5p3u; Tue, 24 Sep 2024 12:57:28 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.174]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4XCfyt6X73z4x9Pt; Tue, 24 Sep 2024 12:57:26 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 52.CA.09642.637B2F66; Tue, 24 Sep 2024 21:57:26 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20240924113446epcas5p2ca1fa34fe7a6b3abefa07079c384552b~4KvSQUAhm0672806728epcas5p27; Tue, 24 Sep 2024 11:34:46 +0000 (GMT) Received: from epsmgmcp1.samsung.com (unknown [182.195.42.82]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20240924113446epsmtrp1ace3ad8048bb4adaa78766994c1318a9~4KvSProCR1551315513epsmtrp1-; Tue, 24 Sep 2024 11:34:46 +0000 (GMT) X-AuditID: b6c32a4b-879fa700000025aa-2a-66f2b7367e95 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgmcp1.samsung.com (Symantec Messaging Gateway) with SMTP id B9.34.19367.6D3A2F66; Tue, 24 Sep 2024 20:34:46 +0900 (KST) Received: from debian.na.corp.samsungelectronics.net (unknown [107.111.150.60]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20240924113446epsmtip1c29625256bbc4c125690ce5defc5d26d~4KvRbfBJt0203802038epsmtip1F; Tue, 24 Sep 2024 11:34:45 +0000 (GMT) From: Badarinarayan Joshi To: kdevops@lists.linux.dev Cc: mcgrof.c@samsung.com, da.gomez@samsung.com, Badarinarayan Joshi Subject: [PATCH] workflow: blkalgn workflow automation Date: Tue, 24 Sep 2024 17:07:35 +0000 Message-ID: <20240924170735.2015812-1-badari.joshi@samsung.com> X-Mailer: git-send-email 2.45.2 Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrBKsWRmVeSWpSXmKPExsWy7bCmhq7Z9k9pBlNOG1l8P3aX2eL95W1M Fj/XvGS2WPBkG7MDi8eLzTMZPfq2rGIMYIrKtslITUxJLVJIzUvOT8nMS7dV8g6Od443NTMw 1DW0tDBXUshLzE21VXLxCdB1y8wB2qSkUJaYUwoUCkgsLlbSt7Mpyi8tSVXIyC8usVVKLUjJ KTAp0CtOzC0uzUvXy0stsTI0MDAyBSpMyM6Y+q2JvWD6H8aKqxNPsDcw/rvL2MXIySEhYCKx 9vRuti5GLg4hgd2MElfmtrJAOJ8YJT49bWOHc/ovToFrWTXtDCtEYiejxO9lkxkhnBYmiflv r7KAVLEBVc1ueQQ0mINDREBa4tomSRCTWSBBonFzEYgpLGAmcWFtIUgxi4CqxKKVt8DG8wrY S6x6vJoZYpW8xMU1z9kg4oISJ2c+ARvODBRv3jqbGWSrhMAidoklJ5+wQjS4SPx8uBHqTmGJ V8e3sEPYUhIv+yGekRBoZpS4t6sBqruHUWL+hX4miCp7idc/djJBHKopsX6XPkRYVmLqqXVM EJv5JHp/P4Eq55XYMQ/GVpPo6Z3DBmHLSMzo2wMV95B492gu2DdCArESP3tPsE5glJ+F5KFZ SB6ahbB5ASPzKkbJ1ILi3PTUYtMC47zUcnjUJufnbmIEJzkt7x2Mjx580DvEyMTBeIhRgoNZ SYR30s2PaUK8KYmVValF+fFFpTmpxYcYTYGhPJFZSjQ5H5hm80riDU0sDUzMzMxMLI3NDJXE eV+3zk0REkhPLEnNTk0tSC2C6WPi4JRqYOKfu9L54OPrywQtt33qO35hZnWSUdmGI/63rnfv v23wfd2ry9+rjwjY1bsu3ea5JO0O//wnCnzZ20uX/nZ5zZFqdDyGSdI/mPP1aof/a1w+hHVV HNqXF9k0Sd116fv/iwz3fZ8jys+y++qqWIV7Ug9Ed+yfIHe+/v6xsz2nQm41RbRqOiqdnCk2 +2luTV18h8yJnuqF3XYfbYUVFvXX2m/5uWJBzYq1OS2rfqvZhX7ZVXTmyTyzi0U+5wQaX7Je ifvq/rXap3+60mS3i6tXHWMLP3Hw+cL5dp0Fz5OfmxV9Kk43Oqi/2SPJueW2D7PMjpO/nr54 fMheLnrKythrB1WW5nwJun5Xfp/WzHv39zOpKLEUZyQaajEXFScCAFt/zz37AwAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFuplluLIzCtJLcpLzFFi42LZdlhJTvfa4k9pBjMnGlt8P3aX2eL95W1M Fj/XvGS2WPBkG7MDi8eLzTMZPfq2rGIMYIrisklJzcksSy3St0vgypj6rYm9YPofxoqrE0+w NzD+u8vYxcjJISFgIrFq2hnWLkYuDiGB7YwSaw/tY4NIyEicubSZFcIWllj57zk7iC0k0MQk 8eCsDojNBtQ8u+URUD0Hh4iAtMS1TZIgYWaBJInPn/tZQcLCAmYSF9YWgoRZBFQlFq28BbaW V8BeYtXj1cwQ0+UlLq55zgYRF5Q4OfMJC8QYeYnmrbOZJzDyzUKSmoUktYCRaRWjaGpBcW56 bnKBoV5xYm5xaV66XnJ+7iZGcIhpBe1gXLb+r94hRiYOxkOMEhzMSiK8k25+TBPiTUmsrEot yo8vKs1JLT7EKM3BoiTOq5zTmSIkkJ5YkpqdmlqQWgSTZeLglGpgEmG4ZMj8LvznuadT3Gbc W3Uw3Zdn058YuwzbZ8/y2XyWvp28/GOrU9Tlcyei9V6xfnqx8GLs9NBJYVc9IuVij+/R6QpM udRtd3J6xDoONmeGkorb37/w3ZF/X2q/IPZMvHuHz9mF8g8SjqVeW5+8dsmPtIB4wf3lWSeD 9H8pLKvd+6Nyj9XzxTdZmQTcMjn5dAwSjc2PR31PLlJqXL11cUwmK9+xfYciauMelZ9PMtJ9 57p1d5T9FOUz2emOO2xCSsu3GYrtZFZezl95cP7RxkuvGyMSN9tOyN70QG2zf6iosZ0rT9r9 yQc4/Tf7fdkn/Kf6aRen3M91x18ytfkfO6/9p3jrqq9H4muf9h5JUWIpzkg01GIuKk4EAEhj Dm6gAgAA X-CMS-MailID: 20240924113446epcas5p2ca1fa34fe7a6b3abefa07079c384552b X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240924113446epcas5p2ca1fa34fe7a6b3abefa07079c384552b References: This is new demo workflow to capture WAF and block alignment details for storage devices. This workflow automates blkalgn from https://github.com/dkruces/bcc/blob/lbs/tools/blkalgn.py Current work is based on https://github.com/dkruces/ansible-playbooks/tree/main/ebpf-playbook/roles/blkalgn Steps involved to run this workload is as follows make defconfig-blkalgn_cfg make make bringup make blkalgn blkalgn workflow can call any other workflow where IO is run, in demo workflow it runs dd command on user defined device /dev/vdc. It captures blkalgn output for disks which have seen some IO and copies locally to workflows/demos/blkalgn/results/ Signed-off-by: Badarinarayan Joshi --- defconfigs/blkalgn_cfg | 407 ++++++++++++++++++ kconfigs/Kconfig.blkalgn | 8 + kconfigs/workflows/Kconfig.demo | 1 + playbooks/blkalgn.yml | 4 + playbooks/demoio.yml | 4 + playbooks/roles/blkalgn/defaults/main.yml | 22 + .../roles/blkalgn/tasks/debian/bdepends.yml | 19 + .../roles/blkalgn/tasks/demo_workload.yml | 11 + .../roles/blkalgn/tasks/install_blkalgn.yml | 28 ++ playbooks/roles/blkalgn/tasks/main.yml | 108 +++++ playbooks/roles/demoio/tasks/main.yml | 13 + workflows/Makefile | 4 + workflows/common/Makefile | 2 + workflows/demos/blkalgn/Kconfig | 10 + workflows/demos/blkalgn/Makefile | 14 + 15 files changed, 655 insertions(+) create mode 100644 defconfigs/blkalgn_cfg create mode 100644 kconfigs/Kconfig.blkalgn create mode 100644 playbooks/blkalgn.yml create mode 100644 playbooks/demoio.yml create mode 100644 playbooks/roles/blkalgn/defaults/main.yml create mode 100644 playbooks/roles/blkalgn/tasks/debian/bdepends.yml create mode 100644 playbooks/roles/blkalgn/tasks/demo_workload.yml create mode 100644 playbooks/roles/blkalgn/tasks/install_blkalgn.yml create mode 100644 playbooks/roles/blkalgn/tasks/main.yml create mode 100644 playbooks/roles/demoio/tasks/main.yml create mode 100644 workflows/demos/blkalgn/Kconfig create mode 100644 workflows/demos/blkalgn/Makefile diff --git a/defconfigs/blkalgn_cfg b/defconfigs/blkalgn_cfg new file mode 100644 index 0000000..58eaf9b --- /dev/null +++ b/defconfigs/blkalgn_cfg @@ -0,0 +1,407 @@ +# +# Automatically generated file; DO NOT EDIT. +# kdevops 5.0.2-g61b3c99-dirty +# +# CONFIG_KDEVOPS_FIRST_RUN is not set +CONFIG_DEFAULT_BLKTESTS_URL="https://github.com/osandov/blktests.git" +CONFIG_DEFAULT_BLKTRACE_URL="https://git.kernel.dk/blktrace" +CONFIG_DEFAULT_DATE_ISO8601="2024-09-24" +CONFIG_DEFAULT_DBENCH_URL="https://github.com/linux-kdevops/dbench.git" +CONFIG_DEFAULT_FSTESTS_GIT_URL="git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git" +CONFIG_DEFAULT_FSTESTS_HTTPS_URL="https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git" +CONFIG_DEFAULT_FSTESTS_GOOGLE_HTTPS_URL="https://kernel.googlesource.com/pub/scm/fs/xfs/xfstests-dev.git" +CONFIG_DEFAULT_FSTESTS_KDEVOPS_GITHUB_HTTPS_URL="https://github.com/linux-kdevops/fstests.git" +CONFIG_DEFAULT_FSTESTS_KDEVOPS_GITLAB_HTTPS_URL="https://gitlab.com/linux-kdevops/fstests.git" +CONFIG_DEFAULT_GIT_URL="https://git.kernel.org/pub/scm/git/git.git" +CONFIG_DEFAULT_KDEVOPS_GITHUB_HTTPS_URL="https://github.com/linux-kdevops/kdevops.git" +CONFIG_DEFAULT_KDEVOPS_GITLAB_HTTPS_URL="https://gitlab.com/linux-kdevops/kdevops.git" +CONFIG_DEFAULT_KDEVOPS_LINUX_HTTPS_URL="https://github.com/linux-kdevops/linux.git" +CONFIG_DEFAULT_QEMU_GITHUB_HTTPS_URL="https://github.com/qemu/qemu.git" +CONFIG_DEFAULT_QEMU_GITHUB_SSH_URL="ssh://git@github.com:qemu/qemu.git" +CONFIG_DEFAULT_QEMU_GITLAB_HTTPS_URL="https://gitlab.com/qemu-project/qemu.git" +CONFIG_DEFAULT_QEMU_GITLAB_SSH_URL="ssh://git@gitlab.com:qemu-project/qemu.git" +CONFIG_DEFAULT_QEMU_JIC23_GITHUB_HTTPS_URL="https://github.com/jic23/qemu.git" +CONFIG_DEFAULT_QEMU_JIC23_GITHUB_SSH_URL="ssh://git@github.com:jic23/qemu.git" +CONFIG_DEFAULT_QEMU_JIC23_GITLAB_HTTPS_URL="https://gitlab.com/jic23/qemu.git" +CONFIG_DEFAULT_QEMU_JIC23_GITLAB_SSH_URL="ssh://git@gitlab.com:jic23/qemu.git" +CONFIG_DEFAULT_LINUX_NEXT_GIT_URL="git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git" +CONFIG_DEFAULT_LINUX_NEXT_HTTPS_URL="https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git" +CONFIG_DEFAULT_LINUX_NEXT_HTTPS_GOOGLE_URL="https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next.git" +CONFIG_DEFAULT_LTP_URL="https://github.com/linux-test-project/ltp.git" +CONFIG_DEFAULT_MCGROF_LINUS_GIT_URL="https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux-next.git" +CONFIG_DEFAULT_MCGROF_LINUS_HTTPS_URL="https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git" +CONFIG_DEFAULT_MCGROF_LINUS_HTTPS_GOOGLE_URL="https://kernel.googlesource.com/pub/scm/linux/kernel/git/mcgrof/linux.git" +CONFIG_DEFAULT_MCGROF_NEXT_GIT_URL="git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux-next.git" +CONFIG_DEFAULT_MCGROF_NEXT_HTTPS_URL="https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux-next.git" +CONFIG_DEFAULT_MCGROF_NEXT_HTTPS_GOOGLE_URL="https://kernel.googlesource.com/pub/scm/linux/kernel/git/mcgrof/linux-next.git" +CONFIG_DEFAULT_NFSTEST_URL="git://git.linux-nfs.org/projects/mora/nfstest.git" +CONFIG_DEFAULT_TORVALDS_GIT_URL="git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git" +CONFIG_DEFAULT_TORVALDS_HTTPS_URL="https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git" +CONFIG_DEFAULT_TORVALDS_HTTPS_GOOGLE_URL="https://kernel.googlesource.com/pub/scm/linux/kernel/git/torvalds/linux.git" +CONFIG_DEFAULT_STABLE_GIT_URL="git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git" +CONFIG_DEFAULT_STABLE_HTTPS_URL="https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git" +CONFIG_DEFAULT_STABLE_HTTPS_GOOGLE_URL="https://kernel.googlesource.com/pub/scm/linux/kernel/git/stable/linux.git" +CONFIG_DEFAULT_VFS_GIT="git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git" +CONFIG_DEFAULT_VFS_HTTPS_GIT="https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git" +CONFIG_DEFAULT_VFS_HTTPS_GOOGLE_URL="https://kernel.googlesource.com/pub/scm/linux/kernel/git/vfs/vfs.git" +CONFIG_DEFAULT_XFSDUMP_URL="https://github.com/linux-kdevops/xfsdump-dev.git" +CONFIG_DEFAULT_XFSPROGS_URL="https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git" +CONFIG_DISTRO_DEBIAN=y +CONFIG_DISTRO_DEBIAN_TRIXIE=y +CONFIG_DISTRO_DEBIAN_HAS_HOP1_SOURCES=y + +# +# Target architecture +# +CONFIG_TARGET_ARCH_X86_64=y +# CONFIG_TARGET_ARCH_ARM64 is not set +# CONFIG_TARGET_ARCH_PPC64LE is not set +# end of Target architecture + +# +# SSH update configuration +# +CONFIG_KDEVOPS_SSH_CONFIG_UPDATE=y +CONFIG_KDEVOPS_SSH_CONFIG="~/.ssh/config" +CONFIG_KDEVOPS_SSH_CONFIG_UPDATE_STRICT=y +CONFIG_KDEVOPS_SSH_CONFIG_UPDATE_BACKUP=y +# end of SSH update configuration + +CONFIG_EXTRA_STORAGE_SUPPORTS_512=y +CONFIG_EXTRA_STORAGE_SUPPORTS_1K=y +CONFIG_EXTRA_STORAGE_SUPPORTS_2K=y +CONFIG_EXTRA_STORAGE_SUPPORTS_4K=y +CONFIG_EXTRA_STORAGE_SUPPORTS_LARGEIO=y +CONFIG_GIT_ALTERNATIVES=y +CONFIG_GIT_LINUX_KDEVOPS_GITHUB=y +# CONFIG_GIT_LINUX_KDEVOPS_GITLAB is not set +# CONFIG_SETUP_POSTFIX_EMAIL_RELAY is not set +# CONFIG_HYPERVISOR_TUNING is not set +CONFIG_ENABLE_LOCAL_LINUX_MIRROR=y +CONFIG_USE_LOCAL_LINUX_MIRROR=y +# CONFIG_INSTALL_LOCAL_LINUX_MIRROR is not set +CONFIG_MIRROR_GIT_WORKS=y +CONFIG_MIRROR_BLKTESTS_URL="https://github.com/osandov/blktests.git" +CONFIG_MIRROR_XFSPROGS_URL="https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git" + +# +# Bring up methods +# +# CONFIG_VAGRANT is not set +CONFIG_GUESTFS=y +# CONFIG_TERRAFORM is not set +# CONFIG_SKIP_BRINGUP is not set +CONFIG_LIBVIRT=y +CONFIG_GUESTFS_HAS_DISTRO_SOURCE_COPY_CAPABILITIES=y +CONFIG_GUESTFS_COPY_SOURCES_FROM_HOST_TO_GUEST=y +CONFIG_GUESTFS_DISTRO_SOURCE_AND_DEST_FILE="/etc/apt/sources.list" +# CONFIG_GUESTFS_FEDORA is not set +CONFIG_GUESTFS_DEBIAN=y +# CONFIG_GUESTFS_DEBIAN_TRIXIE is not set +CONFIG_GUESTFS_DEBIAN_BUSTER=y +CONFIG_GUESTFS_DEBIAN_HOST_UPDATES_GUEST=y +CONFIG_GUESTFS_DEBIAN_IMAGE_NAME="debian-12" +CONFIG_GUESTFS_DEBIAN_BOX_SHORT="debian12" +CONFIG_GUESTFS_DEBIAN_COPY_HOST_SOURCES=y +CONFIG_VIRT_BUILDER_OS_VERSION="debian-12" +# CONFIG_LIBVIRT_INSTALL is not set +# CONFIG_LIBVIRT_CONFIGURE is not set +# CONFIG_LIBVIRT_VERIFY is not set +CONFIG_LIBVIRT_URI_SYSTEM=y +# CONFIG_LIBVIRT_URI_SESSION is not set +# CONFIG_LIBVIRT_URI_CUSTOM is not set +CONFIG_LIBVIRT_URI_PATH="qemu:///system" +CONFIG_LIBVIRT_SYSTEM_URI_PATH="qemu:///system" +CONFIG_LIBVIRT_QEMU_GROUP="libvirt-qemu" +CONFIG_KDEVOPS_STORAGE_POOL_PATH="/xfs1" +CONFIG_QEMU_BIN_PATH="/usr/bin/qemu-system-x86_64" +CONFIG_LIBVIRT_URI="qemu:///system" +CONFIG_LIBVIRT_SYSTEM_URI="qemu:///system" +CONFIG_USE_LIBVIRT_MIRROR=y +# CONFIG_QEMU_BUILD is not set +# CONFIG_QEMU_USE_DEVELOPMENT_VERSION is not set +CONFIG_QEMU_BIN_PATH_LIBVIRT="/usr/bin/qemu-system-x86_64" +CONFIG_QEMU_INSTALL_DIR_LIBVIRT="/usr/bin" +# CONFIG_LIBVIRT_LARGE_CPU is not set +# CONFIG_LIBVIRT_VCPUS_2 is not set +# CONFIG_LIBVIRT_VCPUS_4 is not set +CONFIG_LIBVIRT_VCPUS_8=y +# CONFIG_LIBVIRT_VCPUS_16 is not set +# CONFIG_LIBVIRT_VCPUS_32 is not set +# CONFIG_LIBVIRT_VCPUS_64 is not set +# CONFIG_LIBVIRT_VCPUS_128 is not set +# CONFIG_LIBVIRT_VCPUS_255 is not set +CONFIG_LIBVIRT_VCPUS_COUNT=8 +# CONFIG_LIBVIRT_MEM_2G is not set +# CONFIG_LIBVIRT_MEM_3G is not set +CONFIG_LIBVIRT_MEM_4G=y +# CONFIG_LIBVIRT_MEM_8G is not set +# CONFIG_LIBVIRT_MEM_16G is not set +# CONFIG_LIBVIRT_MEM_32G is not set +CONFIG_LIBVIRT_MEM_MB=4096 +CONFIG_LIBVIRT_MACHINE_TYPE_Q35=y +CONFIG_LIBVIRT_HOST_PASSTHROUGH=y +# CONFIG_LIBVIRT_EXTRA_STORAGE_DRIVE_NVME is not set +CONFIG_LIBVIRT_EXTRA_STORAGE_DRIVE_VIRTIO=y +# CONFIG_LIBVIRT_EXTRA_STORAGE_DRIVE_IDE is not set +CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_512=y +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_1K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_2K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_4K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_8K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_16K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_32K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_64K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_128K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_256K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_512K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_1M is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_2M is not set +CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_512=y +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_1K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_2K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_4K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_8K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_16K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_32K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_64K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_128K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_256K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_512K is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_1M is not set +# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_2M is not set +# CONFIG_LIBVIRT_AIO_MODE_NATIVE is not set +# CONFIG_LIBVIRT_AIO_MODE_THREADS is not set +CONFIG_LIBVIRT_AIO_MODE_IO_URING=y +CONFIG_LIBVIRT_AIO_MODE="io_uring" +CONFIG_LIBVIRT_AIO_CACHE_MODE_NONE=y +# CONFIG_LIBVIRT_AIO_CACHE_MODE_WRITETHROUGH is not set +# CONFIG_LIBVIRT_AIO_CACHE_MODE_WRITEBACK is not set +# CONFIG_LIBVIRT_AIO_CACHE_MODE_DIRECTSYNC is not set +# CONFIG_LIBVIRT_AIO_CACHE_MODE_UNSAFE is not set +CONFIG_LIBVIRT_AIO_CACHE_MODE="none" +# CONFIG_LIBVIRT_EXTRA_DRIVE_FORMAT_QCOW2 is not set +CONFIG_LIBVIRT_EXTRA_DRIVE_FORMAT_RAW=y +CONFIG_LIBVIRT_STORAGE_POOL_PATH_INFER_ADVANCED=y +# CONFIG_LIBVIRT_STORAGE_POOL_PATH_CUSTOM_CWD is not set +# CONFIG_LIBVIRT_STORAGE_POOL_PATH_CUSTOM_DEFAULT_DISTRO is not set +# CONFIG_LIBVIRT_STORAGE_POOL_PATH_CUSTOM_MANUAL is not set +CONFIG_LIBVIRT_STORAGE_POOL_PATH_AUTO="/xfs1" +CONFIG_LIBVIRT_STORAGE_POOL_CREATE=y +CONFIG_LIBVIRT_STORAGE_POOL_NAME="xfs1" +# CONFIG_QEMU_ENABLE_EXTRA_DRIVE_LARGEIO is not set +CONFIG_QEMU_LARGEIO_DRIVE_BASE_SIZE=10240 +CONFIG_QEMU_LARGEIO_COMPAT_SIZE=512 +CONFIG_QEMU_LARGEIO_MAX_POW_LIMIT=12 +# end of Bring up methods + +# +# Bring up goals +# +CONFIG_KDEVOPS_TRY_REFRESH_REPOS=y +# CONFIG_RHEL_REGISTER is not set +CONFIG_KDEVOPS_CUSTOM_YUM_REPOFILE="" +CONFIG_KDEVOPS_TRY_UPDATE_SYSTEMS=y +# CONFIG_KDEVOPS_TRY_INSTALL_KDEV_TOOLS is not set +CONFIG_KDEVOPS_SETUP_RDMA_NONE=y +# CONFIG_KDEVOPS_SETUP_RDMA_SIW is not set +# CONFIG_KDEVOPS_SETUP_RDMA_RXE is not set +# CONFIG_KDEVOPS_SETUP_KTLS is not set +# CONFIG_KDEVOPS_SETUP_NFSD is not set +# CONFIG_KDEVOPS_SETUP_SMBD is not set +# CONFIG_KDEVOPS_SETUP_KRB5 is not set +# CONFIG_KDEVOPS_ENABLE_ISCSI is not set +# end of Bring up goals + +# +# Node sysctl configuration +# +# CONFIG_SYSCTL_TUNING is not set +# end of Node sysctl configuration + +# +# Target workflows +# +CONFIG_WORKFLOWS=y + +# +# Shared workflow configuration +# + +# +# Shared workflow data partition +# +CONFIG_WORKFLOW_DATA_DEVICE_ENABLE_CUSTOM=y +CONFIG_WORKFLOW_DATA_DEVICE_CUSTOM="/dev/vdc" +CONFIG_WORKFLOW_DATA_DEVICE="/dev/disk/by-id/virtio-kdevops0" +CONFIG_WORKFLOW_DATA_PATH="/data" +CONFIG_WORKFLOW_INFER_USER_AND_GROUP=y +# CONFIG_WORKFLOW_DATA_FSTYPE_XFS is not set +# CONFIG_WORKFLOW_DATA_FSTYPE_EXT4 is not set +CONFIG_WORKFLOW_DATA_FSTYPE_BTRFS=y +CONFIG_WORKFLOW_DATA_FSTYPE="btrfs" +CONFIG_WORKFLOW_DATA_LABEL="data" +# end of Shared workflow data partition + +# CONFIG_WORKFLOW_MAKE_CMD_OVERRIDE is not set +# CONFIG_WORKFLOW_KDEVOPS_GIT_CUSTOM is not set +CONFIG_WORKFLOW_KDEVOPS_GIT="git://192.168.122.1/mirror/kdevops.git" +CONFIG_WORKFLOW_KDEVOPS_GIT_VERSION="HEAD" +CONFIG_HAVE_MIRROR_KDEVOPS=y +CONFIG_WORKFLOW_KDEVOPS_GIT_DATA="{{data_path}}/kdevops" +CONFIG_WORKFLOW_KDEVOPS_DIR="{{data_path}}/kdevops" +# CONFIG_WORKFLOW_KOTD_ENABLE is not set +# CONFIG_HAVE_DISTRO_CUSTOM_KOTD_REPO is not set +# end of Shared workflow configuration + +CONFIG_WORKFLOW_LINUX_DISTRO=y +# CONFIG_WORKFLOW_LINUX_CUSTOM is not set +CONFIG_WORKFLOWS_TESTS=y +CONFIG_WORKFLOWS_TESTS_DEMOS=y +# CONFIG_WORKFLOWS_REBOOT_LIMIT is not set +CONFIG_WORKFLOW_HAVE_BLKALGN=y +CONFIG_WORKFLOWS_LINUX_TESTS=y +CONFIG_WORKFLOWS_DEDICATED_WORKFLOW=y +CONFIG_KDEVOPS_WORKFLOW_DEDICATE_FSTESTS=y +# CONFIG_KDEVOPS_WORKFLOW_DEDICATE_BLKTESTS is not set +# CONFIG_KDEVOPS_WORKFLOW_DEDICATE_CXL is not set +# CONFIG_KDEVOPS_WORKFLOW_DEDICATE_PYNFS is not set +# CONFIG_KDEVOPS_WORKFLOW_DEDICATE_SELFTESTS is not set +# CONFIG_KDEVOPS_WORKFLOW_DEDICATE_GITR is not set +# CONFIG_KDEVOPS_WORKFLOW_DEDICATE_LTP is not set +# CONFIG_KDEVOPS_WORKFLOW_DEDICATE_NFSTEST is not set +# CONFIG_KDEVOPS_WORKFLOW_DEDICATE_SYSBENCH is not set +CONFIG_KDEVOPS_WORKFLOW_ENABLE_FSTESTS=y + +# +# Configure and run fstests +# +CONFIG_HAVE_DISTRO_PREFERS_FSTESTS_WATCHDOG=y +CONFIG_HAVE_DISTRO_PREFERS_FSTESTS_WATCHDOG_KILL=y +CONFIG_FSTESTS_GENERATE_SIMPLE_CONFIG_ENABLE=y +CONFIG_FSTESTS_XFS=y +# CONFIG_FSTESTS_BTRFS is not set +# CONFIG_FSTESTS_EXT4 is not set +# CONFIG_FSTESTS_NFS is not set +# CONFIG_FSTESTS_CIFS is not set +# CONFIG_FSTESTS_TMPFS is not set +CONFIG_FSTESTS_FSTYP="xfs" +CONFIG_FSTESTS_TFB_COPY_ENABLE=y +CONFIG_FSTESTS_WATCHDOG=y +CONFIG_FSTESTS_WATCHDOG_CHECK_TIME=5 +CONFIG_FSTESTS_WATCHDOG_MAX_NEW_TEST_TIME=60 +CONFIG_FSTESTS_WATCHDOG_HUNG_MULTIPLIER_LONG_TESTS=10 +CONFIG_FSTESTS_WATCHDOG_HUNG_FAST_TEST_MAX_TIME=5 +CONFIG_FSTESTS_WATCHDOG_KILL_TASKS_ON_HANG=y +# CONFIG_FSTESTS_WATCHDOG_RESET_HUNG_SYSTEMS is not set + +# +# Configure how to test XFS +# +CONFIG_HAVE_DISTRO_XFS_PREFERS_MANUAL=y +# CONFIG_FSTESTS_XFS_BUILD_CUSTOM_XFSPROGS is not set +CONFIG_HAVE_MIRROR_XFSPROGS=y +CONFIG_FSTESTS_XFS_XFSPROGS_DATA="{{data_path}}/xfsprogs" +CONFIG_FSTESTS_XFS_XFSPROGS_INSTALL_PATH="/usr/local/" +# CONFIG_FSTESTS_XFS_BUILD_XFSDUMP is not set +CONFIG_HAVE_MIRROR_XFSDUMP=y +# CONFIG_FSTESTS_XFS_QUOTA_ENABLED is not set +# CONFIG_FSTESTS_XFS_SECTION_LOGDEV_ENABLED is not set +# CONFIG_FSTESTS_XFS_SECTION_RTDEV_ENABLED is not set +CONFIG_FSTESTS_XFS_MANUAL_COVERAGE=y +# CONFIG_FSTESTS_XFS_SECTION_CRC is not set +CONFIG_FSTESTS_XFS_SECTION_NOCRC_ENABLED=y +# CONFIG_FSTESTS_XFS_ENABLE_LBS is not set +# CONFIG_FSTESTS_XFS_SECTION_NOCRC is not set +# CONFIG_FSTESTS_XFS_SECTION_NOCRC_512 is not set +# CONFIG_FSTESTS_XFS_SECTION_NOCRC_1K is not set +# CONFIG_FSTESTS_XFS_SECTION_NOCRC_2K is not set +CONFIG_FSTESTS_XFS_SECTION_NOCRC_4K=y +# CONFIG_FSTESTS_XFS_SECTION_REFLINK_ENABLED is not set +# end of Configure how to test XFS + +CONFIG_HAVE_MIRROR_FSTESTS=y +CONFIG_HAVE_MIRROR_KDEVOPS_FSTESTS=y +CONFIG_FSTESTS_GIT="git://192.168.122.1/mirror/kdevops/fstests.git" +CONFIG_FSTESTS_DATA="{{data_path}}/fstests" +CONFIG_FSTESTS_GIT_VERSION="HEAD" +CONFIG_FSTESTS_DATA_TARGET="/var/lib/xfstests" +# CONFIG_FSTESTS_TESTDEV_UNDER_DEVELOPMENT is not set +CONFIG_FSTESTS_TESTDEV_SPARSEFILE_GENERATION=y +# CONFIG_FSTESTS_TESTDEV_NVME_PARTITION_EUIS is not set +# CONFIG_FSTESTS_TESTDEV_NVME_EUIS is not set +CONFIG_FSTESTS_SPARSE_DEV="/dev/disk/by-id/virtio-kdevops1" +# CONFIG_FSTESTS_SPARSE_XFS is not set +CONFIG_FSTESTS_SPARSE_BTRFS=y +# CONFIG_FSTESTS_SPARSE_EXT4 is not set +CONFIG_FSTESTS_SPARSE_FSTYPE="btrfs" +CONFIG_FSTESTS_SPARSE_LABEL="sparsefiles" +CONFIG_FSTESTS_SPARSE_FILE_PATH="/media/sparsefiles" +CONFIG_FSTESTS_SPARSE_FILE_SIZE="20G" +CONFIG_FSTESTS_SPARSE_FILENAME_PREFIX="sparse-disk" +CONFIG_FSTESTS_TEST_DEV="/dev/loop16" +CONFIG_FSTESTS_TEST_DIR="/media/test" +CONFIG_FSTESTS_SCRATCH_DEV_POOL="/dev/loop5 /dev/loop6 /dev/loop7 /dev/loop8 /dev/loop9 /dev/loop10 /dev/loop11 /dev/loop12" +CONFIG_FSTESTS_SCRATCH_MNT="/media/scratch" +CONFIG_FSTESTS_LOGWRITES_DEV="/dev/loop15" +CONFIG_FSTESTS_SCRATCH_LOGDEV="/dev/loop15" +CONFIG_FSTESTS_SETUP_SYSTEM=y +CONFIG_FSTESTS_RUN_TESTS=y +CONFIG_FSTESTS_RUN_AUTO_GROUP_TESTS=y +CONFIG_FSTESTS_RUN_CUSTOM_GROUP_TESTS="auto" +CONFIG_FSTESTS_EXCLUDE_TEST_GROUPS="" +# CONFIG_FSTESTS_ENABLE_RUN_CUSTOM_TESTS is not set +CONFIG_FSTESTS_RUN_LARGE_DISK_TESTS=y +# CONFIG_FSTESTS_ENABLE_SOAK_DURATION is not set +CONFIG_FSTESTS_SOAK_DURATION=0 +# CONFIG_FSTESTS_ENABLE_JOURNAL is not set +# end of Configure and run fstests + +CONFIG_KDEVOPS_WORKFLOW_GIT_CLONES_KDEVOPS_GIT=y +# end of Target workflows + +# +# Kdevops configuration +# +CONFIG_HAVE_CUSTOM_DISTRO_HOST_PREFIX=y +CONFIG_HAVE_DISTRO_PREFERS_CUSTOM_HOST_PREFIX=y +CONFIG_CUSTOM_DISTRO_HOST_PREFIX="debian12" +CONFIG_KDEVOPS_USE_DISTRO_HOSTS_PREFIX=y +CONFIG_KDEVOPS_HOSTS_PREFIX="debian12" +CONFIG_KDEVOPS_HOST_PREFIX="debian12" +# CONFIG_KDEVOPS_BASELINE_AND_DEV is not set + +# +# Ansible post-bring up provisioning configuration +# +CONFIG_KDEVOPS_PLAYBOOK_DIR="playbooks" +CONFIG_KDEVOPS_ANSIBLE_PROVISION_ENABLE=y +CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK="devconfig.yml" +CONFIG_KDEVOPS_DEVCONFIG_ENABLE=y +CONFIG_KDEVOPS_DEVCONFIG_ENABLE_CONSOLE=y +CONFIG_KDEVOPS_DEVCONFIG_KERNEL_CONSOLE_SETTINGS="console=tty0 console=tty1 console=ttyS0,115200n8" +CONFIG_KDEVOPS_DEVCONFIG_GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --parity=no --stop=1" +CONFIG_KDEVOPS_GRUB_TIMEOUT=2 +CONFIG_KDEVOPS_DEVCONFIG_ENABLE_SYSTEMD_WATCHDOG=y +CONFIG_KDEVOPS_DEVCONFIG_SYSTEMD_WATCHDOG_TIMEOUT_RUNTIME="5min" +CONFIG_KDEVOPS_DEVCONFIG_SYSTEMD_WATCHDOG_TIMEOUT_REBOOT="10min" +CONFIG_KDEVOPS_DEVCONFIG_SYSTEMD_WATCHDOG_TIMEOUT_KEXEC="5min" +CONFIG_KDEVOPS_ANSIBLE_INVENTORY_FILE="hosts" +CONFIG_KDEVOPS_PYTHON_INTERPRETER="/usr/bin/python3" +CONFIG_KDEVOPS_PYTHON_OLD_INTERPRETER="/usr/bin/python2" +# CONFIG_DEVCONFIG_ENABLE_SYSTEMD_JOURNAL_REMOTE is not set +# CONFIG_DEVCONFIG_ENABLE_SYSTEMD_TIMESYNCD is not set +# CONFIG_KDEVOPS_DEVCONFIG_ENABLE_CUSTOM_REPOS is not set +# end of Ansible post-bring up provisioning configuration + +# +# Kernel continous integration configuration +# +CONFIG_KERNEL_CI_DEFAULT_STEADY_STATE_GOAL=100 +CONFIG_KERNEL_CI=y +CONFIG_KERNEL_CI_ENABLE_STEADY_STATE=y +CONFIG_KERNEL_CI_STEADY_STATE_GOAL=100 +CONFIG_KERNEL_CI_STEADY_STATE_INCREMENTAL=y +# CONFIG_KERNEL_CI_EMAIL_REPORT is not set +# end of Kernel continous integration configuration +# end of Kdevops configuration diff --git a/kconfigs/Kconfig.blkalgn b/kconfigs/Kconfig.blkalgn new file mode 100644 index 0000000..4381192 --- /dev/null +++ b/kconfigs/Kconfig.blkalgn @@ -0,0 +1,8 @@ +config ENABLE_BLKALGN + bool "Enable blkalgn installation in targets" + default "n" + help + Enabling blkalgn helps installation and running blkalgn + as part of any suitable workflow. This helps in showing + alignment of IO issued to the NVMe device and provide + write amplification factor of the NVMe SSD diff --git a/kconfigs/workflows/Kconfig.demo b/kconfigs/workflows/Kconfig.demo index 98344f7..6b4f9a5 100644 --- a/kconfigs/workflows/Kconfig.demo +++ b/kconfigs/workflows/Kconfig.demo @@ -1 +1,2 @@ source "workflows/demos/reboot-limit/Kconfig" +source "workflows/demos/blkalgn/Kconfig" diff --git a/playbooks/blkalgn.yml b/playbooks/blkalgn.yml new file mode 100644 index 0000000..b72bd9f --- /dev/null +++ b/playbooks/blkalgn.yml @@ -0,0 +1,4 @@ +--- +- hosts: all + roles: + - role: blkalgn diff --git a/playbooks/demoio.yml b/playbooks/demoio.yml new file mode 100644 index 0000000..b137d34 --- /dev/null +++ b/playbooks/demoio.yml @@ -0,0 +1,4 @@ +--- +- hosts: all + roles: + - role: demoio diff --git a/playbooks/roles/blkalgn/defaults/main.yml b/playbooks/roles/blkalgn/defaults/main.yml new file mode 100644 index 0000000..5805e15 --- /dev/null +++ b/playbooks/roles/blkalgn/defaults/main.yml @@ -0,0 +1,22 @@ +--- +# Build Dependencies +blkalgn_bdepends: + - git + - python-is-python3 + - python3-bpfcc + +# Fetch +blkalgn_repo: https://github.com/dkruces/bcc.git +blkalgn_src: "{{ ansible_user_dir }}/src/bcc/git" +blkalgn_version: lbs + +# Install +blkalgn_destdir: /usr/local/bin + +workflow_have_blkalgn: False +blkalgn_install_path: '/usr/local/bin/blkalgn' + +blkalgn_data: "{{ data_path }}/blkalgn_data" +blkalgn_log: "blkalgn.txt" +blkalgn_pid_file: "/tmp/pid" + diff --git a/playbooks/roles/blkalgn/tasks/debian/bdepends.yml b/playbooks/roles/blkalgn/tasks/debian/bdepends.yml new file mode 100644 index 0000000..424bc67 --- /dev/null +++ b/playbooks/roles/blkalgn/tasks/debian/bdepends.yml @@ -0,0 +1,19 @@ +- name: Provisioning Build Dependencies + become: yes + become_method: sudo + become_user: root + ansible.builtin.apt: + name: "{{ blkalgn_bdepends }}" + update_cache: yes + +- name: Install kernel headers for the kernel + become: yes + become_method: sudo + apt: + pkg: + - linux-headers-{{ansible_kernel}} + state: present + when: '"rc" not in ansible_kernel' + + + diff --git a/playbooks/roles/blkalgn/tasks/demo_workload.yml b/playbooks/roles/blkalgn/tasks/demo_workload.yml new file mode 100644 index 0000000..25e81be --- /dev/null +++ b/playbooks/roles/blkalgn/tasks/demo_workload.yml @@ -0,0 +1,11 @@ +--- +- name: Run Taregt workflow for which blkalgn is to be determined + become: yes + become_method: sudo + shell: "dd if=/dev/vdb of=/dev/vdb bs=1M count=10000 status=progress" + async: 1800 + poll: 1 + register: dd_command + when: 'workflow_have_blkalgn|bool' + + diff --git a/playbooks/roles/blkalgn/tasks/install_blkalgn.yml b/playbooks/roles/blkalgn/tasks/install_blkalgn.yml new file mode 100644 index 0000000..411d65b --- /dev/null +++ b/playbooks/roles/blkalgn/tasks/install_blkalgn.yml @@ -0,0 +1,28 @@ +--- +- name: Build dependencies + import_tasks: debian/bdepends.yml + when: ansible_facts['os_family']|lower == 'debian' and 'workflow_have_blkalgn|bool' + +- name: Fetch Sources + ansible.builtin.git: + dest: "{{ blkalgn_src }}" + repo: "{{ blkalgn_repo }}" + update: yes + version: "{{ blkalgn_version }}" + delay: 5 + register: result + retries: 3 + until: not result.failed + when: 'workflow_have_blkalgn|bool' + +- name: Copy blkalgn to {{ blkalgn_destdir }} + become: yes + become_method: sudo + ansible.builtin.copy: + src: "{{ blkalgn_src }}/tools/blkalgn.py" + dest: "{{ blkalgn_destdir }}/blkalgn" + mode: '0755' + owner: root + group: root + remote_src: yes + when: 'workflow_have_blkalgn|bool' diff --git a/playbooks/roles/blkalgn/tasks/main.yml b/playbooks/roles/blkalgn/tasks/main.yml new file mode 100644 index 0000000..23e8590 --- /dev/null +++ b/playbooks/roles/blkalgn/tasks/main.yml @@ -0,0 +1,108 @@ +--- +- name: Import optional extra_args file + include_vars: "{{ item }}" + ignore_errors: yes + with_first_found: + - files: + - "../extra_vars.yml" + - "../extra_vars.yaml" + - "../extra_vars.json" + skip: true + tags: [ 'vars' ] + +- name: Installation check + become: yes + become_method: sudo + stat: + path: "{{ blkalgn_install_path }}" + register: blkalgn_install_check + when: 'workflow_have_blkalgn|bool' + +- name: Install blkalgn and related tools + ansible.builtin.import_tasks: install_blkalgn.yml + when: + - 'workflow_have_blkalgn|bool' + - 'not blkalgn_install_check.stat.exists' + +- name: Creating data partitions + ansible.builtin.include_role: + name: create_data_partition + tags: [ 'data_partition' ] + when: 'workflow_have_blkalgn|bool' + +- name: Create the blkalgn data collection directory for each host + become: yes + become_method: sudo + file: + path: "{{ blkalgn_data }}/{{ ansible_ssh_host }}" + state: directory + tags: [ 'install', 'first_run' ] + when: 'workflow_have_blkalgn|bool' + +- name: Set the file to collect blkalgn results + become: yes + become_method: sudo + set_fact: + blkalgn_output_file: "{{ blkalgn_data}}/{{ ansible_ssh_host }}/{{ blkalgn_log }}" + tags: [ 'vars' ] + when: 'workflow_have_blkalgn|bool' + +- name: Start blkalgn + become: yes + become_method: sudo + shell: "blkalgn --output {{ blkalgn_output_file }} & echo $! > {{ blkalgn_pid_file }}" + async: 1 + poll: 0 + register: blkalgn_bg + when: 'workflow_have_blkalgn|bool' + +- name: Get pid of blkalgn + become: yes + become_method: sudo + shell: "cat {{blkalgn_pid_file}}" + register: blkalgn_pid + changed_when: false + when: 'workflow_have_blkalgn|bool' + +- name: Run demo workload + ansible.builtin.include_role: + name: demoio + register: demo_command + when: 'workflow_have_blkalgn|bool' + +- name: Stop blkalgn after IO workflow finishes + become: yes + become_method: sudo + shell: "kill -TERM '{{ blkalgn_pid.stdout }}'" + when: + - 'demo_command is succeeded' + - 'workflow_have_blkalgn|bool' + +- name: Set the path where we collect our local reboot-limit results + set_fact: + blkalgn_local_results_dir: "../workflows/demos/blkalgn/results" + tags: [ 'vars' ] + when: 'workflow_have_blkalgn|bool' + + +- name: Create the local results directory + local_action: file path="{{ blkalgn_local_results_dir }}/" state=directory + run_once: true + tags: [ 'first_run' ] + when: 'workflow_have_blkalgn|bool' + +- name: Copy the latest results over when we're done + tags: [ 'copy_results' ] + become: yes + become_flags: 'su - -c' + become_method: sudo + fetch: + src: "{{ item }}" + dest: "{{ blkalgn_local_results_dir }}/{{ item | regex_replace(blkalgn_data | regex_escape()) | regex_replace('^/', '') }}" + flat: yes + with_items: + - "{{ blkalgn_output_file }}" + when: + - 'workflow_have_blkalgn|bool' + + diff --git a/playbooks/roles/demoio/tasks/main.yml b/playbooks/roles/demoio/tasks/main.yml new file mode 100644 index 0000000..1714374 --- /dev/null +++ b/playbooks/roles/demoio/tasks/main.yml @@ -0,0 +1,13 @@ +--- +- include_role: + name: common + when: + - infer_uid_and_group|bool + +- name: Run Taregt workflow for which blkalgn is to be determined + become: yes + become_method: sudo + shell: "dd if={{data_device_custom}} of={{data_device_custom}} bs=1M count=10000 status=progress" + async: 1800 + poll: 1 + register: demo_command diff --git a/workflows/Makefile b/workflows/Makefile index d2b31d1..eae31f1 100644 --- a/workflows/Makefile +++ b/workflows/Makefile @@ -54,6 +54,10 @@ ifeq (y,$(CONFIG_KDEVOPS_WORKFLOW_ENABLE_SYSBENCH)) include workflows/sysbench/Makefile endif # CONFIG_KDEVOPS_WORKFLOW_ENABLE_SYSBENCH == y +ifeq (y,$(CONFIG_WORKFLOW_HAVE_BLKALGN)) +include workflows/demos/blkalgn/Makefile +endif # CONFIG_WORKFLOW_HAVE_BLKALGN == y + ANSIBLE_EXTRA_ARGS += $(WORKFLOW_ARGS) ANSIBLE_EXTRA_ARGS_SEPARATED += $(WORKFLOW_ARGS_SEPARATED) ANSIBLE_EXTRA_ARGS_DIRECT += $(WORKFLOW_ARGS_DIRECT) diff --git a/workflows/common/Makefile b/workflows/common/Makefile index 3636831..b66ba8d 100644 --- a/workflows/common/Makefile +++ b/workflows/common/Makefile @@ -2,6 +2,7 @@ # How we create the partition for the workflow data partition WORKFLOW_DATA_DEVICE:=$(subst ",,$(CONFIG_WORKFLOW_DATA_DEVICE)) +WORKFLOW_DATA_DEVICE_CUSTOM:=$(subst ",,$(CONFIG_WORKFLOW_DATA_DEVICE_CUSTOM)) WORKFLOW_DATA_PATH:=$(subst ",,$(CONFIG_WORKFLOW_DATA_PATH)) WORKFLOW_DATA_FSTYPE:=$(subst ",,$(CONFIG_WORKFLOW_DATA_FSTYPE)) WORKFLOW_DATA_LABEL:=$(subst ",,$(CONFIG_WORKFLOW_DATA_LABEL)) @@ -12,6 +13,7 @@ WORKFLOW_KDEVOPS_GIT_DATA:=$(subst ",,$(CONFIG_WORKFLOW_KDEVOPS_GIT_DATA)) WORKFLOW_KDEVOPS_DIR:=$(subst ",,$(CONFIG_WORKFLOW_KDEVOPS_DIR)) WORKFLOW_ARGS += data_device=$(WORKFLOW_DATA_DEVICE) +WORKFLOW_ARGS += data_device_custom=$(WORKFLOW_DATA_DEVICE_CUSTOM) WORKFLOW_ARGS += data_path=$(WORKFLOW_DATA_PATH) WORKFLOW_ARGS += data_fstype=$(WORKFLOW_DATA_FSTYPE) WORKFLOW_ARGS += data_label=$(WORKFLOW_DATA_LABEL) diff --git a/workflows/demos/blkalgn/Kconfig b/workflows/demos/blkalgn/Kconfig new file mode 100644 index 0000000..6d27acc --- /dev/null +++ b/workflows/demos/blkalgn/Kconfig @@ -0,0 +1,10 @@ +config WORKFLOW_HAVE_BLKALGN + bool "Enable blkalgn tool" + default "n" + help + Enabling this feature helps run blkalgn ebpf tool that records writes to + required storage device and produce histogram representing the block sizes + of the transmitted data and the alignment of the commands. + e.g. sudo blkalgn --disk nvme0n2 --ops Write + It also provides WWAF values for the given SSD. + diff --git a/workflows/demos/blkalgn/Makefile b/workflows/demos/blkalgn/Makefile new file mode 100644 index 0000000..147e85a --- /dev/null +++ b/workflows/demos/blkalgn/Makefile @@ -0,0 +1,14 @@ +# SPDX-License-Identifier: copyleft-next-0.3.1 + +ifeq (y,$(CONFIG_WORKFLOW_HAVE_BLKALGN)) +DYNAMIC_RUNTIME_VARS += , "workflow_have_blkalgn": True +endif + +FSTESTS_BASELINE_EXTRA := + +blkalgn: $(FSTESTS_BASELINE_EXTRA) + $(Q)ansible-playbook $(ANSIBLE_VERBOSE) -l localhost,baseline,dev \ + -f 30 -i hosts playbooks/blkalgn.yml \ + --extra-vars '{ $(DYNAMIC_RUNTIME_VARS) }' + +