From patchwork Mon Dec 4 13:25:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13478416 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="q/9HY8Er" Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8788B172B; Mon, 4 Dec 2023 05:25:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701696315; x=1733232315; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=yPvyICIrXgjBgg3g9jnJM6sXA3S2318Kul8tGM07Dsc=; b=q/9HY8ErTVozq4IsvFj7quvSn5PFR0MJzdKB3n63BqvhclvuxZO4mRMj LnAYlyof3NKYMt8tH6VADsmWm40IZZGwBkScz+4e+WDvzXNVDnf6lsZHQ MtpCsbtNHt6u+kTQ3iCN9O6IMjCnx5KSDmgJ+y4v2gdLz5sYDaOnqaLnB 3qVYRiuUqK7lX0GWiXzK7wWBhTV6/FV1ojFs66xkpJHxJM1YYs1ib2qLn NbxC+DHtCJsUAx3/M5tAbT1+A00loHfSOG+xMThszCJNVavNi94coGh7L vjj9CgOdccdIvvcwLgk1d9dAbA8sUjM7RqaGGWolYbq+DJtoqtQCoeisU g==; X-CSE-ConnectionGUID: An3CZmX6SyGnYdqnO1C+AQ== X-CSE-MsgGUID: a6ARp4ZcSyC7E54f5EBSaQ== X-IronPort-AV: E=Sophos;i="6.04,249,1695657600"; d="scan'208";a="3929117" Received: from uls-op-cesaip01.wdc.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 04 Dec 2023 21:25:14 +0800 IronPort-SDR: 5hW71apbeebqQPcQlky6HPzNiapBsQBD0Fto3mCXIk0vh9WU/0Iaj4f7WXWBJdbjVcGnsV9whs s4IdbXkMv8zg== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 04 Dec 2023 04:36:17 -0800 IronPort-SDR: dQj7x7GdHaRVgdbFjUcm+TXI1ZKgn/6F8h03FgGlurQM2uNQgI+3yZpf+Xs5nJAP8By9IosI5h vk8X5a8gacUQ== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 04 Dec 2023 05:25:13 -0800 From: Johannes Thumshirn Date: Mon, 04 Dec 2023 05:25:04 -0800 Subject: [PATCH 1/7] btrfs: add fstest for stripe-tree metadata with 4k write Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231204-btrfs-raid-v1-1-b254eb1bcff8@wdc.com> References: <20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com> In-Reply-To: <20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com> To: Anand Jain Cc: fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701696311; l=4096; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=yPvyICIrXgjBgg3g9jnJM6sXA3S2318Kul8tGM07Dsc=; b=OUQl8hfHkY3E/iUDnShaJnJEWOnZkMDxEm0OAlXG5blcgkYt/BtbWvhRj1OU0SDADGLMPecap MPOtblsQiGdAUNtfb1q8u3IEJohn4TiJfEjiYE5GlvxrD+EsWQVyXZS X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Test a simple 4k write on all RAID profiles currently supported with the raid-stripe-tree. Signed-off-by: Johannes Thumshirn --- tests/btrfs/302 | 55 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/302.out | 58 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 113 insertions(+) diff --git a/tests/btrfs/302 b/tests/btrfs/302 new file mode 100755 index 000000000000..1d6693beff4c --- /dev/null +++ b/tests/btrfs/302 @@ -0,0 +1,55 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved. +# +# FS QA Test 302 +# +# Test on-disk layout of RAID Stripe Tree Metadata +# +. ./common/preamble +_begin_fstest auto quick raid remount volume raid-stripe-tree + +. ./common/filter +. ./common/filter.btrfs + +_supported_fs btrfs +_require_test +_require_btrfs_command inspect-internal dump-tree +_require_btrfs_mkfs_feature "raid-stripe-tree" +_require_scratch_dev_pool 4 +_require_xfs_io_command "pwrite" +_require_xfs_io_command "fsync" +_require_btrfs_fs_feature "raid_stripe_tree" + +test_4k_write() +{ + profile=$1 + ndevs=$2 + + _scratch_dev_pool_get $ndevs + + echo "==== Testing $profile ====" + _scratch_pool_mkfs -d $profile -m $profile + _scratch_mount + + $XFS_IO_PROG -fc "pwrite 0 4k" "$SCRATCH_MNT/foo" | _filter_xfs_io + + _scratch_cycle_mount + md5sum "$SCRATCH_MNT/foo" | _filter_scratch + + _scratch_unmount + + $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\ + _filter_btrfs_version | _filter_stripe_tree + + _scratch_dev_pool_put +} + +echo "= Test basic 4k write =" +test_4k_write raid0 2 +test_4k_write raid1 2 +test_4k_write raid10 4 + +# success, all done +status=0 +exit diff --git a/tests/btrfs/302.out b/tests/btrfs/302.out new file mode 100644 index 000000000000..149630e69501 --- /dev/null +++ b/tests/btrfs/302.out @@ -0,0 +1,58 @@ +QA output created by 302 += Test basic 4k write = +==== Testing raid0 ==== +wrote 4096/4096 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +5fed275e7617a806f94c173746a2a723 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid1 ==== +wrote 4096/4096 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +5fed275e7617a806f94c173746a2a723 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid10 ==== +wrote 4096/4096 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +5fed275e7617a806f94c173746a2a723 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid From patchwork Mon Dec 4 13:25:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13478417 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="h7lCAQEe" Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E3DF3586; Mon, 4 Dec 2023 05:25:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701696317; x=1733232317; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=bta9REt3bHwPEDy1BJE9ZxbKV2feDDxPxrEqVnHL1BQ=; b=h7lCAQEepyGkMM+0aimTbwHkNEzfEtlhkxbf1XE1vGbVZZyBVIustGQx S6NG1R2FLToFC7FvWNJxcUVH0Q3uY+LDEA6Mgz9yyjGW7lPMZEP2FE/6D aOB4R7ueuk7/LGXohSVu9cl5XgLL8F58Y/Eq9I6hnHhYkD49qlD09vo/t u06fwyamfsdv6ONT5sdMSX0ZLoKD+cSUmhDebRKWxmdhxql+zCwMLjA3r LnLXbyBfLfO1bm19kD30mVOiw0C8yvFgPYG4OkFtOYufkMVZrdSxW5euP oTbHnEK7AnC+6wHHDLCK86qtrRRGrt/jz3MbMzucEn4gLlCYX6u6ymZSj A==; X-CSE-ConnectionGUID: HqdNhMTvTYuDhFLkg9Rldg== X-CSE-MsgGUID: JWt/OR1ERYuFl0kfk5w4rA== X-IronPort-AV: E=Sophos;i="6.04,249,1695657600"; d="scan'208";a="3929120" Received: from uls-op-cesaip01.wdc.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 04 Dec 2023 21:25:15 +0800 IronPort-SDR: 9u5bh2ozKl16Cmwbs3kulzg//yS28fRIY1ncIRrA15rSAwjhBOu0PNUJ0DlBGM/zcEBYdTpusW QXnovPxeb9/g== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 04 Dec 2023 04:36:17 -0800 IronPort-SDR: wJLG3GhTqEwks5dQ32ioNrFRKLzNvuAeBz4buMQxqh8PNl6dG+aL9izYcfb0OqG2p4Sejb1rGG 68HtOd+bwu6Q== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 04 Dec 2023 05:25:14 -0800 From: Johannes Thumshirn Date: Mon, 04 Dec 2023 05:25:05 -0800 Subject: [PATCH 2/7] btrfs: add fstest for 8k write spanning two stripes on raid-stripe-tree Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231204-btrfs-raid-v1-2-b254eb1bcff8@wdc.com> References: <20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com> In-Reply-To: <20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com> To: Anand Jain Cc: fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701696311; l=5504; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=bta9REt3bHwPEDy1BJE9ZxbKV2feDDxPxrEqVnHL1BQ=; b=Q5PCrYRy30rIMLKzIDwEJgC0MaXhpQsNmFxgcR1VizSxT5shdo1F2A57vWTOqLwcGsvSmEjYR Zr7bXqYrOsCDjc7BmCptbpdF9N1neKfYEhkyb1BYR8kET9KNvrCcgDH X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Add a test-case writing 8k to a raid-stripe-tree formatted filesystem with one stripe pre-filled to 60k so the 8k are split into a 4k write finishing stripe 1 and a 4k write starting the next stripe. Signed-off-by: Johannes Thumshirn --- tests/btrfs/303 | 59 ++++++++++++++++++++++++++++++++++++++ tests/btrfs/303.out | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 141 insertions(+) diff --git a/tests/btrfs/303 b/tests/btrfs/303 new file mode 100755 index 000000000000..71ed9dffce5a --- /dev/null +++ b/tests/btrfs/303 @@ -0,0 +1,59 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved. +# +# FS QA Test 303 +# +# Test on-disk layout of RAID Stripe Tree Metadata +# +. ./common/preamble +_begin_fstest auto quick raid remount volume raid-stripe-tree + +. ./common/filter +. ./common/filter.btrfs + +_supported_fs btrfs +_require_test +_require_btrfs_command inspect-internal dump-tree +_require_btrfs_mkfs_feature "raid-stripe-tree" +_require_scratch_dev_pool 4 +_require_xfs_io_command "pwrite" +_require_xfs_io_command "fsync" +_require_btrfs_fs_feature "raid_stripe_tree" + +test_8k_new_stripe() +{ + profile=$1 + ndevs=$2 + + _scratch_dev_pool_get $ndevs + + echo "==== Testing $profile ====" + _scratch_pool_mkfs -d $profile -m $profile + _scratch_mount + + # Fill the first stripe up to 64k - 4k + $XFS_IO_PROG -fc "pwrite 0 60k" -c fsync "$SCRATCH_MNT/bar" | _filter_xfs_io + + # The actual 8k write + $XFS_IO_PROG -fc "pwrite 0 8k" "$SCRATCH_MNT/foo" | _filter_xfs_io + + _scratch_cycle_mount + md5sum "$SCRATCH_MNT/foo" | _filter_scratch + + _scratch_unmount + + $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\ + _filter_btrfs_version | _filter_stripe_tree + + _scratch_dev_pool_put +} + +echo "= Test 8k write to a new file so that 4k start a new stripe =" +test_8k_new_stripe raid0 2 +test_8k_new_stripe raid1 2 +test_8k_new_stripe raid10 4 + +# success, all done +status=0 +exit diff --git a/tests/btrfs/303.out b/tests/btrfs/303.out new file mode 100644 index 000000000000..1259d0f4a587 --- /dev/null +++ b/tests/btrfs/303.out @@ -0,0 +1,82 @@ +QA output created by 303 += Test 8k write to a new file so that 4k start a new stripe = +==== Testing raid0 ==== +wrote 61440/61440 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +9d3940adb41dd525e008a847e01b15f4 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 61440) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 2 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid1 ==== +wrote 61440/61440 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +9d3940adb41dd525e008a847e01b15f4 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 61440) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 8192) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid10 ==== +wrote 61440/61440 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +9d3940adb41dd525e008a847e01b15f4 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 61440) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 2 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 3 physical XXXXXXXXX + stripe 1 devid 4 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid From patchwork Mon Dec 4 13:25:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13478419 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="LoZ3iISg" Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2F4D1739; Mon, 4 Dec 2023 05:25:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701696338; x=1733232338; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=JoF+ZfHIjhHZHR/UcIEiH5N4/5754pd9YZRZITs13RE=; b=LoZ3iISgEgffygB0n20ELNL4Fx484UTGzQG+kl8lAusIOuI5JrLDJ1sd GYaekVq47xK/QuUg8rH5h2nYQK+XPhEsuVwxmralePv382y/PmmazJGrm U+jFzt5BRCmE+BkLAhoXsCTMlxppBkWWSjwyK4zMHXpXWh89ABugbq0+R BTvVxU1H03k2NJzKuA7icGkPRixTH/Qq3gG9xKyCWISNDouAI6CB0UQK3 4nJL4kwmaV8CyQOdRLw1vS3y0JqMnZh2K9krapah1TXRvMupbEJRTc63z kzVvOas7jKj2FCGWXrM2EwBRaWdsqng161sTbnIVrrLL0lob88Dp/zpUP A==; X-CSE-ConnectionGUID: KeVkF4JHS9GLGXcpqfsj8g== X-CSE-MsgGUID: Lwhxu+oNQP+GaLejkAxGnQ== X-IronPort-AV: E=Sophos;i="6.04,249,1695657600"; d="scan'208";a="3929122" Received: from uls-op-cesaip01.wdc.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 04 Dec 2023 21:25:15 +0800 IronPort-SDR: 6YqBnzj4nOeFuzmV9AGuFvSxtAGs54KNzFOXiiA/jbu5Xf8X+xaCMDzhg5UwcsjQU4a9phrcB8 bSsk8F3hSU7w== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 04 Dec 2023 04:36:18 -0800 IronPort-SDR: nDzczwv6F+G2iWB+SkEgr3AnnDDUciFBz+riPrXgrJ1hWINvRPXuOHjXn3emOusHBhrYax49cg JjwJTpXPwsDg== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 04 Dec 2023 05:25:15 -0800 From: Johannes Thumshirn Date: Mon, 04 Dec 2023 05:25:06 -0800 Subject: [PATCH 3/7] btrfs: add fstest for writing to a file at an offset with RST Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231204-btrfs-raid-v1-3-b254eb1bcff8@wdc.com> References: <20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com> In-Reply-To: <20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com> To: Anand Jain Cc: fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701696311; l=5139; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=JoF+ZfHIjhHZHR/UcIEiH5N4/5754pd9YZRZITs13RE=; b=nQCaP9GQH76TS07/jWZcmQI+0ZTO5KAV6Ew6+oZpTCt8em6EOZYfn6RRBHyLh6DoPvA4vOliF qip/01DZR+jA9cwIRfNWAnOwAIJ58tsCFZm4MyCHvrcUJ00eR6ZJ0Iu X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Add a fstest writing 4k at offset 64k to a file with one RAID tripe already pre-filled for a raid-stripe-tree formatted file system. Signed-off-by: Johannes Thumshirn --- tests/btrfs/304 | 57 ++++++++++++++++++++++++++++++++++++++++ tests/btrfs/304.out | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 132 insertions(+) diff --git a/tests/btrfs/304 b/tests/btrfs/304 new file mode 100755 index 000000000000..5ef7b2382f7c --- /dev/null +++ b/tests/btrfs/304 @@ -0,0 +1,57 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved. +# +# FS QA Test 304 +# +# Test on-disk layout of RAID Stripe Tree Metadata +# +. ./common/preamble +_begin_fstest auto quick raid remount volume raid-stripe-tree + +. ./common/filter +. ./common/filter.btrfs + +_supported_fs btrfs +_require_test +_require_btrfs_command inspect-internal dump-tree +_require_btrfs_mkfs_feature "raid-stripe-tree" +_require_scratch_dev_pool 4 +_require_xfs_io_command "pwrite" +_require_xfs_io_command "fsync" +_require_btrfs_fs_feature "raid_stripe_tree" + +test_4k_write_64koff() +{ + profile=$1 + ndevs=$2 + + _scratch_dev_pool_get $ndevs + + echo "==== Testing $profile ====" + _scratch_pool_mkfs -d $profile -m $profile + _scratch_mount + + # precondition one stripe + $XFS_IO_PROG -fc "pwrite 0 64k" "$SCRATCH_MNT/bar" | _filter_xfs_io + + $XFS_IO_PROG -fc "pwrite 64k 4k" "$SCRATCH_MNT/foo" | _filter_xfs_io + + _scratch_cycle_mount + md5sum "$SCRATCH_MNT/foo" | _filter_scratch + + _scratch_unmount + + $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\ + _filter_btrfs_version | _filter_stripe_tree + + _scratch_dev_pool_put +} +echo "= Test 4k write to an empty file at offset 64k with one stripe prefilled =" +test_4k_write_64koff raid0 2 +test_4k_write_64koff raid1 2 +test_4k_write_64koff raid10 4 + +# success, all done +status=0 +exit diff --git a/tests/btrfs/304.out b/tests/btrfs/304.out new file mode 100644 index 000000000000..e94deec1ab21 --- /dev/null +++ b/tests/btrfs/304.out @@ -0,0 +1,75 @@ +QA output created by 304 += Test 4k write to an empty file at offset 64k with one stripe prefilled = +==== Testing raid0 ==== +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +381b0e7d72cb4f75286fe2b445e8d92a SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid1 ==== +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +381b0e7d72cb4f75286fe2b445e8d92a SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid10 ==== +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +381b0e7d72cb4f75286fe2b445e8d92a SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 3 physical XXXXXXXXX + stripe 1 devid 4 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid From patchwork Mon Dec 4 13:25:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13478420 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="q1pl2fPP" Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 75A61423B; Mon, 4 Dec 2023 05:25:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701696339; x=1733232339; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=YNpmRMEKHZVnVD0dNtGJOvoR99Yv/K6r9YAe1acPrx4=; b=q1pl2fPP9KMmjiM0g01BtAZCIpYvw10vv7WnxHhZRLwy8haQPYqgJQwY kSShLZ+yYqFEzg6uTgWbNpkd2U2J7b4+2Mjou1CR9TJQcPpXK+HHFgFRh ZYzRf5/Bxdk1RVRSnxCG1p7kLS+s56EUjWnMqoUl1EtK3mUFKuWZ7MGZv nn9hZKFFBWxAdS3z/GSmTtGcZZ0vTlU3PG6ixhWb3x/23LS89yY3uXpul kbu/oCpkCmmlTf7mjW8ENIj0CGxqCcagf6VfUxjskR6YtjBUUlRFBNQer Ftpj5Qivh7sZeQCJq9/xGmx3WyrX6/TrkL/mfmfK1AmtiC7kzl/sAOvUT A==; X-CSE-ConnectionGUID: KAsHWfkfRDGh5TLb0GFIzQ== X-CSE-MsgGUID: hOl9s7ijTYKNmgdEmzp8rg== X-IronPort-AV: E=Sophos;i="6.04,249,1695657600"; d="scan'208";a="3929125" Received: from uls-op-cesaip01.wdc.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 04 Dec 2023 21:25:16 +0800 IronPort-SDR: tCy/lqpa6OgIpQteIqetPqYIOApzNUvuXwVTL7pl4Pds1d4krWPyJOSNLIBPj7HRF+j2XgP/tK +AKLMAnakXGA== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 04 Dec 2023 04:36:19 -0800 IronPort-SDR: RQuNeRdXlKh8fSTrkOYA52+tOKrqDXEPCVXuaTFkoJ9MWC1X597xfwA0sHfSAOvnA8Tdj8XPP3 cqNYqro/3bkQ== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 04 Dec 2023 05:25:15 -0800 From: Johannes Thumshirn Date: Mon, 04 Dec 2023 05:25:07 -0800 Subject: [PATCH 4/7] btrfs: add fstests to write 128k to a RST filesystem Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231204-btrfs-raid-v1-4-b254eb1bcff8@wdc.com> References: <20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com> In-Reply-To: <20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com> To: Anand Jain Cc: fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701696311; l=4439; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=YNpmRMEKHZVnVD0dNtGJOvoR99Yv/K6r9YAe1acPrx4=; b=c1T2qOi/vL0vCYBmCrgS8kchBLS+9YXH3s9aYGCBiZSEiYKx/vn+5qL9xAgwQTcQQ8CtKsZp1 kYUst/fgcJ1A4Y5jnbkcro3t86txpsbvzgjh+/bqVlNqCH9eZwtV1eO X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Add a test writing 128k to a file on an empty filesystem formatted with a raid-stripe-tree. Signed-off-by: Johannes Thumshirn --- tests/btrfs/305 | 55 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/305.out | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 120 insertions(+) diff --git a/tests/btrfs/305 b/tests/btrfs/305 new file mode 100755 index 000000000000..8b84fccd692b --- /dev/null +++ b/tests/btrfs/305 @@ -0,0 +1,55 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved. +# +# FS QA Test 305 +# +# Test on-disk layout of RAID Stripe Tree Metadata +# +. ./common/preamble +_begin_fstest auto quick raid remount volume raid-stripe-tree + +. ./common/filter +. ./common/filter.btrfs + +_supported_fs btrfs +_require_test +_require_btrfs_command inspect-internal dump-tree +_require_btrfs_mkfs_feature "raid-stripe-tree" +_require_scratch_dev_pool 4 +_require_xfs_io_command "pwrite" +_require_xfs_io_command "fsync" +_require_btrfs_fs_feature "raid_stripe_tree" + +test_128k_write() +{ + profile=$1 + ndevs=$2 + + _scratch_dev_pool_get $ndevs + + echo "==== Testing $profile ====" + _scratch_pool_mkfs -d $profile -m $profile + _scratch_mount + + $XFS_IO_PROG -fc "pwrite 0 128k" "$SCRATCH_MNT/foo" | _filter_xfs_io + + _scratch_cycle_mount + md5sum "$SCRATCH_MNT/foo" | _filter_scratch + + _scratch_unmount + + $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\ + _filter_btrfs_version | _filter_stripe_tree + + _scratch_dev_pool_put +} + +echo "= Test 128k write to empty file =" +test_128k_write raid0 2 +test_128k_write raid1 2 +test_128k_write raid10 4 + +# success, all done +status=0 +exit diff --git a/tests/btrfs/305.out b/tests/btrfs/305.out new file mode 100644 index 000000000000..0dbd3beca77f --- /dev/null +++ b/tests/btrfs/305.out @@ -0,0 +1,65 @@ +QA output created by 305 += Test 128k write to empty file = +==== Testing raid0 ==== +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid1 ==== +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 131072) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid10 ==== +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 3 physical XXXXXXXXX + stripe 1 devid 4 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid From patchwork Mon Dec 4 13:25:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13478423 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="J39hg9Kt" Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CD03319BE; Mon, 4 Dec 2023 05:26:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701696361; x=1733232361; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=3gOUQ3Yg/ep6Z862GfpB6Ilddrw1ibOZJS9Kb3XAvk0=; b=J39hg9KtSNBXWtJAaX5bg8cQqBPKqJHk9rVxMPg4c8DHIT9JrQ2B0elt R6XP962kESG4H9JP8d8F7mM6m8U63bD4DPA3eD5BBvzwZEntwCxSr2SN8 hs6Iw2Tr9LZyyiSsZtn//zlMEfjtACutB6K60NgCgX3EdlH9uWgf1u6ZW Bcdx3/wZJUkQnKBEBL0SEgVlh+qRsVxw3aRB7Ioc/TpgSq0CEvDps+B1Z mqLebDm8NcWmPJtWd7+uU3Wdp9Mw4cz84uWx65K0G+MYzg64MfFWEK+kz +wYOGbje9B37YIHF5w8BG2ScP27Jq24WWGDp+71hCK0y5l79ksgbr4CQj Q==; X-CSE-ConnectionGUID: tljxr+g4SKqADWt8FiIGgg== X-CSE-MsgGUID: zvCnSQNET8ejP67SUHzcnw== X-IronPort-AV: E=Sophos;i="6.04,249,1695657600"; d="scan'208";a="3929127" Received: from uls-op-cesaip01.wdc.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 04 Dec 2023 21:25:17 +0800 IronPort-SDR: M/HaP2arbfQQGau5QqCoY6ADcIZ1gqG87n3tcciNHE10WRKnARnV4duQ5lhIPkGMTdVg3OateW XguieZNEcN/g== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 04 Dec 2023 04:36:20 -0800 IronPort-SDR: lIEZ4f1U2ZclIJEdvQlC7LJVYO6JWYtJC1PHfiCllbKEzHZD+JztfTv9bpYexZDT9chlZScV0m GTIJESg6HZpA== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 04 Dec 2023 05:25:16 -0800 From: Johannes Thumshirn Date: Mon, 04 Dec 2023 05:25:08 -0800 Subject: [PATCH 5/7] btrfs: add fstest for overwriting a file partially with RST Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231204-btrfs-raid-v1-5-b254eb1bcff8@wdc.com> References: <20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com> In-Reply-To: <20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com> To: Anand Jain Cc: fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701696311; l=6598; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=3gOUQ3Yg/ep6Z862GfpB6Ilddrw1ibOZJS9Kb3XAvk0=; b=pTnj+8+ktcmjkuDKCXLbcuCnYhtVcuChE65ot5MuDY23WQy7pKLVVwnoZJ4UpZe70NDtAwmx5 J3Zdjg8SpEbDwyHxWCuuMFUTW4RBM1xVox7/PRCVL+wF0n6bHW3gXS5 X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Add a test writing 128k to an empty file with one stripe already pre-filled on-disk. Then overwrite a portion of the file in the middle. Signed-off-by: Johannes Thumshirn --- tests/btrfs/306 | 57 ++++++++++++++++++++++++++++ tests/btrfs/306.out | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 163 insertions(+) diff --git a/tests/btrfs/306 b/tests/btrfs/306 new file mode 100755 index 000000000000..67c691e47215 --- /dev/null +++ b/tests/btrfs/306 @@ -0,0 +1,57 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved. +# +# FS QA Test 306 +# +# Test on-disk layout of RAID Stripe Tree Metadata +# +. ./common/preamble +_begin_fstest auto quick raid remount volume raid-stripe-tree + +. ./common/filter +. ./common/filter.btrfs + +_supported_fs btrfs +_require_test +_require_btrfs_command inspect-internal dump-tree +_require_btrfs_mkfs_feature "raid-stripe-tree" +_require_scratch_dev_pool 4 +_require_xfs_io_command "pwrite" +_require_xfs_io_command "fsync" +_require_btrfs_fs_feature "raid_stripe_tree" + +test_128k_write_overwrite() +{ + profile=$1 + ndevs=$2 + + _scratch_dev_pool_get $ndevs + + echo "==== Testing $profile ====" + _scratch_pool_mkfs -d $profile -m $profile + _scratch_mount + + $XFS_IO_PROG -fc "pwrite -W 0 32k" "$SCRATCH_MNT/bar" | _filter_xfs_io + $XFS_IO_PROG -fc "pwrite -W 0 128k" "$SCRATCH_MNT/foo" | _filter_xfs_io + $XFS_IO_PROG -fc "pwrite -W 64k 8k" "$SCRATCH_MNT/foo" | _filter_xfs_io + + _scratch_cycle_mount + md5sum "$SCRATCH_MNT/foo" | _filter_scratch + + _scratch_unmount + + $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\ + _filter_btrfs_version | _filter_stripe_tree + + _scratch_dev_pool_put +} + +echo "= Test 128k write to empty file with 1st stripe partially prefilled then overwrite =" +test_128k_write_overwrite raid0 2 +test_128k_write_overwrite raid1 2 +test_128k_write_overwrite raid10 4 + +# success, all done +status=0 +exit diff --git a/tests/btrfs/306.out b/tests/btrfs/306.out new file mode 100644 index 000000000000..52f3d68f77ff --- /dev/null +++ b/tests/btrfs/306.out @@ -0,0 +1,106 @@ +QA output created by 306 += Test 128k write to empty file with 1st stripe partially prefilled then overwrite = +==== Testing raid0 ==== +wrote 32768/32768 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 2 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 2 physical XXXXXXXXX + item 3 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 4 key (XXXXXX RAID_STRIPE 8192) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid1 ==== +wrote 32768/32768 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 131072) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 2 key (XXXXXX RAID_STRIPE 8192) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid10 ==== +wrote 32768/32768 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 2 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 3 physical XXXXXXXXX + stripe 1 devid 4 physical XXXXXXXXX + item 3 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 4 key (XXXXXX RAID_STRIPE 8192) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid From patchwork Mon Dec 4 13:25:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13478422 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="PdKSxena" Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 436221BD3; Mon, 4 Dec 2023 05:26:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701696363; x=1733232363; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=0vFVxyKpqAp7DKayL6Ik1UXOmUX6REN6nOAs9orTBSo=; b=PdKSxena9VF4a0IVe0N3XOGlUyvHKNPQqGTCngxpMVgesPjcjuermx+H GpxJ6r95NJ3PWc7WbfNPle6gs97TtohLB7s1lm67uXJPAI+lnW+VPZtWY tGvxuPzR2vD3QxAZlkYGDUeBGNip28V55Mz/RrePKn7RpbbOgm5FJ40+S pgN78tztgtTMzjYfC+4/Np5i+11D1cNr85XnfFIopmO3GebwWEG/nnt9K XvycGki+A1B3UdwUEk54OSSZi4kTXxzbwEAZdAfCQREzENgRc3CjhYOhd /yCNha56zbkT0Q3CtpA1vizlxnxb3D58yKN1N0KKZSWJwSKOitTH8EJZs A==; X-CSE-ConnectionGUID: i6JTjpMCRcWeMM/ZN5ZMRg== X-CSE-MsgGUID: FBmLLc5PRQymz+hyTuSoQA== X-IronPort-AV: E=Sophos;i="6.04,249,1695657600"; d="scan'208";a="3929134" Received: from uls-op-cesaip01.wdc.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 04 Dec 2023 21:25:19 +0800 IronPort-SDR: SlbzCfDgY6rqJF47o4f2h2b5lUJbI/2u43W5X2E+a17ABxi5u42msF2XC0foXCOlCNbLJ+GcPy 02DtHEcpZXDQ== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 04 Dec 2023 04:36:22 -0800 IronPort-SDR: J8BjInL79iaOnoc38YL82nOnQ8UhWuBTBiwW9DPApqst4JwI00/8TDnwmc9vtc8SnMymKL+6TK Zn6u+voJUf6A== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 04 Dec 2023 05:25:17 -0800 From: Johannes Thumshirn Date: Mon, 04 Dec 2023 05:25:09 -0800 Subject: [PATCH 6/7] common: add filter for btrfs raid-stripe dump Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231204-btrfs-raid-v1-6-b254eb1bcff8@wdc.com> References: <20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com> In-Reply-To: <20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com> To: Anand Jain Cc: fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701696311; l=1491; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=0vFVxyKpqAp7DKayL6Ik1UXOmUX6REN6nOAs9orTBSo=; b=Zk65r4w5MZcoUK9r5+7MvpnuHn9bYAWbImFlEfU7zYEzSNK3VDlgywPPDAXNjrdszkbWYyqYU A7O3Y9pg13IBbxtXeWA7zZ8bkei0nmZmoc7Und6fBHNZKBamzk/x5UF X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Signed-off-by: Johannes Thumshirn --- common/filter.btrfs | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/common/filter.btrfs b/common/filter.btrfs index 02c6b92dfa94..2003ba7b7015 100644 --- a/common/filter.btrfs +++ b/common/filter.btrfs @@ -125,5 +125,19 @@ _filter_btrfs_cloner_error() sed -e "s/\(clone failed:\) Operation not supported/\1 Invalid argument/g" } +# filter output of "btrfs inspect-internal dump-tree -t raid-stripe" +_filter_stripe_tree() +{ + sed -E -e "s/leaf [0-9]+ items [0-9]+ free space [0-9]+ generation [0-9]+ owner RAID_STRIPE_TREE/leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE/" \ + -e "s/leaf [0-9]+ flags 0x1\(WRITTEN\) backref revision 1/leaf XXXXXXXXX flags 0x1\(WRITTEN\) backref revision 1/" \ + -e "s/checksum stored [0-9a-f]+/checksum stored /" \ + -e "s/checksum calced [0-9a-f]+/checksum calced /" \ + -e "s/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}//" \ + -e "s/item ([0-9]+) key \([0-9]+ RAID_STRIPE ([0-9]+)\) itemoff [0-9]+ itemsize ([0-9]+)/item \1 key \(XXXXXX RAID_STRIPE \2\) itemoff XXXXX itemsize \3/" \ + -e "s/stripe ([0-9]+) devid ([0-9]+) physical [0-9]+/stripe \1 devid \2 physical XXXXXXXXX/" \ + -e "s/total bytes [0-9]+/total bytes XXXXXXXX/" \ + -e "s/bytes used [0-9]+/bytes used XXXXXX/" +} + # make sure this script returns success /bin/true From patchwork Mon Dec 4 13:25:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13478418 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="jteBBCUp" Received: from esa5.hgst.iphmx.com (esa5.hgst.iphmx.com [216.71.153.144]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BCC2D35B6; Mon, 4 Dec 2023 05:25:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701696319; x=1733232319; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=vH5ll84B3/EfTeO+vugCYebs+7bGkX44oA4L317bX/A=; b=jteBBCUpo/tNEnNMO0PS8xi23CBh6wU3CQrDEgZH4yRj5TIj5PmgWnG0 etqyp16YPMVHPHGoSG7euah2KNDiSS9cvKseAZzy3wX9DhnIf8VCcXZ3x oQbUk8ZOOgYpYCzLAmz3MpHfkL3mWL9FkD578u6cTp3i8p9dLpvHuJXN1 /EVV4DPVeHV0m/dcglta+AzuHtme//QaK+cakn6B6FXoUrCRSlc2hU0l2 IrQ66rOYSEHWlt2syKMZoPU1eUGmz3ckDczeXoPVpxNf2jMOspwnVIYxD H/xsK3nt2qdJn2kbnXNRtYKJtwSBWeCNKH7EE6//BkgQCT5+Qj+amwn2W w==; X-CSE-ConnectionGUID: qdVcmyJLQsye99AlREJovg== X-CSE-MsgGUID: 9BfX78tiTRu1/7wTGF+MFA== X-IronPort-AV: E=Sophos;i="6.04,249,1695657600"; d="scan'208";a="4141615" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 04 Dec 2023 21:25:18 +0800 IronPort-SDR: 1rty2LbmL80pjgaNzr91kShOC2+nMS9z5Pv3HeB2k7rbrWDQyk2k34xg0EJOy5NvzbgVHsbjey 4IkKS0DPNTrQ== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 04 Dec 2023 04:30:41 -0800 IronPort-SDR: vbppo8dpg3t/8RKwne5JR0TMid1XwRJZidvPj6V9QtG8v3rwYyyI+bHaaxk7Hd6qICfW3GgcB0 CIL/LNZQseeA== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 04 Dec 2023 05:25:18 -0800 From: Johannes Thumshirn Date: Mon, 04 Dec 2023 05:25:10 -0800 Subject: [PATCH 7/7] fstests: doc: add new raid-stripe-tree group Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231204-btrfs-raid-v1-7-b254eb1bcff8@wdc.com> References: <20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com> In-Reply-To: <20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com> To: Anand Jain Cc: fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701696311; l=705; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=vH5ll84B3/EfTeO+vugCYebs+7bGkX44oA4L317bX/A=; b=WLJZYxUAvnUaR+04bV5daOaxqgNFd23RJlHvFeFr8IZcoeqT/I+kCbXAQYdyOBAOlKgIiCm8B RCFUc/VGjZpBgkck+sRvxoM/l65XvP7z44tDLyi+/7Uu3Azdiurb2XK X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Add a new test group for testing the raid-stripe-tree feature of btrfs with fstests. Signed-off-by: Johannes Thumshirn --- doc/group-names.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/group-names.txt b/doc/group-names.txt index c3dcca375537..9c1624868518 100644 --- a/doc/group-names.txt +++ b/doc/group-names.txt @@ -94,6 +94,7 @@ punch fallocate FALLOC_FL_PUNCH_HOLE qgroup btrfs qgroup feature quota filesystem usage quotas raid btrfs RAID +raid-stripe-tree btrfs raid-stripe-tree feature read_repair btrfs error correction on read failure realtime XFS realtime volumes recoveryloop crash recovery loops