From patchwork Fri Apr 23 11:26:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 12220139 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6092C433B4 for ; Fri, 23 Apr 2021 11:26:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 56002613E4 for ; Fri, 23 Apr 2021 11:26:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229479AbhDWL1R (ORCPT ); Fri, 23 Apr 2021 07:27:17 -0400 Received: from esa4.hgst.iphmx.com ([216.71.154.42]:21581 "EHLO esa4.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229928AbhDWL1R (ORCPT ); Fri, 23 Apr 2021 07:27:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1619177201; x=1650713201; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bVK+caetvJJtwehTs2HyCVjZz9IBwItr31BdpSGpynQ=; b=oCkF4TE1J+5xR4pHCpslM9IYljTIICAwV6YdLobmsusiJ/8L9urtbooq Oxgqa7y3kMfLCNea+QCwItEWLf9lgHIhtf9tGNY2aHlbogWZi8goAJQOV VRqQMjZZ4LSrF8csdmkuZHeRRWWUlEKxZ2L/w23HOAnc4hVpbRv2B4Ojv TD9f29yTpgk3Yalr67AHS/BVlCzxgYc715qZ/jcZ14+y9yrEj8hDB4BUU lAMe7g8F3qJqMQ978YUrkyV1iuhqgFC/ntipShXzHTQjFbUKnlA5IkFum Ji0WnsQB5w97MFlTT1U8wpiWueTfdIHUShmKq1hTWdAOJnIjanC1niM4E w==; IronPort-SDR: nZ6UL0H9LwRXZveAAKmiDgw1+V9vyaSXjh6I6haI26+Jo2g7PwH7WV/XcY4vbxs8vTAw6hAFdI AMN69602B8i+ivUNw3aUlzOvEke5AfmurEu9+DXuQ7+lSWfby16ycnyl4JqA3Pps7O3s463EGZ QJsFObDqEcTV1hS5EwZxjVih5FGSryChDZxVsD95PkWZKhgKBYUwE9j65iC3Hhrxfg0Eijodbo XmLeudTSU6SZH6AvJpmMA75MTBLIFoIDxx//y2OxEMPsSv3EbKK3kQ1340jXNS/Bv/TyrU5Inf 3HE= X-IronPort-AV: E=Sophos;i="5.82,245,1613404800"; d="scan'208";a="165365023" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 23 Apr 2021 19:26:41 +0800 IronPort-SDR: 5I/2PFd+qdBVxH/sR0C1GXkWOMloPFCZqrYmLj3siqE4Hx2Q4l3NrImMuYeK7dUHXm1uTTNvdm sNy2vZkgcHEcEa4XuIjogBlUz5y3KOVsAiKEH5dOMWF94YzMc94tITGDN3O4Gr+IYg6msaBt7C y9aMQeR98kRpI4sQKeh1ohK+AryuMvhbDJ7ORRKBA8GLKqp6n5utuNyxXO8SKpqmgGolvyTLGI Mpu/9GMLZTgkKX33S+aBVYBnqh7FL1+ESABoagI58186oxJVBiCY/R636oQPbppiibbvVO78yQ r6tp4H+p8yWfPCi27OYvyrpW Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Apr 2021 04:07:15 -0700 IronPort-SDR: K7Dv7fqpZHAkQEuKOWsf86RNmRJ58aZb9b/jZDPEAF1Jfl39W/9nTxhKSB7JsLVo3aKwswuqZg yuWyebv7uRrL1TNiMhA7NeKX0whAiUkG2gaSJQA8C4fGTL0OzC4I809GL/hND7Cxi+0mPyUVPm 2Cb6Q/wGn56QvWtIyUz94kYUf744qSEsv0y7t6DgztyTsmQojChuyHmxhuRFPi1D3RsVi+AkhU 92KTQFjHq1F2wFYtSmTBR/j6vzt/oWkEVXUOce4qRD6OlvMXJ/hUKkROP/Jek8U8Owg2IMiPww wb8= WDCIronportException: Internal Received: from unknown (HELO redsun60.ssa.fujisawa.hgst.com) ([10.149.66.36]) by uls-op-cesaip02.wdc.com with ESMTP; 23 Apr 2021 04:26:40 -0700 From: Johannes Thumshirn To: Eryu Guan Cc: fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Naohiro Aota Subject: [PATCH 1/4] fstests: add missing checks of fallocate feature Date: Fri, 23 Apr 2021 20:26:31 +0900 Message-Id: <20210423112634.6067-2-johannes.thumshirn@wdc.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210423112634.6067-1-johannes.thumshirn@wdc.com> References: <20210423112634.6067-1-johannes.thumshirn@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Naohiro Aota Many test cases use xfs_io -c 'falloc' but forgot to add _require_xfs_io_command "falloc". This will fail the test case if we run the test case on a file system without fallcoate support e.g. F2FS. While we believe that normal fallocate(mode = 0) is always supported on Linux, it is not true. Fallocate is disabled in several implementations of zoned block support for file systems because the pre-allocated region will break the sequential writing rule. Currently, several test cases unconditionally call fallocate(). Let's add _require_xfs_io_command "falloc" to properly check the feature is supported by a testing file system. Signed-off-by: Naohiro Aota --- tests/btrfs/013 | 1 + tests/btrfs/016 | 1 + tests/btrfs/025 | 1 + tests/btrfs/034 | 1 + tests/btrfs/037 | 1 + tests/btrfs/046 | 1 + tests/btrfs/107 | 1 + tests/ext4/001 | 1 + tests/f2fs/001 | 1 + tests/generic/456 | 1 + tests/xfs/042 | 1 + tests/xfs/114 | 1 + tests/xfs/118 | 1 + tests/xfs/331 | 1 + tests/xfs/341 | 1 + tests/xfs/342 | 1 + tests/xfs/423 | 1 + 17 files changed, 17 insertions(+) diff --git a/tests/btrfs/013 b/tests/btrfs/013 index 9252c82a2076..5e03ed4a4b4b 100755 --- a/tests/btrfs/013 +++ b/tests/btrfs/013 @@ -33,6 +33,7 @@ trap "_cleanup ; exit \$status" 0 1 2 3 15 # real QA test starts here _supported_fs btrfs _require_scratch +_require_xfs_io_command "falloc" rm -f $seqres.full diff --git a/tests/btrfs/016 b/tests/btrfs/016 index 8fd237cbdb64..015ec17f93d6 100755 --- a/tests/btrfs/016 +++ b/tests/btrfs/016 @@ -35,6 +35,7 @@ _supported_fs btrfs _require_test _require_scratch _require_fssum +_require_xfs_io_command "falloc" _scratch_mkfs > /dev/null 2>&1 diff --git a/tests/btrfs/025 b/tests/btrfs/025 index 42cd7cefe825..5c8140552bfb 100755 --- a/tests/btrfs/025 +++ b/tests/btrfs/025 @@ -31,6 +31,7 @@ _cleanup() # real QA test starts here _supported_fs btrfs _require_scratch +_require_xfs_io_command "falloc" rm -f $seqres.full diff --git a/tests/btrfs/034 b/tests/btrfs/034 index bc7a4aae3886..07c84c347d3b 100755 --- a/tests/btrfs/034 +++ b/tests/btrfs/034 @@ -28,6 +28,7 @@ _cleanup() # real QA test starts here _supported_fs btrfs _require_scratch +_require_xfs_io_command "falloc" rm -f $seqres.full diff --git a/tests/btrfs/037 b/tests/btrfs/037 index 1cfaf5be58c8..9ef199a93413 100755 --- a/tests/btrfs/037 +++ b/tests/btrfs/037 @@ -35,6 +35,7 @@ _cleanup() # real QA test starts here _supported_fs btrfs _require_scratch +_require_xfs_io_command "falloc" rm -f $seqres.full diff --git a/tests/btrfs/046 b/tests/btrfs/046 index 882db8eacc4e..a1d82e1cdd54 100755 --- a/tests/btrfs/046 +++ b/tests/btrfs/046 @@ -37,6 +37,7 @@ _cleanup() _supported_fs btrfs _require_test _require_scratch +_require_xfs_io_command "falloc" _require_fssum rm -f $seqres.full diff --git a/tests/btrfs/107 b/tests/btrfs/107 index e57c9dead499..80db5ab9822d 100755 --- a/tests/btrfs/107 +++ b/tests/btrfs/107 @@ -34,6 +34,7 @@ rm -f $seqres.full _supported_fs btrfs _require_scratch +_require_xfs_io_command "falloc" # Use 64K file size to match any sectorsize # And with a unaligned tailing range to ensure it will be at least 2 pages diff --git a/tests/ext4/001 b/tests/ext4/001 index bbb74f1ea5bc..9650303d15b5 100755 --- a/tests/ext4/001 +++ b/tests/ext4/001 @@ -29,6 +29,7 @@ trap "_cleanup ; exit \$status" 0 1 2 3 15 # real QA test starts here _supported_fs ext4 +_require_xfs_io_command "falloc" _require_xfs_io_command "fzero" _require_test diff --git a/tests/f2fs/001 b/tests/f2fs/001 index 98bd2682d60f..0753a09b5576 100755 --- a/tests/f2fs/001 +++ b/tests/f2fs/001 @@ -36,6 +36,7 @@ _cleanup() _supported_fs f2fs _require_scratch +_require_xfs_io_command "falloc" testfile=$SCRATCH_MNT/testfile dummyfile=$SCRATCH_MNT/dummyfile diff --git a/tests/generic/456 b/tests/generic/456 index 2f9df5e5edc4..65667d449dd3 100755 --- a/tests/generic/456 +++ b/tests/generic/456 @@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # real QA test starts here _supported_fs generic _require_scratch +_require_xfs_io_command "falloc" _require_dm_target flakey _require_xfs_io_command "falloc" "-k" _require_xfs_io_command "fzero" diff --git a/tests/xfs/042 b/tests/xfs/042 index b55d642c5170..fcd5181cf590 100755 --- a/tests/xfs/042 +++ b/tests/xfs/042 @@ -31,6 +31,7 @@ trap "_cleanup ; exit \$status" 0 1 2 3 15 # real QA test starts here _supported_fs xfs +_require_xfs_io_command "falloc" _require_scratch diff --git a/tests/xfs/114 b/tests/xfs/114 index b936452461c6..3f5575a61dfb 100755 --- a/tests/xfs/114 +++ b/tests/xfs/114 @@ -32,6 +32,7 @@ _cleanup() _supported_fs xfs _require_test_program "punch-alternating" _require_xfs_scratch_rmapbt +_require_xfs_io_command "falloc" _require_xfs_io_command "fcollapse" _require_xfs_io_command "finsert" diff --git a/tests/xfs/118 b/tests/xfs/118 index 5e23617b39dd..9a431821aa62 100755 --- a/tests/xfs/118 +++ b/tests/xfs/118 @@ -41,6 +41,7 @@ _supported_fs xfs _require_scratch _require_command "$XFS_FSR_PROG" "xfs_fsr" +_require_xfs_io_command "falloc" # 50M _scratch_mkfs_sized $((50 * 1024 * 1024)) >> $seqres.full 2>&1 diff --git a/tests/xfs/331 b/tests/xfs/331 index 4ea54e2a534b..8e92b2e36a8d 100755 --- a/tests/xfs/331 +++ b/tests/xfs/331 @@ -33,6 +33,7 @@ _require_xfs_scratch_rmapbt _require_scratch_reflink _require_xfs_io_command "falloc" _require_test_program "punch-alternating" +_require_xfs_io_command "falloc" rm -f "$seqres.full" diff --git a/tests/xfs/341 b/tests/xfs/341 index e1fbe588d9eb..8bf05087e1ba 100755 --- a/tests/xfs/341 +++ b/tests/xfs/341 @@ -31,6 +31,7 @@ _require_realtime _require_xfs_scratch_rmapbt _require_test_program "punch-alternating" _disable_dmesg_check +_require_xfs_io_command "falloc" rm -f "$seqres.full" diff --git a/tests/xfs/342 b/tests/xfs/342 index 2be5f7698f01..4db222d65fb2 100755 --- a/tests/xfs/342 +++ b/tests/xfs/342 @@ -30,6 +30,7 @@ _supported_fs xfs _require_realtime _require_xfs_scratch_rmapbt _require_test_program "punch-alternating" +_require_xfs_io_command "falloc" rm -f "$seqres.full" diff --git a/tests/xfs/423 b/tests/xfs/423 index 8d51a9a60585..183c9cf5eded 100755 --- a/tests/xfs/423 +++ b/tests/xfs/423 @@ -35,6 +35,7 @@ _cleanup() _supported_fs xfs _require_test_program "punch-alternating" _require_xfs_io_command "scrub" +_require_xfs_io_command "falloc" _require_scratch echo "Format and populate" From patchwork Fri Apr 23 11:26:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 12220141 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE75BC43462 for ; Fri, 23 Apr 2021 11:26:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6C6BB61404 for ; Fri, 23 Apr 2021 11:26:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230463AbhDWL1S (ORCPT ); Fri, 23 Apr 2021 07:27:18 -0400 Received: from esa4.hgst.iphmx.com ([216.71.154.42]:21581 "EHLO esa4.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230072AbhDWL1S (ORCPT ); Fri, 23 Apr 2021 07:27:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1619177202; x=1650713202; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=z60KLyNq2AWMe9YzqvZt4u/Lx64f02p6j1PomXmYUGs=; b=SqgABVnGPxPSDT+YQa8llyIatWqMVuCMuzFCmftg21PR/bjKZ6TVDQWz 8SW3kmKBoH0R6gOVLO/IaSrBZ3S70moCa8LWLyoGDo5KTq3i8iVVjJfS1 6KLmAgVfsPqSQtEJFh2ScrMFk3n2uc8uxxT7WUNWO0O/OY1F/yNsr9Te0 qLPvWk9pWxi3l8Lu8Sp5c14NM0sEvSJCo7LLKr9s9KVNycskwErNdTvAf /dNCttBeSCJrLiAy6cYMak0JhPgF/hcDY1il+KkPbIJ56qxu/X8hN7CiS Bwmhb3ZWknKCu37h8BerpFQstFyHuEsfzBvDswMeTtT1+7oRRporatJPZ g==; IronPort-SDR: cIy/vm5R9dUW6NfHqWCUUZVPSwRZaBD0ehxlFibpuscET9WsVTpThxiypeK+W0uL2xsUYHAq37 C4Ce3XMxlGjgIYpDwH+PYnWjPHAfEia53a35VhGqodFGHexnPAbFPLGlxofW97HLx5MZEYSHFZ GrCfNuZSUcZVgNku0USzzKCE1DZ0Y7O6g8GS8JEK+57SG/mS0kHak7s926ETSbj/9ZeF7e62oO /+3KcbL1Ubb3Xplzo/nIgXKGUg5QqMn4kdUQDJX8jSSxh/K40BdlDApQ8FwoSYPSmZK3qzYWiT Aag= X-IronPort-AV: E=Sophos;i="5.82,245,1613404800"; d="scan'208";a="165365026" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 23 Apr 2021 19:26:42 +0800 IronPort-SDR: 23mLjuL/p6XRqKDS9qqu/5Wx/dLhQ8J6VYRHabKVAxe7ctyUQXLxKVeFOb1xOmVRIwN2UQlKfX xdpqicR2gk1E+eVVJpRKpR7ulsK+qujbMnjtGLMbUIXqZMCm95XJ1r85vmD9oknGW3oSuF6ech LaaNL2KLpu26R33VUbwX/MLQIDMTsPewBn6buTv0eos5r3iJC9UJguxYBgH0eW1xe+VNWtBex4 d5Pomisuh2BdmX5KyG96qv9TC2bRkQ+2iSez/llHxIxvanE+4SZUWBTXWk6Ze1vUNOHBEQHfx5 mEAgIeBgAp0AjJ7fx4S2wm9x Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Apr 2021 04:07:16 -0700 IronPort-SDR: z6NFRIOzId/X6P4Q9cKU9tYVVZkcKpcw7sss4wAgIerFaiKKRqaiZMnxtIZt+TLd/coRBNU87p 0pypJUXW7Yivt0qEPadP1VH4/TqyW482FaveyKT9Y+rJK/q5rfzt+BeB7iMDj6WU/a9pufECmL 9H+gLxDnbpW9pNPvBJefjIdAv41STNPz44fleO1zStlZhxqTRvBLmPf1hNKUZ8Cr5AZ+uvvBKp LGHf5qpfph1nXJiiELNwAC+CWEHRFHdH3GkX89oxCVZNmxHebFstJ9BC9T7J1rdiIms+SLj6aS 0Xg= WDCIronportException: Internal Received: from unknown (HELO redsun60.ssa.fujisawa.hgst.com) ([10.149.66.36]) by uls-op-cesaip02.wdc.com with ESMTP; 23 Apr 2021 04:26:41 -0700 From: Johannes Thumshirn To: Eryu Guan Cc: fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Naohiro Aota , Johannes Thumshirn Subject: [PATCH 2/4] btrfs: require discard functionality from scratch device Date: Fri, 23 Apr 2021 20:26:32 +0900 Message-Id: <20210423112634.6067-3-johannes.thumshirn@wdc.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210423112634.6067-1-johannes.thumshirn@wdc.com> References: <20210423112634.6067-1-johannes.thumshirn@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org Some test cases for btrfs require the scratch device to support discard. Check if the scratch device does support discard before trying to execute the test. Signed-off-by: Johannes Thumshirn --- common/rc | 8 ++++++++ tests/btrfs/116 | 1 + tests/btrfs/156 | 1 + 3 files changed, 10 insertions(+) diff --git a/common/rc b/common/rc index 11ff7635700b..76a7265e23ba 100644 --- a/common/rc +++ b/common/rc @@ -3587,6 +3587,14 @@ _require_batched_discard() $FSTRIM_PROG $1 > /dev/null 2>&1 || _notrun "FITRIM not supported on $1" } +_require_scratch_discard() +{ + local sdev="$(_short_dev $SCRATCH_DEV)" + local discard=$(cat /sys/block/$sdev/queue/discard_granularity) + + [ $discard -gt 0 ] || _notrun "discard not supported" +} + _require_dumpe2fs() { if [ -z "$DUMPE2FS_PROG" ]; then diff --git a/tests/btrfs/116 b/tests/btrfs/116 index 3ed097eccf03..f4db439caef8 100755 --- a/tests/btrfs/116 +++ b/tests/btrfs/116 @@ -29,6 +29,7 @@ _cleanup() # real QA test starts here _supported_fs btrfs _require_scratch +_require_scratch_discard rm -f $seqres.full diff --git a/tests/btrfs/156 b/tests/btrfs/156 index 89c80e7161e2..56206d99c801 100755 --- a/tests/btrfs/156 +++ b/tests/btrfs/156 @@ -42,6 +42,7 @@ rm -f $seqres.full _supported_fs btrfs _require_scratch _require_fstrim +_require_scratch_discard # 1024fs size fs_size=$((1024 * 1024 * 1024)) From patchwork Fri Apr 23 11:26:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 12220143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0220C43600 for ; Fri, 23 Apr 2021 11:26:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 99060600CC for ; Fri, 23 Apr 2021 11:26:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231185AbhDWL1U (ORCPT ); Fri, 23 Apr 2021 07:27:20 -0400 Received: from esa4.hgst.iphmx.com ([216.71.154.42]:21581 "EHLO esa4.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230072AbhDWL1T (ORCPT ); Fri, 23 Apr 2021 07:27:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1619177203; x=1650713203; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=HVL8OW7L0ggpKA0j/PbdNF5sAD6Isv4zfpQMacmVhVc=; b=LUF4XjzcKl33lUMpMry/oEGIRsATymVDoWaU5qk37qsoCLwBe59ebD+8 hdIvrHpBW9bDgXqDfBjtGvrCxpbaSVGAI13xONpjD0GMO5jTG2AgsjGR3 9Jx58iHCkvz/iiZrqvMXLBc6O0+FCgi/DKZmrvGV2XfMsNnnt3aQbdIZH I1xiJboGbT8ipz4LUjhgTI8OwHol9Yg0CdRrKQNHPCLDi7pMD1tkozLRE cpFsox96dnQMkoSSYpA8/noaAKpppWwyr6PhVyRriWppAby6QLeZ6HZVw /quk6ZE+b9wR7JUXZA721Nqx5ezTFzExMauph0VYVMalznokr1Jr8OejX A==; IronPort-SDR: CD3b8p9kWfrvs9hLR9lEuhOk1QdV/QyEbOpyyTXBWT//2/muBtHPuXa5CnDvi0bYcP7kNKL7Wb gVOPlewbIg/mZWbwiJco9TJ7lGjmOsIZya7Vz3LZSQCAm0HDMhdM/+XbekDvyZ693lAT3NBCAy LKjNYQfrBlfL3maRB0v+zEZeBlQm0g3CUWl3UqySiBkTJQT8gSQQ8KHaVArNnTHk16e5Q+Vqbt i+zuY+4+26evRQmc9j9duGSJSOwVHHjXb0Wh0Hise47qeJgzbN4OALHX5YBRsbEwj8DXL9dHXq GhI= X-IronPort-AV: E=Sophos;i="5.82,245,1613404800"; d="scan'208";a="165365027" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 23 Apr 2021 19:26:43 +0800 IronPort-SDR: D/Nq6ZfLfJKl6ps0WxugGtTZerFL6OWvTUKr+MHbuU0ZyVAz7fJd38o47EWdzmI4bCemQ0AbGa OcS5W3cRbTCuKOTD+s82mdoFU1y94vZtldFDXOfXhY19u+znt1ccSwtWS2kRDf118KZdsfxzN5 tO3klh49BpASoJO0JtrLD6TpMzvHAevfxv28QEKgn1Tk6Pvh/ZYCU4SdzihV4k/ZrpCKGmfyHV xdwx+sXlUJrSclU3QgNqCxz+Srnp03OYSwxMx7VF6AOnooKpVZhSpk3dCl/X85W1fxSjGTbsIQ NbFGeagRnvnnYS+50if1/Ggo Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Apr 2021 04:07:17 -0700 IronPort-SDR: XSmQreHo4nP5+/HtmjOipsUjEs3LRCa4OOo97BUIVwXGkTFb85fxEIG7LVONYmxUAyF0er6tZy DFekOSa/IA8vpHji39JjKnmrCysCcS4veIfsVXRi1VKiuTkON16Y3n0Q/pGH++3limhMscHIgV xioirSVpWPNp1FK9kQAenU4duDq/Zyc24hkllbLILprio29bvJ5VZM5GqqGjnNGnXlt8+sClgO 6PscS8XqA39GGXGsKw9miNgPA5tMPnNCblfq8x1yHxb6hzBdX2aTlfvJVvEK4CqLjUx0BlIYyS xlk= WDCIronportException: Internal Received: from unknown (HELO redsun60.ssa.fujisawa.hgst.com) ([10.149.66.36]) by uls-op-cesaip02.wdc.com with ESMTP; 23 Apr 2021 04:26:42 -0700 From: Johannes Thumshirn To: Eryu Guan Cc: fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Naohiro Aota Subject: [PATCH 3/4] common/rc: introduce zone check commands Date: Fri, 23 Apr 2021 20:26:33 +0900 Message-Id: <20210423112634.6067-4-johannes.thumshirn@wdc.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210423112634.6067-1-johannes.thumshirn@wdc.com> References: <20210423112634.6067-1-johannes.thumshirn@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Naohiro Aota Introduce some zone related helper functions: _zone_type(), _require_zoned_device(), and _require_non_zoned_device(). They all take a device path as an argument. _zone_type() return the zone type of the device according to the value returned from "/sys/block//queue/zoned". See Documentation/ABI/testing/sysfs-block for a detail. _require_zoned_device() checks if the device is zoned. If not, it skips the current test. _require_non_zoned_device() does the opposite. Signed-off-by: Naohiro Aota --- common/rc | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/common/rc b/common/rc index 76a7265e23ba..18436e33dcf3 100644 --- a/common/rc +++ b/common/rc @@ -1931,6 +1931,50 @@ _require_dm_target() fi } +_zone_type() +{ + local target=$1 + if [ -z $target ]; then + echo "Usage: _zone_type " + exit 1 + fi + local sdev=`_short_dev $target` + + if [ -e /sys/block/${sdev}/queue/zoned ]; then + cat /sys/block/${sdev}/queue/zoned + else + echo none + fi +} + +_require_zoned_device() +{ + local target=$1 + if [ -z $target ]; then + echo "Usage: _require_zoned_device " + exit 1 + fi + + local type=`_zone_type ${target}` + if [ "${type}" = "none" ]; then + _notrun "this test require zoned block device" + fi +} + +_require_non_zoned_device() +{ + local target=$1 + if [ -z $target ]; then + echo "Usage: _require_non_zoned_device " + exit 1 + fi + + local type=`_zone_type ${target}` + if [ "${type}" != "none" ]; then + _notrun "this test require non-zoned block device" + fi +} + # this test requires the ext4 kernel support crc feature on scratch device # _require_scratch_ext4_crc() From patchwork Fri Apr 23 11:26:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 12220145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5525C4360C for ; Fri, 23 Apr 2021 11:26:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A076B613DE for ; Fri, 23 Apr 2021 11:26:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230072AbhDWL1V (ORCPT ); Fri, 23 Apr 2021 07:27:21 -0400 Received: from esa4.hgst.iphmx.com ([216.71.154.42]:21581 "EHLO esa4.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234057AbhDWL1U (ORCPT ); Fri, 23 Apr 2021 07:27:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1619177204; x=1650713204; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bxiPv/1BNGAdoDbko2VzFXpcz1eDsMxIlem4AaAkOM8=; b=iYY+rSanSEPO5j5NC7g+18RzduVWu6enOt6KxHqV4nHML8Lc/CxGgnDm EfuWgNr0zqv+FklaqjHrz7jLARZUIYfHx8W8TMwb3JwyqKIyZWajJr/uP 54tRw5qmGevx0J6C1qZjQ/6M9b7hy2UoiIOMJ40rcA/mIgfL6ibHCVuIG V91AKkb133X3quvSFgC++rf981yHmBxpiwcWZgfemFZc2ofQJQM/L88I6 ZIRCv+vANdIaeoFcqjgCEgwkMsKrLDmFj0srHovYTD5vVdeJIs/C3iA6l AaeCF+NHDJF5Qgma8BFIeG0PMzhEttw5Ylg8L9kr103lEWSIuclzlrYKn A==; IronPort-SDR: QaylUxn/kiMFy6haPXb+x4fnK3oxrkfbx6HyRNXOkgYoxmXAP4GTNIlDv5I3N3jf8C0azndyiL YmJ5OO7XfATPqcQxQwPRUP/7tPJpCZ9cLroUVz3Xn/SZ5UCcISlsNC8HBQZe/BgY5ZiqBccNeR tvGAPse9Pk4CL2dnnVaM7MuksVKzccmNlEnEhrWq6nnDNwg12BfEPu0guaHDPd6mga0gDnqOog ISgz7Y6jzsK3FLRzHUFRPqbyURgOvSxs2iWhYwS6JF9ZP9BUmhYIXAkX/B6yWp2FuHsVUS9YEV P98= X-IronPort-AV: E=Sophos;i="5.82,245,1613404800"; d="scan'208";a="165365032" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 23 Apr 2021 19:26:44 +0800 IronPort-SDR: toB5FTCOMNoloflTg9Qm/JpxTTpnzWwLe2nJi8b73HvmGJaxjNwmu6FRf+c3/3970okZT9VqK6 Lxt3Cc07D+o7kB1qi3iYk2jKI+vndQixeFA/AAj3QRY8lpeS58Ge0YV/enLRNf4RwGGD186IwH R7RKwTddVWv6kDcPd5KhWUmoKvm477F9kFh/3pW/TbNGJyr2VcubJWhytC0VDAR+W+m8R5b9pU FWyoSs23PbYvBkLBUSnJ/gyfY6QjF35H9Bq3qRhwB6aDTab1lPwgapcSYlYEd/sovEPow1PlTR aKJq6xsJtRwzJ2fkTh1c9NMl Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Apr 2021 04:07:18 -0700 IronPort-SDR: +uWgW7CzCdFOdTjyT1ynXrt2FLLVoaxrSpoFpxE9UOYhzsF4jKpaMZda5NVr5OnBvmN4/Q3xHn HtG+Fj8AcoitDWg3o3031cRkKcjGDlesDt8VpOQvlIxJrvKym04KeucNp1yKMF0EQb+iqdThTw BOOtOG/z+71ELOkczm29pomZpZngWMYJk9v2O/PwLQ1/mcO2vxU8dCZA6X6D3Gq4mgP6cHvSWy 0LGjYtznqV0tJsrj8+0YIskkEByZrkFcyFVInDXLbvPKhNnnJKtL35wjVGdKtyEFKP+BroycTY coI= WDCIronportException: Internal Received: from unknown (HELO redsun60.ssa.fujisawa.hgst.com) ([10.149.66.36]) by uls-op-cesaip02.wdc.com with ESMTP; 23 Apr 2021 04:26:43 -0700 From: Johannes Thumshirn To: Eryu Guan Cc: fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Naohiro Aota , Johannes Thumshirn Subject: [PATCH 4/4] btrfs: add test for zone auto reclaim Date: Fri, 23 Apr 2021 20:26:34 +0900 Message-Id: <20210423112634.6067-5-johannes.thumshirn@wdc.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210423112634.6067-1-johannes.thumshirn@wdc.com> References: <20210423112634.6067-1-johannes.thumshirn@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org Add a test for the patch titled "btrfs: zoned: automatically reclaim zones". This test creates a two file on a newly created FS in a way that when we delete the first one, an auto reclaim process will be triggered by the FS. After the reclaim process, it verifies that the data was moved to another zone and old zone was successfully reset. Signed-off-by: Johannes Thumshirn --- common/config | 1 + tests/btrfs/236 | 102 ++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/236.out | 2 + tests/btrfs/group | 1 + 4 files changed, 106 insertions(+) create mode 100755 tests/btrfs/236 create mode 100644 tests/btrfs/236.out diff --git a/common/config b/common/config index a47e462c7792..1a26934985dd 100644 --- a/common/config +++ b/common/config @@ -226,6 +226,7 @@ export FSVERITY_PROG="$(type -P fsverity)" export OPENSSL_PROG="$(type -P openssl)" export ACCTON_PROG="$(type -P accton)" export E2IMAGE_PROG="$(type -P e2image)" +export BLKZONE_PROG="$(type -P blkzone)" # use 'udevadm settle' or 'udevsettle' to wait for lv to be settled. # newer systems have udevadm command but older systems like RHEL5 don't. diff --git a/tests/btrfs/236 b/tests/btrfs/236 new file mode 100755 index 000000000000..18964699f68e --- /dev/null +++ b/tests/btrfs/236 @@ -0,0 +1,102 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021 Western Digital Corporation. All Rights Reserved. +# +# FS QA Test 236 +# +# Test that zone autoreclaim works as expected, that is: if the dirty +# threashold is exceeded the data gets relocated to new block group and the +# old block group gets deleted. On block group deletion, the underlying device +# zone also needs to be reset. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here + +_supported_fs btrfs +_require_scratch +_require_btrfs_command inspect-internal dump-tree +_require_command "$BLKZONE_PROG" blkzone +_require_zoned_device "$SCRATCH_DEV" + +get_data_bg() +{ + $BTRFS_UTIL_PROG inspect-internal dump-tree -t CHUNK $SCRATCH_DEV |\ + grep -A 1 "CHUNK_ITEM" | grep -B 1 "type DATA" |\ + grep -Eo "CHUNK_ITEM [[:digit:]]+" | cut -d ' ' -f 2 +} + +zonesize=$(cat /sys/block/$(_short_dev $SCRATCH_DEV)/queue/chunk_sectors) +zonesize=$((zonesize << 9)) + +_scratch_mkfs >/dev/null 2>&1 +_scratch_mount -o commit=5 + +uuid=$(findmnt -n -o UUID "$SCRATCH_MNT") +reclaim_threshold=75 +echo $reclaim_threshold > /sys/fs/btrfs/"$uuid"/bg_reclaim_threshold +fill_percent=$((reclaim_threshold + 2)) +rest_percent=$((90 - fill_percent)) # make sure we're not creating a new BG +fill_size=$((zonesize * fill_percent / 100)) +rest=$((zonesize * rest_percent / 100)) + +# step 1, fill FS over $fillsize +$XFS_IO_PROG -fdc "pwrite -W 0 $fill_size" $SCRATCH_MNT/$seq.test1 >> $seqres.full +$XFS_IO_PROG -fdc "pwrite -W 0 $rest" $SCRATCH_MNT/$seq.test2 >> $seqres.full +sleep 5 + +zones_before=$($BLKZONE_PROG report $SCRATCH_DEV | grep -v -e em -e nw | wc -l) +echo "Before reclaim: $zones_before zones open" >> $seqres.full +old_data_zone=$(get_data_bg) +old_data_zone=$((old_data_zone >> 9)) +printf "Old data zone 0x%x\n" $old_data_zone >> $seqres.full + +# step 2, delete the 1st $fill_size sized file to trigger reclaim +rm $SCRATCH_MNT/$seq.test1 +$BTRFS_UTIL_PROG filesystem sync $SCRATCH_MNT +sleep 10 + +# check that we don't have more zones open than before +zones_after=$($BLKZONE_PROG report $SCRATCH_DEV | grep -v -e em -e nw | wc -l) +echo "After reclaim: $zones_after zones open" >> $seqres.full + +# Check that old data zone was reset +old_wptr=$($BLKZONE_PROG report -o $old_data_zone -c 1 $SCRATCH_DEV |\ + grep -Eo "wptr 0x[[:xdigit:]]+" | cut -d ' ' -f 2) +if [ "$old_wptr" != "0x000000" ]; then + _fail "Old wptr still at $old_wptr" +fi + +new_data_zone=$(get_data_bg) +new_data_zone=$((new_data_zone >> 9)) +printf "New data zone 0x%x\n" $new_data_zone >> $seqres.full + +# Check that data was really relocated to a different zone +if [ $old_data_zone -eq $new_data_zone ]; then + _fail "New zone same as old zone" +fi + +# success, all done +echo "Silence is golden" +status=0 +exit diff --git a/tests/btrfs/236.out b/tests/btrfs/236.out new file mode 100644 index 000000000000..b6b6e0cad9a7 --- /dev/null +++ b/tests/btrfs/236.out @@ -0,0 +1,2 @@ +QA output created by 236 +Silence is golden diff --git a/tests/btrfs/group b/tests/btrfs/group index 331dd432fac3..62c9c761e974 100644 --- a/tests/btrfs/group +++ b/tests/btrfs/group @@ -238,3 +238,4 @@ 233 auto quick subvolume 234 auto quick compress rw 235 auto quick send +236 auto quick balance