From patchwork Wed Aug 23 15:49:17 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amir Goldstein X-Patchwork-Id: 9917671 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A2CA0603FA for ; Wed, 23 Aug 2017 15:49:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 94B47285E6 for ; Wed, 23 Aug 2017 15:49:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 897FF285FD; Wed, 23 Aug 2017 15:49:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F3585285E6 for ; Wed, 23 Aug 2017 15:49:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932323AbdHWPtJ (ORCPT ); Wed, 23 Aug 2017 11:49:09 -0400 Received: from mail-wr0-f194.google.com ([209.85.128.194]:38032 "EHLO mail-wr0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932320AbdHWPtE (ORCPT ); Wed, 23 Aug 2017 11:49:04 -0400 Received: by mail-wr0-f194.google.com with SMTP id k10so305541wre.5; Wed, 23 Aug 2017 08:49:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=qv00C0Q3aKF5EnTdwIcbajmWAyHgbi8mwyNMjTPGW0E=; b=hejSPKpm+6/H0ZYeXLbxjcSNDbjoXN6Ml5CQ4y1Fw8UZ6XKIoJB2AJ9UimOzOPerLT DUN1UGG0S2tMTpxV9maQQyz43Z2lbgAN/MiF3CNR0yE5J3hVlebuvw0IU1sVLQDcvyJO upb5lu3eGhxcDyUpntwakatG6HjkRSTLpIIzBzvXTctIGM1ityllExlxlNmuGnEPgym2 067/NvArRy0BjGB1eZ+qRTzBtCjW1ExvyTC7QCQqQWzZjHu5lQCTCz09Ru6XiCqKhEix /1vChEb1uK650GcL4fBLuOTQWHTS3WDksTE65SZy8t7atrH6ULNvQaCLKggUAZq6UZyf GmtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=qv00C0Q3aKF5EnTdwIcbajmWAyHgbi8mwyNMjTPGW0E=; b=ifWjM7pg0z3HgfjRPOWG75Fm1tsfLrYYuHrJT8K/FgFLnlcNjVjzPz1fwJf886AxB4 CNqJW7rJK+kvyqxOmi90nO9M1R70o7ufKmXFshB7wt/n/vH8AxhpM5dey9NqcLsEvP8C /1y9muD2diH4Pu22M5jEcnQ1u5ZlMZ+A47xZ4KxhiCSOGsNsJB9kybaXRhD+aPAELgkz tWXU2GRTGlvWLGf5SfGN0UIB2x7m8P8iIWW6Xtl9tVOj3H/OPzNZpWVhwzQ4qtQ6phI1 oKZLW6Kef/+plyjqJU1ayCPgcHoVwWat7S8HjHGondqJDarUS5zGOaQmPTiNC3gEv/hC 6nRA== X-Gm-Message-State: AHYfb5h92gseBAHjYAdQiE/Y2Lyg7LMaYNAoGwUsOl/FJ8E7ohw8nVa8 Fbx6yPnwDTe0PA== X-Received: by 10.223.144.103 with SMTP id h94mr1841265wrh.216.1503503342713; Wed, 23 Aug 2017 08:49:02 -0700 (PDT) Received: from localhost.localdomain (bzq-79-182-137-119.red.bezeqint.net. [79.182.137.119]) by smtp.gmail.com with ESMTPSA id a6sm1938879wrh.51.2017.08.23.08.49.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 23 Aug 2017 08:49:01 -0700 (PDT) From: Amir Goldstein To: Eryu Guan Cc: Josef Bacik , Christoph Hellwig , fstests@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: [RFC][PATCH 8/8] fstests: add crash consistency fsx test using dm-log-writes Date: Wed, 23 Aug 2017 18:49:17 +0300 Message-Id: <1503503357-26234-9-git-send-email-amir73il@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1503503357-26234-1-git-send-email-amir73il@gmail.com> References: <1503503357-26234-1-git-send-email-amir73il@gmail.com> Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP DO NOT MERGE!!! this test fails most likely due to test bug. The random seed values in this patch fail the test consistently on ext4 always with the same fsck error. btrfs also fails, but with slightly different fsck errors each run. xfs fails sometimes on file checksum error. Cherry-picked the test from commit 70d41e17164b in Josef Bacik's fstests tree (https://github.com/josefbacik/fstests). Quoting from Josef's commit message: The test just runs some ops and exits, then finds all of the good buffers in the directory we provided and: - replays up to the mark given - mounts the file system and compares the md5sum - unmounts and fsck's to check for metadata integrity dm-log-writes will pretend to do discard and the replay-log tool will replay it properly depending on the underlying device, either by writing 0's or actually calling the discard ioctl, so I've enabled discard in the test for maximum fun. [Amir:] - Removed unneeded _test_falloc_support dynamic FSX_OPTS - Added place holders for using constant random seeds - Add test to new 'replay' group Cc: Josef Bacik Signed-off-by: Amir Goldstein --- tests/generic/500 | 128 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/500.out | 2 + tests/generic/group | 1 + 3 files changed, 131 insertions(+) create mode 100755 tests/generic/500 create mode 100644 tests/generic/500.out diff --git a/tests/generic/500 b/tests/generic/500 new file mode 100755 index 0000000..b56eab1 --- /dev/null +++ b/tests/generic/500 @@ -0,0 +1,128 @@ +#! /bin/bash +# FS QA Test No. 500 +# +# Run fsx with log writes to verify power fail safeness. +# +#----------------------------------------------------------------------- +# Copyright (c) 2015 Facebook. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_log_writes +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/dmlogwrites + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch_nocheck +_require_log_writes + +rm -f $seqres.full +rm -rf $TEST_DIR/fsxtests + +check_files() +{ + local _name=$1 + + # Now look for our files + for i in $(find $SANITY_DIR -type f | grep $_name | grep mark) + do + local filename=$(basename $i) + local mark="${filename##*.}" + local expected_size=`_ls_l -h $i | awk '{ print $5 }'` + echo "checking $filename ($expected_size)" >> $seqres.full + _replay_log $filename + _scratch_mount + local expected_md5=$(md5sum $i | cut -f 1 -d ' ') + local md5=$(md5sum $SCRATCH_MNT/$_name | cut -f 1 -d ' ') + local size=`_ls_l -h $SCRATCH_MNT/$_name | awk '{ print $5 }'` + [ "${md5}x" != "${expected_md5}x" ] && _fatal "$filename ($size) md5sum mismatched" + _scratch_unmount + _check_scratch_fs + done +} + +SANITY_DIR=$TEST_DIR/fsxtests +mkdir $SANITY_DIR + +# Create the log +_init_log_writes + +_log_writes_mkfs >> $seqres.full 2>&1 + +# Log writes emulates discard support, turn it on for maximum crying. +_mount_log_writes -o discard + +FSX_OPTS="-N 300 -P $SANITY_DIR -i $LOGWRITES_DMDEV" +# Set random seeds for fsx runs (0 for timestamp + pid) +S1=2885 +S2=2886 +S3=2887 +S4=2888 +# Run fsx for a while +run_check $here/ltp/fsx $FSX_OPTS -S $S1 $SCRATCH_MNT/testfile1 & +run_check $here/ltp/fsx $FSX_OPTS -S $S2 $SCRATCH_MNT/testfile2 & +run_check $here/ltp/fsx $FSX_OPTS -S $S3 $SCRATCH_MNT/testfile3 & +run_check $here/ltp/fsx $FSX_OPTS -S $S4 $SCRATCH_MNT/testfile4 & +wait +test1_md5=$(md5sum $SCRATCH_MNT/testfile1 | cut -f 1 -d ' ') +test2_md5=$(md5sum $SCRATCH_MNT/testfile2 | cut -f 1 -d ' ') +test3_md5=$(md5sum $SCRATCH_MNT/testfile3 | cut -f 1 -d ' ') +test4_md5=$(md5sum $SCRATCH_MNT/testfile4 | cut -f 1 -d ' ') + +# Unmount the scratch dir and tear down the log writes target +_unmount_log_writes +_log_writes_mark end +_log_writes_remove + +for i in testfile1 testfile2 testfile3 testfile4 +do + check_files $i +done + +# Check the end +_replay_log end +_scratch_mount +md5=$(md5sum $SCRATCH_MNT/testfile1 | cut -f 1 -d ' ') +[ "${md5}x" != "${test1_md5}x" ] && _fatal "testfile1 end md5sum mismatched" +md5=$(md5sum $SCRATCH_MNT/testfile2 | cut -f 1 -d ' ') +[ "${md5}x" != "${test2_md5}x" ] && _fatal "testfile2 end md5sum mismatched" +md5=$(md5sum $SCRATCH_MNT/testfile3 | cut -f 1 -d ' ') +[ "${md5}x" != "${test3_md5}x" ] && _fatal "testfile3 end md5sum mismatched" +md5=$(md5sum $SCRATCH_MNT/testfile4 | cut -f 1 -d ' ') +[ "${md5}x" != "${test4_md5}x" ] && _fatal "testfile4 end md5sum mismatched" +_scratch_unmount +_check_scratch_fs + +echo "Silence is golden" +status=0 +exit + diff --git a/tests/generic/500.out b/tests/generic/500.out new file mode 100644 index 0000000..883b2ca --- /dev/null +++ b/tests/generic/500.out @@ -0,0 +1,2 @@ +QA output created by 500 +Silence is golden diff --git a/tests/generic/group b/tests/generic/group index 044ec3f..2396b72 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -453,3 +453,4 @@ 448 auto quick rw 449 auto quick acl enospc 450 auto quick rw +500 auto log replay