From patchwork Mon Oct 29 16:50:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jayashree Mohan X-Patchwork-Id: 10659773 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 43915174F for ; Mon, 29 Oct 2018 16:50:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2EB8D29C43 for ; Mon, 29 Oct 2018 16:50:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2C67029CCA; Mon, 29 Oct 2018 16:50:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7AFB429CF1 for ; Mon, 29 Oct 2018 16:50:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727715AbeJ3BkC (ORCPT ); Mon, 29 Oct 2018 21:40:02 -0400 Received: from mail-vk1-f195.google.com ([209.85.221.195]:39989 "EHLO mail-vk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727519AbeJ3BkC (ORCPT ); Mon, 29 Oct 2018 21:40:02 -0400 Received: by mail-vk1-f195.google.com with SMTP id h20-v6so2226982vke.7 for ; Mon, 29 Oct 2018 09:50:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=Qf9Dj0X74v+A0ddVskXkE8lpOM/yRxbJM2p7fVK7pJs=; b=ZGXJq5By66GDKbhTSMABsWcv8fFR63as5072DY8jp5L3aGxB4i2ZFJjJTfRkA9EAlE C1ZvgBi5UVvQai+4MP2/7gM3+CBArl+gIIFj9os/A8e6uiYmXngNuFZsDphf3bhTDY61 EXQPQZIWLGRF/PPAGAzLi4YqqsBsh4bAAlySSkVt9BBgoUhG708jvYc5SFnM1OmWzSx0 ac8wV8AN0zV0al1OpakB5oz6kss526xIoBgEClEU4CFnM60k6NqLe2jjP7SJprrlfP+X kuMMlv6D2i6fzsDehHpmVZzA+tFAQPtI/w3FQZsHK8ifcYyih4AjuB2uB9De47ipmCSN kPMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=Qf9Dj0X74v+A0ddVskXkE8lpOM/yRxbJM2p7fVK7pJs=; b=fSPjbjvAkbCxIfkzyRkTxaDRKhyYsGrTMqxfXAFy1GFSdMRPV2Louf25UB+9hlqSs3 acdTrQ/+CEsRbwiDjrdO0w3Hm47706XGR3RRVg2sge/0E85fE7c/ZLWzWLqET1A87jDO zWC6J/19erDJnt4U5cVxzIizULVG5QBRO3LhmWKUEwzCpZoeRz5XsBOeW62RJkUp9dwm m60Hq0EU94eWEZKSH2TTA8PQQ8vOZprXxM+LLv7Rrq1w/yB8c9k9vPSEiNxWC6b46fSA 66Tezf13xB3w87on/2FbgaOn9PzCa0562UFPdxgydGQXFFbWWoCzamx4DlXDghwPXH00 TT8w== X-Gm-Message-State: AGRZ1gItWwOWV47RMWuAFuWQpHMyvm9ADn2xf+kiNK7xoYynn4fra4mG FNcqMwYElNJJ5hCEl5qHlOdC38xtUp/K1mns1fZWFMiZ X-Google-Smtp-Source: AJdET5e3F+lZ7kkOXf/fUzPHwVrZwDZulEZgtp/PdoaSYSHI39uwFD6T7MpLGlufMVHL0xnZPt3sGCRVi/+VIkDWzGU= X-Received: by 2002:a1f:e9c7:: with SMTP id g190mr2296983vkh.69.1540831836895; Mon, 29 Oct 2018 09:50:36 -0700 (PDT) MIME-Version: 1.0 From: Jayashree Mohan Date: Mon, 29 Oct 2018 11:50:39 -0500 Message-ID: Subject: [PATCH] fstest: CrashMonkey tests ported to xfstest To: fstests Cc: Vijaychidambaram Velayudhan Pillai , "Theodore Ts'o" , Dave Chinner , Amir Goldstein , Eryu Guan Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hi all, As discussed previously, I have ported a few CrashMonkey tests into xfstest test-case for your review. I have written a patch for testing simple "hard link" behaviour. This patch batches 37 test cases generated by CrashMonkey into one xfstest test, all of them testing hard-link behaviour. These 37 tests have been grouped based on the similarities; there are comments in the patch to describe what each group is testing for. On a high-level, we are testing the creation of hard links between files in same directory and files across two directories, while allowing fsync of either the files involved, their parent directories, or unrelated sibling files. We aim to convert the other CrashMonkey tests in the same way, batched by the type of file-system operation we test. This particular test took 10 seconds to run on my VM (Ubuntu 16.04, 2GB RAM, single core) on 100MB scratch partition. I have added per test case timer in this patch, which shows each test case takes about 0.25 seconds (to run, flakey_remount, test and cleanup). This test passes clean on ext4, xfs and F2FS. It will show three failed cases as of kernel 4.16 on btrfs (I think it's not yet patched on 4.19, so you should see the failure on the newest kernel as well). Thinking more about it, none of the CrashMonkey test cases check for resource exhaustion and hence we don't need more than 100MB of scratch device. If the per-test external overhead due to fsck/scrub etc depends on the scratch partition size, we can speed up the CrashMonkey tests by allowing a new device like scratch(much smaller in size - about 100-200MB). Additionally, our tests also do not need fsck to be run at the end. Is there a way to skip performing fsck after each test (if its something configurable in the test file) ? If this sort of patch seems fine to you, I can go ahead and port the other CrashMonkey tests in the same way. After batching, I hope there would be around 10-12 test files like the one attached here. Patch below: --- Thanks, Jayashree Mohan diff --git a/tests/generic/517-link b/tests/generic/517-link new file mode 100755 index 0000000..791b6c0 --- /dev/null +++ b/tests/generic/517-link @@ -0,0 +1,162 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2018 The University of Texas at Austin. All Rights Reserved. +# +# FS QA Test 517-link +# +# Test if we create a hard link to a file and persist either of the files, all the names persist. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + _cleanup_flakey + cd / + rm -f $tmp.* +} + +init_start_time=$(date +%s.%N) +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/dmflakey + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch +_require_dm_target flakey + +# initialize scratch device +_scratch_mkfs >>$seqres.full 2>&1 +_require_metadata_journaling $SCRATCH_DEV +_init_flakey +_mount_flakey +init_end_time=$(date +%s.%N) +init_time_elapsed=$(echo "$init_end_time-$init_start_time" | bc) +echo "Initialization time = $init_time_elapsed" >> $seqres.full + +stat_opt='-c "blocks: %b size: %s inode: %i links: %h"' +test_num=0 +total_time=0 + +cleanup() +{ + rm -rf $SCRATCH_MNT/* + sync +} + +# create a hard link $2 to file $1, and fsync $3, followed by power-cu +test_link() +{ + start_time=$(date +%s.%N) + test_num=$((test_num + 1)) + sibling=0 + before="" + after="" + echo -ne "\n=== Test $test_num : link $1 $2 " | tee -a $seqres.full + + # now execute the workload + mkdir -p "${1%/*}" && mkdir -p "${2%/*}" && touch "$1" + ln $1 $2 + + if [ -z "$3" ] + then + echo -ne " with sync ===\n" + sync + before=`stat "$stat_opt" $1` + else + echo " with fsync $3 ===" + + # If the file being persisted is a sibling, create it first + if [ ! -f $3 ] + then + sibling=1 + touch $3 + fi + + $XFS_IO_PROG -c "fsync" $3 | _filter_xfs_io + + if [ $sibling -ne 1 ] + then + before=`stat "$stat_opt" $1` + fi + fi + + _flakey_drop_and_remount | tee -a $seqres.full + + if [ -f $1 ] + then + after=`stat "$stat_opt" $1` + fi + + if [ "$before" != "$after" ] && [ $sibling -ne 1 ] + then + echo "Before: $before" | tee -a $seqres.full + echo "After: $after" | tee -a $seqres.full + fi + + cleanup + end_time=$(date +%s.%N) + time_elapsed=$(echo "$end_time-$start_time" | bc) + echo " Elapsed time : $time_elapsed" >> $seqres.full + total_time=$(echo "$total_time+$time_elapsed" | bc) + echo " Total time : $total_time" >> $seqres.full +} + +# run the link test for different combinations + +test_start_time=$(date +%s.%N) +# Group 1: Both files within root directory +for file_name in $SCRATCH_MNT $SCRATCH_MNT/foo $SCRATCH_MNT/bar; do + test_link $SCRATCH_MNT/foo $SCRATCH_MNT/bar $file_name +done +test_link $SCRATCH_MNT/foo $SCRATCH_MNT/bar + +# Group 2: Create hard link in a sub directory +for file_name in $SCRATCH_MNT $SCRATCH_MNT/foo $SCRATCH_MNT/bar $SCRATCH_MNT/A $SCRATCH_MNT/A/bar $SCRATCH_MNT/A/foo; do + test_link $SCRATCH_MNT/foo $SCRATCH_MNT/A/bar $file_name +done +test_link $SCRATCH_MNT/foo $SCRATCH_MNT/A/bar + +# Group 3: Create hard link in parent directory +for file_name in $SCRATCH_MNT $SCRATCH_MNT/foo $SCRATCH_MNT/bar $SCRATCH_MNT/A $SCRATCH_MNT/A/bar $SCRATCH_MNT/A/foo; do + test_link $SCRATCH_MNT/A/foo $SCRATCH_MNT/bar $file_name +done +test_link $SCRATCH_MNT/A/foo $SCRATCH_MNT/bar + +# Group 4: Both files within a directory other than root +for file_name in $SCRATCH_MNT $SCRATCH_MNT/A $SCRATCH_MNT/A/bar $SCRATCH_MNT/A/foo; do + test_link $SCRATCH_MNT/A/foo $SCRATCH_MNT/A/bar $file_name +done +test_link $SCRATCH_MNT/A/foo $SCRATCH_MNT/A/bar + +#Group 5: Exercise name reuse : Link file in sub-directory +for file_name in $SCRATCH_MNT $SCRATCH_MNT/foo $SCRATCH_MNT/bar $SCRATCH_MNT/A $SCRATCH_MNT/A/bar $SCRATCH_MNT/A/foo; do + test_link $SCRATCH_MNT/bar $SCRATCH_MNT/A/bar $file_name +done +test_link $SCRATCH_MNT/bar $SCRATCH_MNT/A/bar + +#Group 6: Exercise name reuse : Link file in parent directory +for file_name in $SCRATCH_MNT $SCRATCH_MNT/foo $SCRATCH_MNT/bar $SCRATCH_MNT/A $SCRATCH_MNT/A/bar $SCRATCH_MNT/A/foo; do + test_link $SCRATCH_MNT/A/bar $SCRATCH_MNT/bar $file_name +done +test_link $SCRATCH_MNT/A/bar $SCRATCH_MNT/bar + +test_end_time=$(date +%s.%N) +test_time_elapsed=$(echo "$test_end_time-$test_start_time" | bc) +echo "Test Elapsed time : $test_time_elapsed" >> $seqres.full + +# success, all done +status=0 +exit diff --git a/tests/generic/517-link.out b/tests/generic/517-link.out new file mode 100644 index 0000000..381b644 --- /dev/null +++ b/tests/generic/517-link.out @@ -0,0 +1,75 @@ +QA output created by 517-link + +=== Test 1 : link /mnt/scratch/foo /mnt/scratch/bar with fsync /mnt/scratch === + +=== Test 2 : link /mnt/scratch/foo /mnt/scratch/bar with fsync /mnt/scratch/foo === + +=== Test 3 : link /mnt/scratch/foo /mnt/scratch/bar with fsync /mnt/scratch/bar === + +=== Test 4 : link /mnt/scratch/foo /mnt/scratch/bar with sync === + +=== Test 5 : link /mnt/scratch/foo /mnt/scratch/A/bar with fsync /mnt/scratch === + +=== Test 6 : link /mnt/scratch/foo /mnt/scratch/A/bar with fsync /mnt/scratch/foo === + +=== Test 7 : link /mnt/scratch/foo /mnt/scratch/A/bar with fsync /mnt/scratch/bar === + +=== Test 8 : link /mnt/scratch/foo /mnt/scratch/A/bar with fsync /mnt/scratch/A === + +=== Test 9 : link /mnt/scratch/foo /mnt/scratch/A/bar with fsync /mnt/scratch/A/bar === + +=== Test 10 : link /mnt/scratch/foo /mnt/scratch/A/bar with fsync /mnt/scratch/A/foo === + +=== Test 11 : link /mnt/scratch/foo /mnt/scratch/A/bar with sync === + +=== Test 12 : link /mnt/scratch/A/foo /mnt/scratch/bar with fsync /mnt/scratch === + +=== Test 13 : link /mnt/scratch/A/foo /mnt/scratch/bar with fsync /mnt/scratch/foo === + +=== Test 14 : link /mnt/scratch/A/foo /mnt/scratch/bar with fsync /mnt/scratch/bar === + +=== Test 15 : link /mnt/scratch/A/foo /mnt/scratch/bar with fsync /mnt/scratch/A === + +=== Test 16 : link /mnt/scratch/A/foo /mnt/scratch/bar with fsync /mnt/scratch/A/bar === + +=== Test 17 : link /mnt/scratch/A/foo /mnt/scratch/bar with fsync /mnt/scratch/A/foo === + +=== Test 18 : link /mnt/scratch/A/foo /mnt/scratch/bar with sync === + +=== Test 19 : link /mnt/scratch/A/foo /mnt/scratch/A/bar with fsync /mnt/scratch === + +=== Test 20 : link /mnt/scratch/A/foo /mnt/scratch/A/bar with fsync /mnt/scratch/A === + +=== Test 21 : link /mnt/scratch/A/foo /mnt/scratch/A/bar with fsync /mnt/scratch/A/bar === + +=== Test 22 : link /mnt/scratch/A/foo /mnt/scratch/A/bar with fsync /mnt/scratch/A/foo === + +=== Test 23 : link /mnt/scratch/A/foo /mnt/scratch/A/bar with sync === + +=== Test 24 : link /mnt/scratch/bar /mnt/scratch/A/bar with fsync /mnt/scratch === + +=== Test 25 : link /mnt/scratch/bar /mnt/scratch/A/bar with fsync /mnt/scratch/foo === + +=== Test 26 : link /mnt/scratch/bar /mnt/scratch/A/bar with fsync /mnt/scratch/bar === + +=== Test 27 : link /mnt/scratch/bar /mnt/scratch/A/bar with fsync /mnt/scratch/A === + +=== Test 28 : link /mnt/scratch/bar /mnt/scratch/A/bar with fsync /mnt/scratch/A/bar === + +=== Test 29 : link /mnt/scratch/bar /mnt/scratch/A/bar with fsync /mnt/scratch/A/foo === + +=== Test 30 : link /mnt/scratch/bar /mnt/scratch/A/bar with sync === + +=== Test 31 : link /mnt/scratch/A/bar /mnt/scratch/bar with fsync /mnt/scratch === + +=== Test 32 : link /mnt/scratch/A/bar /mnt/scratch/bar with fsync /mnt/scratch/foo === + +=== Test 33 : link /mnt/scratch/A/bar /mnt/scratch/bar with fsync /mnt/scratch/bar === + +=== Test 34 : link /mnt/scratch/A/bar /mnt/scratch/bar with fsync /mnt/scratch/A === + +=== Test 35 : link /mnt/scratch/A/bar /mnt/scratch/bar with fsync /mnt/scratch/A/bar === + +=== Test 36 : link /mnt/scratch/A/bar /mnt/scratch/bar with fsync /mnt/scratch/A/foo === + +=== Test 37 : link /mnt/scratch/A/bar /mnt/scratch/bar with sync === diff --git a/tests/generic/group b/tests/generic/group index 47de978..dc04152 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -519,3 +519,4 @@ 514 auto quick clone 515 auto quick clone 516 auto quick dedupe clone +517-link crash