From patchwork Mon Jan 16 12:42:25 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amir Goldstein X-Patchwork-Id: 9518637 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B3F1D6020B for ; Mon, 16 Jan 2017 12:42:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9976C28119 for ; Mon, 16 Jan 2017 12:42:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8D76C28364; Mon, 16 Jan 2017 12:42:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F22A828119 for ; Mon, 16 Jan 2017 12:42:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751075AbdAPMmm (ORCPT ); Mon, 16 Jan 2017 07:42:42 -0500 Received: from mail-wm0-f67.google.com ([74.125.82.67]:36340 "EHLO mail-wm0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751106AbdAPMml (ORCPT ); Mon, 16 Jan 2017 07:42:41 -0500 Received: by mail-wm0-f67.google.com with SMTP id r126so30571322wmr.3; Mon, 16 Jan 2017 04:42:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=HSGui7Zj5IRk4+2zU9sIjOvm3/uYX/sEYoM8Ek9k9xY=; b=V7Suf2fzxxlJUNBEHdVBKQYvRRA9xoGA7kCLmezVAMJlvV7mQmoYxD/JyCYHpD4seR i/pX3xEZWV9+Rbyv/o44xV94YASxB+wuHqMPCMLeYuyOF8j3adHdMgbr7l7EK/uuMEpo xUWi9HAM3IwPsmN/QSIcplGwmcbk6oPJbw4fstQ+09/UJ76Zjk1MVOLEHzsheYVCw8WI EdMXbIx6uQbY12oYKeRCrGflSmhBUYIor7BAgIs7+BqZVP16zVBp78SgYA1A8lZ+P4up FRlBaCwH0S/mORDvIyTgEmGaWWBDYTr80MBP9l+Vvs/tp0pxy6NJi49A7ZXoUVXkwRmW 4oFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=HSGui7Zj5IRk4+2zU9sIjOvm3/uYX/sEYoM8Ek9k9xY=; b=GdiNDSZbxnrFghEzdIj9JJZwnLzxECra9fu+VaKzox0CucUckrz+4wavJQKCv4yIEH PGtIE5M2EgcKTro0awszzwghMRGvHDYcjtjqCcWKJjSu2XWa+J0Lel/5oc/t3kV8heZu MBu+ZzgeO+zN82CChe5PXPmKwxw9ctk8728y/XGaoCeo8Qmt3um1IMd1yp/1QujuSaKF suLatnZ2QKCRta4POHFvv/xrl6htxX+DaGXQzhi5dc235FSyZ+9PtaUrCEjypmknSBea oWpZyK0CKZA+9THLmY14wdtD8ij+fe1bNd0TuxfmRQv+VovxKMGA9EQYWn6hB5kD2Qie 7Ypg== X-Gm-Message-State: AIkVDXIdX1QBlUE+2yThmgFmAgDzYU8NZBE/WwpJzQhD1KrmUY9bYzv8ZmdMEaUwj6V6Yw== X-Received: by 10.28.131.132 with SMTP id f126mr1212606wmd.61.1484570559859; Mon, 16 Jan 2017 04:42:39 -0800 (PST) Received: from amir-VirtualBox.ctera.local (bzq-82-81-193-202.cablep.bezeqint.net. [82.81.193.202]) by smtp.gmail.com with ESMTPSA id a13sm28685790wma.0.2017.01.16.04.42.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 16 Jan 2017 04:42:38 -0800 (PST) From: Amir Goldstein To: Eryu Guan Cc: Miklos Szeredi , linux-unionfs@vger.kernel.org, fstests@vger.kernel.org Subject: [PATCH v2] overlay: test concurrent copy up Date: Mon, 16 Jan 2017 14:42:25 +0200 Message-Id: <1484570545-31448-1-git-send-email-amir73il@gmail.com> X-Mailer: git-send-email 2.7.4 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Run 4 process pairs, each competing over copy up of 1K files in 1 directory. One opponent touches all files in the directory and the other truncates all files in the directory. This test does NOT check for concurrent copy up support. It only fails on unexpected errors of any of the touch/truncate operations. The test full output should demonstrate the expected results - for kernel with concurrent copy up support, truncate workers are not delayed by touch workers. As a result, truncate workers will finish their work much sooner than a test run without concurrent copy up support. Signed-off-by: Amir Goldstein --- tests/overlay/021 | 135 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/overlay/021.out | 2 + tests/overlay/group | 1 + 3 files changed, 138 insertions(+) create mode 100755 tests/overlay/021 create mode 100644 tests/overlay/021.out diff --git a/tests/overlay/021 b/tests/overlay/021 new file mode 100755 index 0000000..c79bb01 --- /dev/null +++ b/tests/overlay/021 @@ -0,0 +1,135 @@ +#! /bin/bash +# FS QA Test 021 +# +# Test concurrent copy up +# +#----------------------------------------------------------------------- +# Copyright (C) 2016 CTERA Networks. All Rights Reserved. +# Author: Amir Goldstein +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs overlay +_supported_os Linux +_require_scratch + +# Remove all files from previous tests +_scratch_mkfs + +# overlay copy_up doesn't deal with sparse file well, holes will be filled by +# zeros, so for the worst case of hitting all the copy up bomb files, we need +# (64*(16+32+64+128)M) free space on $SCRATCH_DEV. +# However, triggering more than a total of 16 copy up bombs would require +# really fast data copy (clone up doesn't take up space at all), so let's be +# conservative and reserve space for 16 data copy ups per directory. +_require_fs_space $SCRATCH_DEV $((16*(16+32+64+128)*1024)) + +lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR +mkdir -p $lowerdir + +testdir=arena +d_low=$lowerdir/$testdir +mkdir -p $d_low + +# Create 4K empty files in 4 directories +echo $FSSTRESS_PROG -d $d_low -p 4 -z -f creat=1 -n 1024 -v >> $seqres.full +$FSSTRESS_PROG -d $d_low -p 4 -z -f creat=1 -n 1024 -v >> $seqres.full 2>&1 +echo "--------------------------------------" >> $seqres.full +echo "Created 1K files in lower directory. " >> $seqres.full + +# Plant 64 copy up bombs in each directory +for f in $d_low/p0/*0; do $XFS_IO_PROG -c "truncate 128m" $f ;done +for f in $d_low/p1/*4; do $XFS_IO_PROG -c "truncate 64m" $f ;done +for f in $d_low/p2/*8; do $XFS_IO_PROG -c "truncate 32m" $f ;done +for f in $d_low/p3/*b; do $XFS_IO_PROG -c "truncate 16m" $f ;done +echo "Created 64*4 copy up bombs. " >> $seqres.full +echo "--------------------------------------" >> $seqres.full + +_scratch_mount + +d_top=$SCRATCH_MNT/$testdir + +# +# Run 2 process teams - 4 pairs of rival processes +# Each process pair competing on copy up of 1K files in 1 directory. +# Team 'touch' players touch all files in readdir order. +# Team 'truncate' players truncates all files in numeric (name) order. +# +# If player from team 'touch' reaches a copy up bomb before player +# from team 'truncate' does, the copy up of (sparse) data will delay +# the end of the process pair match. +# +# If copy up of bomb is not concurrent with other copy ups, then +# 'touch' player p0 with the largest copy up bombs will delay players +# of both teams and all matches will take longer. +# +# If copy up is concurrent with copy ups in different directories, +# process pair 3 match will be over first and process pair 0 match +# will be over last. +# +# If copy up of data is concurrent with other copy ups on the same directory, +# then all the 'touch' team players will finish far behind their 'truncate' +# opponenets. +# +# This test doesn't verify any of those conditions, it will only fail +# on unexpected errors of any of the touch/truncate operations. +# The test full output should demonstrate the expected game results, +# as described above and depending on concurrent copy up support in kernel. +# +cd $d_top +echo "--------------------------------------" >> $seqres.full +echo "Go team touch!! " >> $seqres.full +find p0 -type f -print -exec touch {} \; >> $seqres.full & +find p1 -type f -print -exec touch {} \; >> $seqres.full & +find p2 -type f -print -exec touch {} \; >> $seqres.full & +find p3 -type f -print -exec touch {} \; >> $seqres.full & +cd - > /dev/null + +echo "--------------------------------------" >> $seqres.full +echo "Go team truncate!! " >> $seqres.full +# Give team 'touch' a 1 second head start. +# Team 'truncate' players should catch up after few copy up bombs. +sleep 1 +$FSSTRESS_PROG -d $d_top -p 4 -z -f creat=1 -n 1024 -v >> $seqres.full & + +wait %1 %2 %3 %4 %5 + +echo "Silence is golden" +status=0 +exit diff --git a/tests/overlay/021.out b/tests/overlay/021.out new file mode 100644 index 0000000..09f4062 --- /dev/null +++ b/tests/overlay/021.out @@ -0,0 +1,2 @@ +QA output created by 021 +Silence is golden diff --git a/tests/overlay/group b/tests/overlay/group index 684ed45..5bcc25e 100644 --- a/tests/overlay/group +++ b/tests/overlay/group @@ -23,3 +23,4 @@ 018 auto quick copyup 019 auto stress 020 auto quick copyup perms +021 auto quick copyup