From patchwork Mon Sep 4 09:38:19 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eryu Guan X-Patchwork-Id: 9936899 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9301660237 for ; Mon, 4 Sep 2017 09:38:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 81778287A4 for ; Mon, 4 Sep 2017 09:38:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7628C287A7; Mon, 4 Sep 2017 09:38:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E75BD287A4 for ; Mon, 4 Sep 2017 09:38:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753476AbdIDJig (ORCPT ); Mon, 4 Sep 2017 05:38:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57402 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753364AbdIDJif (ORCPT ); Mon, 4 Sep 2017 05:38:35 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 82668C047B69; Mon, 4 Sep 2017 09:38:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 82668C047B69 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=eguan@redhat.com Received: from localhost (dhcp-12-147.nay.redhat.com [10.66.12.147]) by smtp.corp.redhat.com (Postfix) with ESMTP id C39217F6B4; Mon, 4 Sep 2017 09:38:34 +0000 (UTC) From: Eryu Guan To: fstests@vger.kernel.org Cc: linux-xfs@vger.kernel.org, Eryu Guan Subject: [PATCH] generic: reserve correct indirect blocks for delalloc write path Date: Mon, 4 Sep 2017 17:38:19 +0800 Message-Id: <20170904093819.28708-1-eguan@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Mon, 04 Sep 2017 09:38:35 +0000 (UTC) Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Test that XFS reserves reasonable indirect blocks for delalloc and speculative allocation, and doesn't cause any fdblocks corruption. This was inspired by an XFS but that too large 'indlen' was returned by xfs_bmap_worst_indlen() which can't fit in a 17 bits value (STARTBLOCKVALBITS is defined as 17), then leaked 1 << 17 blocks in sb_fdblocks. This was only seen on XFS with rmapbt feature enabled, but nothing prevents the test from being a generic test. Signed-off-by: Eryu Guan --- This reproduces sb_fdblocks corruption issue for me with 1k block size rmapbt enabled XFS on x86_64 and ppc64 hosts. Other block size XFS pass the test now. tests/generic/456 | 113 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/456.out | 2 + tests/generic/group | 1 + 3 files changed, 116 insertions(+) create mode 100755 tests/generic/456 create mode 100644 tests/generic/456.out diff --git a/tests/generic/456 b/tests/generic/456 new file mode 100755 index 000000000000..b878c67c6b0e --- /dev/null +++ b/tests/generic/456 @@ -0,0 +1,113 @@ +#! /bin/bash +# FS QA Test 456 +# +# Test that XFS reserves reasonable indirect blocks for delalloc and +# speculative allocation, and doesn't cause any fdblocks corruption. +# +# This was inspired by an XFS but that too large 'indlen' was returned by +# xfs_bmap_worst_indlen() which can't fit in a 17 bits value (STARTBLOCKVALBITS +# is defined as 17), then leaked 1 << 17 blocks in sb_fdblocks. +# +# This was only seen on XFS with rmapbt feature enabled, but nothing prevents +# the test from being a generic test. +# +#----------------------------------------------------------------------- +# Copyright (c) 2017 Red Hat Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +testfile=$SCRATCH_MNT/1G_file.$seq +file_size=$((1024 * 1024 * 1024)) +trap "_cleanup; exit \$status" 0 1 2 3 15 + +saved_dirty_background_ratio=0 +saved_dirty_ratio=0 + +save_dirty_ratio() +{ + saved_dirty_background_ratio=`cat /proc/sys/vm/dirty_background_ratio` + saved_dirty_ratio=`cat /proc/sys/vm/dirty_ratio` +} + +set_dirty_ratio() +{ + echo 100 > /proc/sys/vm/dirty_background_ratio + echo 100 > /proc/sys/vm/dirty_ratio +} + +restore_dirty_ratio() +{ + if [ $saved_dirty_background_ratio -ne 0 ]; then + echo $saved_dirty_background_ratio > /proc/sys/vm/dirty_background_ratio + fi + if [ $saved_dirty_ratio -ne 0 ]; then + echo $saved_dirty_ratio > /proc/sys/vm/dirty_ratio + fi +} + +_cleanup() +{ + cd / + restore_dirty_ratio + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs generic +_supported_os Linux +# test with scratch device, because test is known to corrupt fs, we don't want +# the corruption affect subsequent tests +_require_scratch + +_scratch_mkfs >>$seqres.full 2>&1 +_scratch_mount + +# need at least 1G free space +_require_fs_space $SCRATCH_MNT $((1024 * 1024)) + +# To reproduce the bug, we need to keep enough dirty data in memory (1G at +# least), so that a large enough delay allocated extent is kept in memory, then +# speculative preallocation could allocate large number of blocks based on the +# existing extent size. +# So we set dirty_background_ratio and dirty_ratio to 100% uncontitionally, +# even if the total memory is less than 1G, there's no harm to run a test on a +# such host. +save_dirty_ratio +set_dirty_ratio + +# buffer write a 1G file, which is enough to trigger the bug, +# _check_filesystems will complain about fs corruption after test +$XFS_IO_PROG -fc "pwrite -b 1m 0 $file_size" $testfile >/dev/null 2>&1 + +echo "Silence is golden" + +# success, all done +status=0 +exit diff --git a/tests/generic/456.out b/tests/generic/456.out new file mode 100644 index 000000000000..248dfeb2ad90 --- /dev/null +++ b/tests/generic/456.out @@ -0,0 +1,2 @@ +QA output created by 456 +Silence is golden diff --git a/tests/generic/group b/tests/generic/group index 6422d51033af..e9f08ab4ef79 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -458,3 +458,4 @@ 453 auto quick dir 454 auto quick attr 455 auto rw +456 auto quick rw