[v6,2/2] generic: add test for DAX MAP_SYNC support

Commit Message

Ross Zwisler Dec. 7, 2017, 11:19 p.m. UTC

This test creates a file and writes to it via an mmap(), but never syncs
via fsync/msync.  This process is tracked via dm-log-writes, then replayed.

If MAP_SYNC is working the dm-log-writes replay will show the test file
with 1 MiB of on-media block allocations.  This is because each allocating
page fault included an implicit metadata sync.  If MAP_SYNC isn't working
(which you can test by removing the "-S" flag to xfs_io mmap) the file
will be smaller or missing entirely.

Note that dm-log-writes doesn't track the data that we write via the
mmap(), so we can't do any data integrity checking.  We can only verify
that the metadata writes for the page faults happened.

Signed-off-by: Ross Zwisler <[e-mail]>

---

Changes since v5:
 - Addressed Eryu's comments.  Thanks for the review.

---
 common/dmlogwrites    | 19 ++++++++++++
 tests/generic/999     | 80 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/999.out |  2 ++
 tests/generic/group   |  1 +
 4 files changed, 102 insertions(+)
 create mode 100755 tests/generic/999
 create mode 100644 tests/generic/999.out

Comments

Eryu Guan Dec. 8, 2017, 6:36 a.m. UTC | #1

On Thu, Dec 07, 2017 at 04:19:50PM -0700, Ross Zwisler wrote:
> This test creates a file and writes to it via an mmap(), but never syncs
> via fsync/msync.  This process is tracked via dm-log-writes, then replayed.
> 
> If MAP_SYNC is working the dm-log-writes replay will show the test file
> with 1 MiB of on-media block allocations.  This is because each allocating
> page fault included an implicit metadata sync.  If MAP_SYNC isn't working
> (which you can test by removing the "-S" flag to xfs_io mmap) the file
> will be smaller or missing entirely.
> 
> Note that dm-log-writes doesn't track the data that we write via the
> mmap(), so we can't do any data integrity checking.  We can only verify
> that the metadata writes for the page faults happened.
> 
> Signed-off-by: Ross Zwisler <[e-mail]>
> 
> ---
> 
> Changes since v5:
>  - Addressed Eryu's comments.  Thanks for the review.

Thanks for the explanation on the dm-log-writes dax support! I agree
that we should keep _require_log_writes_dax for now till dm-log-writes
gains full dax support. I added some comments about this above the
helper definition and queued the patchset for next fstests update. (Test
was re-numbered as generic/470, BTW.)

Thanks,
Eryu
--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Ross Zwisler Dec. 8, 2017, 5:47 p.m. UTC | #2

On Fri, Dec 08, 2017 at 02:36:10PM +0800, Eryu Guan wrote:

> (Test was re-numbered as generic/470, BTW.)

Thanks!  For future reference, does the pattern of us submitting tests with
high numbers (generic/999) to avoid merge conflicts and asking you to renumber
them when you merge work for you?  Or would you prefer that we number our
tests to the next available, which may change from submission to submission?
--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Eryu Guan Dec. 10, 2017, 6:15 a.m. UTC | #3

On Fri, Dec 08, 2017 at 10:47:47AM -0700, Ross Zwisler wrote:
> On Fri, Dec 08, 2017 at 02:36:10PM +0800, Eryu Guan wrote:
> 
> > (Test was re-numbered as generic/470, BTW.)
> 
> Thanks!  For future reference, does the pattern of us submitting tests with
> high numbers (generic/999) to avoid merge conflicts and asking you to renumber
> them when you merge work for you?  Or would you prefer that we number our
> tests to the next available, which may change from submission to submission?

For patch that adds a single test, either way is fine, I need to edit
the patch anyway on conflicts, as the group file conflicts. For patch or
patchset that adds multiple tests, starting with a high test seq number
would be better, I only need to edit the group file by hand, not the seq
numbers in each test (that can be done by ./tools/mvtest script).

But overall, the starting seq number doesn't matter that much. OTOH,
basing new tests on top of latest master as much as possible would be
perfered, that reduces the possibility of conflicts, as I only need to
resolve conflicts within all the new tests.

Thanks,
Eryu
--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/common/dmlogwrites b/common/dmlogwrites
index 05829dbc..c235ee49 100644
--- a/common/dmlogwrites
+++ b/common/dmlogwrites
@@ -28,6 +28,25 @@  _require_log_writes()
 	_require_test_program "log-writes/replay-log"
 }
 
+_require_log_writes_dax()
+{
+	[ -z "$LOGWRITES_DEV" -o ! -b "$LOGWRITES_DEV" ] && \
+		_notrun "This test requires a valid \$LOGWRITES_DEV"
+
+	_require_dm_target log-writes
+	_require_test_program "log-writes/replay-log"
+
+	_log_writes_init
+	_log_writes_mkfs > /dev/null 2>&1
+	_log_writes_mount -o dax
+	# Check options to be sure. XFS ignores dax option
+	# and goes on if dev underneath does not support dax.
+	_fs_options $LOGWRITES_DMDEV | grep -qw "dax" || \
+		_notrun "$LOGWRITES_DMDEV $FSTYP does not support -o dax"
+	_log_writes_unmount
+	_log_writes_remove
+}
+
 _log_writes_init()
 {
 	local BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV`
diff --git a/tests/generic/999 b/tests/generic/999
new file mode 100755
index 00000000..e20faa8f
--- /dev/null
+++ b/tests/generic/999
@@ -0,0 +1,80 @@ 
+#! /bin/bash
+# FS QA Test No. 999
+#
+# Use dm-log-writes to verify that MAP_SYNC actually syncs metadata during
+# page faults.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2017 Intel Corporation.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	_log_writes_cleanup
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/dmlogwrites
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+_require_log_writes_dax
+_require_xfs_io_command "mmap" "-S"
+_require_xfs_io_command "log_writes"
+
+_log_writes_init
+_log_writes_mkfs >> $seqres.full 2>&1
+_log_writes_mount -o dax
+
+LEN=$((1024 * 1024)) # 1 MiB
+
+$XFS_IO_PROG -t -c "truncate $LEN" -c "mmap -S 0 $LEN" -c "mwrite 0 $LEN" \
+	-c "log_writes -d $LOGWRITES_NAME -m preunmap" \
+	-f $SCRATCH_MNT/test
+
+# Unmount the scratch dir and tear down the log writes target
+_log_writes_unmount
+_log_writes_remove
+_check_scratch_fs
+
+# destroy previous filesystem so we can be sure our rebuild works
+_scratch_mkfs >> $seqres.full 2>&1
+
+# check pre-unmap state
+_log_writes_replay_log preunmap
+_scratch_mount
+
+# We should see $SCRATCH_MNT/test as having 1 MiB in block allocations
+du -sh $SCRATCH_MNT/test | _filter_scratch | _filter_spaces
+
+status=0
+exit
diff --git a/tests/generic/999.out b/tests/generic/999.out
new file mode 100644
index 00000000..611289b6
--- /dev/null
+++ b/tests/generic/999.out
@@ -0,0 +1,2 @@ 
+QA output created by 999
+1.0M SCRATCH_MNT/test
diff --git a/tests/generic/group b/tests/generic/group
index 6c3bb03a..ae88aa03 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -472,3 +472,4 @@ 
 467 auto quick exportfs
 468 shutdown auto quick metadata
 469 auto quick
+999 auto quick dax