| Message ID | 20181121032702.3048-1-yehs2007@zoho.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | realize dax_operations for dm-snapshot | expand |
On Tue, Nov 20, 2018 at 7:27 PM Huaisheng Ye <yehs2007@zoho.com> wrote: > > From: Huaisheng Ye <yehs1@lenovo.com> > > Changes > v1->v2: > Add NULL funtions for origin_dax_direct_access and > origin_dax_copy_from/to_iter in order to avoid building > error when CONFIG_DAX_DRIVER has NOT been enabled. > > [v1]: https://lkml.org/lkml/2018/11/20/759 > > This series patches are used to realize the dax_operations for dm-snapshot > with persistent memory device. How does this interact with mmap write faults if the mapping is dax and the page needs to be cow'd?
From: Dan Williams <dan.j.williams@intel.com> Sent: Monday, November 26, 2018 5:00 AM> > On Tue, Nov 20, 2018 at 7:27 PM Huaisheng Ye <yehs2007@zoho.com> wrote: > > > > From: Huaisheng Ye <yehs1@lenovo.com> > > > > Changes > > v1->v2: > > Add NULL funtions for origin_dax_direct_access and > > origin_dax_copy_from/to_iter in order to avoid building > > error when CONFIG_DAX_DRIVER has NOT been enabled. > > > > [v1]: https://lkml.org/lkml/2018/11/20/759 > > > > This series patches are used to realize the dax_operations for dm-snapshot > > with persistent memory device. > > How does this interact with mmap write faults if the mapping is dax > and the page needs to be cow'd? Hi Dan, Based on my understanding, I think dm-snapshot is impossible to work with file mapping in DAX way. When file mapping has been used with DAX, userspace could get the pointer To access persistent memory directly by page fault. Here is the call trace. 74 [ 7677.897354] Call Trace: 75 [ 7677.900538] dump_stack+0x67/0x82 76 [ 7677.904695] __pmem_direct_access+0xa9/0x101 [nd_pmem] 77 [ 7677.910922] ? linear_ctr+0x12a/0x12a [dm_mod] 78 [ 7677.916340] pmem_dax_direct_access+0x30/0x37 [nd_pmem] 79 [ 7677.922641] dax_direct_access+0x30/0x58 80 [ 7677.927480] linear_dax_direct_access+0x66/0x71 [dm_mod] 81 [ 7677.933848] dm_dax_direct_access+0x9c/0xf1 [dm_mod] 82 [ 7677.939856] ? origin_dax_copy_from_iter+0x88/0x88 [dm_snapshot] 83 [ 7677.947032] dax_direct_access+0x30/0x58 84 [ 7677.951876] origin_dax_direct_access+0x6a/0x75 [dm_snapshot] 85 [ 7677.958753] dm_dax_direct_access+0x9c/0xf1 [dm_mod] 86 [ 7677.964738] dax_direct_access+0x30/0x58 87 [ 7677.969542] dax_iomap_pfn+0x84/0x10d 88 [ 7677.974061] dax_iomap_pte_fault+0x4a9/0x773 89 [ 7677.979271] dax_iomap_fault+0x21/0x36 90 [ 7677.983895] ext2_dax_fault+0x70/0x9a [ext2] 91 [ 7677.989061] __do_fault+0x1d/0x74 92 [ 7677.993159] __handle_mm_fault+0xf04/0x17a4 93 [ 7677.998225] handle_mm_fault+0x1a0/0x204 94 [ 7678.003035] __do_page_fault+0x39b/0x4d3 95 [ 7678.007839] do_page_fault+0xfc/0x11b 96 [ 7678.012316] ? page_fault+0x8/0x30 97 [ 7678.016498] page_fault+0x1e/0x30 The application in userspace could directly read or write the data of the file content by mmap in DAX way. dm-snapshot works at kernel space, so it doesn't have chance to be aware of the modification of FS. I think in this case, perhaps userspace should take the responsibility for snapshot when file mapping has been used in DAX way. Just like what NVML has done as CLFLUSH for cache lines flush. Correct me if anything is not accurate. Cheers, Huaisheng Ye
From: Huaisheng Ye <yehs1@lenovo.com> Changes v1->v2: Add NULL funtions for origin_dax_direct_access and origin_dax_copy_from/to_iter in order to avoid building error when CONFIG_DAX_DRIVER has NOT been enabled. [v1]: https://lkml.org/lkml/2018/11/20/759 This series patches are used to realize the dax_operations for dm-snapshot with persistent memory device. Here are the steps about how to verify the function. 1. Configure the persistent memory to fs-dax mode and create namespace with ndctl; 2. find them in /dev; # ndctl list { "dev":"namespace0.0", "mode":"fsdax", "map":"dev", "size":132118478848, "sector_size":512, "blockdev":"pmem0", "name":"yhs_pmem0", "numa_node":0 }, 3. create lv_pmem (here is 4G size) for testing; # pvcreate /dev/pmem0 # vgcreate vg_pmem /dev/pmem0 # lvcreate -L 4G -n lv_pmem vg_pmem 4. create filesystem (ext2 or ext4) to /dev/pmem0 # mkfs.ext2 -b 4096 /dev/vg_pmem/lv_pmem 5. mount pmem with DAX way; # mkdir /mnt/lv_pmem # mount -o dax /dev/vg_pmem/lv_pmem /mnt/lv_pmem/ 6. cp some files to /mnt/lv_pmem; # cp linear_table03.log /mnt/lv_pmem/ # cp test0.log /mnt/lv_pmem/ 7. create snapshot for test (here I limit it to 1G size); # lvcreate -L 1G -n snap_pmem -s /dev/vg_pmem/lv_pmem 8. modify the files copied with vim or copy more other new files; # vim /mnt/lv_pmem/test0.log 9. umount the pmem device; # umount /mnt/lv_pmem/ 10.merge the snapshot back to origin; # lvconvert --merge /dev/vg_pmem/snap_pmem 11.mount pmem device again for checking the content of files; # mount -o dax /dev/vg_pmem/lv_pmem /mnt/lv_pmem/ Huaisheng Ye (3): dm: enable dax_operations for dm-snapshot dm: expand hc_map in mapped_device for lack of map dm: expand valid types for dm-ioctl drivers/md/dm-core.h | 1 + drivers/md/dm-ioctl.c | 4 +++- drivers/md/dm-snap.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++++-- drivers/md/dm.c | 15 +++++++++++++++ 4 files changed, 68 insertions(+), 3 deletions(-)