| Message ID | 1611853955-32167-1-git-send-email-sergei.shtepa@veeam.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | block: blk_interposer v3 | expand |
On 2021/01/29 2:23, Sergei Shtepa wrote: > Hi all, > > I`m ready to suggest the blk_interposer again. > blk_interposer allows to intercept bio requests, remap bio to > another devices or add new bios. > > This version has support from device mapper. > > For the dm-linear device creation command, the `noexcl` parameter > has been added, which allows to open block devices without > FMODE_EXCL mode. It allows to create dm-linear device on a block > device with an already mounted file system. > The new ioctl DM_DEV_REMAP allows to enable and disable bio > interception. > > Thus, it is possible to add the dm-device to the block layer stack > without reconfiguring and rebooting. Please add the changelog here instead of adding it in each patch. And keep the changelog for previous versions too (i.e. v1->v2 in this case) so that the changes overall can be tracked. The proper formatting for the title should be [PATCH v3 X/Y] instead of adding v3 in the title itself. With git format-patch, you can use "-v 3" option to format this for you, or --subject-prefix="PATCH v3" option. > > > Sergei Shtepa (2): > block: blk_interposer > [dm] blk_interposer for dm-linear > > block/bio.c | 2 + > block/blk-core.c | 29 +++ > block/blk-mq.c | 13 ++ > block/genhd.c | 82 ++++++++ > drivers/md/dm-core.h | 46 +++- > drivers/md/dm-ioctl.c | 39 ++++ > drivers/md/dm-linear.c | 17 +- > drivers/md/dm-table.c | 12 +- > drivers/md/dm.c | 383 ++++++++++++++++++++++++++++++++-- > drivers/md/dm.h | 2 +- > include/linux/blk-mq.h | 1 + > include/linux/blk_types.h | 6 +- > include/linux/device-mapper.h | 7 + > include/linux/genhd.h | 19 ++ > include/uapi/linux/dm-ioctl.h | 15 +- > 15 files changed, 643 insertions(+), 30 deletions(-) >
On 1/28/21 9:12 AM, Sergei Shtepa wrote: > I`m ready to suggest the blk_interposer again. > blk_interposer allows to intercept bio requests, remap bio to > another devices or add new bios. > > This version has support from device mapper. > > For the dm-linear device creation command, the `noexcl` parameter > has been added, which allows to open block devices without > FMODE_EXCL mode. It allows to create dm-linear device on a block > device with an already mounted file system. > The new ioctl DM_DEV_REMAP allows to enable and disable bio > interception. > > Thus, it is possible to add the dm-device to the block layer stack > without reconfiguring and rebooting. What functionality does this driver provide that is not yet available in a RAID level 1 (mirroring) driver + a custom dm driver? My understanding is that there are already two RAID level 1 drivers in the kernel tree and that both driver support sending bio's to two different block devices. Thanks, Bart.
The 02/01/2021 18:45, Bart Van Assche wrote: > On 1/28/21 9:12 AM, Sergei Shtepa wrote: > > I`m ready to suggest the blk_interposer again. > > blk_interposer allows to intercept bio requests, remap bio to > > another devices or add new bios. > > > > This version has support from device mapper. > > > > For the dm-linear device creation command, the `noexcl` parameter > > has been added, which allows to open block devices without > > FMODE_EXCL mode. It allows to create dm-linear device on a block > > device with an already mounted file system. > > The new ioctl DM_DEV_REMAP allows to enable and disable bio > > interception. > > > > Thus, it is possible to add the dm-device to the block layer stack > > without reconfiguring and rebooting. > > What functionality does this driver provide that is not yet available in > a RAID level 1 (mirroring) driver + a custom dm driver? My understanding > is that there are already two RAID level 1 drivers in the kernel tree > and that both driver support sending bio's to two different block devices. > > Thanks, > > Bart. Hi Bart. The proposed patch is not realy aimed at RAID1. Creating a new dm device in the non-FMODE_EXCL mode and then remaping bio requests from the regular block device to the new DM device using the blk_interposer will allow to use device mapper for regular devices. For dm-linear, there is not much benefit from using blk_interposer. This is a good and illustrative example. Later, using blk-interposer, it will be possible to connect the dm-cache "on the fly" without having to reboot and/or reconfigure. My intention is to let users use dm-snap to create snapshots of any device. blk-interposer will allow to add new features to Device Mapper. As per Daniel's advice I want to add a documentation, I'm working on it now. The documentation will also contain a description of new features that blk_interposer will add to Device Mapper Thanks.
On Mon, Feb 01 2021 at 1:18pm -0500, Sergei Shtepa <sergei.shtepa@veeam.com> wrote: > The 02/01/2021 18:45, Bart Van Assche wrote: > > On 1/28/21 9:12 AM, Sergei Shtepa wrote: > > > I`m ready to suggest the blk_interposer again. > > > blk_interposer allows to intercept bio requests, remap bio to > > > another devices or add new bios. > > > > > > This version has support from device mapper. > > > > > > For the dm-linear device creation command, the `noexcl` parameter > > > has been added, which allows to open block devices without > > > FMODE_EXCL mode. It allows to create dm-linear device on a block > > > device with an already mounted file system. > > > The new ioctl DM_DEV_REMAP allows to enable and disable bio > > > interception. > > > > > > Thus, it is possible to add the dm-device to the block layer stack > > > without reconfiguring and rebooting. > > > > What functionality does this driver provide that is not yet available in > > a RAID level 1 (mirroring) driver + a custom dm driver? My understanding > > is that there are already two RAID level 1 drivers in the kernel tree > > and that both driver support sending bio's to two different block devices. > > > > Thanks, > > > > Bart. > > Hi Bart. > > The proposed patch is not realy aimed at RAID1. > > Creating a new dm device in the non-FMODE_EXCL mode and then remaping bio > requests from the regular block device to the new DM device using > the blk_interposer will allow to use device mapper for regular devices. > For dm-linear, there is not much benefit from using blk_interposer. > This is a good and illustrative example. Later, using blk-interposer, > it will be possible to connect the dm-cache "on the fly" without having > to reboot and/or reconfigure. > My intention is to let users use dm-snap to create snapshots of any device. > blk-interposer will allow to add new features to Device Mapper. > > As per Daniel's advice I want to add a documentation, I'm working on it now. > The documentation will also contain a description of new features that > blk_interposer will add to Device Mapper More Documentation is fine, but the code needs to be improved and fully formed before you start trying to polish with Documentation -- definitely don't put time to Documentation that is speculative! You'd do well to focus on an implementation that doesn't require an extra clone if interposed device will use DM (DM core already handles cloning all incoming bios). Mike
The 02/01/2021 21:50, Mike Snitzer wrote: > On Mon, Feb 01 2021 at 1:18pm -0500, > Sergei Shtepa <sergei.shtepa@veeam.com> wrote: > > > The 02/01/2021 18:45, Bart Van Assche wrote: > > > On 1/28/21 9:12 AM, Sergei Shtepa wrote: > > > > I`m ready to suggest the blk_interposer again. > > > > blk_interposer allows to intercept bio requests, remap bio to > > > > another devices or add new bios. > > > > > > > > This version has support from device mapper. > > > > > > > > For the dm-linear device creation command, the `noexcl` parameter > > > > has been added, which allows to open block devices without > > > > FMODE_EXCL mode. It allows to create dm-linear device on a block > > > > device with an already mounted file system. > > > > The new ioctl DM_DEV_REMAP allows to enable and disable bio > > > > interception. > > > > > > > > Thus, it is possible to add the dm-device to the block layer stack > > > > without reconfiguring and rebooting. > > > > > > What functionality does this driver provide that is not yet available in > > > a RAID level 1 (mirroring) driver + a custom dm driver? My understanding > > > is that there are already two RAID level 1 drivers in the kernel tree > > > and that both driver support sending bio's to two different block devices. > > > > > > Thanks, > > > > > > Bart. > > > > Hi Bart. > > > > The proposed patch is not realy aimed at RAID1. > > > > Creating a new dm device in the non-FMODE_EXCL mode and then remaping bio > > requests from the regular block device to the new DM device using > > the blk_interposer will allow to use device mapper for regular devices. > > For dm-linear, there is not much benefit from using blk_interposer. > > This is a good and illustrative example. Later, using blk-interposer, > > it will be possible to connect the dm-cache "on the fly" without having > > to reboot and/or reconfigure. > > My intention is to let users use dm-snap to create snapshots of any device. > > blk-interposer will allow to add new features to Device Mapper. > > > > As per Daniel's advice I want to add a documentation, I'm working on it now. > > The documentation will also contain a description of new features that > > blk_interposer will add to Device Mapper > > More Documentation is fine, but the code needs to be improved and fully > formed before you start trying to polish with Documentation -- > definitely don't put time to Documentation that is speculative! > > You'd do well to focus on an implementation that doesn't require an > extra clone if interposed device will use DM (DM core already handles > cloning all incoming bios). > > Mike > Hi Mike. I agree that quality code is more important than documentation. And I think this code has already solved the problem of a bio extra cloning. +static void dm_remap_fn(void *context, struct dm_rb_range *node, struct bio *bio) +{ + struct mapped_device *md = context; + + /* Set acceptor device. */ + bio->bi_disk = md->disk; + + /* Remap disks offset */ + bio->bi_iter.bi_sector -= node->start; + + /* + * bio should be resubmitted. + * We can just add bio to bio_list of the current process. + * current->bio_list must be initialized when this function is called. + * If call submit_bio_noacct(), the bio will be checked twice. + */ + BUG_ON(!current->bio_list); + bio_list_add(¤t->bio_list[0], bio); +}