From patchwork Wed Apr 21 16:45:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Shtepa X-Patchwork-Id: 12216527 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 460C2C43461 for ; Wed, 21 Apr 2021 16:46:42 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9BF2A61456 for ; Wed, 21 Apr 2021 16:46:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9BF2A61456 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=veeam.com Authentication-Results: mail.kernel.org; spf=tempfail smtp.mailfrom=dm-devel-bounces@redhat.com Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-93-Oc8xPbgDM_ycaeR3hdGJNw-1; Wed, 21 Apr 2021 12:46:32 -0400 X-MC-Unique: Oc8xPbgDM_ycaeR3hdGJNw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 038306DD5A; Wed, 21 Apr 2021 16:46:15 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6879B6091B; Wed, 21 Apr 2021 16:46:13 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id E0EDE4ED7A; Wed, 21 Apr 2021 16:46:09 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 13LGk703013303 for ; Wed, 21 Apr 2021 12:46:08 -0400 Received: by smtp.corp.redhat.com (Postfix) id 05A205F274; Wed, 21 Apr 2021 16:46:07 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast03.extmail.prod.ext.rdu2.redhat.com [10.11.55.19]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C54E2F66BB for ; Wed, 21 Apr 2021 16:46:04 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 04F2480270A for ; Wed, 21 Apr 2021 16:46:04 +0000 (UTC) Received: from mx2.veeam.com (mx2.veeam.com [64.129.123.6]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-194-h4QNxq_4NFKpSoLbEaQFnA-1; Wed, 21 Apr 2021 12:45:59 -0400 X-MC-Unique: h4QNxq_4NFKpSoLbEaQFnA-1 Received: from mail.veeam.com (prgmbx01.amust.local [172.24.0.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx2.veeam.com (Postfix) with ESMTPS id 5EBE742406; Wed, 21 Apr 2021 12:45:55 -0400 (EDT) Received: from prgdevlinuxpatch01.amust.local (172.24.14.5) by prgmbx01.amust.local (172.24.0.171) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.858.5; Wed, 21 Apr 2021 18:45:53 +0200 From: Sergei Shtepa To: Christoph Hellwig , Hannes Reinecke , Mike Snitzer , Alasdair Kergon , Alexander Viro , Jens Axboe , , , , Date: Wed, 21 Apr 2021 19:45:41 +0300 Message-ID: <1619023545-23431-1-git-send-email-sergei.shtepa@veeam.com> MIME-Version: 1.0 X-Originating-IP: [172.24.14.5] X-ClientProxiedBy: prgmbx02.amust.local (172.24.0.172) To prgmbx01.amust.local (172.24.0.171) X-EsetResult: clean, is OK X-EsetId: 37303A29D2A50B59677566 X-Veeam-MMEX: True X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-loop: dm-devel@redhat.com Cc: pavel.tide@veeam.com, sergei.shtepa@veeam.com Subject: [dm-devel] [PATCH v9 0/4] block device interposer X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dm-devel-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com A new version of a block device interposer (blk_interposer). In this series of patches, I have tried to take into account the comments made by Mike to the previous version. First of all, this applies to more detailed explanations of the commits. Indeed, the changes in blk-core.c and dm.c may seem complicated, but they are no more complicated than the rest of the code in these files. Removed the [interpose] option for block devices opened by the DM target. Instead, the dm_get_device_ex() function is added, which allows to explicitly specify which devices can be used for the interposer and which can not. Additional testing has revealed a problem with suspending and resuming DM targets attached via blk_interposer. This has been fixed. History: v8 - https://patchwork.kernel.org/project/linux-block/cover/1617968884-15149-1-git-send-email-sergei.shtepa@veeam.com/ * The attaching and detaching to interposed device moved to __dm_suspend() and __dm_resume() functions. * Redesigned the submit_bio_noacct() function and added a lock for the block device interposer. * Adds [interpose] option to block device patch in dm table. * Fix origin_map() then o->split_binary value is zero. v7 - https://patchwork.kernel.org/project/linux-block/cover/1615563895-28565-1-git-send-email-sergei.shtepa@veeam.com/ * the request interception mechanism. Now the interposer is a block device that receives requests instead of the original device; * code design fixes. v6 - https://patchwork.kernel.org/project/linux-block/cover/1614774618-22410-1-git-send-email-sergei.shtepa@veeam.com/ * designed for 5.12; * thanks to the new design of the bio structure in v5.12, it is possible to perform interception not for the entire disk, but for each block device; * instead of the new ioctl DM_DEV_REMAP_CMD and the 'noexcl' option, the DM_INTERPOSED_FLAG flag for the ioctl DM_TABLE_LOAD_CMD is applied. v5 - https://patchwork.kernel.org/project/linux-block/cover/1612881028-7878-1-git-send-email-sergei.shtepa@veeam.com/ * rebase for v5.11-rc7; * patch set organization; * fix defects in documentation; * add some comments; * change mutex names for better code readability; * remove calling bd_unlink_disk_holder() for targets with non-exclusive flag; * change type for struct dm_remap_param from uint8_t to __u8. v4 - https://patchwork.kernel.org/project/linux-block/cover/1612367638-3794-1-git-send-email-sergei.shtepa@veeam.com/ Mostly changes were made, due to Damien's comments: * on the design of the code; * by the patch set organization; * bug with passing a wrong parameter to dm_get_device(); * description of the 'noexcl' parameter in the linear.rst. Also added remap_and_filter.rst. v3 - https://patchwork.kernel.org/project/linux-block/cover/1611853955-32167-1-git-send-email-sergei.shtepa@veeam.com/ In this version, I already suggested blk_interposer to apply to dm-linear. Problems were solved: * Interception of bio requests from a specific device on the disk, not from the entire disk. To do this, we added the dm_interposed_dev structure and an interval tree to store these structures. * Implemented ioctl DM_DEV_REMAP_CMD. A patch with changes in the lvm2 project was sent to the team lvm-devel@redhat.com. * Added the 'noexcl' option for dm-linear, which allows you to open the underlying block-device without FMODE_EXCL mode. v2 - https://patchwork.kernel.org/project/linux-block/cover/1607518911-30692-1-git-send-email-sergei.shtepa@veeam.com/ I tried to suggest blk_interposer without using it in device mapper, but with the addition of a sample of its use. It was then that I learned about the maintainers' attitudes towards the samples directory :). v1 - https://lwn.net/ml/linux-block/20201119164924.74401-1-hare@suse.de/ This Hannes's patch can be considered as a starting point, since this is where the interception mechanism and the term blk_interposer itself appeared. It became clear that blk_interposer can be useful for device mapper. before v1 - https://patchwork.kernel.org/project/linux-block/cover/1603271049-20681-1-git-send-email-sergei.shtepa@veeam.com/ I tried to offer a rather cumbersome blk-filter and a monster-like blk-snap module for creating snapshots. Sergei Shtepa (4): Adds blk_interposer Applying the blk_interposer in the block device layer Add blk_interposer in DM Using dm_get_device_ex() instead of dm_get_device() block/bio.c | 2 + block/blk-core.c | 194 ++++++++++++++------------- block/genhd.c | 52 ++++++++ drivers/md/dm-cache-target.c | 5 +- drivers/md/dm-core.h | 1 + drivers/md/dm-delay.c | 3 +- drivers/md/dm-dust.c | 3 +- drivers/md/dm-era-target.c | 4 +- drivers/md/dm-flakey.c | 3 +- drivers/md/dm-ioctl.c | 59 ++++++++- drivers/md/dm-linear.c | 3 +- drivers/md/dm-log-writes.c | 3 +- drivers/md/dm-snap.c | 3 +- drivers/md/dm-table.c | 21 ++- drivers/md/dm-writecache.c | 3 +- drivers/md/dm.c | 242 ++++++++++++++++++++++++++++++---- drivers/md/dm.h | 8 +- fs/block_dev.c | 3 + include/linux/blk_types.h | 6 + include/linux/blkdev.h | 32 +++++ include/linux/device-mapper.h | 11 +- include/uapi/linux/dm-ioctl.h | 6 + 22 files changed, 530 insertions(+), 137 deletions(-) --- 2.20.1 -- dm-devel mailing list dm-devel@redhat.com https://listman.redhat.com/mailman/listinfo/dm-devel