From patchwork Fri Mar 12 15:44:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Shtepa X-Patchwork-Id: 12135033 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63E55C4332D for ; Fri, 12 Mar 2021 15:46:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2E58264FDD for ; Fri, 12 Mar 2021 15:46:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232778AbhCLPpl (ORCPT ); Fri, 12 Mar 2021 10:45:41 -0500 Received: from mx4.veeam.com ([104.41.138.86]:49948 "EHLO mx4.veeam.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232583AbhCLPpV (ORCPT ); Fri, 12 Mar 2021 10:45:21 -0500 Received: from mail.veeam.com (prgmbx01.amust.local [172.24.0.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx4.veeam.com (Postfix) with ESMTPS id C04F1114A90; Fri, 12 Mar 2021 18:45:17 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=veeam.com; s=mx4; t=1615563917; bh=E7vklZS3KQKvlQW65s1wvDn6+2dH/1XjVlF3nm5Ihtk=; h=From:To:CC:Subject:Date:In-Reply-To:References:From; b=SdtDNGUABLkZFqgXedaW5RLH3FOTjPK8hBnQe3Hc/iMoyHErKJ+DD1rc8rjM7ouV6 TFcQ5frFhf4IULEwLTohxz51zG0cgCtZ35Jvvm8aHkszbiLwOhfPvPtrcDdBRUsPOy D9QdwhbxiQlJXnBSgkTajzEienrF87YoWlr/Qu8A= Received: from prgdevlinuxpatch01.amust.local (172.24.14.5) by prgmbx01.amust.local (172.24.0.171) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.721.2; Fri, 12 Mar 2021 16:45:15 +0100 From: Sergei Shtepa To: Christoph Hellwig , Mike Snitzer , Alasdair Kergon , Hannes Reinecke , Jens Axboe , , , , CC: , Subject: [PATCH v7 1/3] block: add blk_mq_is_queue_frozen() Date: Fri, 12 Mar 2021 18:44:53 +0300 Message-ID: <1615563895-28565-2-git-send-email-sergei.shtepa@veeam.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1615563895-28565-1-git-send-email-sergei.shtepa@veeam.com> References: <1615563895-28565-1-git-send-email-sergei.shtepa@veeam.com> MIME-Version: 1.0 X-Originating-IP: [172.24.14.5] X-ClientProxiedBy: prgmbx02.amust.local (172.24.0.172) To prgmbx01.amust.local (172.24.0.171) X-EsetResult: clean, is OK X-EsetId: 37303A29D2A50B58627366 X-Veeam-MMEX: True Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org blk_mq_is_queue_frozen() allow to assert that the queue is frozen. Signed-off-by: Sergei Shtepa --- block/blk-mq.c | 13 +++++++++++++ include/linux/blk-mq.h | 1 + 2 files changed, 14 insertions(+) diff --git a/block/blk-mq.c b/block/blk-mq.c index d4d7c1caa439..2f188a865024 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -161,6 +161,19 @@ int blk_mq_freeze_queue_wait_timeout(struct request_queue *q, } EXPORT_SYMBOL_GPL(blk_mq_freeze_queue_wait_timeout); +bool blk_mq_is_queue_frozen(struct request_queue *q) +{ + bool frozen; + + mutex_lock(&q->mq_freeze_lock); + frozen = percpu_ref_is_dying(&q->q_usage_counter) && + percpu_ref_is_zero(&q->q_usage_counter); + mutex_unlock(&q->mq_freeze_lock); + + return frozen; +} +EXPORT_SYMBOL_GPL(blk_mq_is_queue_frozen); + /* * Guarantee no request is in use, so we can change any data structure of * the queue afterward. diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 2c473c9b8990..6f01971abf7b 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -533,6 +533,7 @@ void blk_freeze_queue_start(struct request_queue *q); void blk_mq_freeze_queue_wait(struct request_queue *q); int blk_mq_freeze_queue_wait_timeout(struct request_queue *q, unsigned long timeout); +bool blk_mq_is_queue_frozen(struct request_queue *q); int blk_mq_map_queues(struct blk_mq_queue_map *qmap); void blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, int nr_hw_queues); From patchwork Fri Mar 12 15:44:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Shtepa X-Patchwork-Id: 12135035 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 655A6C43332 for ; Fri, 12 Mar 2021 15:46:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4DC3864FFA for ; Fri, 12 Mar 2021 15:46:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232907AbhCLPpm (ORCPT ); Fri, 12 Mar 2021 10:45:42 -0500 Received: from mx4.veeam.com ([104.41.138.86]:50032 "EHLO mx4.veeam.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232878AbhCLPpX (ORCPT ); Fri, 12 Mar 2021 10:45:23 -0500 Received: from mail.veeam.com (prgmbx01.amust.local [172.24.0.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx4.veeam.com (Postfix) with ESMTPS id 741A9874F3; Fri, 12 Mar 2021 18:45:19 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=veeam.com; s=mx4; t=1615563919; bh=ZsS9OaFDK0F73QmlFcb2IgsRKU3DBf+S0NM5uzcqHR0=; h=From:To:CC:Subject:Date:In-Reply-To:References:From; b=dMYiXSmfV8U0INVNZkEnHxS0D6qqWRjuUbs/GKZLMo94CpQMN53kd3NFvqQ1ttdZg HN2Y1unepikb3wRQaXOqEvgyMI7C8Zzgsv9xjQVr/vbd6BmVlvqdFO0OIn8kRDHCa+ Bn7oW8WTJZdJTcyYq0zZGm9h2Ul1mS0r8tg1DWzM= Received: from prgdevlinuxpatch01.amust.local (172.24.14.5) by prgmbx01.amust.local (172.24.0.171) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.721.2; Fri, 12 Mar 2021 16:45:17 +0100 From: Sergei Shtepa To: Christoph Hellwig , Mike Snitzer , Alasdair Kergon , Hannes Reinecke , Jens Axboe , , , , CC: , Subject: [PATCH v7 2/3] block: add bdev_interposer Date: Fri, 12 Mar 2021 18:44:54 +0300 Message-ID: <1615563895-28565-3-git-send-email-sergei.shtepa@veeam.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1615563895-28565-1-git-send-email-sergei.shtepa@veeam.com> References: <1615563895-28565-1-git-send-email-sergei.shtepa@veeam.com> MIME-Version: 1.0 X-Originating-IP: [172.24.14.5] X-ClientProxiedBy: prgmbx02.amust.local (172.24.0.172) To prgmbx01.amust.local (172.24.0.171) X-EsetResult: clean, is OK X-EsetId: 37303A29D2A50B58627366 X-Veeam-MMEX: True Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org bdev_interposer allows to redirect bio requests to another devices. Signed-off-by: Sergei Shtepa --- block/bio.c | 2 ++ block/blk-core.c | 57 +++++++++++++++++++++++++++++++++++++++ block/genhd.c | 54 +++++++++++++++++++++++++++++++++++++ include/linux/blk_types.h | 3 +++ include/linux/blkdev.h | 9 +++++++ 5 files changed, 125 insertions(+) diff --git a/block/bio.c b/block/bio.c index a1c4d2900c7a..0bfbf06475ee 100644 --- a/block/bio.c +++ b/block/bio.c @@ -640,6 +640,8 @@ void __bio_clone_fast(struct bio *bio, struct bio *bio_src) bio_set_flag(bio, BIO_THROTTLED); if (bio_flagged(bio_src, BIO_REMAPPED)) bio_set_flag(bio, BIO_REMAPPED); + if (bio_flagged(bio_src, BIO_INTERPOSED)) + bio_set_flag(bio, BIO_INTERPOSED); bio->bi_opf = bio_src->bi_opf; bio->bi_ioprio = bio_src->bi_ioprio; bio->bi_write_hint = bio_src->bi_write_hint; diff --git a/block/blk-core.c b/block/blk-core.c index fc60ff208497..da1abc4c27a9 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1018,6 +1018,55 @@ static blk_qc_t __submit_bio_noacct_mq(struct bio *bio) return ret; } +static noinline blk_qc_t submit_bio_interposed(struct bio *bio) +{ + blk_qc_t ret = BLK_QC_T_NONE; + struct bio_list bio_list[2] = { }; + struct gendisk *orig_disk; + + if (current->bio_list) { + bio_list_add(¤t->bio_list[0], bio); + return BLK_QC_T_NONE; + } + + orig_disk = bio->bi_bdev->bd_disk; + if (unlikely(bio_queue_enter(bio))) + return BLK_QC_T_NONE; + + current->bio_list = bio_list; + + do { + struct block_device *interposer = bio->bi_bdev->bd_interposer; + + if (unlikely(!interposer)) { + /* interposer was removed */ + bio_list_add(¤t->bio_list[0], bio); + break; + } + /* assign bio to interposer device */ + bio_set_dev(bio, interposer); + bio_set_flag(bio, BIO_INTERPOSED); + + if (!submit_bio_checks(bio)) + break; + /* + * Because the current->bio_list is initialized, + * the submit_bio callback will always return BLK_QC_T_NONE. + */ + interposer->bd_disk->fops->submit_bio(bio); + } while (false); + + current->bio_list = NULL; + + blk_queue_exit(orig_disk->queue); + + /* Resubmit remaining bios */ + while ((bio = bio_list_pop(&bio_list[0]))) + ret = submit_bio_noacct(bio); + + return ret; +} + /** * submit_bio_noacct - re-submit a bio to the block device layer for I/O * @bio: The bio describing the location in memory and on the device. @@ -1029,6 +1078,14 @@ static blk_qc_t __submit_bio_noacct_mq(struct bio *bio) */ blk_qc_t submit_bio_noacct(struct bio *bio) { + /* + * Checking the BIO_INTERPOSED flag is necessary so that the bio + * created by the bdev_interposer do not get to it for processing. + */ + if (bdev_has_interposer(bio->bi_bdev) && + !bio_flagged(bio, BIO_INTERPOSED)) + return submit_bio_interposed(bio); + if (!submit_bio_checks(bio)) return BLK_QC_T_NONE; diff --git a/block/genhd.c b/block/genhd.c index c55e8f0fced1..c840ecffea68 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -30,6 +30,11 @@ static struct kobject *block_depr; DECLARE_RWSEM(bdev_lookup_sem); +/* + * Prevents different block-layer interposers from attaching or detaching + * to the block device at the same time. + */ +static DEFINE_MUTEX(bdev_interposer_attach_lock); /* for extended dynamic devt allocation, currently only one major is used */ #define NR_EXT_DEVT (1 << MINORBITS) @@ -1940,3 +1945,52 @@ static void disk_release_events(struct gendisk *disk) WARN_ON_ONCE(disk->ev && disk->ev->block != 1); kfree(disk->ev); } + +int bdev_interposer_attach(struct block_device *original, + struct block_device *interposer) +{ + int ret = 0; + + if (WARN_ON(((!original) || (!interposer)))) + return -EINVAL; + /* + * interposer should be simple, no a multi-queue device + */ + if (!interposer->bd_disk->fops->submit_bio) + return -EINVAL; + + if (WARN_ON(!blk_mq_is_queue_frozen(original->bd_disk->queue))) + return -EPERM; + + mutex_lock(&bdev_interposer_attach_lock); + + if (bdev_has_interposer(original)) + ret = -EBUSY; + else { + original->bd_interposer = bdgrab(interposer); + if (!original->bd_interposer) + ret = -ENODEV; + } + + mutex_unlock(&bdev_interposer_attach_lock); + + return ret; +} +EXPORT_SYMBOL_GPL(bdev_interposer_attach); + +void bdev_interposer_detach(struct block_device *original) +{ + if (WARN_ON(!original)) + return; + + if (WARN_ON(!blk_mq_is_queue_frozen(original->bd_disk->queue))) + return; + + mutex_lock(&bdev_interposer_attach_lock); + if (bdev_has_interposer(original)) { + bdput(original->bd_interposer); + original->bd_interposer = NULL; + } + mutex_unlock(&bdev_interposer_attach_lock); +} +EXPORT_SYMBOL_GPL(bdev_interposer_detach); diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index db026b6ec15a..13bda4732cf5 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -19,6 +19,7 @@ struct io_context; struct cgroup_subsys_state; typedef void (bio_end_io_t) (struct bio *); struct bio_crypt_ctx; +struct bdev_interposer; struct block_device { sector_t bd_start_sect; @@ -46,6 +47,7 @@ struct block_device { spinlock_t bd_size_lock; /* for bd_inode->i_size updates */ struct gendisk * bd_disk; struct backing_dev_info *bd_bdi; + struct block_device *bd_interposer; /* The counter of freeze processes */ int bd_fsfreeze_count; @@ -304,6 +306,7 @@ enum { BIO_CGROUP_ACCT, /* has been accounted to a cgroup */ BIO_TRACKED, /* set if bio goes through the rq_qos path */ BIO_REMAPPED, + BIO_INTERPOSED, /* bio was reassigned to another block device */ BIO_FLAG_LAST }; diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index bc6bc8383b43..90f62b4197da 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -2031,4 +2031,13 @@ int fsync_bdev(struct block_device *bdev); int freeze_bdev(struct block_device *bdev); int thaw_bdev(struct block_device *bdev); +static inline bool bdev_has_interposer(struct block_device *bdev) +{ + return (bdev->bd_interposer != NULL); +}; + +int bdev_interposer_attach(struct block_device *original, + struct block_device *interposer); +void bdev_interposer_detach(struct block_device *original); + #endif /* _LINUX_BLKDEV_H */ From patchwork Fri Mar 12 15:44:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Shtepa X-Patchwork-Id: 12135037 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5A76C43333 for ; Fri, 12 Mar 2021 15:46:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9D2956500C for ; Fri, 12 Mar 2021 15:46:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232939AbhCLPpm (ORCPT ); Fri, 12 Mar 2021 10:45:42 -0500 Received: from mx4.veeam.com ([104.41.138.86]:50058 "EHLO mx4.veeam.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232257AbhCLPp1 (ORCPT ); Fri, 12 Mar 2021 10:45:27 -0500 Received: from mail.veeam.com (prgmbx01.amust.local [172.24.0.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx4.veeam.com (Postfix) with ESMTPS id F1A3B88818; Fri, 12 Mar 2021 18:45:21 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=veeam.com; s=mx4; t=1615563922; bh=JVhti3gwjryftiNb90ufxdKmut15OEOP1ZGRIcCJTRo=; h=From:To:CC:Subject:Date:In-Reply-To:References:From; b=VLUrxYkmxG7VbZ/0/GA+kzVESCHEKJAIJK5/FTv/1cFvCYGiubE303by1mEgfDbiY UOix5UAbirdDNNhYM2tJ590nVgLJXwSWLnia77a1oInYTUp7VvcEwfvDkWuHvw/2po dlDq97Q25J8MusOvS1zE4naStzU90MLPIP3XBm3c= Received: from prgdevlinuxpatch01.amust.local (172.24.14.5) by prgmbx01.amust.local (172.24.0.171) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.721.2; Fri, 12 Mar 2021 16:45:19 +0100 From: Sergei Shtepa To: Christoph Hellwig , Mike Snitzer , Alasdair Kergon , Hannes Reinecke , Jens Axboe , , , , CC: , Subject: [PATCH v7 3/3] dm: add DM_INTERPOSED_FLAG Date: Fri, 12 Mar 2021 18:44:55 +0300 Message-ID: <1615563895-28565-4-git-send-email-sergei.shtepa@veeam.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1615563895-28565-1-git-send-email-sergei.shtepa@veeam.com> References: <1615563895-28565-1-git-send-email-sergei.shtepa@veeam.com> MIME-Version: 1.0 X-Originating-IP: [172.24.14.5] X-ClientProxiedBy: prgmbx02.amust.local (172.24.0.172) To prgmbx01.amust.local (172.24.0.171) X-EsetResult: clean, is OK X-EsetId: 37303A29D2A50B58627366 X-Veeam-MMEX: True Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org DM_INTERPOSED_FLAG allow to create DM targets on "the fly". Underlying block device opens without a flag FMODE_EXCL. DM target receives bio from the original device via bdev_interposer. Signed-off-by: Sergei Shtepa --- drivers/md/dm-core.h | 3 ++ drivers/md/dm-ioctl.c | 13 ++++++++ drivers/md/dm-table.c | 61 +++++++++++++++++++++++++++++------ drivers/md/dm.c | 38 +++++++++++++++------- include/linux/device-mapper.h | 1 + include/uapi/linux/dm-ioctl.h | 6 ++++ 6 files changed, 101 insertions(+), 21 deletions(-) diff --git a/drivers/md/dm-core.h b/drivers/md/dm-core.h index 5953ff2bd260..9eae419c7b18 100644 --- a/drivers/md/dm-core.h +++ b/drivers/md/dm-core.h @@ -114,6 +114,9 @@ struct mapped_device { bool init_tio_pdu:1; struct srcu_struct io_barrier; + + /* attach target via block-layer interposer */ + bool is_interposed; }; void disable_discard(struct mapped_device *md); diff --git a/drivers/md/dm-ioctl.c b/drivers/md/dm-ioctl.c index 5e306bba4375..2b4c9557c283 100644 --- a/drivers/md/dm-ioctl.c +++ b/drivers/md/dm-ioctl.c @@ -1267,6 +1267,11 @@ static inline fmode_t get_mode(struct dm_ioctl *param) return mode; } +static inline bool get_interposer_flag(struct dm_ioctl *param) +{ + return (param->flags & DM_INTERPOSED_FLAG); +} + static int next_target(struct dm_target_spec *last, uint32_t next, void *end, struct dm_target_spec **spec, char **target_params) { @@ -1293,6 +1298,10 @@ static int populate_table(struct dm_table *table, DMWARN("populate_table: no targets specified"); return -EINVAL; } + if (table->md->is_interposed && (param->target_count != 1)) { + DMWARN("%s: with interposer should be specified only one target", __func__); + return -EINVAL; + } for (i = 0; i < param->target_count; i++) { @@ -1338,6 +1347,8 @@ static int table_load(struct file *filp, struct dm_ioctl *param, size_t param_si if (!md) return -ENXIO; + md->is_interposed = get_interposer_flag(param); + r = dm_table_create(&t, get_mode(param), param->target_count, md); if (r) goto err; @@ -2098,6 +2109,8 @@ int __init dm_early_create(struct dm_ioctl *dmi, if (r) goto err_hash_remove; + md->is_interposed = get_interposer_flag(dmi); + /* add targets */ for (i = 0; i < dmi->target_count; i++) { r = dm_table_add_target(t, spec_array[i]->target_type, diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c index 95391f78b8d5..f6e2eb3f8949 100644 --- a/drivers/md/dm-table.c +++ b/drivers/md/dm-table.c @@ -225,12 +225,13 @@ void dm_table_destroy(struct dm_table *t) /* * See if we've already got a device in the list. */ -static struct dm_dev_internal *find_device(struct list_head *l, dev_t dev) +static struct dm_dev_internal *find_device(struct list_head *l, dev_t dev, bool is_interposed) { struct dm_dev_internal *dd; list_for_each_entry (dd, l, list) - if (dd->dm_dev->bdev->bd_dev == dev) + if ((dd->dm_dev->bdev->bd_dev == dev) && + (dd->dm_dev->is_interposed == is_interposed)) return dd; return NULL; @@ -358,6 +359,18 @@ dev_t dm_get_dev_t(const char *path) } EXPORT_SYMBOL_GPL(dm_get_dev_t); +static inline void dm_disk_freeze(struct gendisk *disk) +{ + blk_mq_freeze_queue(disk->queue); + blk_mq_quiesce_queue(disk->queue); +} + +static inline void dm_disk_unfreeze(struct gendisk *disk) +{ + blk_mq_unquiesce_queue(disk->queue); + blk_mq_unfreeze_queue(disk->queue); +} + /* * Add a device to the list, or just increment the usage count if * it's already present. @@ -385,7 +398,7 @@ int dm_get_device(struct dm_target *ti, const char *path, fmode_t mode, return -ENODEV; } - dd = find_device(&t->devices, dev); + dd = find_device(&t->devices, dev, t->md->is_interposed); if (!dd) { dd = kmalloc(sizeof(*dd), GFP_KERNEL); if (!dd) @@ -398,15 +411,38 @@ int dm_get_device(struct dm_target *ti, const char *path, fmode_t mode, refcount_set(&dd->count, 1); list_add(&dd->list, &t->devices); - goto out; - } else if (dd->dm_dev->mode != (mode | dd->dm_dev->mode)) { r = upgrade_mode(dd, mode, t->md); if (r) return r; + refcount_inc(&dd->count); } - refcount_inc(&dd->count); -out: + + if (t->md->is_interposed) { + struct block_device *original = dd->dm_dev->bdev; + struct block_device *interposer = t->md->disk->part0; + + if ((ti->begin != 0) || (ti->len < bdev_nr_sectors(original))) { + dm_put_device(ti, dd->dm_dev); + DMERR("The interposer device should not be less than the original."); + return -EINVAL; + } + + /* + * Attach mapped interposer device to original. + * It is quite convenient that device mapper creates + * one disk for one block device. + */ + dm_disk_freeze(original->bd_disk); + r = bdev_interposer_attach(original, interposer); + dm_disk_unfreeze(original->bd_disk); + if (r) { + dm_put_device(ti, dd->dm_dev); + DMERR("Failed to attach dm interposer."); + return r; + } + } + *result = dd->dm_dev; return 0; } @@ -446,6 +482,7 @@ void dm_put_device(struct dm_target *ti, struct dm_dev *d) { int found = 0; struct list_head *devices = &ti->table->devices; + struct mapped_device *md = ti->table->md; struct dm_dev_internal *dd; list_for_each_entry(dd, devices, list) { @@ -456,11 +493,17 @@ void dm_put_device(struct dm_target *ti, struct dm_dev *d) } if (!found) { DMWARN("%s: device %s not in table devices list", - dm_device_name(ti->table->md), d->name); + dm_device_name(md), d->name); return; } + if (md->is_interposed) { + dm_disk_freeze(d->bdev->bd_disk); + bdev_interposer_detach(d->bdev); + dm_disk_unfreeze(d->bdev->bd_disk); + } + if (refcount_dec_and_test(&dd->count)) { - dm_put_table_device(ti->table->md, d); + dm_put_table_device(md, d); list_del(&dd->list); kfree(dd); } diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 50b693d776d6..466bf70a66b0 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -762,16 +762,24 @@ static int open_table_device(struct table_device *td, dev_t dev, BUG_ON(td->dm_dev.bdev); - bdev = blkdev_get_by_dev(dev, td->dm_dev.mode | FMODE_EXCL, _dm_claim_ptr); - if (IS_ERR(bdev)) - return PTR_ERR(bdev); + if (md->is_interposed) { - r = bd_link_disk_holder(bdev, dm_disk(md)); - if (r) { - blkdev_put(bdev, td->dm_dev.mode | FMODE_EXCL); - return r; + bdev = blkdev_get_by_dev(dev, td->dm_dev.mode, NULL); + if (IS_ERR(bdev)) + return PTR_ERR(bdev); + } else { + bdev = blkdev_get_by_dev(dev, td->dm_dev.mode | FMODE_EXCL, _dm_claim_ptr); + if (IS_ERR(bdev)) + return PTR_ERR(bdev); + + r = bd_link_disk_holder(bdev, dm_disk(md)); + if (r) { + blkdev_put(bdev, td->dm_dev.mode | FMODE_EXCL); + return r; + } } + td->dm_dev.is_interposed = md->is_interposed; td->dm_dev.bdev = bdev; td->dm_dev.dax_dev = dax_get_by_host(bdev->bd_disk->disk_name); return 0; @@ -785,20 +793,26 @@ static void close_table_device(struct table_device *td, struct mapped_device *md if (!td->dm_dev.bdev) return; - bd_unlink_disk_holder(td->dm_dev.bdev, dm_disk(md)); - blkdev_put(td->dm_dev.bdev, td->dm_dev.mode | FMODE_EXCL); + if (td->dm_dev.is_interposed) + blkdev_put(td->dm_dev.bdev, td->dm_dev.mode); + else { + bd_unlink_disk_holder(td->dm_dev.bdev, dm_disk(md)); + blkdev_put(td->dm_dev.bdev, td->dm_dev.mode | FMODE_EXCL); + } put_dax(td->dm_dev.dax_dev); td->dm_dev.bdev = NULL; td->dm_dev.dax_dev = NULL; } static struct table_device *find_table_device(struct list_head *l, dev_t dev, - fmode_t mode) + fmode_t mode, bool is_interposed) { struct table_device *td; list_for_each_entry(td, l, list) - if (td->dm_dev.bdev->bd_dev == dev && td->dm_dev.mode == mode) + if (td->dm_dev.bdev->bd_dev == dev && + td->dm_dev.mode == mode && + td->dm_dev.is_interposed == is_interposed) return td; return NULL; @@ -811,7 +825,7 @@ int dm_get_table_device(struct mapped_device *md, dev_t dev, fmode_t mode, struct table_device *td; mutex_lock(&md->table_devices_lock); - td = find_table_device(&md->table_devices, dev, mode); + td = find_table_device(&md->table_devices, dev, mode, md->is_interposed); if (!td) { td = kmalloc_node(sizeof(*td), GFP_KERNEL, md->numa_node_id); if (!td) { diff --git a/include/linux/device-mapper.h b/include/linux/device-mapper.h index 7f4ac87c0b32..76a6dfb1cb29 100644 --- a/include/linux/device-mapper.h +++ b/include/linux/device-mapper.h @@ -159,6 +159,7 @@ struct dm_dev { struct block_device *bdev; struct dax_device *dax_dev; fmode_t mode; + bool is_interposed; char name[16]; }; diff --git a/include/uapi/linux/dm-ioctl.h b/include/uapi/linux/dm-ioctl.h index fcff6669137b..fc4d06bb3dbb 100644 --- a/include/uapi/linux/dm-ioctl.h +++ b/include/uapi/linux/dm-ioctl.h @@ -362,4 +362,10 @@ enum { */ #define DM_INTERNAL_SUSPEND_FLAG (1 << 18) /* Out */ +/* + * If set, the underlying device should open without FMODE_EXCL + * and attach mapped device via bdev_interposer. + */ +#define DM_INTERPOSED_FLAG (1 << 19) /* In */ + #endif /* _LINUX_DM_IOCTL_H */