From patchwork Wed Sep 16 03:53:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 11778899 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 539E7139F for ; Wed, 16 Sep 2020 03:54:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 382E1221E5 for ; Wed, 16 Sep 2020 03:54:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600228463; bh=gkyBV96JQe05ZTIpHO/yhCZume0/K4OfWBC+0QarbAE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=DJ7jZFU9a7vX/KlBR2DZ7i49IUcVEgPPdmqVOGYfFzoFotdWliwu/WitAPEm0fLWU Qes092azgGE+BMrdM9G969JzP+HoySuzPaympFUTAMJUoWqYaV91lxfFmOcpKH4027 kiksivJmIPY1eUwqrgHkcneSIF+cRuJSACyy/pK0= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726136AbgIPDyN (ORCPT ); Tue, 15 Sep 2020 23:54:13 -0400 Received: from mail.kernel.org ([198.145.29.99]:50374 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726119AbgIPDyH (ORCPT ); Tue, 15 Sep 2020 23:54:07 -0400 Received: from sol.attlocal.net (172-10-235-113.lightspeed.sntcca.sbcglobal.net [172.10.235.113]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 7385A221E3; Wed, 16 Sep 2020 03:54:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600228446; bh=gkyBV96JQe05ZTIpHO/yhCZume0/K4OfWBC+0QarbAE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nGT1plsjhxPd2lZmB1uTSE51w+k1bJWhkoFfS2OleyELN0/9eH71rWlWLsuxKWKRZ 1akoka6Z/z+KDjhf0Sc0mYu5K37oYvxyHm5bqHwCGSYPp94zNh7r3cx7TloNqhzHO8 /hdFMp52Wk5a68l6M1xb8qmUOb+tqH5F9Ypcelxo= From: Eric Biggers To: linux-block@vger.kernel.org, Jens Axboe Cc: dm-devel@redhat.com, Satya Tangirala , Miaohe Lin Subject: [PATCH v2 1/3] block: make bio_crypt_clone() able to fail Date: Tue, 15 Sep 2020 20:53:13 -0700 Message-Id: <20200916035315.34046-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200916035315.34046-1-ebiggers@kernel.org> References: <20200916035315.34046-1-ebiggers@kernel.org> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Eric Biggers bio_crypt_clone() assumes its gfp_mask argument always includes __GFP_DIRECT_RECLAIM, so that the mempool_alloc() will always succeed. However, bio_crypt_clone() might be called with GFP_ATOMIC via setup_clone() in drivers/md/dm-rq.c, or with GFP_NOWAIT via kcryptd_io_read() in drivers/md/dm-crypt.c. Neither case is currently reachable with a bio that actually has an encryption context. However, it's fragile to rely on this. Just make bio_crypt_clone() able to fail, analogous to bio_integrity_clone(). Reported-by: Miaohe Lin Cc: Satya Tangirala Signed-off-by: Eric Biggers Reviewed-by: Satya Tangirala Reviewed-by: Mike Snitzer --- block/bio.c | 20 +++++++++----------- block/blk-crypto.c | 5 ++++- block/bounce.c | 19 +++++++++---------- drivers/md/dm.c | 7 ++++--- include/linux/blk-crypto.h | 20 ++++++++++++++++---- 5 files changed, 42 insertions(+), 29 deletions(-) diff --git a/block/bio.c b/block/bio.c index a9931f23d9332..b42e046b12eb3 100644 --- a/block/bio.c +++ b/block/bio.c @@ -713,20 +713,18 @@ struct bio *bio_clone_fast(struct bio *bio, gfp_t gfp_mask, struct bio_set *bs) __bio_clone_fast(b, bio); - bio_crypt_clone(b, bio, gfp_mask); + if (bio_crypt_clone(b, bio, gfp_mask) < 0) + goto err_put; - if (bio_integrity(bio)) { - int ret; - - ret = bio_integrity_clone(b, bio, gfp_mask); - - if (ret < 0) { - bio_put(b); - return NULL; - } - } + if (bio_integrity(bio) && + bio_integrity_clone(b, bio, gfp_mask) < 0) + goto err_put; return b; + +err_put: + bio_put(b); + return NULL; } EXPORT_SYMBOL(bio_clone_fast); diff --git a/block/blk-crypto.c b/block/blk-crypto.c index 2d5e60023b08b..a3f27a19067c9 100644 --- a/block/blk-crypto.c +++ b/block/blk-crypto.c @@ -95,10 +95,13 @@ void __bio_crypt_free_ctx(struct bio *bio) bio->bi_crypt_context = NULL; } -void __bio_crypt_clone(struct bio *dst, struct bio *src, gfp_t gfp_mask) +int __bio_crypt_clone(struct bio *dst, struct bio *src, gfp_t gfp_mask) { dst->bi_crypt_context = mempool_alloc(bio_crypt_ctx_pool, gfp_mask); + if (!dst->bi_crypt_context) + return -ENOMEM; *dst->bi_crypt_context = *src->bi_crypt_context; + return 0; } EXPORT_SYMBOL_GPL(__bio_crypt_clone); diff --git a/block/bounce.c b/block/bounce.c index 431be88a02405..162a6eee89996 100644 --- a/block/bounce.c +++ b/block/bounce.c @@ -267,22 +267,21 @@ static struct bio *bounce_clone_bio(struct bio *bio_src, gfp_t gfp_mask, break; } - bio_crypt_clone(bio, bio_src, gfp_mask); + if (bio_crypt_clone(bio, bio_src, gfp_mask) < 0) + goto err_put; - if (bio_integrity(bio_src)) { - int ret; - - ret = bio_integrity_clone(bio, bio_src, gfp_mask); - if (ret < 0) { - bio_put(bio); - return NULL; - } - } + if (bio_integrity(bio_src) && + bio_integrity_clone(bio, bio_src, gfp_mask) < 0) + goto err_put; bio_clone_blkg_association(bio, bio_src); blkcg_bio_issue_init(bio); return bio; + +err_put: + bio_put(bio); + return NULL; } static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig, diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 3dedd9cc4fb65..5487c3ff74b51 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1326,14 +1326,15 @@ static int clone_bio(struct dm_target_io *tio, struct bio *bio, sector_t sector, unsigned len) { struct bio *clone = &tio->clone; + int r; __bio_clone_fast(clone, bio); - bio_crypt_clone(clone, bio, GFP_NOIO); + r = bio_crypt_clone(clone, bio, GFP_NOIO); + if (r < 0) + return r; if (bio_integrity(bio)) { - int r; - if (unlikely(!dm_target_has_integrity(tio->ti->type) && !dm_target_passes_integrity(tio->ti->type))) { DMWARN("%s: the target %s doesn't support integrity data.", diff --git a/include/linux/blk-crypto.h b/include/linux/blk-crypto.h index e82342907f2b1..69b24fe92cbf1 100644 --- a/include/linux/blk-crypto.h +++ b/include/linux/blk-crypto.h @@ -112,12 +112,24 @@ static inline bool bio_has_crypt_ctx(struct bio *bio) #endif /* CONFIG_BLK_INLINE_ENCRYPTION */ -void __bio_crypt_clone(struct bio *dst, struct bio *src, gfp_t gfp_mask); -static inline void bio_crypt_clone(struct bio *dst, struct bio *src, - gfp_t gfp_mask) +int __bio_crypt_clone(struct bio *dst, struct bio *src, gfp_t gfp_mask); +/** + * bio_crypt_clone - clone bio encryption context + * @dst: destination bio + * @src: source bio + * @gfp_mask: memory allocation flags + * + * If @src has an encryption context, clone it to @dst. + * + * Return: 0 on success, -ENOMEM if out of memory. -ENOMEM is only possible if + * @gfp_mask doesn't include %__GFP_DIRECT_RECLAIM. + */ +static inline int bio_crypt_clone(struct bio *dst, struct bio *src, + gfp_t gfp_mask) { if (bio_has_crypt_ctx(src)) - __bio_crypt_clone(dst, src, gfp_mask); + return __bio_crypt_clone(dst, src, gfp_mask); + return 0; } #endif /* __LINUX_BLK_CRYPTO_H */ From patchwork Wed Sep 16 03:53:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 11778895 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1EC50746 for ; Wed, 16 Sep 2020 03:54:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0674A22207 for ; Wed, 16 Sep 2020 03:54:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600228457; bh=3YnrepG0AoOP07GUAq1nq3E1pIVE5+36kfYE5z5wXGk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=p09yuUajTFaBhT5euoSyl8E0OGtzrhX20Kn7dWDyzM67eyM4jvVdSxdx/oiPvCSCS NvdINzCsTgQP6eDUSUMl8ZD9SRIA8ju67IZytJmKSNIJBfILwCNJbRl3ta3nLHYjIE o6I4EZLfbdoeZNGS2Nx+dWnJGw34eG9BDtLyJVSs= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726133AbgIPDyN (ORCPT ); Tue, 15 Sep 2020 23:54:13 -0400 Received: from mail.kernel.org ([198.145.29.99]:50384 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726125AbgIPDyH (ORCPT ); Tue, 15 Sep 2020 23:54:07 -0400 Received: from sol.attlocal.net (172-10-235-113.lightspeed.sntcca.sbcglobal.net [172.10.235.113]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AF638221E5; Wed, 16 Sep 2020 03:54:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600228446; bh=3YnrepG0AoOP07GUAq1nq3E1pIVE5+36kfYE5z5wXGk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=1CbJITxFpguU2m/lUQTS9SdTngkkcyaXDywEXO85xA7RTNucFaYd3mnpNixjzpiPg HrfFnnVcxvdxHJXgupsvpM56L6s+j5DsRW1NS9TwH3h3oD1B23iJHUUnc1ITCh5QHk ApE2wnUw3oowhCpU5hlrtS/T4KWuvxcU2aOhymNM= From: Eric Biggers To: linux-block@vger.kernel.org, Jens Axboe Cc: dm-devel@redhat.com, Satya Tangirala , Miaohe Lin Subject: [PATCH v2 2/3] block: make blk_crypto_rq_bio_prep() able to fail Date: Tue, 15 Sep 2020 20:53:14 -0700 Message-Id: <20200916035315.34046-3-ebiggers@kernel.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200916035315.34046-1-ebiggers@kernel.org> References: <20200916035315.34046-1-ebiggers@kernel.org> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Eric Biggers blk_crypto_rq_bio_prep() assumes its gfp_mask argument always includes __GFP_DIRECT_RECLAIM, so that the mempool_alloc() will always succeed. However, blk_crypto_rq_bio_prep() might be called with GFP_ATOMIC via setup_clone() in drivers/md/dm-rq.c. This case isn't currently reachable with a bio that actually has an encryption context. However, it's fragile to rely on this. Just make blk_crypto_rq_bio_prep() able to fail. Cc: Miaohe Lin Suggested-by: Satya Tangirala Signed-off-by: Eric Biggers Reviewed-by: Satya Tangirala Reviewed-by: Mike Snitzer --- block/blk-core.c | 8 +++++--- block/blk-crypto-internal.h | 21 ++++++++++++++++----- block/blk-crypto.c | 18 +++++++----------- block/blk-mq.c | 7 ++++++- 4 files changed, 34 insertions(+), 20 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index ca3f0f00c9435..fbeaa49f6fe2c 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1620,8 +1620,10 @@ int blk_rq_prep_clone(struct request *rq, struct request *rq_src, if (rq->bio) { rq->biotail->bi_next = bio; rq->biotail = bio; - } else + } else { rq->bio = rq->biotail = bio; + } + bio = NULL; } /* Copy attributes of the original request to the clone request. */ @@ -1634,8 +1636,8 @@ int blk_rq_prep_clone(struct request *rq, struct request *rq_src, rq->nr_phys_segments = rq_src->nr_phys_segments; rq->ioprio = rq_src->ioprio; - if (rq->bio) - blk_crypto_rq_bio_prep(rq, rq->bio, gfp_mask); + if (rq->bio && blk_crypto_rq_bio_prep(rq, rq->bio, gfp_mask) < 0) + goto free_and_out; return 0; diff --git a/block/blk-crypto-internal.h b/block/blk-crypto-internal.h index d2b0f565d83cb..0d36aae538d7b 100644 --- a/block/blk-crypto-internal.h +++ b/block/blk-crypto-internal.h @@ -142,13 +142,24 @@ static inline void blk_crypto_free_request(struct request *rq) __blk_crypto_free_request(rq); } -void __blk_crypto_rq_bio_prep(struct request *rq, struct bio *bio, - gfp_t gfp_mask); -static inline void blk_crypto_rq_bio_prep(struct request *rq, struct bio *bio, - gfp_t gfp_mask) +int __blk_crypto_rq_bio_prep(struct request *rq, struct bio *bio, + gfp_t gfp_mask); +/** + * blk_crypto_rq_bio_prep - Prepare a request's crypt_ctx when its first bio + * is inserted + * @rq: The request to prepare + * @bio: The first bio being inserted into the request + * @gfp_mask: Memory allocation flags + * + * Return: 0 on success, -ENOMEM if out of memory. -ENOMEM is only possible if + * @gfp_mask doesn't include %__GFP_DIRECT_RECLAIM. + */ +static inline int blk_crypto_rq_bio_prep(struct request *rq, struct bio *bio, + gfp_t gfp_mask) { if (bio_has_crypt_ctx(bio)) - __blk_crypto_rq_bio_prep(rq, bio, gfp_mask); + return __blk_crypto_rq_bio_prep(rq, bio, gfp_mask); + return 0; } /** diff --git a/block/blk-crypto.c b/block/blk-crypto.c index a3f27a19067c9..bbe7974fd74f0 100644 --- a/block/blk-crypto.c +++ b/block/blk-crypto.c @@ -283,20 +283,16 @@ bool __blk_crypto_bio_prep(struct bio **bio_ptr) return false; } -/** - * __blk_crypto_rq_bio_prep - Prepare a request's crypt_ctx when its first bio - * is inserted - * - * @rq: The request to prepare - * @bio: The first bio being inserted into the request - * @gfp_mask: gfp mask - */ -void __blk_crypto_rq_bio_prep(struct request *rq, struct bio *bio, - gfp_t gfp_mask) +int __blk_crypto_rq_bio_prep(struct request *rq, struct bio *bio, + gfp_t gfp_mask) { - if (!rq->crypt_ctx) + if (!rq->crypt_ctx) { rq->crypt_ctx = mempool_alloc(bio_crypt_ctx_pool, gfp_mask); + if (!rq->crypt_ctx) + return -ENOMEM; + } *rq->crypt_ctx = *bio->bi_crypt_context; + return 0; } /** diff --git a/block/blk-mq.c b/block/blk-mq.c index e04b759add758..9ec0e7149ae69 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1940,13 +1940,18 @@ void blk_mq_flush_plug_list(struct blk_plug *plug, bool from_schedule) static void blk_mq_bio_to_request(struct request *rq, struct bio *bio, unsigned int nr_segs) { + int err; + if (bio->bi_opf & REQ_RAHEAD) rq->cmd_flags |= REQ_FAILFAST_MASK; rq->__sector = bio->bi_iter.bi_sector; rq->write_hint = bio->bi_write_hint; blk_rq_bio_prep(rq, bio, nr_segs); - blk_crypto_rq_bio_prep(rq, bio, GFP_NOIO); + + /* This can't fail, since GFP_NOIO includes __GFP_DIRECT_RECLAIM. */ + err = blk_crypto_rq_bio_prep(rq, bio, GFP_NOIO); + WARN_ON_ONCE(err); blk_account_io_start(rq); } From patchwork Wed Sep 16 03:53:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 11778897 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8057714F6 for ; Wed, 16 Sep 2020 03:54:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5E299221E5 for ; Wed, 16 Sep 2020 03:54:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600228457; bh=sKfndHdiXCcM6425hNhDhw2SooX9ZNQ+cnCmVRd8T4k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=FggZmQM2A3+MtqaSZj6PzM+qzHsNd68VNlwtW2HRImWFamG9qHxrj/CrP9Le24ZNk dvM7vb8Vdsdf7IXjFrWV+kAmA9ZILNRRuiVxXcFJ/bXep6gmdEOQ7OiOYgduu52LZr ST5rDrkmm7hPwaAgIJlld4pbp2AEanT/Cf51fcfA= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726125AbgIPDyN (ORCPT ); Tue, 15 Sep 2020 23:54:13 -0400 Received: from mail.kernel.org ([198.145.29.99]:50396 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726128AbgIPDyH (ORCPT ); Tue, 15 Sep 2020 23:54:07 -0400 Received: from sol.attlocal.net (172-10-235-113.lightspeed.sntcca.sbcglobal.net [172.10.235.113]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id EC4FA221E7; Wed, 16 Sep 2020 03:54:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600228447; bh=sKfndHdiXCcM6425hNhDhw2SooX9ZNQ+cnCmVRd8T4k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XoetrHzn1PwlFoJ5BRHB0T4PvB0DmKVyDgeMIJK8lH19VxnnSnaCUxpMLxhcD6svC oqEtSGiGSieXVjistOdWBFbFNqg+ewFRGTApVxQAqW204F4CfHLNVJsWq1xsqmkn/K UUph6N5aO5ovsoGw+SmuzUrVzNva8xXYLC/KK+Aw= From: Eric Biggers To: linux-block@vger.kernel.org, Jens Axboe Cc: dm-devel@redhat.com, Satya Tangirala , Miaohe Lin Subject: [PATCH v2 3/3] block: warn if !__GFP_DIRECT_RECLAIM in bio_crypt_set_ctx() Date: Tue, 15 Sep 2020 20:53:15 -0700 Message-Id: <20200916035315.34046-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200916035315.34046-1-ebiggers@kernel.org> References: <20200916035315.34046-1-ebiggers@kernel.org> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Eric Biggers bio_crypt_set_ctx() assumes its gfp_mask argument always includes __GFP_DIRECT_RECLAIM, so that the mempool_alloc() will always succeed. For now this assumption is still fine, since no callers violate it. Making bio_crypt_set_ctx() able to fail would add unneeded complexity. However, if a caller didn't use __GFP_DIRECT_RECLAIM, it would be very hard to notice the bug. Make it easier by adding a WARN_ON_ONCE(). Cc: Miaohe Lin Cc: Satya Tangirala Signed-off-by: Eric Biggers Reviewed-by: Satya Tangirala --- block/blk-crypto.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/block/blk-crypto.c b/block/blk-crypto.c index bbe7974fd74f0..5da43f0973b46 100644 --- a/block/blk-crypto.c +++ b/block/blk-crypto.c @@ -81,7 +81,15 @@ subsys_initcall(bio_crypt_ctx_init); void bio_crypt_set_ctx(struct bio *bio, const struct blk_crypto_key *key, const u64 dun[BLK_CRYPTO_DUN_ARRAY_SIZE], gfp_t gfp_mask) { - struct bio_crypt_ctx *bc = mempool_alloc(bio_crypt_ctx_pool, gfp_mask); + struct bio_crypt_ctx *bc; + + /* + * The caller must use a gfp_mask that contains __GFP_DIRECT_RECLAIM so + * that the mempool_alloc() can't fail. + */ + WARN_ON_ONCE(!(gfp_mask & __GFP_DIRECT_RECLAIM)); + + bc = mempool_alloc(bio_crypt_ctx_pool, gfp_mask); bc->bc_key = key; memcpy(bc->bc_dun, dun, sizeof(bc->bc_dun));