From patchwork Wed Dec 14 18:55:19 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Wheeler X-Patchwork-Id: 9474535 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 3165A60571 for ; Wed, 14 Dec 2016 19:10:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1C03B286F1 for ; Wed, 14 Dec 2016 19:10:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0E27428734; Wed, 14 Dec 2016 19:10:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 91CDE286F1 for ; Wed, 14 Dec 2016 19:10:45 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id uBEJ9YvN059026; Wed, 14 Dec 2016 14:09:35 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id uBEJ9YqP012190 for ; Wed, 14 Dec 2016 14:09:34 -0500 Received: from mx1.redhat.com (ext-mx10.extmail.prod.ext.phx2.redhat.com [10.5.110.39]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id uBEJ9Yiw014711 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Wed, 14 Dec 2016 14:09:34 -0500 Received: from mail.ewheeler.net (mx.ewheeler.net [66.155.3.69]) by mx1.redhat.com (Postfix) with ESMTP id B166F61B96 for ; Wed, 14 Dec 2016 19:09:32 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.ewheeler.net (Postfix) with ESMTP id 2B046A03B0 for ; Wed, 14 Dec 2016 11:04:26 -0800 (PST) X-Virus-Scanned: amavisd-new at ewheeler.net Received: from mail.ewheeler.net ([127.0.0.1]) by localhost (mail.ewheeler.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id d4MDGCuMhNgV for ; Wed, 14 Dec 2016 11:04:25 -0800 (PST) Received: by mail.ewheeler.net (Postfix, from userid 503) id 8B2DBA0414; Wed, 14 Dec 2016 11:04:25 -0800 (PST) Date: Wed, 14 Dec 2016 10:55:19 -0800 (PST) From: Eric Wheeler X-X-Sender: lists@mail.ewheeler.net To: Mikulas Patocka , Alasdair Kergon , Mike Snitzer , Shaohua Li , Jens Axboe , linux-block@vger.kernel.org, dm-devel@redhat.com Message-ID: User-Agent: Alpine 2.11 (LRH 23 2013-08-11) MIME-Version: 1.0 Status: R X-Status: X-Keywords: X-UID: 2 X-Greylist: Delayed for 00:05:03 by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 14 Dec 2016 19:09:32 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 14 Dec 2016 19:09:32 +0000 (UTC) for IP:'66.155.3.69' DOMAIN:'mx.ewheeler.net' HELO:'mail.ewheeler.net' FROM:'dm-devel@lists.ewheeler.net' RCPT:'' X-RedHat-Spam-Score: -2.299 (BAYES_50, RP_MATCHES_RCVD) 66.155.3.69 mx.ewheeler.net 66.155.3.69 mx.ewheeler.net X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Scanned-By: MIMEDefang 2.78 on 10.5.110.39 X-loop: dm-devel@redhat.com Subject: [dm-devel] [PATCH] dm-crypt: fix lost ioprio when queuing crypto bios from task with ioprio X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Virus-Scanned: ClamAV using ClamSMTP Since dm-crypt queues writes (and sometimes reads) to a different kernel thread (workqueue), the bios will dispatch from tasks with different io_context->ioprio settings than the submitting task, thus giving incorrect ioprio hints to the io scheduler. By assigning the ioprio to the bio before queuing to a workqueue, the original submitting task's io_context->ioprio setting can be retained through the life of the bio. We only set the bio's ioprio in dm-crypt if not already set (by somewhere else, higher in the stack). Signed-off-by: Eric Wheeler Cc: Mikulas Patocka Cc: Alasdair Kergon Cc: Mike Snitzer Cc: Shaohua Li Cc: Jens Axboe Cc: stable@vger.kernel.org --- block/bio.c | 1 + drivers/md/dm-crypt.c | 11 +++++++++-- include/linux/bio.h | 23 +++++++++++++++++++++++ 3 files changed, 33 insertions(+), 2 deletions(-) diff --git a/block/bio.c b/block/bio.c index db85c57..1a529ff 100644 --- a/block/bio.c +++ b/block/bio.c @@ -584,6 +584,7 @@ void __bio_clone_fast(struct bio *bio, struct bio *bio_src) bio->bi_iter = bio_src->bi_iter; bio->bi_io_vec = bio_src->bi_io_vec; + bio_set_prio(bio, bio_prio(bio_src)); bio_clone_blkcg_association(bio, bio_src); } EXPORT_SYMBOL(__bio_clone_fast); diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index 9b99ee9..ea7e102 100644 --- a/drivers/md/dm-crypt.c +++ b/drivers/md/dm-crypt.c @@ -1133,6 +1133,7 @@ static void clone_init(struct dm_crypt_io *io, struct bio *clone) clone->bi_private = io; clone->bi_end_io = crypt_endio; clone->bi_bdev = cc->dev->bdev; + bio_set_prio(clone, bio_prio(io->base_bio)); bio_set_op_attrs(clone, bio_op(io->base_bio), bio_flags(io->base_bio)); } @@ -2070,10 +2071,16 @@ static int crypt_map(struct dm_target *ti, struct bio *bio) io->ctx.req = (struct skcipher_request *)(io + 1); if (bio_data_dir(io->base_bio) == READ) { - if (kcryptd_io_read(io, GFP_NOWAIT)) + if (kcryptd_io_read(io, GFP_NOWAIT)) { + if (!ioprio_valid(bio_prio(io->base_bio))) + bio_set_task_prio(io->base_bio, current); kcryptd_queue_read(io); - } else + } + } else { + if (!ioprio_valid(bio_prio(io->base_bio))) + bio_set_task_prio(io->base_bio, current); kcryptd_queue_crypt(io); + } return DM_MAPIO_SUBMITTED; } diff --git a/include/linux/bio.h b/include/linux/bio.h index 97cb48f..0b4f8bb 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -62,6 +62,29 @@ #define bio_sectors(bio) ((bio)->bi_iter.bi_size >> 9) #define bio_end_sector(bio) ((bio)->bi_iter.bi_sector + bio_sectors((bio))) +/* Set the bio's ioprio to that of the task's io_context->ioprio, if any. + * Always returns io_context->ioprio (or 0 if none); bio can be NULL. + */ +static inline unsigned short bio_set_task_prio( + struct bio *bio, struct task_struct *task) +{ + struct io_context *ioc; + unsigned short ioprio = 0; + + ioc = get_task_io_context(current, GFP_NOIO, NUMA_NO_NODE); + if (ioc) { + if (ioprio_valid(ioc->ioprio)) { + ioprio = ioc->ioprio; + put_io_context(ioc); + } + + if (bio != NULL && ioprio_valid(ioprio)) + bio_set_prio(bio, ioc->ioprio); + } + + return ioprio; +} + /* * Check whether this bio carries any data or not. A NULL bio is allowed. */