From patchwork Tue Dec 12 11:11:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyu Jin X-Patchwork-Id: 13488990 X-Patchwork-Delegate: snitzer@redhat.com Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A0BB5CD22 for ; Tue, 12 Dec 2023 11:12:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dk6rsGb/" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-1d2eb06ab11so21599015ad.1 for ; Tue, 12 Dec 2023 03:12:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702379536; x=1702984336; darn=lists.linux.dev; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kk71sp729csxdUViLno20l6lcdfkMyL/Wdv2yhvpzTA=; b=dk6rsGb/pqxKuJb3o7ntdoVr7Xcw8wFXVLDxvhXxGSrxWOYbgaZdSGZBkBvtIIRENh FC6H5Wy+rhc1S6jXS73zP3YGwUJWvgVQw9vYSYVOQ4YeRdzu7sAUxVFxqW0qjHGHdkao VG1NxqoFbwOqUkZnDmwOcncFWNidGuc05CKNZ8lhi1XdRgSmkE1uxtDgx4lvXnc+zZdA LNGGenBxQZY1fmUbeEs2q8I1RLKrKOESK1wjYUxqJP7dtZ/lKx7PWxJGAVsNJ/9mpnC0 +BZLV7JeemVRN76HBN3nWHYsmD55esWoLyFVM0QsIPL53jk/SxNgftorRwrQ9iJTBzhe 065A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702379536; x=1702984336; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kk71sp729csxdUViLno20l6lcdfkMyL/Wdv2yhvpzTA=; b=UDy4cWHEtjaDoCa22VPFnqGGOWTTLexuRVy7glGEHiGTxkO01cQ3P6YJkvhRVypbUl 59VTsfa5q8J82vEYRPNwUDDrPzehB1bJCNLnsCwpXLO/TRxZV/VvOS+Gep/UJE69k3lQ DLTo2ns/0sOhMui7y5m+cILHN4Y2WKZuE5kNRjm9oJUHQVpnHF9yHUS3mx3UEk/A3WeR tuhsYWWcARzuFiAbNlOBirj7SjJEH5D5Oa4ac0brEs14yr54vvX0CCcoG7acwdjcBk8T Ze3IEXsOlwgKi0o/CNVXLQk1hhcw6Uy+LAf3vvjzMaeb9pfKErq1elLqOACHump3N5Qu 56TA== X-Gm-Message-State: AOJu0YyaWNUuXiG1rBs6Hq2lhUr4Se94UvtHDu+MDbjIt+qJ5tpQ0LEf tg+/w35x8U8IDx9ytNBfIBo= X-Google-Smtp-Source: AGHT+IH+ypC0oT6XrZH3uyP+6+iEZx9SsOkdE5qJlw+adVZmAWQ+SlcAzutITFQmVX46kV8vnEcdig== X-Received: by 2002:a17:902:6b02:b0:1d0:90bf:cbb7 with SMTP id o2-20020a1709026b0200b001d090bfcbb7mr2387161plk.66.1702379535806; Tue, 12 Dec 2023 03:12:15 -0800 (PST) Received: from ubuntu.. ([117.18.48.102]) by smtp.gmail.com with ESMTPSA id 21-20020a170902ee5500b001d0511c990csm8345453plo.237.2023.12.12.03.12.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:12:15 -0800 (PST) From: Hongyu Jin To: agk@redhat.com, snitzer@kernel.org, mpatocka@redhat.com, axboe@kernel.dk, ebiggers@kernel.org Cc: zhiguo.niu@unisoc.com, ke.wang@unisoc.com, yibin.ding@unisoc.com, hongyu.jin@unisoc.com, linux-kernel@vger.kernel.org, dm-devel@lists.linux.dev, linux-block@vger.kernel.org Subject: [PATCH v4 1/5] block: Fix bio IO priority setting Date: Tue, 12 Dec 2023 19:11:46 +0800 Message-Id: <20231212111150.18155-2-hongyu.jin.cn@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> References: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Hongyu Jin Move bio_set_ioprio() into submit_bio(): 1. Only call bio_set_ioprio() once to set the priority of original bio, the bio that cloned and splited from original bio will auto inherit the priority of original bio in clone process. 2. The IO priority can be passed to module that implement struct gendisk::fops::submit_bio, help resolve some of the IO priority loss issues. This patch depends on commit 82b74cac2849 ("blk-ioprio: Convert from rqos policy to direct call") Fixes: a78418e6a04c ("block: Always initialize bio IO priority on submit") Co-developed-by: Yibin Ding Signed-off-by: Yibin Ding Signed-off-by: Hongyu Jin --- block/blk-core.c | 10 ++++++++++ block/blk-mq.c | 11 ----------- 2 files changed, 10 insertions(+), 11 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index fdf25b8d6e78..68158c327aea 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -49,6 +49,7 @@ #include "blk-pm.h" #include "blk-cgroup.h" #include "blk-throttle.h" +#include "blk-ioprio.h" struct dentry *blk_debugfs_root; @@ -809,6 +810,14 @@ void submit_bio_noacct(struct bio *bio) } EXPORT_SYMBOL(submit_bio_noacct); +static void bio_set_ioprio(struct bio *bio) +{ + /* Nobody set ioprio so far? Initialize it based on task's nice value */ + if (IOPRIO_PRIO_CLASS(bio->bi_ioprio) == IOPRIO_CLASS_NONE) + bio->bi_ioprio = get_current_ioprio(); + blkcg_set_ioprio(bio); +} + /** * submit_bio - submit a bio to the block device layer for I/O * @bio: The &struct bio which describes the I/O @@ -831,6 +840,7 @@ void submit_bio(struct bio *bio) count_vm_events(PGPGOUT, bio_sectors(bio)); } + bio_set_ioprio(bio); submit_bio_noacct(bio); } EXPORT_SYMBOL(submit_bio); diff --git a/block/blk-mq.c b/block/blk-mq.c index e2d11183f62e..a6e2609df9c9 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -40,7 +40,6 @@ #include "blk-stat.h" #include "blk-mq-sched.h" #include "blk-rq-qos.h" -#include "blk-ioprio.h" static DEFINE_PER_CPU(struct llist_head, blk_cpu_done); static DEFINE_PER_CPU(call_single_data_t, blk_cpu_csd); @@ -2922,14 +2921,6 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q, return rq; } -static void bio_set_ioprio(struct bio *bio) -{ - /* Nobody set ioprio so far? Initialize it based on task's nice value */ - if (IOPRIO_PRIO_CLASS(bio->bi_ioprio) == IOPRIO_CLASS_NONE) - bio->bi_ioprio = get_current_ioprio(); - blkcg_set_ioprio(bio); -} - /** * blk_mq_submit_bio - Create and send a request to block device. * @bio: Bio pointer. @@ -2963,8 +2954,6 @@ void blk_mq_submit_bio(struct bio *bio) if (!bio_integrity_prep(bio)) return; - bio_set_ioprio(bio); - rq = blk_mq_get_cached_request(q, plug, &bio, nr_segs); if (!rq) { if (!bio) From patchwork Tue Dec 12 11:11:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyu Jin X-Patchwork-Id: 13488991 X-Patchwork-Delegate: snitzer@redhat.com Received: from mail-pg1-f173.google.com (mail-pg1-f173.google.com [209.85.215.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 368F85CD1F for ; Tue, 12 Dec 2023 11:12:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ezrIbJHf" Received: by mail-pg1-f173.google.com with SMTP id 41be03b00d2f7-5c701bd9a3cso1010928a12.0 for ; Tue, 12 Dec 2023 03:12:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702379540; x=1702984340; darn=lists.linux.dev; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3C4XGNOSQh2qdBXT10pq8W1umCRdPJt9fc3zqaA6RvE=; b=ezrIbJHf+yx+8b3G8Dr9mYsgdx3PbOJ9jY6J3utNxOkd63MmBCAOEhdoRe+mXIlxZe pw/wKNpufkBC8CsLaTiXUhoEqBeNwJO+mjIufVnSKVJIJM371D+r7EnHYZvWDMM67Kxl zojJuA1iABTMLuIZHQitofyvnA6dDlIaw5Q28kajBToO3zXsDjDryYeRrYDruwOuvkqJ gQJRnRzpLi2fDHP9rVUUTlXnae31pd//hOeSFe+iQHuJHZWXMIF0LyroL7yO1amOG51q cDIVageeRs7vW8UIqqQbHCzQwhW4U1tCQxvfWosAbp+BHsQlaVBHMuvzWMlwz3B2mQ6K V80g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702379540; x=1702984340; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3C4XGNOSQh2qdBXT10pq8W1umCRdPJt9fc3zqaA6RvE=; b=Fgxestge7hYCBczcG8IbwApwES8kO0MvV562qrABTmCf4xu3Kzg31XfC+g2mj7UGgK 4uUUIiaNqJHExzw493qR/+fY/PFBX1dbdgZxKiD62SayWGyouuORYy5QSrzN7mB5xh59 xNlmPgf9i9QmZviARx1e5xH2+QI3m9Dibmo5ERyGkGFIGrsjGVI0Eg/oSZAAFt/NaIOU FpaMX7saq/QVDUXM/+2x0+SQZjIk+kFV3QgOwlxpsNgEIB6kw87oJs94jn0KcSYYhEiV I0BD4ZjWsc/OAJw1xaE1WL1FfgxzjeRKVTrPjrJjBURGY7vG8qFsX0lEgqQeoP9PlnjR DuJA== X-Gm-Message-State: AOJu0YwakbDZahEDzj7vzQERYDhyM030SqmgXSoPAVDBsN2FMijzIt63 TrGDpkIQR4kK+sl8iTiAZUI= X-Google-Smtp-Source: AGHT+IH2Bn0L9xCh0Tvq4bOuPTmr0lgYXz5vcEeU0cMWIvfZTFc/5sRqby35e2TOr2XjIqb2AQ+L7w== X-Received: by 2002:a05:6a20:3d83:b0:190:7b07:b7c9 with SMTP id s3-20020a056a203d8300b001907b07b7c9mr2389253pzi.66.1702379540250; Tue, 12 Dec 2023 03:12:20 -0800 (PST) Received: from ubuntu.. ([117.18.48.102]) by smtp.gmail.com with ESMTPSA id 21-20020a170902ee5500b001d0511c990csm8345453plo.237.2023.12.12.03.12.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:12:19 -0800 (PST) From: Hongyu Jin To: agk@redhat.com, snitzer@kernel.org, mpatocka@redhat.com, axboe@kernel.dk, ebiggers@kernel.org Cc: zhiguo.niu@unisoc.com, ke.wang@unisoc.com, yibin.ding@unisoc.com, hongyu.jin@unisoc.com, linux-kernel@vger.kernel.org, dm-devel@lists.linux.dev, linux-block@vger.kernel.org Subject: [PATCH v4 2/5] dm: Support I/O priority for dm_io() Date: Tue, 12 Dec 2023 19:11:47 +0800 Message-Id: <20231212111150.18155-3-hongyu.jin.cn@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> References: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Hongyu Jin Add ioprio field in struct dm_io_region, by this field specific I/O priority when call dm_io(). Co-developed-by: Yibin Ding Signed-off-by: Yibin Ding Signed-off-by: Hongyu Jin --- drivers/md/dm-bufio.c | 3 +++ drivers/md/dm-integrity.c | 5 +++++ drivers/md/dm-io.c | 1 + drivers/md/dm-log.c | 1 + drivers/md/dm-raid1.c | 2 ++ drivers/md/dm-snap-persistent.c | 1 + drivers/md/dm-writecache.c | 4 ++++ include/linux/dm-io.h | 2 ++ 8 files changed, 19 insertions(+) diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c index 62eb27639c9b..7f82262aed54 100644 --- a/drivers/md/dm-bufio.c +++ b/drivers/md/dm-bufio.c @@ -1269,6 +1269,7 @@ static void use_dmio(struct dm_buffer *b, enum req_op op, sector_t sector, .bdev = b->c->bdev, .sector = sector, .count = n_sectors, + .ioprio = IOPRIO_DEFAULT, }; if (b->data_mode != DATA_MODE_VMALLOC) { @@ -2125,6 +2126,7 @@ int dm_bufio_issue_flush(struct dm_bufio_client *c) .bdev = c->bdev, .sector = 0, .count = 0, + .ioprio = IOPRIO_DEFAULT, }; if (WARN_ON_ONCE(dm_bufio_in_request())) @@ -2149,6 +2151,7 @@ int dm_bufio_issue_discard(struct dm_bufio_client *c, sector_t block, sector_t c .bdev = c->bdev, .sector = block_to_sector(c, block), .count = block_to_sector(c, count), + .ioprio = IOPRIO_DEFAULT, }; if (WARN_ON_ONCE(dm_bufio_in_request())) diff --git a/drivers/md/dm-integrity.c b/drivers/md/dm-integrity.c index e85c688fd91e..7cba183abdce 100644 --- a/drivers/md/dm-integrity.c +++ b/drivers/md/dm-integrity.c @@ -543,6 +543,7 @@ static int sync_rw_sb(struct dm_integrity_c *ic, blk_opf_t opf) io_loc.bdev = ic->meta_dev ? ic->meta_dev->bdev : ic->dev->bdev; io_loc.sector = ic->start; io_loc.count = SB_SECTORS; + io_loc.ioprio = IOPRIO_DEFAULT; if (op == REQ_OP_WRITE) { sb_set_version(ic); @@ -1070,6 +1071,7 @@ static void rw_journal_sectors(struct dm_integrity_c *ic, blk_opf_t opf, io_loc.bdev = ic->meta_dev ? ic->meta_dev->bdev : ic->dev->bdev; io_loc.sector = ic->start + SB_SECTORS + sector; io_loc.count = n_sectors; + io_loc.ioprio = IOPRIO_DEFAULT; r = dm_io(&io_req, 1, &io_loc, NULL); if (unlikely(r)) { @@ -1187,6 +1189,7 @@ static void copy_from_journal(struct dm_integrity_c *ic, unsigned int section, u io_loc.bdev = ic->dev->bdev; io_loc.sector = target; io_loc.count = n_sectors; + io_loc.ioprio = IOPRIO_DEFAULT; r = dm_io(&io_req, 1, &io_loc, NULL); if (unlikely(r)) { @@ -1515,6 +1518,7 @@ static void dm_integrity_flush_buffers(struct dm_integrity_c *ic, bool flush_dat fr.io_reg.bdev = ic->dev->bdev, fr.io_reg.sector = 0, fr.io_reg.count = 0, + fr.io_reg.ioprio = IOPRIO_DEFAULT, fr.ic = ic; init_completion(&fr.comp); r = dm_io(&fr.io_req, 1, &fr.io_reg, NULL); @@ -2738,6 +2742,7 @@ static void integrity_recalc(struct work_struct *w) io_loc.bdev = ic->dev->bdev; io_loc.sector = get_data_sector(ic, area, offset); io_loc.count = n_sectors; + io_loc.ioprio = IOPRIO_DEFAULT; r = dm_io(&io_req, 1, &io_loc, NULL); if (unlikely(r)) { diff --git a/drivers/md/dm-io.c b/drivers/md/dm-io.c index f053ce245814..b40f0a432981 100644 --- a/drivers/md/dm-io.c +++ b/drivers/md/dm-io.c @@ -354,6 +354,7 @@ static void do_region(const blk_opf_t opf, unsigned int region, &io->client->bios); bio->bi_iter.bi_sector = where->sector + (where->count - remaining); bio->bi_end_io = endio; + bio->bi_ioprio = where->ioprio; store_io_and_region_in_bio(bio, io, region); if (op == REQ_OP_DISCARD || op == REQ_OP_WRITE_ZEROES) { diff --git a/drivers/md/dm-log.c b/drivers/md/dm-log.c index f9f84236dfcd..e0dacdcd94f1 100644 --- a/drivers/md/dm-log.c +++ b/drivers/md/dm-log.c @@ -309,6 +309,7 @@ static int flush_header(struct log_c *lc) .bdev = lc->header_location.bdev, .sector = 0, .count = 0, + .ioprio = IOPRIO_DEFAULT, }; lc->io_req.bi_opf = REQ_OP_WRITE | REQ_PREFLUSH; diff --git a/drivers/md/dm-raid1.c b/drivers/md/dm-raid1.c index ddcb2bc4a617..2de9b1377de3 100644 --- a/drivers/md/dm-raid1.c +++ b/drivers/md/dm-raid1.c @@ -275,6 +275,7 @@ static int mirror_flush(struct dm_target *ti) io[i].bdev = m->dev->bdev; io[i].sector = 0; io[i].count = 0; + io[i].ioprio = IOPRIO_DEFAULT; } error_bits = -1; @@ -475,6 +476,7 @@ static void map_region(struct dm_io_region *io, struct mirror *m, io->bdev = m->dev->bdev; io->sector = map_sector(m, bio); io->count = bio_sectors(bio); + io->ioprio = bio_prio(bio); } static void hold_bio(struct mirror_set *ms, struct bio *bio) diff --git a/drivers/md/dm-snap-persistent.c b/drivers/md/dm-snap-persistent.c index 15649921f2a9..4aa70b71f1da 100644 --- a/drivers/md/dm-snap-persistent.c +++ b/drivers/md/dm-snap-persistent.c @@ -236,6 +236,7 @@ static int chunk_io(struct pstore *ps, void *area, chunk_t chunk, blk_opf_t opf, .bdev = dm_snap_cow(ps->store->snap)->bdev, .sector = ps->store->chunk_size * chunk, .count = ps->store->chunk_size, + .ioprio = IOPRIO_DEFAULT, }; struct dm_io_request io_req = { .bi_opf = opf, diff --git a/drivers/md/dm-writecache.c b/drivers/md/dm-writecache.c index 074cb785eafc..135d1268246f 100644 --- a/drivers/md/dm-writecache.c +++ b/drivers/md/dm-writecache.c @@ -515,6 +515,7 @@ static void ssd_commit_flushed(struct dm_writecache *wc, bool wait_for_ios) region.bdev = wc->ssd_dev->bdev; region.sector = (sector_t)i * (BITMAP_GRANULARITY >> SECTOR_SHIFT); region.count = (sector_t)(j - i) * (BITMAP_GRANULARITY >> SECTOR_SHIFT); + region.ioprio = IOPRIO_DEFAULT; if (unlikely(region.sector >= wc->metadata_sectors)) break; @@ -555,6 +556,7 @@ static void ssd_commit_superblock(struct dm_writecache *wc) region.bdev = wc->ssd_dev->bdev; region.sector = 0; region.count = max(4096U, wc->block_size) >> SECTOR_SHIFT; + region.ioprio = IOPRIO_DEFAULT; if (unlikely(region.sector + region.count > wc->metadata_sectors)) region.count = wc->metadata_sectors - region.sector; @@ -590,6 +592,7 @@ static void writecache_disk_flush(struct dm_writecache *wc, struct dm_dev *dev) region.bdev = dev->bdev; region.sector = 0; region.count = 0; + region.ioprio = IOPRIO_DEFAULT; req.bi_opf = REQ_OP_WRITE | REQ_PREFLUSH; req.mem.type = DM_IO_KMEM; req.mem.ptr.addr = NULL; @@ -984,6 +987,7 @@ static int writecache_read_metadata(struct dm_writecache *wc, sector_t n_sectors region.bdev = wc->ssd_dev->bdev; region.sector = wc->start_sector; region.count = n_sectors; + region.ioprio = IOPRIO_DEFAULT; req.bi_opf = REQ_OP_READ | REQ_SYNC; req.mem.type = DM_IO_VMA; req.mem.ptr.vma = (char *)wc->memory_map; diff --git a/include/linux/dm-io.h b/include/linux/dm-io.h index 7595142f3fc5..227ee6d77c70 100644 --- a/include/linux/dm-io.h +++ b/include/linux/dm-io.h @@ -20,6 +20,8 @@ struct dm_io_region { struct block_device *bdev; sector_t sector; sector_t count; /* If this is zero the region is ignored. */ + /* Set it to IOPRIO_DEFAULT if you don't know what value to set */ + unsigned short ioprio; }; struct page_list { From patchwork Tue Dec 12 11:11:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyu Jin X-Patchwork-Id: 13488992 X-Patchwork-Delegate: snitzer@redhat.com Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B73B55CD37 for ; Tue, 12 Dec 2023 11:12:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MauCDSBT" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-1d069b1d127so31646655ad.0 for ; Tue, 12 Dec 2023 03:12:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702379545; x=1702984345; darn=lists.linux.dev; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4yqIhLm4LzKCMhpYyEZh+xPSurdrvcNEreCvT9oRThc=; b=MauCDSBTDHcEYDVVHrmRcoUWviDNJro6Aobiow+NG8qBLZ3FmvwZnTfXEmWwWJjNZc pnK5S5crAJMS7lthIAH6rcSdTTwmFLaN2DHDsn4fiKB2XQz58vA40RMBy51VPQK/w70A Z6m/W0tP4Uwf7XB4LA85Z/pNQpK66s+HLlvyVvu6ZRAQ1w+vLDhj6z02E2XY4Du2nyBK RUhwJ8TXCw668lQl6pfSU8Ugm1VhrR4lenZpKirorlut8b3Kf/EQwgffrimzzvPDkHne bmKz7HxW+ipg9cE88vTz9bjcdVfzou9JKKPgT68dpgqOfvUJf3gJ3du5cEklqmG3Csqv eIZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702379545; x=1702984345; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4yqIhLm4LzKCMhpYyEZh+xPSurdrvcNEreCvT9oRThc=; b=FFwcyAYpPTxwRhknC+csjbPyOWF+sU8f4K+1c9Aa6eu1iceiYKF45ni1wNg3NIBdVp hqh6ANkxaZxs7igqXB6YEcNKVIEZmKO1tohbw6Kc4bueM1teOEdfgvr87ksBB16BaOPN tzBX9Z5LAQ/KgR70fcvqteIB0NuTIlB2rrrXTmG3Gq0O0IqqXIavzbbWWf59J2Fa8T/Y eYHMITVhaKgP7T43rBgylGLqujapr0PlTOWOQMLRPyUkA6MgphlRiEQhwoXGphDJZ6N+ 1nq52J7T46gX4n4KQuYK8WubiBUr26NEfhsQIeljecIFnF0o/hcTLJsLoNCsgCXWroy6 H0Hg== X-Gm-Message-State: AOJu0YxetodXVs4tfCHikN7LXdtYZHHgJU5hUZCf32JB4gT1fpEsZlD9 +DNlEdVaJFp/kJAbi1CWSvk= X-Google-Smtp-Source: AGHT+IGJkbSmMBCSDQCY4qtO/eaJOlAMLjKcM057nI0ijY6goUpjdvEP/pMjSUfVMw9aAZp0rt041g== X-Received: by 2002:a17:90a:d205:b0:286:c398:841b with SMTP id o5-20020a17090ad20500b00286c398841bmr2773861pju.58.1702379544881; Tue, 12 Dec 2023 03:12:24 -0800 (PST) Received: from ubuntu.. ([117.18.48.102]) by smtp.gmail.com with ESMTPSA id 21-20020a170902ee5500b001d0511c990csm8345453plo.237.2023.12.12.03.12.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:12:24 -0800 (PST) From: Hongyu Jin To: agk@redhat.com, snitzer@kernel.org, mpatocka@redhat.com, axboe@kernel.dk, ebiggers@kernel.org Cc: zhiguo.niu@unisoc.com, ke.wang@unisoc.com, yibin.ding@unisoc.com, hongyu.jin@unisoc.com, linux-kernel@vger.kernel.org, dm-devel@lists.linux.dev, linux-block@vger.kernel.org Subject: [PATCH v4 3/5] dm-bufio: Support I/O priority Date: Tue, 12 Dec 2023 19:11:48 +0800 Message-Id: <20231212111150.18155-4-hongyu.jin.cn@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> References: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Hongyu Jin Add I/O priority parameter for dm_bufio_read() and dm_bufio_prefetch(). Co-developed-by: Yibin Ding Signed-off-by: Yibin Ding Signed-off-by: Hongyu Jin --- drivers/md/dm-bufio.c | 33 ++++++++++--------- drivers/md/dm-ebs-target.c | 8 ++--- drivers/md/dm-integrity.c | 2 +- drivers/md/dm-snap-persistent.c | 4 +-- drivers/md/dm-verity-fec.c | 4 +-- drivers/md/dm-verity-target.c | 4 +-- drivers/md/persistent-data/dm-block-manager.c | 6 ++-- include/linux/dm-bufio.h | 6 ++-- 8 files changed, 34 insertions(+), 33 deletions(-) diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c index 7f82262aed54..739f5dc52432 100644 --- a/drivers/md/dm-bufio.c +++ b/drivers/md/dm-bufio.c @@ -1256,7 +1256,7 @@ static void dmio_complete(unsigned long error, void *context) } static void use_dmio(struct dm_buffer *b, enum req_op op, sector_t sector, - unsigned int n_sectors, unsigned int offset) + unsigned int n_sectors, unsigned int offset, unsigned short ioprio) { int r; struct dm_io_request io_req = { @@ -1296,7 +1296,7 @@ static void bio_complete(struct bio *bio) } static void use_bio(struct dm_buffer *b, enum req_op op, sector_t sector, - unsigned int n_sectors, unsigned int offset) + unsigned int n_sectors, unsigned int offset, unsigned short ioprio) { struct bio *bio; char *ptr; @@ -1304,13 +1304,14 @@ static void use_bio(struct dm_buffer *b, enum req_op op, sector_t sector, bio = bio_kmalloc(1, GFP_NOWAIT | __GFP_NORETRY | __GFP_NOWARN); if (!bio) { - use_dmio(b, op, sector, n_sectors, offset); + use_dmio(b, op, sector, n_sectors, offset, ioprio); return; } bio_init(bio, b->c->bdev, bio->bi_inline_vecs, 1, op); bio->bi_iter.bi_sector = sector; bio->bi_end_io = bio_complete; bio->bi_private = b; + bio->bi_ioprio = ioprio; ptr = (char *)b->data + offset; len = n_sectors << SECTOR_SHIFT; @@ -1333,7 +1334,7 @@ static inline sector_t block_to_sector(struct dm_bufio_client *c, sector_t block return sector; } -static void submit_io(struct dm_buffer *b, enum req_op op, +static void submit_io(struct dm_buffer *b, enum req_op op, unsigned short ioprio, void (*end_io)(struct dm_buffer *, blk_status_t)) { unsigned int n_sectors; @@ -1363,9 +1364,9 @@ static void submit_io(struct dm_buffer *b, enum req_op op, } if (b->data_mode != DATA_MODE_VMALLOC) - use_bio(b, op, sector, n_sectors, offset); + use_bio(b, op, sector, n_sectors, offset, ioprio); else - use_dmio(b, op, sector, n_sectors, offset); + use_dmio(b, op, sector, n_sectors, offset, ioprio); } /* @@ -1421,7 +1422,7 @@ static void __write_dirty_buffer(struct dm_buffer *b, b->write_end = b->dirty_end; if (!write_list) - submit_io(b, REQ_OP_WRITE, write_endio); + submit_io(b, REQ_OP_WRITE, IOPRIO_DEFAULT, write_endio); else list_add_tail(&b->write_list, write_list); } @@ -1435,7 +1436,7 @@ static void __flush_write_list(struct list_head *write_list) struct dm_buffer *b = list_entry(write_list->next, struct dm_buffer, write_list); list_del(&b->write_list); - submit_io(b, REQ_OP_WRITE, write_endio); + submit_io(b, REQ_OP_WRITE, IOPRIO_DEFAULT, write_endio); cond_resched(); } blk_finish_plug(&plug); @@ -1817,7 +1818,7 @@ static void read_endio(struct dm_buffer *b, blk_status_t status) * and uses dm_bufio_mark_buffer_dirty to write new data back). */ static void *new_read(struct dm_bufio_client *c, sector_t block, - enum new_flag nf, struct dm_buffer **bp) + enum new_flag nf, struct dm_buffer **bp, unsigned short ioprio) { int need_submit = 0; struct dm_buffer *b; @@ -1870,7 +1871,7 @@ static void *new_read(struct dm_bufio_client *c, sector_t block, return NULL; if (need_submit) - submit_io(b, REQ_OP_READ, read_endio); + submit_io(b, REQ_OP_READ, ioprio, read_endio); wait_on_bit_io(&b->state, B_READING, TASK_UNINTERRUPTIBLE); @@ -1890,17 +1891,17 @@ static void *new_read(struct dm_bufio_client *c, sector_t block, void *dm_bufio_get(struct dm_bufio_client *c, sector_t block, struct dm_buffer **bp) { - return new_read(c, block, NF_GET, bp); + return new_read(c, block, NF_GET, bp, IOPRIO_DEFAULT); } EXPORT_SYMBOL_GPL(dm_bufio_get); void *dm_bufio_read(struct dm_bufio_client *c, sector_t block, - struct dm_buffer **bp) + struct dm_buffer **bp, unsigned short ioprio) { if (WARN_ON_ONCE(dm_bufio_in_request())) return ERR_PTR(-EINVAL); - return new_read(c, block, NF_READ, bp); + return new_read(c, block, NF_READ, bp, ioprio); } EXPORT_SYMBOL_GPL(dm_bufio_read); @@ -1910,12 +1911,12 @@ void *dm_bufio_new(struct dm_bufio_client *c, sector_t block, if (WARN_ON_ONCE(dm_bufio_in_request())) return ERR_PTR(-EINVAL); - return new_read(c, block, NF_FRESH, bp); + return new_read(c, block, NF_FRESH, bp, IOPRIO_DEFAULT); } EXPORT_SYMBOL_GPL(dm_bufio_new); void dm_bufio_prefetch(struct dm_bufio_client *c, - sector_t block, unsigned int n_blocks) + sector_t block, unsigned int n_blocks, unsigned short ioprio) { struct blk_plug plug; @@ -1951,7 +1952,7 @@ void dm_bufio_prefetch(struct dm_bufio_client *c, dm_bufio_unlock(c); if (need_submit) - submit_io(b, REQ_OP_READ, read_endio); + submit_io(b, REQ_OP_READ, ioprio, read_endio); dm_bufio_release(b); cond_resched(); diff --git a/drivers/md/dm-ebs-target.c b/drivers/md/dm-ebs-target.c index 435b45201f4d..8198c8a7b416 100644 --- a/drivers/md/dm-ebs-target.c +++ b/drivers/md/dm-ebs-target.c @@ -84,7 +84,7 @@ static int __ebs_rw_bvec(struct ebs_c *ec, enum req_op op, struct bio_vec *bv, /* Avoid reading for writes in case bio vector's page overwrites block completely. */ if (op == REQ_OP_READ || buf_off || bv_len < dm_bufio_get_block_size(ec->bufio)) - ba = dm_bufio_read(ec->bufio, block, &b); + ba = dm_bufio_read(ec->bufio, block, &b, IOPRIO_DEFAULT); else ba = dm_bufio_new(ec->bufio, block, &b); @@ -194,13 +194,13 @@ static void __ebs_process_bios(struct work_struct *ws) bio_list_for_each(bio, &bios) { block1 = __sector_to_block(ec, bio->bi_iter.bi_sector); if (bio_op(bio) == REQ_OP_READ) - dm_bufio_prefetch(ec->bufio, block1, __nr_blocks(ec, bio)); + dm_bufio_prefetch(ec->bufio, block1, __nr_blocks(ec, bio), IOPRIO_DEFAULT); else if (bio_op(bio) == REQ_OP_WRITE && !(bio->bi_opf & REQ_PREFLUSH)) { block2 = __sector_to_block(ec, bio_end_sector(bio)); if (__block_mod(bio->bi_iter.bi_sector, ec->u_bs)) - dm_bufio_prefetch(ec->bufio, block1, 1); + dm_bufio_prefetch(ec->bufio, block1, 1, IOPRIO_DEFAULT); if (__block_mod(bio_end_sector(bio), ec->u_bs) && block2 != block1) - dm_bufio_prefetch(ec->bufio, block2, 1); + dm_bufio_prefetch(ec->bufio, block2, 1, IOPRIO_DEFAULT); } } diff --git a/drivers/md/dm-integrity.c b/drivers/md/dm-integrity.c index 7cba183abdce..a2853c24a259 100644 --- a/drivers/md/dm-integrity.c +++ b/drivers/md/dm-integrity.c @@ -1421,7 +1421,7 @@ static int dm_integrity_rw_tag(struct dm_integrity_c *ic, unsigned char *tag, se if (unlikely(r)) return r; - data = dm_bufio_read(ic->bufio, *metadata_block, &b); + data = dm_bufio_read(ic->bufio, *metadata_block, &b, IOPRIO_DEFAULT); if (IS_ERR(data)) return PTR_ERR(data); diff --git a/drivers/md/dm-snap-persistent.c b/drivers/md/dm-snap-persistent.c index 4aa70b71f1da..eb6943fc7024 100644 --- a/drivers/md/dm-snap-persistent.c +++ b/drivers/md/dm-snap-persistent.c @@ -525,7 +525,7 @@ static int read_exceptions(struct pstore *ps, if (unlikely(pf_chunk >= dm_bufio_get_device_size(client))) break; - dm_bufio_prefetch(client, pf_chunk, 1); + dm_bufio_prefetch(client, pf_chunk, 1, IOPRIO_DEFAULT); prefetch_area++; if (unlikely(!prefetch_area)) break; @@ -534,7 +534,7 @@ static int read_exceptions(struct pstore *ps, chunk = area_location(ps, ps->current_area); - area = dm_bufio_read(client, chunk, &bp); + area = dm_bufio_read(client, chunk, &bp, IOPRIO_DEFAULT); if (IS_ERR(area)) { r = PTR_ERR(area); goto ret_destroy_bufio; diff --git a/drivers/md/dm-verity-fec.c b/drivers/md/dm-verity-fec.c index 3ef9f018da60..715173cbf0ee 100644 --- a/drivers/md/dm-verity-fec.c +++ b/drivers/md/dm-verity-fec.c @@ -68,7 +68,7 @@ static u8 *fec_read_parity(struct dm_verity *v, u64 rsb, int index, block = div64_u64_rem(position, v->fec->io_size, &rem); *offset = (unsigned int)rem; - res = dm_bufio_read(v->fec->bufio, block, buf); + res = dm_bufio_read(v->fec->bufio, block, buf, IOPRIO_DEFAULT); if (IS_ERR(res)) { DMERR("%s: FEC %llu: parity read failed (block %llu): %ld", v->data_dev->name, (unsigned long long)rsb, @@ -247,7 +247,7 @@ static int fec_read_bufs(struct dm_verity *v, struct dm_verity_io *io, bufio = v->bufio; } - bbuf = dm_bufio_read(bufio, block, &buf); + bbuf = dm_bufio_read(bufio, block, &buf, IOPRIO_DEFAULT); if (IS_ERR(bbuf)) { DMWARN_LIMIT("%s: FEC %llu: read failed (%llu): %ld", v->data_dev->name, diff --git a/drivers/md/dm-verity-target.c b/drivers/md/dm-verity-target.c index 26adcfea0302..0038e168f3d7 100644 --- a/drivers/md/dm-verity-target.c +++ b/drivers/md/dm-verity-target.c @@ -307,7 +307,7 @@ static int verity_verify_level(struct dm_verity *v, struct dm_verity_io *io, return -EAGAIN; } } else - data = dm_bufio_read(v->bufio, hash_block, &buf); + data = dm_bufio_read(v->bufio, hash_block, &buf, IOPRIO_DEFAULT); if (IS_ERR(data)) return PTR_ERR(data); @@ -717,7 +717,7 @@ static void verity_prefetch_io(struct work_struct *work) } no_prefetch_cluster: dm_bufio_prefetch(v->bufio, hash_block_start, - hash_block_end - hash_block_start + 1); + hash_block_end - hash_block_start + 1, IOPRIO_DEFAULT); } kfree(pw); diff --git a/drivers/md/persistent-data/dm-block-manager.c b/drivers/md/persistent-data/dm-block-manager.c index 0e010e1204aa..86a4f73d2f3d 100644 --- a/drivers/md/persistent-data/dm-block-manager.c +++ b/drivers/md/persistent-data/dm-block-manager.c @@ -474,7 +474,7 @@ int dm_bm_read_lock(struct dm_block_manager *bm, dm_block_t b, void *p; int r; - p = dm_bufio_read(bm->bufio, b, (struct dm_buffer **) result); + p = dm_bufio_read(bm->bufio, b, (struct dm_buffer **) result, IOPRIO_DEFAULT); if (IS_ERR(p)) return PTR_ERR(p); @@ -510,7 +510,7 @@ int dm_bm_write_lock(struct dm_block_manager *bm, if (dm_bm_is_read_only(bm)) return -EPERM; - p = dm_bufio_read(bm->bufio, b, (struct dm_buffer **) result); + p = dm_bufio_read(bm->bufio, b, (struct dm_buffer **) result, IOPRIO_DEFAULT); if (IS_ERR(p)) return PTR_ERR(p); @@ -624,7 +624,7 @@ EXPORT_SYMBOL_GPL(dm_bm_flush); void dm_bm_prefetch(struct dm_block_manager *bm, dm_block_t b) { - dm_bufio_prefetch(bm->bufio, b, 1); + dm_bufio_prefetch(bm->bufio, b, 1, IOPRIO_DEFAULT); } bool dm_bm_is_read_only(struct dm_block_manager *bm) diff --git a/include/linux/dm-bufio.h b/include/linux/dm-bufio.h index 75e7d8cbb532..d270d48891f7 100644 --- a/include/linux/dm-bufio.h +++ b/include/linux/dm-bufio.h @@ -11,6 +11,7 @@ #define _LINUX_DM_BUFIO_H #include +#include #include /*----------------------------------------------------------------*/ @@ -62,7 +63,7 @@ void dm_bufio_set_sector_offset(struct dm_bufio_client *c, sector_t start); * it dirty. */ void *dm_bufio_read(struct dm_bufio_client *c, sector_t block, - struct dm_buffer **bp); + struct dm_buffer **bp, unsigned short ioprio); /* * Like dm_bufio_read, but return buffer from cache, don't read @@ -84,8 +85,7 @@ void *dm_bufio_new(struct dm_bufio_client *c, sector_t block, * I/O to finish. */ void dm_bufio_prefetch(struct dm_bufio_client *c, - sector_t block, unsigned int n_blocks); - + sector_t block, unsigned int n_blocks, unsigned short ioprio); /* * Release a reference obtained with dm_bufio_{read,get,new}. The data * pointer and dm_buffer pointer is no longer valid after this call. From patchwork Tue Dec 12 11:11:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyu Jin X-Patchwork-Id: 13488993 X-Patchwork-Delegate: snitzer@redhat.com Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2EFB75AB8F for ; Tue, 12 Dec 2023 11:12:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Am8KI+Er" Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-6ce94f62806so2966408b3a.1 for ; Tue, 12 Dec 2023 03:12:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702379549; x=1702984349; darn=lists.linux.dev; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vN+RlAGZeGH2zMVmYb6ghj5GDWJcjTyD9/i+J4cSzo8=; b=Am8KI+ErpqYYc0CL0k0ZIFm6y/xjwrOFbkmGY3Dkgz3cPOBWy+x2pXRBaq2YEHwsug 4psA6q2n2XYJmlTa/V3qsZrBk87IN7Fgs+aUShry3m97A2s1aYnqiRfT65/xrXPpo0Zm ImINsEkbLwlPBB77zuJ+Tsb7tuCE8+xJ3dDsBxXhe6mhLqT5HrKn3lZiS3UEtTNsaHq+ VzcBEXphmd0Hqi3vDmAE4kQ2HJqrF/764VZcO0dnaTb7NARCLVcagsqw3wG9+O3ksyeG bdeurEMB5jDXHimvI1O2dTZ0Ip+R1k08s9cW9EbfDljZWRWAudjtBDSKPosUVSG5FD3Q aP7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702379549; x=1702984349; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vN+RlAGZeGH2zMVmYb6ghj5GDWJcjTyD9/i+J4cSzo8=; b=UDwb0HlFXbx4VC6grQYqeop7PaciT3zfv2qZIPjzEFWHxHWvvTUg8RHs81F7zJB6qO YlJiWolH7ImW9AJjfVBPzX1nw9QxOQH5tITjfKRiCLRr3PIpEHzzIMMqIM/5HNpk9Vj6 HUUupA7N+oLslKws6tUefyqX8tfwhKhWjVbJUOGx39nhAU83Q4keoDBL+mMz+2OEAXVB G3E9gBfPbQISUkX82zs4jwI7MbRDWdNw3CLcOY3Law/5Y4H2szj8DTaiY5WuLAQEY4/8 cVBAw8+Xfjxegpwq/JRKVGnBik7VcHd/bLyhhsLpLcaP+myIEczRuAIurf2Pbhq4cLrt cwcw== X-Gm-Message-State: AOJu0YyaEQXq+WEFZZB45x1slkuTiI2dDw5XcSCerDg7tQ5CF8VaXKCR t7I31ozPrHGeDZ2jJ825Ri3SFxAS0Vk= X-Google-Smtp-Source: AGHT+IH8HknLDzaW0Q9TDSRdg3SQu4DIMs5eNiFa7zAa7YAtIDMAqdyYIiPkKRRbitlR1WMMmVqmNQ== X-Received: by 2002:a05:6a20:4287:b0:186:e53:b64b with SMTP id o7-20020a056a20428700b001860e53b64bmr3221043pzj.47.1702379549391; Tue, 12 Dec 2023 03:12:29 -0800 (PST) Received: from ubuntu.. ([117.18.48.102]) by smtp.gmail.com with ESMTPSA id 21-20020a170902ee5500b001d0511c990csm8345453plo.237.2023.12.12.03.12.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:12:29 -0800 (PST) From: Hongyu Jin To: agk@redhat.com, snitzer@kernel.org, mpatocka@redhat.com, axboe@kernel.dk, ebiggers@kernel.org Cc: zhiguo.niu@unisoc.com, ke.wang@unisoc.com, yibin.ding@unisoc.com, hongyu.jin@unisoc.com, linux-kernel@vger.kernel.org, dm-devel@lists.linux.dev, linux-block@vger.kernel.org Subject: [PATCH v4 4/5] dm verity: Fix I/O priority lost when read FEC and hash Date: Tue, 12 Dec 2023 19:11:49 +0800 Message-Id: <20231212111150.18155-5-hongyu.jin.cn@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> References: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Hongyu Jin To fix this problem, when read FEC and hash from disk, I/O priority are inconsistent with data block and blocked by other I/O with low I/O priority. Make I/O for FEC and hash has same I/O priority with original data I/O. Co-developed-by: Yibin Ding Signed-off-by: Yibin Ding Signed-off-by: Hongyu Jin --- drivers/md/dm-verity-fec.c | 3 ++- drivers/md/dm-verity-target.c | 8 ++++++-- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/md/dm-verity-fec.c b/drivers/md/dm-verity-fec.c index 715173cbf0ee..6a5a679e7e8a 100644 --- a/drivers/md/dm-verity-fec.c +++ b/drivers/md/dm-verity-fec.c @@ -209,6 +209,7 @@ static int fec_read_bufs(struct dm_verity *v, struct dm_verity_io *io, u8 *bbuf, *rs_block; u8 want_digest[HASH_MAX_DIGESTSIZE]; unsigned int n, k; + struct bio *bio = dm_bio_from_per_bio_data(io, v->ti->per_io_data_size); if (neras) *neras = 0; @@ -247,7 +248,7 @@ static int fec_read_bufs(struct dm_verity *v, struct dm_verity_io *io, bufio = v->bufio; } - bbuf = dm_bufio_read(bufio, block, &buf, IOPRIO_DEFAULT); + bbuf = dm_bufio_read(bufio, block, &buf, bio_prio(bio)); if (IS_ERR(bbuf)) { DMWARN_LIMIT("%s: FEC %llu: read failed (%llu): %ld", v->data_dev->name, diff --git a/drivers/md/dm-verity-target.c b/drivers/md/dm-verity-target.c index 0038e168f3d7..8c911b6722ce 100644 --- a/drivers/md/dm-verity-target.c +++ b/drivers/md/dm-verity-target.c @@ -51,6 +51,7 @@ static DEFINE_STATIC_KEY_FALSE(use_tasklet_enabled); struct dm_verity_prefetch_work { struct work_struct work; struct dm_verity *v; + unsigned short ioprio; sector_t block; unsigned int n_blocks; }; @@ -293,6 +294,7 @@ static int verity_verify_level(struct dm_verity *v, struct dm_verity_io *io, int r; sector_t hash_block; unsigned int offset; + struct bio *bio = dm_bio_from_per_bio_data(io, v->ti->per_io_data_size); verity_hash_at_level(v, block, level, &hash_block, &offset); @@ -307,7 +309,7 @@ static int verity_verify_level(struct dm_verity *v, struct dm_verity_io *io, return -EAGAIN; } } else - data = dm_bufio_read(v->bufio, hash_block, &buf, IOPRIO_DEFAULT); + data = dm_bufio_read(v->bufio, hash_block, &buf, bio_prio(bio)); if (IS_ERR(data)) return PTR_ERR(data); @@ -717,7 +719,7 @@ static void verity_prefetch_io(struct work_struct *work) } no_prefetch_cluster: dm_bufio_prefetch(v->bufio, hash_block_start, - hash_block_end - hash_block_start + 1, IOPRIO_DEFAULT); + hash_block_end - hash_block_start + 1, pw->ioprio); } kfree(pw); @@ -728,6 +730,7 @@ static void verity_submit_prefetch(struct dm_verity *v, struct dm_verity_io *io) sector_t block = io->block; unsigned int n_blocks = io->n_blocks; struct dm_verity_prefetch_work *pw; + struct bio *bio = dm_bio_from_per_bio_data(io, v->ti->per_io_data_size); if (v->validated_blocks) { while (n_blocks && test_bit(block, v->validated_blocks)) { @@ -751,6 +754,7 @@ static void verity_submit_prefetch(struct dm_verity *v, struct dm_verity_io *io) pw->v = v; pw->block = block; pw->n_blocks = n_blocks; + pw->ioprio = bio_prio(bio); queue_work(v->verify_wq, &pw->work); } From patchwork Tue Dec 12 11:11:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyu Jin X-Patchwork-Id: 13488994 X-Patchwork-Delegate: snitzer@redhat.com Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 663FA5CD1B for ; Tue, 12 Dec 2023 11:12:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IA7WBEYW" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-1d0c94397c0so31319325ad.2 for ; Tue, 12 Dec 2023 03:12:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702379554; x=1702984354; darn=lists.linux.dev; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fRWJvmM8+hpVsGJoyvu2YuvVcQVJUfQcuTmMveSE8/s=; b=IA7WBEYWwgnfPWTL7G3pzd1ONuF4VH9/4mA8oECGDkUbnkpBrUnSPaHf3bOuJiPTem AJGCZzanMet2fAE5gnVcwMyx/U3W+71KWKtfhS1hym8l3E2mUb/xmiNIg1N9Sz4Bh79g Mkrikn0bMNaGmpeK59b3SAmz+6R7Y1YGmgC0PbDatGb4c2UuXPaT7BNDdfcseMfUN1mT YdgkbyVm9boqJqtnzfvmzBfPtFdW3RpsfHvHX7rpW6n/ziQZdgwVPcNCU3DbNc0K3/gz GHuXMbpos4FUQe8mLoVTcBzExbzmWuFaz57gRcxWegT6bic/9ZaYu6P8LtvItm/j4DjL +Jdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702379554; x=1702984354; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fRWJvmM8+hpVsGJoyvu2YuvVcQVJUfQcuTmMveSE8/s=; b=r+40NQ0WaO5/RXjKnO2xaaYlTxW48+T5rJYS5qpNNAhydT1xb/6vWOZrtZkYLsfWqz NrewgNkZ2zXkQKFtj993tEjXS571WVRNOdr4KULsyzv9GhSu20bZkfhfWX9lEwvSZ42T jr/8Y5n2XaCSN/q0wjMMvBSl8tht8XiVrwS4O7rOCNSUcucjaUY8irpdKbZqO62R2gEy /pzzSfdXDOmEzC28EaBx7W6RJlQG+Z/ax4ba+ncX9WFgu9bKoBX1t4Ojrsx0GBBtL6qn 3GxkxeJaE0Skf20Us1XFyK9HdjGfGqJEe0TMhHxqLGK2T+O+5KAhb5o7EHQVHbZzPrzi IzDA== X-Gm-Message-State: AOJu0YzG7QY65mondJaIxNOmeM5dSI7q6b3qzhtYDvwmkj4ajPLoIwfW ajYWuuEAMztDoVEZ+392IaI= X-Google-Smtp-Source: AGHT+IFlcy+h/DMb4yTLMEPWYaQdXpLguXXRtdGTaHrTLhSYejeIi4auC/X/p0G23rVz46hOssocww== X-Received: by 2002:a17:902:780a:b0:1d3:3876:bfe with SMTP id p10-20020a170902780a00b001d338760bfemr169241pll.90.1702379553803; Tue, 12 Dec 2023 03:12:33 -0800 (PST) Received: from ubuntu.. ([117.18.48.102]) by smtp.gmail.com with ESMTPSA id 21-20020a170902ee5500b001d0511c990csm8345453plo.237.2023.12.12.03.12.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:12:33 -0800 (PST) From: Hongyu Jin To: agk@redhat.com, snitzer@kernel.org, mpatocka@redhat.com, axboe@kernel.dk, ebiggers@kernel.org Cc: zhiguo.niu@unisoc.com, ke.wang@unisoc.com, yibin.ding@unisoc.com, hongyu.jin@unisoc.com, linux-kernel@vger.kernel.org, dm-devel@lists.linux.dev, linux-block@vger.kernel.org Subject: [PATCH v4 5/5] dm-crypt: Fix lost ioprio when queuing write bios Date: Tue, 12 Dec 2023 19:11:50 +0800 Message-Id: <20231212111150.18155-6-hongyu.jin.cn@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> References: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Hongyu Jin The original submitting bio->bi_ioprio setting can be retained by struct dm_crypt_io::base_bio, we set the original bio's ioprio to the cloned bio for write. Link: https://lore.kernel.org/dm-devel/alpine.LRH.2.11.1612141049250.13402@mail.ewheeler.net Signed-off-by: Hongyu Jin --- drivers/md/dm-crypt.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index 6de107aff331..7149da6555b8 100644 --- a/drivers/md/dm-crypt.c +++ b/drivers/md/dm-crypt.c @@ -1683,6 +1683,7 @@ static struct bio *crypt_alloc_buffer(struct dm_crypt_io *io, unsigned int size) GFP_NOIO, &cc->bs); clone->bi_private = io; clone->bi_end_io = crypt_endio; + clone->bi_ioprio = io->base_bio->bi_ioprio; remaining_size = size;