From patchwork Thu Jan 27 22:56:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12727490 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF7CDC433F5 for ; Thu, 27 Jan 2022 22:56:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235026AbiA0W4y (ORCPT ); Thu, 27 Jan 2022 17:56:54 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:25893 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229801AbiA0W4x (ORCPT ); Thu, 27 Jan 2022 17:56:53 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643324213; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=LPT/xXbA8dMnjNDPrd3foEhpTUzqW1ZlXYJLQcHQMDc=; b=QFWkEuNGXxAoOEVqrh+qeAgt6FELWRpJ4VUWy928ylA4HEL1laFctIx7PxTld1oh5HyCsl kR7SiUj4j2ieQ+U+7FzAKuA328YNy1KgLtm2AX5QCtPFkO4CvnX7myYT/awdj907uCMaK3 P2L6fsmxAqZOrVSb5w2vFlsMXh3f3Uk= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-150-ftDUR_S0NhKAK_i8oxWjrA-1; Thu, 27 Jan 2022 17:56:51 -0500 X-MC-Unique: ftDUR_S0NhKAK_i8oxWjrA-1 Received: by mail-qt1-f198.google.com with SMTP id x5-20020ac84d45000000b002cf826b1a18so3336408qtv.2 for ; Thu, 27 Jan 2022 14:56:51 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=LPT/xXbA8dMnjNDPrd3foEhpTUzqW1ZlXYJLQcHQMDc=; b=13iO9zfYstyChK7gN/OW+3HVsyWnGjKG8AyGLajp3OBv5X76+II7dtYk9Ov9PZHrtH ViPHiQlDkHQCXDRDhzavMTUmCVIvPVQCyY/7HRqewj3GbKRqZIEWsAmnqfhogx7NRQS7 APitIxBbrZotwXvG7hv0EyC/fP7fqxpELEZ6bgKRqk7Uu0Mb/949eu3gbXKcm8iefi7l ffVJpoGox3v9bSbQh9EkwcKsTjbsyDWhH9rtLrGU5puCDasUknrMMmn67P45jLVVpD4r tO6jeX0N4O3Z53wlu974YaMJVj0fJv91/8kkHcEM9Ih7H7EY2kXbtPBUB9zIEmXZac6y B9aA== X-Gm-Message-State: AOAM532uM5f6615kyOHi2h+4Y4rf9FwzE8IYe1vCLS417xDMlydvV/84 9uUEg2O3D8rZpxCKXVIeHi+IzUX3faYJZn9q41DyOhK390HjOi+ef+Wxlgj47lH6e+0876tbiOH TsVcWWUBRAnk/O98t2VfoKQ== X-Received: by 2002:a05:620a:141a:: with SMTP id d26mr4086407qkj.403.1643324211288; Thu, 27 Jan 2022 14:56:51 -0800 (PST) X-Google-Smtp-Source: ABdhPJxvpuCni4WvV5PNyzRGhC0OXvCVSabWWqJ2hiDnj4Ya0PLPWe4YLLKbvK6oGvLtvu8SjLW8yw== X-Received: by 2002:a05:620a:141a:: with SMTP id d26mr4086396qkj.403.1643324211073; Thu, 27 Jan 2022 14:56:51 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id i5sm2205277qkn.19.2022.01.27.14.56.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Jan 2022 14:56:50 -0800 (PST) From: Mike Snitzer To: axboe@kernel.dk Cc: hch@lst.de, dm-devel@redhat.com, linux-block@vger.kernel.org Subject: [PATCH v2 1/3] block: add bio_start_io_acct_time() to control start_time Date: Thu, 27 Jan 2022 17:56:46 -0500 Message-Id: <20220127225648.28729-2-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220127225648.28729-1-snitzer@redhat.com> References: <20220127225648.28729-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org bio_start_io_acct_time() interface is like bio_start_io_acct() that allows start_time to be passed in. This gives drivers the ability to defer starting accounting until after IO is issued (but possibily not entirely due to bio splitting). Signed-off-by: Mike Snitzer --- block/blk-core.c | 25 +++++++++++++++++++------ include/linux/blkdev.h | 1 + 2 files changed, 20 insertions(+), 6 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 97f8bc8d3a79..d93e3bb9a769 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1061,20 +1061,32 @@ void update_io_ticks(struct block_device *part, unsigned long now, bool end) } static unsigned long __part_start_io_acct(struct block_device *part, - unsigned int sectors, unsigned int op) + unsigned int sectors, unsigned int op, + unsigned long start_time) { const int sgrp = op_stat_group(op); - unsigned long now = READ_ONCE(jiffies); part_stat_lock(); - update_io_ticks(part, now, false); + update_io_ticks(part, start_time, false); part_stat_inc(part, ios[sgrp]); part_stat_add(part, sectors[sgrp], sectors); part_stat_local_inc(part, in_flight[op_is_write(op)]); part_stat_unlock(); - return now; + return start_time; +} + +/** + * bio_start_io_acct_time - start I/O accounting for bio based drivers + * @bio: bio to start account for + * @start_time: start time that should be passed back to bio_end_io_acct(). + */ +void bio_start_io_acct_time(struct bio *bio, unsigned long start_time) +{ + __part_start_io_acct(bio->bi_bdev, bio_sectors(bio), + bio_op(bio), start_time); } +EXPORT_SYMBOL_GPL(bio_start_io_acct_time); /** * bio_start_io_acct - start I/O accounting for bio based drivers @@ -1084,14 +1096,15 @@ static unsigned long __part_start_io_acct(struct block_device *part, */ unsigned long bio_start_io_acct(struct bio *bio) { - return __part_start_io_acct(bio->bi_bdev, bio_sectors(bio), bio_op(bio)); + return __part_start_io_acct(bio->bi_bdev, bio_sectors(bio), + bio_op(bio), jiffies); } EXPORT_SYMBOL_GPL(bio_start_io_acct); unsigned long disk_start_io_acct(struct gendisk *disk, unsigned int sectors, unsigned int op) { - return __part_start_io_acct(disk->part0, sectors, op); + return __part_start_io_acct(disk->part0, sectors, op, jiffies); } EXPORT_SYMBOL(disk_start_io_acct); diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 9c95df26fc26..f35aea98bc35 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1258,6 +1258,7 @@ unsigned long disk_start_io_acct(struct gendisk *disk, unsigned int sectors, void disk_end_io_acct(struct gendisk *disk, unsigned int op, unsigned long start_time); +void bio_start_io_acct_time(struct bio *bio, unsigned long start_time); unsigned long bio_start_io_acct(struct bio *bio); void bio_end_io_acct_remapped(struct bio *bio, unsigned long start_time, struct block_device *orig_bdev); From patchwork Thu Jan 27 22:56:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12727491 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60B71C433EF for ; Thu, 27 Jan 2022 22:56:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235038AbiA0W44 (ORCPT ); Thu, 27 Jan 2022 17:56:56 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:38714 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235304AbiA0W44 (ORCPT ); Thu, 27 Jan 2022 17:56:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643324215; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=tQTNouHLD2pqrJUsY0rGuooqQ0qZGduHNcta+zU07XE=; b=Hy656/MPtbbDSXvaGv9sEUwLf8U8UijJgFTOO83RJBuuwSjnTKg7fFEH6xeEn+kMK64Ryf JhoheFQYnKo4hv4/7kr3vh+5Tr9G3zWDusHSo4cXIOIZ66ifNEtpWU9RVK+lLFYKo2A7ey tC0yvFCSbOe+f+uyJrGgU+TbrBdhoIY= Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-441-zzUAaXg0OyGZUmLir0lFnA-1; Thu, 27 Jan 2022 17:56:53 -0500 X-MC-Unique: zzUAaXg0OyGZUmLir0lFnA-1 Received: by mail-qk1-f198.google.com with SMTP id u17-20020a05620a431100b004765c0dc33cso3511454qko.14 for ; Thu, 27 Jan 2022 14:56:53 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=tQTNouHLD2pqrJUsY0rGuooqQ0qZGduHNcta+zU07XE=; b=Y6Xe8Rsm/D3p+bU9xvZHEWLGvP426PvkRQwC4yrMF/0XXjb+eWYbrzKis5i2Pdu5M8 FOhcMsxtw9Aur7uexVU6isAY8oMCqjsAdsUDyo4Z0ts5v4idxlcqzJkDeehfRoLwAWN6 TmPi66iR1GFHrx7lq7V1P6SgQH9mq21v6iqbeKX2jP8fZFl79VwxWIEE6jb9btTPJUXo 5KRb6MDnUb3xr4jXpF62uJjfjQDqaFqWyldZReoXLdIodU2ZZd1JWsrtlr72iUJKK5q1 dm85FvLIXIiAtoP9Y3FKN9gc8od5k9nQSJKIU1tgB0nj/2yWlg7nVtA15pp/jqRz9P3W 3jzA== X-Gm-Message-State: AOAM531lRVeVpWEHhoyr/4ab58n44w81U+1Gbqgb9y6K/LS0XKh+xjnV H2n7XFsEUafAFfRCHRQtF0IYJKt06wQMS1nHerJc3G+d7t7oM2hJJIV7K3NryyXc1Cg+AudQeS7 QWedT2JBD7jIdFXYFfRlkZQ== X-Received: by 2002:a05:620a:103c:: with SMTP id a28mr4150249qkk.417.1643324212904; Thu, 27 Jan 2022 14:56:52 -0800 (PST) X-Google-Smtp-Source: ABdhPJyL2H63eKsh0Qu7Yz6ytqgIu4GetPzYuDGznITAsL1ZAj2qOMKyljpt5v1X/OwIwEtcD0UfYw== X-Received: by 2002:a05:620a:103c:: with SMTP id a28mr4150241qkk.417.1643324212703; Thu, 27 Jan 2022 14:56:52 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id p134sm2016512qke.29.2022.01.27.14.56.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Jan 2022 14:56:52 -0800 (PST) From: Mike Snitzer To: axboe@kernel.dk Cc: hch@lst.de, dm-devel@redhat.com, linux-block@vger.kernel.org Subject: [PATCH v2 2/3] dm: revert partial fix for redundant bio-based IO accounting Date: Thu, 27 Jan 2022 17:56:47 -0500 Message-Id: <20220127225648.28729-3-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220127225648.28729-1-snitzer@redhat.com> References: <20220127225648.28729-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Reverts a1e1cb72d9649 ("dm: fix redundant IO accounting for bios that need splitting") because it was too narrow in scope (only addressed redundant 'sectors[]' accounting and not ios, nsecs[], etc). Cc: stable@vger.kernel.org Signed-off-by: Mike Snitzer --- drivers/md/dm.c | 15 --------------- 1 file changed, 15 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index c0ae8087c602..9849114b3c08 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1442,9 +1442,6 @@ static void init_clone_info(struct clone_info *ci, struct mapped_device *md, ci->sector = bio->bi_iter.bi_sector; } -#define __dm_part_stat_sub(part, field, subnd) \ - (part_stat_get(part, field) -= (subnd)) - /* * Entry point to split a bio into clones and submit them to the targets. */ @@ -1480,18 +1477,6 @@ static void __split_and_process_bio(struct mapped_device *md, GFP_NOIO, &md->queue->bio_split); ci.io->orig_bio = b; - /* - * Adjust IO stats for each split, otherwise upon queue - * reentry there will be redundant IO accounting. - * NOTE: this is a stop-gap fix, a proper fix involves - * significant refactoring of DM core's bio splitting - * (by eliminating DM's splitting and just using bio_split) - */ - part_stat_lock(); - __dm_part_stat_sub(dm_disk(md)->part0, - sectors[op_stat_group(bio_op(bio))], ci.sector_count); - part_stat_unlock(); - bio_chain(b, bio); trace_block_split(b, bio->bi_iter.bi_sector); submit_bio_noacct(bio); From patchwork Thu Jan 27 22:56:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12727492 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08CCDC433FE for ; Thu, 27 Jan 2022 22:56:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235304AbiA0W45 (ORCPT ); Thu, 27 Jan 2022 17:56:57 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:21565 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229801AbiA0W44 (ORCPT ); Thu, 27 Jan 2022 17:56:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643324216; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=vJ6TnJPKd3jnAAKSRYmEdyBJB9c5u4F9myAf76vl6jE=; b=J33ymQjexydbPNM+4OytE1i7B+RyBTbZCFi54KA18Nh1d/eIN3syCoBk8sYpelZTPmXTfD tclDovBggLdaSRfQyCDZ1l3VKBcV7PSZbJjxxnu3lEhyOvoCeipJHzdUUHzDhKImJJhPvW eRTQkciFwSEy7gY+W7YM/VzhBQFs+Bw= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-196-2x-7KD3sOnOFI60Zk27uSw-1; Thu, 27 Jan 2022 17:56:55 -0500 X-MC-Unique: 2x-7KD3sOnOFI60Zk27uSw-1 Received: by mail-qv1-f70.google.com with SMTP id e14-20020a0cf34e000000b0042623ad325aso91198qvm.16 for ; Thu, 27 Jan 2022 14:56:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vJ6TnJPKd3jnAAKSRYmEdyBJB9c5u4F9myAf76vl6jE=; b=Oufe10Z/SRzc5vOca1WlnvbvFZ1cwZsKza0+ANPlRe5UsQxF/qQUE43X31bjcmxS2U 9dZ2szcK5neLdQbi5CxqtYj0T6j3800BXWqDYkd9J6z8sdR2wTlF7WfwTay6AC/QC/UQ Nu31ZSob8IbNuq7/GyOnriSwEX+nUJX3fESz+rCiyEdwFREil2Ju+cGsE0KdywSnpBk8 w+8SzvgaUUGpyWbt8gdiokd6UCGTdONFv1JwLaIrrobJqle9cGN3syX8bLywKWv7DIk2 HY0dTE/dkmNiQTrKw86f+4jDASx1Cqw6i99Z0cs5q4mcm3c5WnCEQuYimNTMqAifEjU4 H4rQ== X-Gm-Message-State: AOAM5301jyRPsRe6DWjrOJFUZAIgAFpHYHPglWFVao7AxPrsAvkp+duG M90XqEjJ90lBEiwcBHMU/0f0D3XmELJV7S+JwkXwpN9fVpzyvmnn2aiT4IggkfSyqLZ9YLneNl7 BpQVpNlTUVK4N5XoPt4oVPw== X-Received: by 2002:a05:620a:450c:: with SMTP id t12mr4251281qkp.343.1643324214621; Thu, 27 Jan 2022 14:56:54 -0800 (PST) X-Google-Smtp-Source: ABdhPJx0nJBNt2XNTVFChnIRxj7wnHOd+t1Jl5N/kzJqu73nkFplu02EOEvuNXsrUdyeMf2Ql9ud7g== X-Received: by 2002:a05:620a:450c:: with SMTP id t12mr4251277qkp.343.1643324214446; Thu, 27 Jan 2022 14:56:54 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id n17sm1149219qkp.89.2022.01.27.14.56.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Jan 2022 14:56:54 -0800 (PST) From: Mike Snitzer To: axboe@kernel.dk Cc: hch@lst.de, dm-devel@redhat.com, linux-block@vger.kernel.org Subject: [PATCH v2 3/3] dm: properly fix redundant bio-based IO accounting Date: Thu, 27 Jan 2022 17:56:48 -0500 Message-Id: <20220127225648.28729-4-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220127225648.28729-1-snitzer@redhat.com> References: <20220127225648.28729-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Record the start_time for a bio but defer the starting block core's IO accounting until after IO is submitted using bio_start_io_acct_time(). This approach avoids the need to mess around with any of the individual IO stats in response to a bio_split() that follows bio submission. Reported-by: Bud Brown Cc: stable@vger.kernel.org Depends-on: f9893e1da2e3 ("block: add bio_start_io_acct_time() to control start_time") Signed-off-by: Mike Snitzer --- drivers/md/dm.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 9849114b3c08..144436301a57 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -489,7 +489,7 @@ static void start_io_acct(struct dm_io *io) struct mapped_device *md = io->md; struct bio *bio = io->orig_bio; - io->start_time = bio_start_io_acct(bio); + __bio_start_io_acct(bio, io->start_time); if (unlikely(dm_stats_used(&md->stats))) dm_stats_account_io(&md->stats, bio_data_dir(bio), bio->bi_iter.bi_sector, bio_sectors(bio), @@ -535,7 +535,7 @@ static struct dm_io *alloc_io(struct mapped_device *md, struct bio *bio) io->md = md; spin_lock_init(&io->endio_lock); - start_io_acct(io); + io->start_time = READ_ONCE(jiffies); return io; } @@ -1482,6 +1482,7 @@ static void __split_and_process_bio(struct mapped_device *md, submit_bio_noacct(bio); } } + start_io_acct(ci.io); /* drop the extra reference count */ dm_io_dec_pending(ci.io, errno_to_blk_status(error));