From patchwork Mon Jun 6 17:56:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12870741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E391CCA482 for ; Mon, 6 Jun 2022 17:57:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229452AbiFFR5J (ORCPT ); Mon, 6 Jun 2022 13:57:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45904 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230322AbiFFR5H (ORCPT ); Mon, 6 Jun 2022 13:57:07 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 129A23207C9; Mon, 6 Jun 2022 10:57:02 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id F03D221B1F; Mon, 6 Jun 2022 17:57:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1654538221; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=W4D2/ytiKcx0QKYW8PMSxMJ4IhfT+jX8mKmVp4uJXSM=; b=UeryIq1BmW3nDkgoDomooUQS8B14hUSNMkPnYQNd9EMTSai/l7fauS2hu+ScZPWS8u47tK ZGBhBq9HeuByUGfbyi5ImBy/nmlKmNZRWPjLegZGeEDu5nZ1TWmTFIli0U4t56yX5S0FqG 6YjfQNYIQpWV9+ndCXQi3kR+Os5h82w= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1654538221; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=W4D2/ytiKcx0QKYW8PMSxMJ4IhfT+jX8mKmVp4uJXSM=; b=ZaZNvvt4FdT760IzOGsUA31Qr072Ijy8KNyAasArznuSATBA5gGogR6rU1GzdNPYjVuXSE 4wnQdsyWT2M0GOAQ== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id D8D192C142; Mon, 6 Jun 2022 17:57:00 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 0F33DA0634; Mon, 6 Jun 2022 19:56:55 +0200 (CEST) From: Jan Kara To: stable@vger.kernel.org Cc: , Paolo Valente , Jens Axboe , Jan Kara , "yukuai (C)" , Christoph Hellwig Subject: [PATCH 1/6] bfq: Avoid merging queues with different parents Date: Mon, 6 Jun 2022 19:56:36 +0200 Message-Id: <20220606175655.8993-1-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220606174118.10992-1-jack@suse.cz> References: <20220606174118.10992-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3153; h=from:subject; bh=A/I/kHI3BdapLCXKTc2SKGZd0HSXEBlfaC3acAvMRuM=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBinj/U2gJtF+YfLeh9VSlGi00oN5882kMh3O9Qv7pu y8SBWsCJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYp4/1AAKCRCcnaoHP2RA2Zh4CA DKB+mFult4bNlb0X5f7txABWLb7OuWlMzPbobhCZKTVu8knO0hv3uGkt35uKLFawt1pyeKcKKnIqRx +MW0D191S9M0mz07raSFNXDsCGhN5UhBUfZh04O00UyC2g4rZuUwya1M75iybZ/Q/IIzL2L3WmCjup v307880KKy1R2yncrOtge88vkJJ0EaNuyRGtxWb6WgESIbxK8Ygv5/V1esxnwcpl7FKI+yBkvPKZtb SxIqjjNpGV0QNUdQfEQUvZ157JA17TnIkDLIsDIYW2OcFeVfXHixgFh5stvJbKRTXV+CTAvuEYxZ3/ T48m+JSdTpS0MJdRpYlX9QXN973eKt X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org commit c1cee4ab36acef271be9101590756ed0c0c374d9 upstream. It can happen that the parent of a bfqq changes between the moment we decide two queues are worth to merge (and set bic->stable_merge_bfqq) and the moment bfq_setup_merge() is called. This can happen e.g. because the process submitted IO for a different cgroup and thus bfqq got reparented. It can even happen that the bfqq we are merging with has parent cgroup that is already offline and going to be destroyed in which case the merge can lead to use-after-free issues such as: BUG: KASAN: use-after-free in __bfq_deactivate_entity+0x9cb/0xa50 Read of size 8 at addr ffff88800693c0c0 by task runc:[2:INIT]/10544 CPU: 0 PID: 10544 Comm: runc:[2:INIT] Tainted: G E 5.15.2-0.g5fb85fd-default #1 openSUSE Tumbleweed (unreleased) f1f3b891c72369aebecd2e43e4641a6358867c70 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a-rebuilt.opensuse.org 04/01/2014 Call Trace: dump_stack_lvl+0x46/0x5a print_address_description.constprop.0+0x1f/0x140 ? __bfq_deactivate_entity+0x9cb/0xa50 kasan_report.cold+0x7f/0x11b ? __bfq_deactivate_entity+0x9cb/0xa50 __bfq_deactivate_entity+0x9cb/0xa50 ? update_curr+0x32f/0x5d0 bfq_deactivate_entity+0xa0/0x1d0 bfq_del_bfqq_busy+0x28a/0x420 ? resched_curr+0x116/0x1d0 ? bfq_requeue_bfqq+0x70/0x70 ? check_preempt_wakeup+0x52b/0xbc0 __bfq_bfqq_expire+0x1a2/0x270 bfq_bfqq_expire+0xd16/0x2160 ? try_to_wake_up+0x4ee/0x1260 ? bfq_end_wr_async_queues+0xe0/0xe0 ? _raw_write_unlock_bh+0x60/0x60 ? _raw_spin_lock_irq+0x81/0xe0 bfq_idle_slice_timer+0x109/0x280 ? bfq_dispatch_request+0x4870/0x4870 __hrtimer_run_queues+0x37d/0x700 ? enqueue_hrtimer+0x1b0/0x1b0 ? kvm_clock_get_cycles+0xd/0x10 ? ktime_get_update_offsets_now+0x6f/0x280 hrtimer_interrupt+0x2c8/0x740 Fix the problem by checking that the parent of the two bfqqs we are merging in bfq_setup_merge() is the same. Link: https://lore.kernel.org/linux-block/20211125172809.GC19572@quack2.suse.cz/ CC: stable@vger.kernel.org Fixes: 430a67f9d616 ("block, bfq: merge bursts of newly-created queues") Tested-by: "yukuai (C)" Signed-off-by: Jan Kara Reviewed-by: Christoph Hellwig Link: https://lore.kernel.org/r/20220401102752.8599-2-jack@suse.cz Signed-off-by: Jens Axboe --- block/bfq-iosched.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index bab5122062f5..e14f421282dd 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -2509,6 +2509,14 @@ bfq_setup_merge(struct bfq_queue *bfqq, struct bfq_queue *new_bfqq) if (process_refs == 0 || new_process_refs == 0) return NULL; + /* + * Make sure merged queues belong to the same parent. Parents could + * have changed since the time we decided the two queues are suitable + * for merging. + */ + if (new_bfqq->entity.parent != bfqq->entity.parent) + return NULL; + bfq_log_bfqq(bfqq->bfqd, bfqq, "scheduling merge with queue %d", new_bfqq->pid); From patchwork Mon Jun 6 17:56:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12870743 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28614CCA486 for ; Mon, 6 Jun 2022 17:57:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230304AbiFFR5K (ORCPT ); Mon, 6 Jun 2022 13:57:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45890 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230313AbiFFR5H (ORCPT ); Mon, 6 Jun 2022 13:57:07 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 126CF3151ED; Mon, 6 Jun 2022 10:57:02 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id F260921B82; Mon, 6 Jun 2022 17:57:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1654538220; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9dNUC5ElJIbZzeLi0chUE2OFIgsfH7PTiQ5PX1L/rm0=; b=mzb9pmVRJklNcSzjV2KezE9JC99jN6Rji69LmfS+hTA6skaXZvxK1M5j20BRbFV9zCarb+ QMrr4U1DxZmIU0PjSJKKh7GSoVIZqPu3c2U5Oja27ibnQmrdoG4Hd0xhiLGVssaTfNfTiq wrUKpc/5+sVCXKl5UZXbR+yHHzUT4HQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1654538220; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9dNUC5ElJIbZzeLi0chUE2OFIgsfH7PTiQ5PX1L/rm0=; b=v0RYa4Ep4J36sU2XL157c5DEDqSnQOMpRR1RpGbRW0BcAtEFP+RMqBfNgKSKijD5TuUMky TgaIYTF9//E9smCg== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id DEDB52C143; Mon, 6 Jun 2022 17:57:00 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 15C2BA0635; Mon, 6 Jun 2022 19:56:55 +0200 (CEST) From: Jan Kara To: stable@vger.kernel.org Cc: , Paolo Valente , Jens Axboe , Jan Kara , "yukuai (C)" , Christoph Hellwig Subject: [PATCH 2/6] bfq: Drop pointless unlock-lock pair Date: Mon, 6 Jun 2022 19:56:37 +0200 Message-Id: <20220606175655.8993-2-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220606174118.10992-1-jack@suse.cz> References: <20220606174118.10992-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1164; h=from:subject; bh=fazGyHZ8AizFT9D2tpzVsapOpu4GnWYR/+Eh7rri6nA=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBinj/VYdWTQHDswBgNchjRCke1hpPpM9cmGOwxdQoo 9VHvj96JATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYp4/1QAKCRCcnaoHP2RA2T0iCA CFRN0ozJFeULDoqCgXP9vq9WxFZz5f4zsDfbI8lNlAqgcXsE8ZU9KMTu10ZNqPbVcurP4hGI0VLU5P l+Q0a7HItNkcAvEo6THMM28yV+pVn29Dn3eGZkmup7ovpHXS+myS44+I8ObKXynhQRFqHZ4SJu4+Da R8mxUpDoxKWWRjfo6eUvoR1xJFT6xdENWI0BHx3CJdge0n9GO/3Y5Es0Ln7OFUMCTy4qgTy1Vova77 chH3VAiNvm/zft4R190AxVCkKQoOYHXy+J0R8Txh2vBSwsA5RrRn9iDK+hGBAXowMx7WP0E3tebgPC thnVsh3nL/VJNCm68Byvw8vzHV8bfR X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org commit fc84e1f941b91221092da5b3102ec82da24c5673 upstream. In bfq_insert_request() we unlock bfqd->lock only to call trace_block_rq_insert() and then lock bfqd->lock again. This is really pointless since tracing is disabled if we really care about performance and even if the tracepoint is enabled, it is a quick call. CC: stable@vger.kernel.org Tested-by: "yukuai (C)" Signed-off-by: Jan Kara Reviewed-by: Christoph Hellwig Link: https://lore.kernel.org/r/20220401102752.8599-5-jack@suse.cz Signed-off-by: Jens Axboe --- block/bfq-iosched.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index e14f421282dd..bad088103279 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -5537,11 +5537,8 @@ static void bfq_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, return; } - spin_unlock_irq(&bfqd->lock); - blk_mq_sched_request_inserted(rq); - spin_lock_irq(&bfqd->lock); bfqq = bfq_init_rq(rq); if (!bfqq || at_head || blk_rq_is_passthrough(rq)) { if (at_head) From patchwork Mon Jun 6 17:56:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12870742 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97A7DCCA484 for ; Mon, 6 Jun 2022 17:57:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230322AbiFFR5J (ORCPT ); Mon, 6 Jun 2022 13:57:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230304AbiFFR5F (ORCPT ); Mon, 6 Jun 2022 13:57:05 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1255B2F9AFF; Mon, 6 Jun 2022 10:57:02 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 072501FAB4; Mon, 6 Jun 2022 17:57:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1654538221; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uHkxjr2Ym/i8mJeccO571n7luRlOdtuAed072azC5j0=; b=ebRitawTLd0gPBOmRE6mDb1/UpAffDAkGfNyh4+GLCdca4kHtHtDsEGYTqLXfWka6fQ5aQ mNCAbEwjeQzjtxwHSIU9hfDRrvy0Mae1b9mFNxKIZojlrSGW8LSYHdFpGspUCM9IpYlyfc bBNYmQ9DePCYaPJ50VGgHd+OtltKQRg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1654538221; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uHkxjr2Ym/i8mJeccO571n7luRlOdtuAed072azC5j0=; b=+gzQphCqvLRn1/cFj3tY59A31gl7eS7FVRhWGvAbJ/P8Ky0X/s9FbGIZg45LGjlfYEczqp SqeZ28Ob0tRe7gDg== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id E27FF2C145; Mon, 6 Jun 2022 17:57:00 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 1B85EA0637; Mon, 6 Jun 2022 19:56:55 +0200 (CEST) From: Jan Kara To: stable@vger.kernel.org Cc: , Paolo Valente , Jens Axboe , Jan Kara , "yukuai (C)" , Christoph Hellwig Subject: [PATCH 3/6] bfq: Remove pointless bfq_init_rq() calls Date: Mon, 6 Jun 2022 19:56:38 +0200 Message-Id: <20220606175655.8993-3-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220606174118.10992-1-jack@suse.cz> References: <20220606174118.10992-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3088; h=from:subject; bh=3klBZKaG8TQ6MLxmPuPqj5KLP4rIpGxj0PEejZkD6sA=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBinj/Wt0fJtlT36dh/RxOKXc22ad0vX7OD4CU6Y4+5 WGNAswyJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYp4/1gAKCRCcnaoHP2RA2dE6B/ 9xCeWgIaVpZPakL3+DdNWrM0/8YmFJwSiIsYqPTeDc5kHxHmhhRasQuedq8ymHfbav7dCnhTBg3eTT h3KS30hO8T1IinL9ShkA67vueIJ0YlRfJ28KSlbhw3s+BOT/tfhc188D97+lLWDja/9YDUXIZfY4PF 9XDxE3g1jO1ckBTLIFhrB+hBbVXZWnzZEQ2c9aPeo4dtlKrcj5TzzCR7RUPd4/7RL+n/Pi3EG5cv/S ooqLiBSai8PRPOJgwu7e+35y4iiYqRxXMs5d19aErTNFL9edRQMPJfrNVdt15qN8mXVkPsjGP7KCGD 3AHhFvLet+vWfcxtYQsdi7gLQaw3cK X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org commit 5f550ede5edf846ecc0067be1ba80514e6fe7f8e upstream. We call bfq_init_rq() from request merging functions where requests we get should have already gone through bfq_init_rq() during insert and anyway we want to do anything only if the request is already tracked by BFQ. So replace calls to bfq_init_rq() with RQ_BFQQ() instead to simply skip requests untracked by BFQ. We move bfq_init_rq() call in bfq_insert_request() a bit earlier to cover request merging and thus can transfer FIFO position in case of a merge. CC: stable@vger.kernel.org Tested-by: "yukuai (C)" Signed-off-by: Jan Kara Reviewed-by: Christoph Hellwig Link: https://lore.kernel.org/r/20220401102752.8599-6-jack@suse.cz Signed-off-by: Jens Axboe --- block/bfq-iosched.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index bad088103279..3b605d8d99bf 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -2267,8 +2267,6 @@ static int bfq_request_merge(struct request_queue *q, struct request **req, return ELEVATOR_NO_MERGE; } -static struct bfq_queue *bfq_init_rq(struct request *rq); - static void bfq_request_merged(struct request_queue *q, struct request *req, enum elv_merge type) { @@ -2277,7 +2275,7 @@ static void bfq_request_merged(struct request_queue *q, struct request *req, blk_rq_pos(req) < blk_rq_pos(container_of(rb_prev(&req->rb_node), struct request, rb_node))) { - struct bfq_queue *bfqq = bfq_init_rq(req); + struct bfq_queue *bfqq = RQ_BFQQ(req); struct bfq_data *bfqd; struct request *prev, *next_rq; @@ -2329,8 +2327,8 @@ static void bfq_request_merged(struct request_queue *q, struct request *req, static void bfq_requests_merged(struct request_queue *q, struct request *rq, struct request *next) { - struct bfq_queue *bfqq = bfq_init_rq(rq), - *next_bfqq = bfq_init_rq(next); + struct bfq_queue *bfqq = RQ_BFQQ(rq), + *next_bfqq = RQ_BFQQ(next); if (!bfqq) return; @@ -5518,6 +5516,8 @@ static inline void bfq_update_insert_stats(struct request_queue *q, unsigned int cmd_flags) {} #endif /* CONFIG_BFQ_CGROUP_DEBUG */ +static struct bfq_queue *bfq_init_rq(struct request *rq); + static void bfq_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, bool at_head) { @@ -5532,6 +5532,7 @@ static void bfq_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, bfqg_stats_update_legacy_io(q, rq); #endif spin_lock_irq(&bfqd->lock); + bfqq = bfq_init_rq(rq); if (blk_mq_sched_try_insert_merge(q, rq)) { spin_unlock_irq(&bfqd->lock); return; @@ -5539,7 +5540,6 @@ static void bfq_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, blk_mq_sched_request_inserted(rq); - bfqq = bfq_init_rq(rq); if (!bfqq || at_head || blk_rq_is_passthrough(rq)) { if (at_head) list_add(&rq->queuelist, &bfqd->dispatch); From patchwork Mon Jun 6 17:56:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12870746 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5797CCA489 for ; Mon, 6 Jun 2022 17:57:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230320AbiFFR5L (ORCPT ); Mon, 6 Jun 2022 13:57:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230325AbiFFR5H (ORCPT ); Mon, 6 Jun 2022 13:57:07 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1290531CC89; Mon, 6 Jun 2022 10:57:02 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 1959D1FAB6; Mon, 6 Jun 2022 17:57:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1654538221; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dbPawdX5B3E2oP+8USedKhNrEeSZHwGBIF1v9bBKK74=; b=d9fftP95D+h379D3aU9+n7a9dWJQ1ztc48AaDQkes9jhv4UT2AOG+DIHvyJcjTjDh/Pmph slCyyELy7mgrJKWENiv+OnQgXug1sueM82pjHyPzmtN9dT2UYKzhWMFnGKMjxGniIetLEQ dvNrFZ2vh7VuK+pxcoyGpr/glzNkheg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1654538221; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dbPawdX5B3E2oP+8USedKhNrEeSZHwGBIF1v9bBKK74=; b=q/QdY6k4dYkQOeR6NwITI+R/k766BN+xHlelDDJJd89dZjoNlYOih7XICW6htUPpyeUgHi sM0cU3nvAb8oosDA== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id E60642C146; Mon, 6 Jun 2022 17:57:00 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 225D6A0638; Mon, 6 Jun 2022 19:56:55 +0200 (CEST) From: Jan Kara To: stable@vger.kernel.org Cc: , Paolo Valente , Jens Axboe , Jan Kara , "yukuai (C)" , Christoph Hellwig Subject: [PATCH 4/6] bfq: Get rid of __bio_blkcg() usage Date: Mon, 6 Jun 2022 19:56:39 +0200 Message-Id: <20220606175655.8993-4-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220606174118.10992-1-jack@suse.cz> References: <20220606174118.10992-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6875; h=from:subject; bh=Rw0dwNVbs0zoYjRv4tJ+QqUwOTZHh+Gw2mfV8IF+vow=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBinj/W3oPd+MsbUKCsjgJV7G1h3BWzeS6lVPEyhrz2 tvCWo2yJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYp4/1gAKCRCcnaoHP2RA2dJHCA DJzh+5bEjwNm/r752tDgnTB0y+tHEyzov8fijOC1VT72Xszcb8ZAPwXp40Mx4CV96Hm93+6/w1it7h eIq7Xi1LpccOBKe4IWa7Y25vCKMi3eH9MkO28bfi/exXYISMgOX+oFX0MCoFgfmK9Llx4Z2EpCvQDI bK9m8Lh6YobBeVhXml+UpnjpHiyCVi1YIQ9M0ghusjwJ8mYrnn1HISebTWxnssCtCDjTAeylarZiDc FED9D3ZBcoTiA7+YVw93gJpg5IQ9Tt+MkRRwWcGUudLUEhmlNQOv9UiW79i23ZZvxZMpwOqG6f+hbh WSnmfiYHwHKB8tjzP+Pg6zDnr0veAu X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org commit 4e54a2493e582361adc3bfbf06c7d50d19d18837 upstream. BFQ usage of __bio_blkcg() is a relict from the past. Furthermore if bio would not be associated with any blkcg, the usage of __bio_blkcg() in BFQ is prone to races with the task being migrated between cgroups as __bio_blkcg() calls at different places could return different blkcgs. Convert BFQ to the new situation where bio->bi_blkg is initialized in bio_set_dev() and thus practically always valid. This allows us to save blkcg_gq lookup and noticeably simplify the code. CC: stable@vger.kernel.org Fixes: 0fe061b9f03c ("blkcg: fix ref count issue with bio_blkcg() using task_css") Tested-by: "yukuai (C)" Signed-off-by: Jan Kara Reviewed-by: Christoph Hellwig Link: https://lore.kernel.org/r/20220401102752.8599-8-jack@suse.cz Signed-off-by: Jens Axboe --- block/bfq-cgroup.c | 63 +++++++++++++++++---------------------------- block/bfq-iosched.c | 10 +------ block/bfq-iosched.h | 3 +-- 3 files changed, 25 insertions(+), 51 deletions(-) diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index 0077f44ad63f..622c9a13e496 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -582,27 +582,11 @@ static void bfq_group_set_parent(struct bfq_group *bfqg, entity->sched_data = &parent->sched_data; } -static struct bfq_group *bfq_lookup_bfqg(struct bfq_data *bfqd, - struct blkcg *blkcg) +static void bfq_link_bfqg(struct bfq_data *bfqd, struct bfq_group *bfqg) { - struct blkcg_gq *blkg; - - blkg = blkg_lookup(blkcg, bfqd->queue); - if (likely(blkg)) - return blkg_to_bfqg(blkg); - return NULL; -} - -struct bfq_group *bfq_find_set_group(struct bfq_data *bfqd, - struct blkcg *blkcg) -{ - struct bfq_group *bfqg, *parent; + struct bfq_group *parent; struct bfq_entity *entity; - bfqg = bfq_lookup_bfqg(bfqd, blkcg); - if (unlikely(!bfqg)) - return NULL; - /* * Update chain of bfq_groups as we might be handling a leaf group * which, along with some of its relatives, has not been hooked yet @@ -619,8 +603,15 @@ struct bfq_group *bfq_find_set_group(struct bfq_data *bfqd, bfq_group_set_parent(curr_bfqg, parent); } } +} - return bfqg; +struct bfq_group *bfq_bio_bfqg(struct bfq_data *bfqd, struct bio *bio) +{ + struct blkcg_gq *blkg = bio->bi_blkg; + + if (!blkg) + return bfqd->root_group; + return blkg_to_bfqg(blkg); } /** @@ -696,25 +687,15 @@ void bfq_bfqq_move(struct bfq_data *bfqd, struct bfq_queue *bfqq, * Move bic to blkcg, assuming that bfqd->lock is held; which makes * sure that the reference to cgroup is valid across the call (see * comments in bfq_bic_update_cgroup on this issue) - * - * NOTE: an alternative approach might have been to store the current - * cgroup in bfqq and getting a reference to it, reducing the lookup - * time here, at the price of slightly more complex code. */ -static struct bfq_group *__bfq_bic_change_cgroup(struct bfq_data *bfqd, - struct bfq_io_cq *bic, - struct blkcg *blkcg) +static void *__bfq_bic_change_cgroup(struct bfq_data *bfqd, + struct bfq_io_cq *bic, + struct bfq_group *bfqg) { struct bfq_queue *async_bfqq = bic_to_bfqq(bic, 0); struct bfq_queue *sync_bfqq = bic_to_bfqq(bic, 1); - struct bfq_group *bfqg; struct bfq_entity *entity; - bfqg = bfq_find_set_group(bfqd, blkcg); - - if (unlikely(!bfqg)) - bfqg = bfqd->root_group; - if (async_bfqq) { entity = &async_bfqq->entity; @@ -766,20 +747,24 @@ static struct bfq_group *__bfq_bic_change_cgroup(struct bfq_data *bfqd, void bfq_bic_update_cgroup(struct bfq_io_cq *bic, struct bio *bio) { struct bfq_data *bfqd = bic_to_bfqd(bic); - struct bfq_group *bfqg = NULL; + struct bfq_group *bfqg = bfq_bio_bfqg(bfqd, bio); uint64_t serial_nr; - rcu_read_lock(); - serial_nr = __bio_blkcg(bio)->css.serial_nr; + serial_nr = bfqg_to_blkg(bfqg)->blkcg->css.serial_nr; /* * Check whether blkcg has changed. The condition may trigger * spuriously on a newly created cic but there's no harm. */ if (unlikely(!bfqd) || likely(bic->blkcg_serial_nr == serial_nr)) - goto out; + return; - bfqg = __bfq_bic_change_cgroup(bfqd, bic, __bio_blkcg(bio)); + /* + * New cgroup for this process. Make sure it is linked to bfq internal + * cgroup hierarchy. + */ + bfq_link_bfqg(bfqd, bfqg); + __bfq_bic_change_cgroup(bfqd, bic, bfqg); /* * Update blkg_path for bfq_log_* functions. We cache this * path, and update it here, for the following @@ -832,8 +817,6 @@ void bfq_bic_update_cgroup(struct bfq_io_cq *bic, struct bio *bio) */ blkg_path(bfqg_to_blkg(bfqg), bfqg->blkg_path, sizeof(bfqg->blkg_path)); bic->blkcg_serial_nr = serial_nr; -out: - rcu_read_unlock(); } /** @@ -1451,7 +1434,7 @@ void bfq_end_wr_async(struct bfq_data *bfqd) bfq_end_wr_async_queues(bfqd, bfqd->root_group); } -struct bfq_group *bfq_find_set_group(struct bfq_data *bfqd, struct blkcg *blkcg) +struct bfq_group *bfq_bio_bfqg(struct bfq_data *bfqd, struct bio *bio) { return bfqd->root_group; } diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 3b605d8d99bf..592d32a46c4c 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -5162,14 +5162,7 @@ static struct bfq_queue *bfq_get_queue(struct bfq_data *bfqd, struct bfq_queue *bfqq; struct bfq_group *bfqg; - rcu_read_lock(); - - bfqg = bfq_find_set_group(bfqd, __bio_blkcg(bio)); - if (!bfqg) { - bfqq = &bfqd->oom_bfqq; - goto out; - } - + bfqg = bfq_bio_bfqg(bfqd, bio); if (!is_sync) { async_bfqq = bfq_async_queue_prio(bfqd, bfqg, ioprio_class, ioprio); @@ -5213,7 +5206,6 @@ static struct bfq_queue *bfq_get_queue(struct bfq_data *bfqd, out: bfqq->ref++; /* get a process reference to this queue */ bfq_log_bfqq(bfqd, bfqq, "get_queue, at end: %p, %d", bfqq, bfqq->ref); - rcu_read_unlock(); return bfqq; } diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index b116ed48d27d..2a4a6f44efff 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -984,8 +984,7 @@ void bfq_bfqq_move(struct bfq_data *bfqd, struct bfq_queue *bfqq, void bfq_init_entity(struct bfq_entity *entity, struct bfq_group *bfqg); void bfq_bic_update_cgroup(struct bfq_io_cq *bic, struct bio *bio); void bfq_end_wr_async(struct bfq_data *bfqd); -struct bfq_group *bfq_find_set_group(struct bfq_data *bfqd, - struct blkcg *blkcg); +struct bfq_group *bfq_bio_bfqg(struct bfq_data *bfqd, struct bio *bio); struct blkcg_gq *bfqg_to_blkg(struct bfq_group *bfqg); struct bfq_group *bfqq_group(struct bfq_queue *bfqq); struct bfq_group *bfq_create_group_hierarchy(struct bfq_data *bfqd, int node); From patchwork Mon Jun 6 17:56:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12870745 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05BEFCCA488 for ; Mon, 6 Jun 2022 17:57:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230253AbiFFR5K (ORCPT ); Mon, 6 Jun 2022 13:57:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45906 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230320AbiFFR5H (ORCPT ); Mon, 6 Jun 2022 13:57:07 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1284031CC88; Mon, 6 Jun 2022 10:57:02 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 6454F21B91; Mon, 6 Jun 2022 17:57:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1654538221; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5SDuL1yHsM9tqahZ4MTx0RfeBwJpM0mHbD0EbAJ/Kms=; b=Nht0RnweMQ17Ddsdp05K38fzLgjJRxF+Fq/RVzG24opijEiasCqBHlr993FHnh7iyyw3tt 0iw3nX5SHIRCEmEk04B88Svi+bCmmkSIPPfBUqWJzye9z+lTdskRDSPrOyetYJlyrpfIC0 Iub6gpSLV02quYm3xxxskde4gKNC8zQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1654538221; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5SDuL1yHsM9tqahZ4MTx0RfeBwJpM0mHbD0EbAJ/Kms=; b=Jlp7hjFIFuptWqYeviQaN61LA6aRCdjwxBDhuYbDchSCi5A2/NQRpiYVTs0E//9W3Ww6Qv Bz6L5vB5xmW3zMDQ== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 545582C141; Mon, 6 Jun 2022 17:57:01 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 28BE9A0639; Mon, 6 Jun 2022 19:56:55 +0200 (CEST) From: Jan Kara To: stable@vger.kernel.org Cc: , Paolo Valente , Jens Axboe , Jan Kara , "yukuai (C)" , Christoph Hellwig Subject: [PATCH 5/6] bfq: Make sure bfqg for which we are queueing requests is online Date: Mon, 6 Jun 2022 19:56:40 +0200 Message-Id: <20220606175655.8993-5-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220606174118.10992-1-jack@suse.cz> References: <20220606174118.10992-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1787; h=from:subject; bh=CGjHdnXjgD/kEx+rwRDpVFYMVyeiH0NpKSH89ZbtK5Q=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBinj/XYYVlTHQfy2/yW+znzcBe9OxDzVmM98FKwdFt aZWTb+KJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYp4/1wAKCRCcnaoHP2RA2axKCA DrZBTOP2vhcgiaz85nwXbcoBNDj6eEt3k2TIgHKj5HC0Fp3ij7xHBeVOT8EuQ5WQ9QtR0w1ayLcKt0 MbfHwo0C4ozdkpX+sbem19TmRtqYJGosrYpLlWEh8FBTS3r1VvpgCw6VWxfwQnenwaACY1nsX1HvrX ig0Nww1ObE+DGAohLPX9ytuksMS7MKKuRqsylUd+G7k62fc9OpSQ11PcvNIU/sjXsQKIwJbrdR+jda lNRhdN+VoOiFqFukun6LKSFCYrZGu/Dzwx7SxJqbjDopdLHc48yyIQIR5IBqQRa3XcFF9LWBEWsNAI pFHOlZzwHFVba3UKHEfvitup+ecaZO X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org commit 075a53b78b815301f8d3dd1ee2cd99554e34f0dd upstream. Bios queued into BFQ IO scheduler can be associated with a cgroup that was already offlined. This may then cause insertion of this bfq_group into a service tree. But this bfq_group will get freed as soon as last bio associated with it is completed leading to use after free issues for service tree users. Fix the problem by making sure we always operate on online bfq_group. If the bfq_group associated with the bio is not online, we pick the first online parent. CC: stable@vger.kernel.org Fixes: e21b7a0b9887 ("block, bfq: add full hierarchical scheduling and cgroups support") Tested-by: "yukuai (C)" Signed-off-by: Jan Kara Reviewed-by: Christoph Hellwig Link: https://lore.kernel.org/r/20220401102752.8599-9-jack@suse.cz Signed-off-by: Jens Axboe --- block/bfq-cgroup.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index 622c9a13e496..be6733558b83 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -608,10 +608,19 @@ static void bfq_link_bfqg(struct bfq_data *bfqd, struct bfq_group *bfqg) struct bfq_group *bfq_bio_bfqg(struct bfq_data *bfqd, struct bio *bio) { struct blkcg_gq *blkg = bio->bi_blkg; + struct bfq_group *bfqg; - if (!blkg) - return bfqd->root_group; - return blkg_to_bfqg(blkg); + while (blkg) { + bfqg = blkg_to_bfqg(blkg); + if (bfqg->online) { + bio_associate_blkg_from_css(bio, &blkg->blkcg->css); + return bfqg; + } + blkg = blkg->parent; + } + bio_associate_blkg_from_css(bio, + &bfqg_to_blkg(bfqd->root_group)->blkcg->css); + return bfqd->root_group; } /** From patchwork Mon Jun 6 17:56:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12870744 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52BECCCA487 for ; Mon, 6 Jun 2022 17:57:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229869AbiFFR5K (ORCPT ); Mon, 6 Jun 2022 13:57:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230253AbiFFR5F (ORCPT ); Mon, 6 Jun 2022 13:57:05 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 115CB2F9AFE; Mon, 6 Jun 2022 10:57:02 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 89C5621B9B; Mon, 6 Jun 2022 17:57:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1654538221; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WUB6r6nT4kAlF/JGpn/A9Zjm6xLLgzr4cLzF47XsUqo=; b=awusVAjcS5Dp6hyKRo+16lMrgtAILerEhpXzJ72fgec/yd38MwbiZjXsxnNaYi5zlE+zZT ifQ8d4t7P3On83LT5tmpqcTDc15wJmVDjrdZwjQ89/XzCdfREoiGC36kvMk7vB+e8uF1/0 fgQsvhWrQ3m8fgqHrPjnNQQaEuIJ2CE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1654538221; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WUB6r6nT4kAlF/JGpn/A9Zjm6xLLgzr4cLzF47XsUqo=; b=NQtLkYxU6om2SuS3q0QvJxLN7AeYikggev3/uFwjteIMnqkXN/lspw1WhKKdwtuP95l+87 fEtgApdX42o2piAw== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 7726B2C142; Mon, 6 Jun 2022 17:57:01 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 2FF4EA063A; Mon, 6 Jun 2022 19:56:55 +0200 (CEST) From: Jan Kara To: stable@vger.kernel.org Cc: , Paolo Valente , Jens Axboe , Jan Kara , Logan Gunthorpe , Donald Buczek , Christoph Hellwig Subject: [PATCH 6/6] block: fix bio_clone_blkg_association() to associate with proper blkcg_gq Date: Mon, 6 Jun 2022 19:56:41 +0200 Message-Id: <20220606175655.8993-6-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220606174118.10992-1-jack@suse.cz> References: <20220606174118.10992-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1860; h=from:subject; bh=irukgdQK/KNSx5BYqIWY1nIBQgWrGIob468B0DBPsT0=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBinj/YiIP/ZWe9jnGi4+zXKuPt708SNJFNSqBtJhZX 3TkPc0WJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYp4/2AAKCRCcnaoHP2RA2UTpCA CgJGmHJfmrXpFOY7IZPQYWH03BAUseGH9lXytArAtcAPpHHiHwmJ4ksEJgHPwqhclrS9bnysju6/S7 VA4bU9GUsL2LcQVB/xeBZP1sg/bFA5p5oKUGhMsqk4/76ERz7wunkfO2UcTpjkwU/KbjuSBI9hDIGe IlcC5wqlEQ80iby6gL5wVrNQX6lfie1fMmjoygEsw1aHImYWLYDon4p607uxWJ+TbjZbS6ZPfeEW7S qg/xB6A7Bh6ZnCWgMK15G7zJr4hMKsn1jEY9sAATPvJersvSj4BzI59szqSMuQlg5lCgoB/QR7i2i+ u2wa8VzsmUrvYd9EZ1E6wRcAZ6hySL X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org commit 22b106e5355d6e7a9c3b5cb5ed4ef22ae585ea94 upstream. Commit d92c370a16cb ("block: really clone the block cgroup in bio_clone_blkg_association") changed bio_clone_blkg_association() to just clone bio->bi_blkg reference from source to destination bio. This is however wrong if the source and destination bios are against different block devices because struct blkcg_gq is different for each bdev-blkcg pair. This will result in IOs being accounted (and throttled as a result) multiple times against the same device (src bdev) while throttling of the other device (dst bdev) is ignored. In case of BFQ the inconsistency can even result in crashes in bfq_bic_update_cgroup(). Fix the problem by looking up correct blkcg_gq for the cloned bio. Reported-by: Logan Gunthorpe Reported-and-tested-by: Donald Buczek Fixes: d92c370a16cb ("block: really clone the block cgroup in bio_clone_blkg_association") CC: stable@vger.kernel.org Reviewed-by: Christoph Hellwig Signed-off-by: Jan Kara Link: https://lore.kernel.org/r/20220602081242.7731-1-jack@suse.cz Signed-off-by: Jens Axboe --- block/blk-cgroup.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index 5b19665bc486..484c6b2dd264 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -1892,12 +1892,8 @@ EXPORT_SYMBOL_GPL(bio_associate_blkg); */ void bio_clone_blkg_association(struct bio *dst, struct bio *src) { - if (src->bi_blkg) { - if (dst->bi_blkg) - blkg_put(dst->bi_blkg); - blkg_get(src->bi_blkg); - dst->bi_blkg = src->bi_blkg; - } + if (src->bi_blkg) + bio_associate_blkg_from_css(dst, &bio_blkcg(src)->css); } EXPORT_SYMBOL_GPL(bio_clone_blkg_association);