From patchwork Fri Apr 1 10:27:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12798062 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 523FAC433FE for ; Fri, 1 Apr 2022 10:28:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241970AbiDAK3t (ORCPT ); Fri, 1 Apr 2022 06:29:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38650 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245580AbiDAK3p (ORCPT ); Fri, 1 Apr 2022 06:29:45 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5EB7126E55E; Fri, 1 Apr 2022 03:27:55 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id CA5851FD01; Fri, 1 Apr 2022 10:27:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648808873; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ve5MoBnU6j7qsA5vAC+9zKpXDeMocKoMXKYujoxLpg4=; b=v360xNS40sg9jA3maj5tikmtk3xjtfuP0oYU3VuaYFEhfEe2tiJG+QHF4vul4mqY2Vcqix 4pi/Vf0fyY4PIgmQjoUjM+i6jgsYSCrZvYauFCsORmTr2lGltSzVTzpAEig7+KcV50+5px f0CrexRRJnZk5sDJw7BrzUwYPhosopc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648808873; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ve5MoBnU6j7qsA5vAC+9zKpXDeMocKoMXKYujoxLpg4=; b=OFru2bu2h5h2L9rBD8NBRaoEEorH5IEUWdu1zy6aCZG7jl6h4s8W48lTxIrRuAqGaOTlg5 YDGEcNZfDBG2yUBg== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id A73A1A3B8A; Fri, 1 Apr 2022 10:27:53 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 7552FA059B; Fri, 1 Apr 2022 12:27:52 +0200 (CEST) From: Jan Kara To: Paolo Valente Cc: , Jens Axboe , "yukuai (C)" , Jan Kara , stable@vger.kernel.org Subject: [PATCH 1/9] bfq: Avoid false marking of bic as stably merged Date: Fri, 1 Apr 2022 12:27:42 +0200 Message-Id: <20220401102752.8599-1-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220401102325.17617-1-jack@suse.cz> References: <20220401102325.17617-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1165; h=from:subject; bh=frZzv2m+qEXWPCVllSYQTbrZO5Wf1OHveC5lfcQSEjY=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRtOe5tyP+Xu0lpBrb9Mxi9IKj7ipa0vOExN+jB1O XjGHoumJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkbTngAKCRCcnaoHP2RA2UQJB/ 4hR6H8kAvK2hWfiweJi69jhHS60vlDa8BWTkn7apkjz2bv0jVSUrQu20U7pJ5pYN3nSgajZkYIko4M S0m2apQca4tynHOS1Kl2Bdh+vY5JrdBRVvIoGIB35R+ZvDxpdDhZ0IDNbtFDW43fkinE24Qgga+5cQ o6SBLn1My3JZ8D2gy1oPHbb0CYvDOfiHhlW1X4n7WmXSShHAYw94V6Qkyg/3GEnDjmqky/e4qfSQ7h 0WXVsIdmZRKg/LKhpILBCqOZpz3ZU1As/gKIQ8sG/4Zk1NFepf8p84NVHfeRC9ORO5Xm8ZJnHu/cIp 0T8p3bQ41L+MEr9xzi62ji4aWcP3TT X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org bfq_setup_cooperator() can mark bic as stably merged even though it decides to not merge its bfqqs (when bfq_setup_merge() returns NULL). Make sure to mark bic as stably merged only if we are really going to merge bfqqs. CC: stable@vger.kernel.org Tested-by: "yukuai (C)" Fixes: 430a67f9d616 ("block, bfq: merge bursts of newly-created queues") Signed-off-by: Jan Kara --- block/bfq-iosched.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 2e0dd68a3cbe..6d122c28086e 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -2895,9 +2895,12 @@ bfq_setup_cooperator(struct bfq_data *bfqd, struct bfq_queue *bfqq, struct bfq_queue *new_bfqq = bfq_setup_merge(bfqq, stable_merge_bfqq); - bic->stably_merged = true; - if (new_bfqq && new_bfqq->bic) - new_bfqq->bic->stably_merged = true; + if (new_bfqq) { + bic->stably_merged = true; + if (new_bfqq->bic) + new_bfqq->bic->stably_merged = + true; + } return new_bfqq; } else return NULL; From patchwork Fri Apr 1 10:27:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12798063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81004C433F5 for ; Fri, 1 Apr 2022 10:28:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245580AbiDAK3t (ORCPT ); Fri, 1 Apr 2022 06:29:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38646 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245560AbiDAK3p (ORCPT ); Fri, 1 Apr 2022 06:29:45 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E1C526E54C; Fri, 1 Apr 2022 03:27:55 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id C53F01FCFF; Fri, 1 Apr 2022 10:27:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648808873; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gfxNY8pkD4wG9bo/byknp9EmBiK4BnZ+HIjHLRJmYLM=; b=OYDInG951v/Tn30EwmyxCwQRTkzdj4fVyrjoLBkVB76VU4DZF9pI5NaKlQGjkCxXX9/TFs 0+QAH5RD2O7D6UnJP6jetX0v3nsL+4PtFe8bsiRwBLw1xtBQ24SoJy0lFrDKQp01RIsMCj JieM1Fs3tqwGdZzLPXuWdqnGdOmYZO8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648808873; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gfxNY8pkD4wG9bo/byknp9EmBiK4BnZ+HIjHLRJmYLM=; b=TdKHCYWkmP88CgSXAG2i3wBJ6Utm5z+jXSFCdBu+uZyjK/aYeOExocq9argQt338vqi4Gt qPlAZVoOyuWEbmDg== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id A4262A3B89; Fri, 1 Apr 2022 10:27:53 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 7C797A0618; Fri, 1 Apr 2022 12:27:52 +0200 (CEST) From: Jan Kara To: Paolo Valente Cc: , Jens Axboe , "yukuai (C)" , Jan Kara , stable@vger.kernel.org Subject: [PATCH 2/9] bfq: Avoid merging queues with different parents Date: Fri, 1 Apr 2022 12:27:43 +0200 Message-Id: <20220401102752.8599-2-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220401102325.17617-1-jack@suse.cz> References: <20220401102325.17617-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2934; h=from:subject; bh=7tmFPjV8M2Lx3B/gsex9ygtAi4GAyO2CUR5hzaPWmrY=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRtOfsoOxGQACw0FePTNQ42iGCUudqsPksxx2yZ3J SOKN4lSJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkbTnwAKCRCcnaoHP2RA2aPMB/ 9LGTA72k9Xo0pW2lCEEkqTs2Y75Vyi3qFmfiiYCZtJOseGMXarUSESGp1AkQ63NRm9s8ztqfmA7A+a hjpnJgANwJ7jr00iR2kVUv7G1Qiyrmrxapbs6qrzYKCIrdPXc/NQBrq07zeY31kRY9l7mhRNYdglSP omJL7N7ASa0vofKfdWDFoP5cjLCOKIuvYfzs8GgiUIfFO050TUlI5L1BiRh0BiVKhd1t6/tW8LA2Qv qhbfiiYusxxmlDZtq6Mm6Fm9VSAtvmgmmsdjLE0OJ+ejc+dqFxIBLldkyLlGAsPmpjKcHg2mAAp+xT 7mvI2HARGmj5DI41cSvzHqQAqTbMda X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org It can happen that the parent of a bfqq changes between the moment we decide two queues are worth to merge (and set bic->stable_merge_bfqq) and the moment bfq_setup_merge() is called. This can happen e.g. because the process submitted IO for a different cgroup and thus bfqq got reparented. It can even happen that the bfqq we are merging with has parent cgroup that is already offline and going to be destroyed in which case the merge can lead to use-after-free issues such as: BUG: KASAN: use-after-free in __bfq_deactivate_entity+0x9cb/0xa50 Read of size 8 at addr ffff88800693c0c0 by task runc:[2:INIT]/10544 CPU: 0 PID: 10544 Comm: runc:[2:INIT] Tainted: G E 5.15.2-0.g5fb85fd-default #1 openSUSE Tumbleweed (unreleased) f1f3b891c72369aebecd2e43e4641a6358867c70 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a-rebuilt.opensuse.org 04/01/2014 Call Trace: dump_stack_lvl+0x46/0x5a print_address_description.constprop.0+0x1f/0x140 ? __bfq_deactivate_entity+0x9cb/0xa50 kasan_report.cold+0x7f/0x11b ? __bfq_deactivate_entity+0x9cb/0xa50 __bfq_deactivate_entity+0x9cb/0xa50 ? update_curr+0x32f/0x5d0 bfq_deactivate_entity+0xa0/0x1d0 bfq_del_bfqq_busy+0x28a/0x420 ? resched_curr+0x116/0x1d0 ? bfq_requeue_bfqq+0x70/0x70 ? check_preempt_wakeup+0x52b/0xbc0 __bfq_bfqq_expire+0x1a2/0x270 bfq_bfqq_expire+0xd16/0x2160 ? try_to_wake_up+0x4ee/0x1260 ? bfq_end_wr_async_queues+0xe0/0xe0 ? _raw_write_unlock_bh+0x60/0x60 ? _raw_spin_lock_irq+0x81/0xe0 bfq_idle_slice_timer+0x109/0x280 ? bfq_dispatch_request+0x4870/0x4870 __hrtimer_run_queues+0x37d/0x700 ? enqueue_hrtimer+0x1b0/0x1b0 ? kvm_clock_get_cycles+0xd/0x10 ? ktime_get_update_offsets_now+0x6f/0x280 hrtimer_interrupt+0x2c8/0x740 Fix the problem by checking that the parent of the two bfqqs we are merging in bfq_setup_merge() is the same. Link: https://lore.kernel.org/linux-block/20211125172809.GC19572@quack2.suse.cz/ CC: stable@vger.kernel.org Fixes: 430a67f9d616 ("block, bfq: merge bursts of newly-created queues") Tested-by: "yukuai (C)" Signed-off-by: Jan Kara --- block/bfq-iosched.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 6d122c28086e..7d00b21ebe5d 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -2758,6 +2758,14 @@ bfq_setup_merge(struct bfq_queue *bfqq, struct bfq_queue *new_bfqq) if (process_refs == 0 || new_process_refs == 0) return NULL; + /* + * Make sure merged queues belong to the same parent. Parents could + * have changed since the time we decided the two queues are suitable + * for merging. + */ + if (new_bfqq->entity.parent != bfqq->entity.parent) + return NULL; + bfq_log_bfqq(bfqq->bfqd, bfqq, "scheduling merge with queue %d", new_bfqq->pid); From patchwork Fri Apr 1 10:27:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12798064 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75E8CC43217 for ; Fri, 1 Apr 2022 10:28:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245729AbiDAK3u (ORCPT ); Fri, 1 Apr 2022 06:29:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38652 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245593AbiDAK3p (ORCPT ); Fri, 1 Apr 2022 06:29:45 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5EF9026E56A; Fri, 1 Apr 2022 03:27:55 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id CB8C721A98; Fri, 1 Apr 2022 10:27:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648808873; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UCkvPmMdlj5gKMX4b0BFS+C6otL8e89DKbtDixo1Hbo=; b=Ko2QF1yIadeBb+JAECXJwhRb74uOUZ0p3M8g2JeWtEwYskUjTGDfl02hHbxUj9cFKrPDfO JhXArnxkelbGo0iWhwt/CqB6p7B1Z2LP/7uUrDy8R2addh6lmXLceQOOKRU9atHVcswtMr oMBxXbjA+8hgVdqMnRrI1Jmq7oLjnIk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648808873; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UCkvPmMdlj5gKMX4b0BFS+C6otL8e89DKbtDixo1Hbo=; b=fT71xnm8VBy1iRrq1QHxp7Oh22p7+tYSlcGE3xHakjqhP95YA8by5tyvuqq5cehGJgID79 awKit219buOtQtBA== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id AA4CDA3B92; Fri, 1 Apr 2022 10:27:53 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 822F9A0619; Fri, 1 Apr 2022 12:27:52 +0200 (CEST) From: Jan Kara To: Paolo Valente Cc: , Jens Axboe , "yukuai (C)" , Jan Kara , stable@vger.kernel.org Subject: [PATCH 3/9] bfq: Split shared queues on move between cgroups Date: Fri, 1 Apr 2022 12:27:44 +0200 Message-Id: <20220401102752.8599-3-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220401102325.17617-1-jack@suse.cz> References: <20220401102325.17617-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3547; h=from:subject; bh=tITHR38wOCpdaxdNhhZ40lzu6OcKz//GDfn54YLu2Tw=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRtOgm2v7JAE7UPLNtQcBXxB32BvEhwmwHfVNZsST how2FiCJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkbToAAKCRCcnaoHP2RA2XFIB/ 9XCvOu1Q0vTruGS3UgE8EBNsZx/+KbGOd38hH4ugYihBvXAyrX4H0r8eeZVFgl2//Ix56ui617goxz h7/c7T5D9qTmW5SCtPS/9HD/M5NZlIlGBv2vN7jTnZKBijhW2sZ52Lv/LwLgJoWffpWEnI5FjAtUgz fagVNDuwIHJ1hHzibhEfYsD1mLe+UtJYpxHmsaPnTlBsU5J8jb0YSnkoamhrq/Mgu1/M8t2ZdquC20 CIjWJxdphFTAR4k00IGZbi5Jz6dSqu8bpejOR2ZYMS9GU6gCvxz1Lg5cux88zao2OWxxVxi05FIdxn n19+71pzuZf/48/RkxwiTGu0bxuMFI X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org When bfqq is shared by multiple processes it can happen that one of the processes gets moved to a different cgroup (or just starts submitting IO for different cgroup). In case that happens we need to split the merged bfqq as otherwise we will have IO for multiple cgroups in one bfqq and we will just account IO time to wrong entities etc. Similarly if the bfqq is scheduled to merge with another bfqq but the merge didn't happen yet, cancel the merge as it need not be valid anymore. CC: stable@vger.kernel.org Fixes: e21b7a0b9887 ("block, bfq: add full hierarchical scheduling and cgroups support") Tested-by: "yukuai (C)" Signed-off-by: Jan Kara --- block/bfq-cgroup.c | 36 +++++++++++++++++++++++++++++++++--- block/bfq-iosched.c | 2 +- block/bfq-iosched.h | 1 + 3 files changed, 35 insertions(+), 4 deletions(-) diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index 420eda2589c0..9352f3cc2377 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -743,9 +743,39 @@ static struct bfq_group *__bfq_bic_change_cgroup(struct bfq_data *bfqd, } if (sync_bfqq) { - entity = &sync_bfqq->entity; - if (entity->sched_data != &bfqg->sched_data) - bfq_bfqq_move(bfqd, sync_bfqq, bfqg); + if (!sync_bfqq->new_bfqq && !bfq_bfqq_coop(sync_bfqq)) { + /* We are the only user of this bfqq, just move it */ + if (sync_bfqq->entity.sched_data != &bfqg->sched_data) + bfq_bfqq_move(bfqd, sync_bfqq, bfqg); + } else { + struct bfq_queue *bfqq; + + /* + * The queue was merged to a different queue. Check + * that the merge chain still belongs to the same + * cgroup. + */ + for (bfqq = sync_bfqq; bfqq; bfqq = bfqq->new_bfqq) + if (bfqq->entity.sched_data != + &bfqg->sched_data) + break; + if (bfqq) { + /* + * Some queue changed cgroup so the merge is + * not valid anymore. We cannot easily just + * cancel the merge (by clearing new_bfqq) as + * there may be other processes using this + * queue and holding refs to all queues below + * sync_bfqq->new_bfqq. Similarly if the merge + * already happened, we need to detach from + * bfqq now so that we cannot merge bio to a + * request from the old cgroup. + */ + bfq_put_cooperator(sync_bfqq); + bfq_release_process_ref(bfqd, sync_bfqq); + bic_set_bfqq(bic, NULL, 1); + } + } } return bfqg; diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 7d00b21ebe5d..89fe3f85eb3c 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -5315,7 +5315,7 @@ static void bfq_put_stable_ref(struct bfq_queue *bfqq) bfq_put_queue(bfqq); } -static void bfq_put_cooperator(struct bfq_queue *bfqq) +void bfq_put_cooperator(struct bfq_queue *bfqq) { struct bfq_queue *__bfqq, *next; diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index 3b83e3d1c2e5..a56763045d19 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -979,6 +979,7 @@ void bfq_weights_tree_remove(struct bfq_data *bfqd, void bfq_bfqq_expire(struct bfq_data *bfqd, struct bfq_queue *bfqq, bool compensate, enum bfqq_expiration reason); void bfq_put_queue(struct bfq_queue *bfqq); +void bfq_put_cooperator(struct bfq_queue *bfqq); void bfq_end_wr_async_queues(struct bfq_data *bfqd, struct bfq_group *bfqg); void bfq_release_process_ref(struct bfq_data *bfqd, struct bfq_queue *bfqq); void bfq_schedule_dispatch(struct bfq_data *bfqd); From patchwork Fri Apr 1 10:27:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12798060 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59F89C433FE for ; Fri, 1 Apr 2022 10:28:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245540AbiDAK3p (ORCPT ); Fri, 1 Apr 2022 06:29:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38630 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241970AbiDAK3o (ORCPT ); Fri, 1 Apr 2022 06:29:44 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5EDB626E566; Fri, 1 Apr 2022 03:27:55 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id C61CA210FC; Fri, 1 Apr 2022 10:27:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648808873; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9kz4yzWuKqKHnpGX5HVClU1VX8JDHF41CaOQeK0bFmM=; b=vase2uVkwsTdL52H0RMuKpv3tNlREid2a/5wuH5ZsF4NY6DXwfXPvlw2kCcqAAuuAjNyPw Q6ATyXdyqbPVqewJZ3rgH/RuLLlQIqzs9mp6O/KjC23ilHT+W4VleJlP7ZhKvWWQt1bP+c 34uDXm4jYXG5tagz4zKhr11tuWz+kLY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648808873; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9kz4yzWuKqKHnpGX5HVClU1VX8JDHF41CaOQeK0bFmM=; b=uw9KWn9V+62jXZyHOqvdLFqKI6OfiQZSyeS1FUa3fbLOApKXvmRfDy/nKeaSJb44VWjmCK UFZlciojuI+gi4BA== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id A4107A3B88; Fri, 1 Apr 2022 10:27:53 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 875CCA061A; Fri, 1 Apr 2022 12:27:52 +0200 (CEST) From: Jan Kara To: Paolo Valente Cc: , Jens Axboe , "yukuai (C)" , Jan Kara , stable@vger.kernel.org Subject: [PATCH 4/9] bfq: Update cgroup information before merging bio Date: Fri, 1 Apr 2022 12:27:45 +0200 Message-Id: <20220401102752.8599-4-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220401102325.17617-1-jack@suse.cz> References: <20220401102325.17617-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1455; h=from:subject; bh=ZXtMvPdme+98TvEer4Fc63SIc5w+HQccSDUrivn0Ttw=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRtOhxr8HA9TU7R2le3a3GLod00df5u+vOKGbBIwA +UOFXbeJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkbToQAKCRCcnaoHP2RA2UA1B/ 4s/xVxYae2jfiby2vEr7yP16xSSAlPNPznE2wAYZdRP1HZsmwXMabpNPa+8rZMMQDNTHcnoULZ7EEh r3xB/PO/vsYCyGUzaB8sIBjaSxFFo3HwjZ+grmYicnY5q7oudywN7LTHlRvkYizRwg25jA2UB4Ascn U5kggrSnQVvPHrRsQQcrtPAGrij8xc0rl285Vf0PRJromv5ls2BtscqpNkF+DZc0KbchCLC/bvCfVb y6o/K1+P8b6fiyIEiLRrGYgyMa0vJWVsjDZSPWAENekVVd4/3XV9juuDmwr5E2ULFl9Q39hiwCbQhS mZL+KVGEXsLOG50KdQQHI2TfBoodDU X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org When the process is migrated to a different cgroup (or in case of writeback just starts submitting bios associated with a different cgroup) bfq_merge_bio() can operate with stale cgroup information in bic. Thus the bio can be merged to a request from a different cgroup or it can result in merging of bfqqs for different cgroups or bfqqs of already dead cgroups and causing possible use-after-free issues. Fix the problem by updating cgroup information in bfq_merge_bio(). CC: stable@vger.kernel.org Fixes: e21b7a0b9887 ("block, bfq: add full hierarchical scheduling and cgroups support") Tested-by: "yukuai (C)" Signed-off-by: Jan Kara --- block/bfq-iosched.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 89fe3f85eb3c..1fc4d4628fba 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -2457,10 +2457,17 @@ static bool bfq_bio_merge(struct request_queue *q, struct bio *bio, spin_lock_irq(&bfqd->lock); - if (bic) + if (bic) { + /* + * Make sure cgroup info is uptodate for current process before + * considering the merge. + */ + bfq_bic_update_cgroup(bic, bio); + bfqd->bio_bfqq = bic_to_bfqq(bic, op_is_sync(bio->bi_opf)); - else + } else { bfqd->bio_bfqq = NULL; + } bfqd->bio_bic = bic; ret = blk_mq_sched_try_merge(q, bio, nr_segs, &free); From patchwork Fri Apr 1 10:27:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12798067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF8E5C4321E for ; Fri, 1 Apr 2022 10:28:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245635AbiDAK3x (ORCPT ); Fri, 1 Apr 2022 06:29:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245615AbiDAK3r (ORCPT ); Fri, 1 Apr 2022 06:29:47 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2167526E570; Fri, 1 Apr 2022 03:27:56 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id A77A81FD04; Fri, 1 Apr 2022 10:27:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648808874; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qOuiYb1lKXflzrbNlr4ffbWrXf/9yvnmykox32bxZwU=; b=ep7ydXvlBedTPae2aigPOIZ7tu2Ft92V7kSszeE6Kxtl+yWuLZSVcdDszCkTAvMu5JlWCa Wj8NlKMS2h879TYNNkKnocd8Lse+pWD5ZLNgO8Gbos44ihumvwoILbW512U8C07eVuBXSV iOqdujo0kQhDgY1q5iIJWhQYgfkvJqE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648808874; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qOuiYb1lKXflzrbNlr4ffbWrXf/9yvnmykox32bxZwU=; b=UEZuT2o0A5mvF2zOFujCYL+I5oC+cH9jRcbxP5+q1yYIFElLVzcglFU5KMQ1lwu4DWilIU BzOr99OONa3PkaDA== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 82246A3B92; Fri, 1 Apr 2022 10:27:54 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 8D79BA061B; Fri, 1 Apr 2022 12:27:52 +0200 (CEST) From: Jan Kara To: Paolo Valente Cc: , Jens Axboe , "yukuai (C)" , Jan Kara , stable@vger.kernel.org Subject: [PATCH 5/9] bfq: Drop pointless unlock-lock pair Date: Fri, 1 Apr 2022 12:27:46 +0200 Message-Id: <20220401102752.8599-5-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220401102325.17617-1-jack@suse.cz> References: <20220401102325.17617-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=908; h=from:subject; bh=f6a//ztGhPlB2z2XpIrLMQ0/PyT+BqDb2fM6yWfWv9E=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRtOiyQ3U5Y+mlC8NHplioNGOhgYJ9xWmqnglRyDH +MzguKeJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkbTogAKCRCcnaoHP2RA2UdHCA CEUQLtCsQKDFr2Em7i2k2huuA1KKXygTe7pFnzk6qUmZy3lu0H5aQptIyMq3a5c4x8xSZexhnnT2Iw E+zDpozVXoi4z2YXee+zborRL9MCRRxx8wOnUI40Fhl+BJRKT6eURHiqc34h0MGnQ4NCVUUVo/4K7t Jmdq0VGTwqYM/9HkqYSNkZ4Jki3HXxj1mQt834quxDMdRKthlk6U4htltt1nm+b6tZ7hZPc0mRF7Ja sfTd20Hp21QKZny5HF4LXpG6+b9YWfvRlxaMRflyV8lmoD8l23pNX/jwR/i/FzfzTl3k+YY/XUjMMy oBG0VS9y5y4OEhDIaRFb0GwiKggNDb X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org In bfq_insert_request() we unlock bfqd->lock only to call trace_block_rq_insert() and then lock bfqd->lock again. This is really pointless since tracing is disabled if we really care about performance and even if the tracepoint is enabled, it is a quick call. CC: stable@vger.kernel.org Tested-by: "yukuai (C)" Signed-off-by: Jan Kara --- block/bfq-iosched.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 1fc4d4628fba..19082e14f3c1 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -6150,11 +6150,8 @@ static void bfq_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, return; } - spin_unlock_irq(&bfqd->lock); - trace_block_rq_insert(rq); - spin_lock_irq(&bfqd->lock); bfqq = bfq_init_rq(rq); if (!bfqq || at_head) { if (at_head) From patchwork Fri Apr 1 10:27:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12798068 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 850EFC43219 for ; Fri, 1 Apr 2022 10:28:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245615AbiDAK3z (ORCPT ); Fri, 1 Apr 2022 06:29:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38666 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245675AbiDAK3r (ORCPT ); Fri, 1 Apr 2022 06:29:47 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1148526E56F; Fri, 1 Apr 2022 03:27:56 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id A3CBA21A9B; Fri, 1 Apr 2022 10:27:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648808874; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZNsnxIKTavjIHNIjtiVoZn5e2Xzy2WQNZ6NrPs/wvPE=; b=U55FlFMkxxe4Ayw50E4HNh0B694mjLaRuw8kMH+XsQvn3j/EzzCIR/xm/ivp9rshDWS3q4 S9uiUhrvkjm8Ehb2Xuu0PFXK//WeiKzjxzKK1wkqoWSPBw3gAHgzOInp5Rg6TJtssu+PcR 6OX6Kn5++V04rdhCNhsmMLEvN9xl4nI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648808874; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZNsnxIKTavjIHNIjtiVoZn5e2Xzy2WQNZ6NrPs/wvPE=; b=VUP7OekIG1gFTWpWDAHnD/SsqDjv4nHQZ++k77EkNEeHgDhGfMO6O9ujnn8W+mDXJVdaOU VstVeiIseielcvAA== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 822F1A3B94; Fri, 1 Apr 2022 10:27:54 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 93C4AA061C; Fri, 1 Apr 2022 12:27:52 +0200 (CEST) From: Jan Kara To: Paolo Valente Cc: , Jens Axboe , "yukuai (C)" , Jan Kara , stable@vger.kernel.org Subject: [PATCH 6/9] bfq: Remove pointless bfq_init_rq() calls Date: Fri, 1 Apr 2022 12:27:47 +0200 Message-Id: <20220401102752.8599-6-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220401102325.17617-1-jack@suse.cz> References: <20220401102325.17617-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2865; h=from:subject; bh=FpjUl5jP9AHT8IEvZDKrq5z048n3tCBdPWcF/ubs6jc=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRtOjUBZ6tsjycaAfA+Aimb2kxDiN2ywujGvSawd3 SHt/IXiJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkbTowAKCRCcnaoHP2RA2dg0B/ 4022/+xJxqhHg5UXlbYZeQTAbhdLNy1n54JPxUFdmJhtgRIAMFvkJS52ru3ABHEY6ShTuvZN0IveXK MYp1OlF983t3PW8sAgIKFQQ41j2amMhUde+feQhhQjoOhwHzt5pmL2h+a/3U7PMTSyYstiES4Q9f4C qrv9gnZXbvw/3CbiNceE4baqOnjSQoTLQb9ubGaH7iqd4EnJS5h3xPvSz5n3q5i1qKH6jGk8o3KK1V k9m45DACuNGt1GlHF+2e7sRTr1iNQpG6098+ZO7+5/5Wrag+aRIL7d7WlggLJR+IkXS1qp1GEpfgzK 1uA+w4PTdoRYCl6ERRAXu4UHc/SYkQ X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org We call bfq_init_rq() from request merging functions where requests we get should have already gone through bfq_init_rq() during insert and anyway we want to do anything only if the request is already tracked by BFQ. So replace calls to bfq_init_rq() with RQ_BFQQ() instead to simply skip requests untracked by BFQ. We move bfq_init_rq() call in bfq_insert_request() a bit earlier to cover request merging and thus can transfer FIFO position in case of a merge. CC: stable@vger.kernel.org Tested-by: "yukuai (C)" Signed-off-by: Jan Kara --- block/bfq-iosched.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 19082e14f3c1..d7cf930b47bb 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -2497,8 +2497,6 @@ static int bfq_request_merge(struct request_queue *q, struct request **req, return ELEVATOR_NO_MERGE; } -static struct bfq_queue *bfq_init_rq(struct request *rq); - static void bfq_request_merged(struct request_queue *q, struct request *req, enum elv_merge type) { @@ -2507,7 +2505,7 @@ static void bfq_request_merged(struct request_queue *q, struct request *req, blk_rq_pos(req) < blk_rq_pos(container_of(rb_prev(&req->rb_node), struct request, rb_node))) { - struct bfq_queue *bfqq = bfq_init_rq(req); + struct bfq_queue *bfqq = RQ_BFQQ(req); struct bfq_data *bfqd; struct request *prev, *next_rq; @@ -2559,8 +2557,8 @@ static void bfq_request_merged(struct request_queue *q, struct request *req, static void bfq_requests_merged(struct request_queue *q, struct request *rq, struct request *next) { - struct bfq_queue *bfqq = bfq_init_rq(rq), - *next_bfqq = bfq_init_rq(next); + struct bfq_queue *bfqq = RQ_BFQQ(rq), + *next_bfqq = RQ_BFQQ(next); if (!bfqq) goto remove; @@ -6129,6 +6127,8 @@ static inline void bfq_update_insert_stats(struct request_queue *q, unsigned int cmd_flags) {} #endif /* CONFIG_BFQ_CGROUP_DEBUG */ +static struct bfq_queue *bfq_init_rq(struct request *rq); + static void bfq_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, bool at_head) { @@ -6144,6 +6144,7 @@ static void bfq_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, bfqg_stats_update_legacy_io(q, rq); #endif spin_lock_irq(&bfqd->lock); + bfqq = bfq_init_rq(rq); if (blk_mq_sched_try_insert_merge(q, rq, &free)) { spin_unlock_irq(&bfqd->lock); blk_mq_free_requests(&free); @@ -6152,7 +6153,6 @@ static void bfq_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, trace_block_rq_insert(rq); - bfqq = bfq_init_rq(rq); if (!bfqq || at_head) { if (at_head) list_add(&rq->queuelist, &bfqd->dispatch); From patchwork Fri Apr 1 10:27:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12798061 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFF68C4332F for ; Fri, 1 Apr 2022 10:28:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245689AbiDAK3s (ORCPT ); Fri, 1 Apr 2022 06:29:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38654 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241970AbiDAK3p (ORCPT ); Fri, 1 Apr 2022 06:29:45 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E176526E56B; Fri, 1 Apr 2022 03:27:55 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 9501C21A9A; Fri, 1 Apr 2022 10:27:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648808874; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YFpkNPF3uJRlZQSWVGOFjXkVj/UMJKrQUqJzQPtsgKw=; b=l+q74p/8d7Bia3xg3pZeot8GChd6NY7lIGHIiIUNRMGNv+tQwk++e0EsIyYRsMt2OPPaHi YeWHCTs42xZRJTJDNFXKa7w7LuWKJYM/ABU71r2i/1EITKAPQWpQiycrJg3G5L2crATXIi Q/LmmcYeYzQG7M4lQXYdIEViHX0c384= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648808874; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YFpkNPF3uJRlZQSWVGOFjXkVj/UMJKrQUqJzQPtsgKw=; b=y0gzdMO1dnTh8nqlLYdGqwMRK2aaMjSYXJzfGLo4fNpyFKwRwkYS0vZNwwdCrpVsYyuZUm HHF2Gz5JcsNc+6Aw== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 7BD0AA3B83; Fri, 1 Apr 2022 10:27:54 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 99E5AA061D; Fri, 1 Apr 2022 12:27:52 +0200 (CEST) From: Jan Kara To: Paolo Valente Cc: , Jens Axboe , "yukuai (C)" , Jan Kara , stable@vger.kernel.org Subject: [PATCH 7/9] bfq: Track whether bfq_group is still online Date: Fri, 1 Apr 2022 12:27:48 +0200 Message-Id: <20220401102752.8599-7-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220401102325.17617-1-jack@suse.cz> References: <20220401102325.17617-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1771; h=from:subject; bh=cGpLXMZhgvdpvOg/vhc4Y6xKtjjgG1ZtO8E3+AAlxnQ=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRtOkm5+xlSK7pmV2/cdmILjXlk+jzRsKtERV+D1k ZI9PNWaJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkbTpAAKCRCcnaoHP2RA2RJeCA DnT+9KPHzYEdE+yW007cB8jin8dzLU6NATqIGWtk0sCkzkJJfhDV/JK0kytrSRSrSpjsWTCDbM3hV8 Uqn27kCIhEEqRggXLHk7TXbwhuIUXRSUT1ZT6Q9RgLIZyeWBWbx8EqH76bUqIhdBHh2raPud8mgRjG SCc7yT3VBGzpA1NysS5VUmMP5yI1a7RGh4Amvypzh1Xv2MBYuBtrnqNZDXSqsKw7XFUOzimOzqd4Rc 1HOxLiAx5y3mR/SRWPvaUfn28qt0C/2ksF3EuoaysXL27CIM7xGEoUgd66nwZkKLvHfgT6rE+Z09GO HSnTJ8CO6U3XmJfx1rDVdfeMQJuKUb X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Track whether bfq_group is still online. We cannot rely on blkcg_gq->online because that gets cleared only after all policies are offlined and we need something that gets updated already under bfqd->lock when we are cleaning up our bfq_group to be able to guarantee that when we see online bfq_group, it will stay online while we are holding bfqd->lock lock. CC: stable@vger.kernel.org Tested-by: "yukuai (C)" Signed-off-by: Jan Kara --- block/bfq-cgroup.c | 3 ++- block/bfq-iosched.h | 2 ++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index 9352f3cc2377..879380c2bc7e 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -557,6 +557,7 @@ static void bfq_pd_init(struct blkg_policy_data *pd) */ bfqg->bfqd = bfqd; bfqg->active_entities = 0; + bfqg->online = true; bfqg->rq_pos_tree = RB_ROOT; } @@ -603,7 +604,6 @@ struct bfq_group *bfq_find_set_group(struct bfq_data *bfqd, struct bfq_entity *entity; bfqg = bfq_lookup_bfqg(bfqd, blkcg); - if (unlikely(!bfqg)) return NULL; @@ -979,6 +979,7 @@ static void bfq_pd_offline(struct blkg_policy_data *pd) put_async_queues: bfq_put_async_queues(bfqd, bfqg); + bfqg->online = false; spin_unlock_irqrestore(&bfqd->lock, flags); /* diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index a56763045d19..4664e2f3e828 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -928,6 +928,8 @@ struct bfq_group { /* reference counter (see comments in bfq_bic_update_cgroup) */ int ref; + /* Is bfq_group still online? */ + bool online; struct bfq_entity entity; struct bfq_sched_data sched_data; From patchwork Fri Apr 1 10:27:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12798069 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82166C433FE for ; Fri, 1 Apr 2022 10:28:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343517AbiDAK34 (ORCPT ); Fri, 1 Apr 2022 06:29:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245676AbiDAK3r (ORCPT ); Fri, 1 Apr 2022 06:29:47 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A23426E57B; Fri, 1 Apr 2022 03:27:56 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id BC6811FD05; Fri, 1 Apr 2022 10:27:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648808874; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p8HInWOn0JhCGNwoQJmTzRAGa2b1vZ1mm3a6v3TxOEk=; b=NHawarvg96YwPJfe8jGHAS35dQ6mnS1pIvyMP3ws8ulU8byrNgeqcz4irLxZXEuGcGRHbk gV670bS6uiOFTgUL88hZpe4+GfD2dSdqT57sVIuhrB9AidgJTyEn754T4CvRYrPsNUs2TN XAH/132sDS/rRNdxzXkWRwAGdLBgei4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648808874; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p8HInWOn0JhCGNwoQJmTzRAGa2b1vZ1mm3a6v3TxOEk=; b=ZQouYWe8OkYwRJ/NLDM5n15237uB4542jZO3foQVEvnhiNRvZoR5ZS6R8LgrksJSr9MQwj EBfX193++MOc75DA== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 84B74A3B95; Fri, 1 Apr 2022 10:27:54 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id A09E1A061E; Fri, 1 Apr 2022 12:27:52 +0200 (CEST) From: Jan Kara To: Paolo Valente Cc: , Jens Axboe , "yukuai (C)" , Jan Kara , stable@vger.kernel.org Subject: [PATCH 8/9] bfq: Get rid of __bio_blkcg() usage Date: Fri, 1 Apr 2022 12:27:49 +0200 Message-Id: <20220401102752.8599-8-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220401102325.17617-1-jack@suse.cz> References: <20220401102325.17617-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6636; h=from:subject; bh=hmCQcGPDWk91gsH0Ff6dqf205Vr5YwBuVPp7vIJKXHA=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRtOlRcq5SUVFcwdhJubzpp07pilAw1qLVVFvCC3T +GEjsv6JATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkbTpQAKCRCcnaoHP2RA2Uf6B/ 9NAj5dovT41inrRQ8Y2MKon+z3fJhzZEmJbYlQ8jJ0P4zuZo8FKMyQyz3XkdrJm9/9xKpLKlU/BJKz psMMCmdjOQbKHYY0xj00TkVHmZz+HTAbqQ/6V+ismZsh2I8rX7uvHk+Js03A7+xkMh33gr4jpLwwrb zSjYHKsfXfTb45CecRxO+iTjdbn6TY2BOmmbA7B+TCqOe2wqf0OvD2rr55cOYNpTt2C3QWjJqGnEJl XOfCn0D9EdG55dzTIFxY/w/6phN+YudB7XgiDH8mrKOW628kkYmHRLzpn0VZaD0AwsMqryqAoH0pf0 JFejz0ZV9632cZ50HZ+V6LtyB55rOa X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org BFQ usage of __bio_blkcg() is a relict from the past. Furthermore if bio would not be associated with any blkcg, the usage of __bio_blkcg() in BFQ is prone to races with the task being migrated between cgroups as __bio_blkcg() calls at different places could return different blkcgs. Convert BFQ to the new situation where bio->bi_blkg is initialized in bio_set_dev() and thus practically always valid. This allows us to save blkcg_gq lookup and noticeably simplify the code. CC: stable@vger.kernel.org Fixes: 0fe061b9f03c ("blkcg: fix ref count issue with bio_blkcg() using task_css") Tested-by: "yukuai (C)" Signed-off-by: Jan Kara --- block/bfq-cgroup.c | 63 +++++++++++++++++---------------------------- block/bfq-iosched.c | 11 +------- block/bfq-iosched.h | 3 +-- 3 files changed, 25 insertions(+), 52 deletions(-) diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index 879380c2bc7e..32d2c2a47480 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -586,27 +586,11 @@ static void bfq_group_set_parent(struct bfq_group *bfqg, entity->sched_data = &parent->sched_data; } -static struct bfq_group *bfq_lookup_bfqg(struct bfq_data *bfqd, - struct blkcg *blkcg) +static void bfq_link_bfqg(struct bfq_data *bfqd, struct bfq_group *bfqg) { - struct blkcg_gq *blkg; - - blkg = blkg_lookup(blkcg, bfqd->queue); - if (likely(blkg)) - return blkg_to_bfqg(blkg); - return NULL; -} - -struct bfq_group *bfq_find_set_group(struct bfq_data *bfqd, - struct blkcg *blkcg) -{ - struct bfq_group *bfqg, *parent; + struct bfq_group *parent; struct bfq_entity *entity; - bfqg = bfq_lookup_bfqg(bfqd, blkcg); - if (unlikely(!bfqg)) - return NULL; - /* * Update chain of bfq_groups as we might be handling a leaf group * which, along with some of its relatives, has not been hooked yet @@ -623,8 +607,15 @@ struct bfq_group *bfq_find_set_group(struct bfq_data *bfqd, bfq_group_set_parent(curr_bfqg, parent); } } +} - return bfqg; +struct bfq_group *bfq_bio_bfqg(struct bfq_data *bfqd, struct bio *bio) +{ + struct blkcg_gq *blkg = bio->bi_blkg; + + if (!blkg) + return bfqd->root_group; + return blkg_to_bfqg(blkg); } /** @@ -714,25 +705,15 @@ void bfq_bfqq_move(struct bfq_data *bfqd, struct bfq_queue *bfqq, * Move bic to blkcg, assuming that bfqd->lock is held; which makes * sure that the reference to cgroup is valid across the call (see * comments in bfq_bic_update_cgroup on this issue) - * - * NOTE: an alternative approach might have been to store the current - * cgroup in bfqq and getting a reference to it, reducing the lookup - * time here, at the price of slightly more complex code. */ -static struct bfq_group *__bfq_bic_change_cgroup(struct bfq_data *bfqd, - struct bfq_io_cq *bic, - struct blkcg *blkcg) +static void *__bfq_bic_change_cgroup(struct bfq_data *bfqd, + struct bfq_io_cq *bic, + struct bfq_group *bfqg) { struct bfq_queue *async_bfqq = bic_to_bfqq(bic, 0); struct bfq_queue *sync_bfqq = bic_to_bfqq(bic, 1); - struct bfq_group *bfqg; struct bfq_entity *entity; - bfqg = bfq_find_set_group(bfqd, blkcg); - - if (unlikely(!bfqg)) - bfqg = bfqd->root_group; - if (async_bfqq) { entity = &async_bfqq->entity; @@ -784,20 +765,24 @@ static struct bfq_group *__bfq_bic_change_cgroup(struct bfq_data *bfqd, void bfq_bic_update_cgroup(struct bfq_io_cq *bic, struct bio *bio) { struct bfq_data *bfqd = bic_to_bfqd(bic); - struct bfq_group *bfqg = NULL; + struct bfq_group *bfqg = bfq_bio_bfqg(bfqd, bio); uint64_t serial_nr; - rcu_read_lock(); - serial_nr = __bio_blkcg(bio)->css.serial_nr; + serial_nr = bfqg_to_blkg(bfqg)->blkcg->css.serial_nr; /* * Check whether blkcg has changed. The condition may trigger * spuriously on a newly created cic but there's no harm. */ if (unlikely(!bfqd) || likely(bic->blkcg_serial_nr == serial_nr)) - goto out; + return; - bfqg = __bfq_bic_change_cgroup(bfqd, bic, __bio_blkcg(bio)); + /* + * New cgroup for this process. Make sure it is linked to bfq internal + * cgroup hierarchy. + */ + bfq_link_bfqg(bfqd, bfqg); + __bfq_bic_change_cgroup(bfqd, bic, bfqg); /* * Update blkg_path for bfq_log_* functions. We cache this * path, and update it here, for the following @@ -850,8 +835,6 @@ void bfq_bic_update_cgroup(struct bfq_io_cq *bic, struct bio *bio) */ blkg_path(bfqg_to_blkg(bfqg), bfqg->blkg_path, sizeof(bfqg->blkg_path)); bic->blkcg_serial_nr = serial_nr; -out: - rcu_read_unlock(); } /** @@ -1469,7 +1452,7 @@ void bfq_end_wr_async(struct bfq_data *bfqd) bfq_end_wr_async_queues(bfqd, bfqd->root_group); } -struct bfq_group *bfq_find_set_group(struct bfq_data *bfqd, struct blkcg *blkcg) +struct bfq_group *bfq_bio_bfqg(struct bfq_data *bfqd, struct bio *bio) { return bfqd->root_group; } diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index d7cf930b47bb..e47c75f1fa0f 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -5726,14 +5726,7 @@ static struct bfq_queue *bfq_get_queue(struct bfq_data *bfqd, struct bfq_queue *bfqq; struct bfq_group *bfqg; - rcu_read_lock(); - - bfqg = bfq_find_set_group(bfqd, __bio_blkcg(bio)); - if (!bfqg) { - bfqq = &bfqd->oom_bfqq; - goto out; - } - + bfqg = bfq_bio_bfqg(bfqd, bio); if (!is_sync) { async_bfqq = bfq_async_queue_prio(bfqd, bfqg, ioprio_class, ioprio); @@ -5779,8 +5772,6 @@ static struct bfq_queue *bfq_get_queue(struct bfq_data *bfqd, if (bfqq != &bfqd->oom_bfqq && is_sync && !respawn) bfqq = bfq_do_or_sched_stable_merge(bfqd, bfqq, bic); - - rcu_read_unlock(); return bfqq; } diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index 4664e2f3e828..978ef5d6fe6a 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -1009,8 +1009,7 @@ void bfq_bfqq_move(struct bfq_data *bfqd, struct bfq_queue *bfqq, void bfq_init_entity(struct bfq_entity *entity, struct bfq_group *bfqg); void bfq_bic_update_cgroup(struct bfq_io_cq *bic, struct bio *bio); void bfq_end_wr_async(struct bfq_data *bfqd); -struct bfq_group *bfq_find_set_group(struct bfq_data *bfqd, - struct blkcg *blkcg); +struct bfq_group *bfq_bio_bfqg(struct bfq_data *bfqd, struct bio *bio); struct blkcg_gq *bfqg_to_blkg(struct bfq_group *bfqg); struct bfq_group *bfqq_group(struct bfq_queue *bfqq); struct bfq_group *bfq_create_group_hierarchy(struct bfq_data *bfqd, int node); From patchwork Fri Apr 1 10:27:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12798066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B716FC433EF for ; Fri, 1 Apr 2022 10:28:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245687AbiDAK3w (ORCPT ); Fri, 1 Apr 2022 06:29:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38664 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245635AbiDAK3r (ORCPT ); Fri, 1 Apr 2022 06:29:47 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED7EF26E56E; Fri, 1 Apr 2022 03:27:55 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id A15131FD03; Fri, 1 Apr 2022 10:27:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648808874; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tA0rKXsxf77nc7Y9wimslw6o3kvPAaCI532S6Fi8Ku8=; b=JTpL/yQ35Uo0cdyHthSMpwo+YYr9z9gbqEVfbnlwtm21i1pPikC5BwnHiJFfwEH4wMQOa/ d4smprMTtGFMDt99t6UwNOX+824oE5uZdUy/814GNlYDYcRh2AZYCbA9OIqJHKBsVMQxHe BNA6ivLPACoKsUlieD372btioLxvB0E= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648808874; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tA0rKXsxf77nc7Y9wimslw6o3kvPAaCI532S6Fi8Ku8=; b=JweEsUcElcwq2AE3MqnVDYuqWGUXa7Ic1doIZQzk+ycT4MEhnss9WChRssxOgY3o6sF41o e28Eb49SIPkFKtDg== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 821DFA3B88; Fri, 1 Apr 2022 10:27:54 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id A709CA061F; Fri, 1 Apr 2022 12:27:52 +0200 (CEST) From: Jan Kara To: Paolo Valente Cc: , Jens Axboe , "yukuai (C)" , Jan Kara , stable@vger.kernel.org Subject: [PATCH 9/9] bfq: Make sure bfqg for which we are queueing requests is online Date: Fri, 1 Apr 2022 12:27:50 +0200 Message-Id: <20220401102752.8599-9-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220401102325.17617-1-jack@suse.cz> References: <20220401102325.17617-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1568; h=from:subject; bh=sgAPcAQrJvyNTY55SSAru4QatmfaUnmjg+zxK4TmvNM=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRtOlKklLhNNV3RGbf7D1KUJYUgBsxgg8XSM1TWU0 HAuk0saJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkbTpQAKCRCcnaoHP2RA2WUuB/ 45d/A4FWTboIVYdZ+QTKDERa161jYw4o75MeQV0qFUxjSiBn1+H1MSA5bPrLHZUGJhfAsD+RMefQIg oxOjs4FUnRMk2utm8NFp39DvIwTryxvAhI85kRUFLpXS5TOQrk6j6WQAeybv4ucYN/opBfH7ikQBEe iDW+Okuhm4+wzbxoEBOitpzukCfF6eK3aky2OeiHz78HU672Df1Gx22XTmXF3mstLzoFP35yonrra6 HZMnZIZCAvtjxOsozE5Dv0E5WnuPq4q0M9r95UaHad4mUF9kltwKmYuluyi9CvyS6QjKruRzVf3l1Z kYLXABUvmFhNAIiYt4qTATjJmfQsi6 X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Bios queued into BFQ IO scheduler can be associated with a cgroup that was already offlined. This may then cause insertion of this bfq_group into a service tree. But this bfq_group will get freed as soon as last bio associated with it is completed leading to use after free issues for service tree users. Fix the problem by making sure we always operate on online bfq_group. If the bfq_group associated with the bio is not online, we pick the first online parent. CC: stable@vger.kernel.org Fixes: e21b7a0b9887 ("block, bfq: add full hierarchical scheduling and cgroups support") Tested-by: "yukuai (C)" Signed-off-by: Jan Kara --- block/bfq-cgroup.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index 32d2c2a47480..09574af83566 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -612,10 +612,19 @@ static void bfq_link_bfqg(struct bfq_data *bfqd, struct bfq_group *bfqg) struct bfq_group *bfq_bio_bfqg(struct bfq_data *bfqd, struct bio *bio) { struct blkcg_gq *blkg = bio->bi_blkg; + struct bfq_group *bfqg; - if (!blkg) - return bfqd->root_group; - return blkg_to_bfqg(blkg); + while (blkg) { + bfqg = blkg_to_bfqg(blkg); + if (bfqg->online) { + bio_associate_blkg_from_css(bio, &blkg->blkcg->css); + return bfqg; + } + blkg = blkg->parent; + } + bio_associate_blkg_from_css(bio, + &bfqg_to_blkg(bfqd->root_group)->blkcg->css); + return bfqd->root_group; } /**