From patchwork Wed Mar 30 12:42:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12795777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D1D9C433EF for ; Wed, 30 Mar 2022 12:45:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345534AbiC3MrE (ORCPT ); Wed, 30 Mar 2022 08:47:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34384 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345327AbiC3Mqk (ORCPT ); Wed, 30 Mar 2022 08:46:40 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97E347DE17; Wed, 30 Mar 2022 05:43:02 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 2E0AC1F38C; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vLbXYHxIS9+MRxNXnLg56MB0MlJ/8dPXaZ7nAX5+HvE=; b=nSRGpGh1Os+Uv34BDMGUSUMsLXwq1Pw83aaeCbaEljTIqam6Na/4GzPRMsLoeeR+1v9M1L Qcf4ncfsayrcRoymZafGv6xOEE6LXbQnO04K3zZIFXEVALQDnsJ4sXxQGuzIuGz2M2xBEG i46MiTQF3TIpcTNGr7OV23sd9p4X3QM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vLbXYHxIS9+MRxNXnLg56MB0MlJ/8dPXaZ7nAX5+HvE=; b=oIMjBwwUpMJ8Zv5bH2C0iC7zL6t72WRs7o/fyyiTtgBwah44F2mcA7gINHIm0Lc70vX6yI emK3fAp5fbbI00AQ== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 1EA50A3B96; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 678C7A0615; Wed, 30 Mar 2022 14:42:56 +0200 (CEST) From: Jan Kara To: Cc: Paolo Valente , Jens Axboe , "yukuai (C)" , Jan Kara , stable@vger.kernel.org Subject: [PATCH 1/9] bfq: Avoid false marking of bic as stably merged Date: Wed, 30 Mar 2022 14:42:44 +0200 Message-Id: <20220330124255.24581-1-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220330123438.32719-1-jack@suse.cz> References: <20220330123438.32719-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1119; h=from:subject; bh=y0+aE+PP4MGA/7TmenAivQcOtrdcGD/YLmt5dn84/Js=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRFBEyII0Q8DmSosqFkx2FseuyFwd22h1KoedIzeV 4ygRUqeJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkRQRAAKCRCcnaoHP2RA2ZLuB/ 9/QQIouvPceN+2Ild6FwsBmpqRWWRfqZzTqWf8oJ5KFzDIMTewhGgS/gOv6MF/9tAWcQftGQl+Z9lV frfC9y6NZAocBTs8CpFS28y0O39kJDSuo0qwy9Ohycfzo4MkNu8Aoz1j6Ksz8FQdQgko2cgk8ybeMf Rgx9mQbpFyKHxWQEgezllQ7kpJUded8PkHXJoJ34OR9HuwvmAu4yv6UKEKNulUIWTvgw2ubeqCvpKu 827U0ZeVZoPUW8lWnJnI6ZGFnJ0lL/EM6FbAYL0OZWViiYTwUpeN3BVVWOyIQfMVxLCuqwZR2wJE/k B59ZWSn4u8J+xyM3QHHcGn4Ixz6D5g X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org bfq_setup_cooperator() can mark bic as stably merged even though it decides to not merge its bfqqs (when bfq_setup_merge() returns NULL). Make sure to mark bic as stably merged only if we are really going to merge bfqqs. CC: stable@vger.kernel.org Fixes: 430a67f9d616 ("block, bfq: merge bursts of newly-created queues") Signed-off-by: Jan Kara --- block/bfq-iosched.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 2e0dd68a3cbe..6d122c28086e 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -2895,9 +2895,12 @@ bfq_setup_cooperator(struct bfq_data *bfqd, struct bfq_queue *bfqq, struct bfq_queue *new_bfqq = bfq_setup_merge(bfqq, stable_merge_bfqq); - bic->stably_merged = true; - if (new_bfqq && new_bfqq->bic) - new_bfqq->bic->stably_merged = true; + if (new_bfqq) { + bic->stably_merged = true; + if (new_bfqq->bic) + new_bfqq->bic->stably_merged = + true; + } return new_bfqq; } else return NULL; From patchwork Wed Mar 30 12:42:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12795785 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 047B8C433EF for ; Wed, 30 Mar 2022 12:47:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241881AbiC3MtN (ORCPT ); Wed, 30 Mar 2022 08:49:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34400 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345436AbiC3Mqy (ORCPT ); Wed, 30 Mar 2022 08:46:54 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 980B77DE18; Wed, 30 Mar 2022 05:43:02 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 32D311F7AD; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t33HL+ZUEFlbU34WO0KCc7WMDxCOogLnEO2lgYt7GyI=; b=tPra7vJ/PLkZjLoyJyJtYoj60GHLORHRUm26M/lmRqA7bZunT1JZDwpKQfeAz+pBHyRfwM TysElPdleLJn+lH0oQcRnEpnasA4mH5PxFj/ecNnbdu86K3OGzqFeeXAmGoFe2ZABANdvq CqQO/U3cQBP5FYGShhkrHj7GpWKUtME= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t33HL+ZUEFlbU34WO0KCc7WMDxCOogLnEO2lgYt7GyI=; b=nOd1pPT6XfPDMMyWbG1wD1lHIGcuMJEOipfq0GGkICTFCaMTzS2NEw1ZgHHTzRxNGxHpTe CLOqcoEqNYhf/hAw== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 19C7FA3B87; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 6D76CA0618; Wed, 30 Mar 2022 14:42:56 +0200 (CEST) From: Jan Kara To: Cc: Paolo Valente , Jens Axboe , "yukuai (C)" , Jan Kara , stable@vger.kernel.org Subject: [PATCH 2/9] bfq: Avoid merging queues with different parents Date: Wed, 30 Mar 2022 14:42:45 +0200 Message-Id: <20220330124255.24581-2-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220330123438.32719-1-jack@suse.cz> References: <20220330123438.32719-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2888; h=from:subject; bh=YqBlQaGG2lClN376Ajy30o6ZjdZ8iHsmFLLyoJCVJMo=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRFBFfmNBeD3h5JnQFvzsn+bZ+enOfvA9fnQFJLLh 6qs1mlWJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkRQRQAKCRCcnaoHP2RA2SNqCA DLTF+x5XZWnpGNEd85aBZ7N00LIP8HRX74AP2CYrngUXsmkSErYh/j8UOjn9V0jivuHPQDSyt7/pvg 1JxCle30Fy0AGnFU28aadcquqQ8ZiEUoKN/uBLuVvsV9WDM6toKq83EEWr7cc7BVGPxb757Z7j4JVb DK2T1WktV+wMN46dg9g7f+7u+oipSegsHp3FzGErkc11KgXB4wRNopS2gzs7i4HGmKdLsk8EOat0qv FBBRHjA1SuUGeV2XcAPXI5i/fy7NiS/+FDB1UjIneZ908+cAetJKk2Nf7n9oKsh66f+YOyHybrbRBg lV+6MCfLXQ4ariBmU7NfJqaBSo0Kxx X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org It can happen that the parent of a bfqq changes between the moment we decide two queues are worth to merge (and set bic->stable_merge_bfqq) and the moment bfq_setup_merge() is called. This can happen e.g. because the process submitted IO for a different cgroup and thus bfqq got reparented. It can even happen that the bfqq we are merging with has parent cgroup that is already offline and going to be destroyed in which case the merge can lead to use-after-free issues such as: BUG: KASAN: use-after-free in __bfq_deactivate_entity+0x9cb/0xa50 Read of size 8 at addr ffff88800693c0c0 by task runc:[2:INIT]/10544 CPU: 0 PID: 10544 Comm: runc:[2:INIT] Tainted: G E 5.15.2-0.g5fb85fd-default #1 openSUSE Tumbleweed (unreleased) f1f3b891c72369aebecd2e43e4641a6358867c70 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a-rebuilt.opensuse.org 04/01/2014 Call Trace: dump_stack_lvl+0x46/0x5a print_address_description.constprop.0+0x1f/0x140 ? __bfq_deactivate_entity+0x9cb/0xa50 kasan_report.cold+0x7f/0x11b ? __bfq_deactivate_entity+0x9cb/0xa50 __bfq_deactivate_entity+0x9cb/0xa50 ? update_curr+0x32f/0x5d0 bfq_deactivate_entity+0xa0/0x1d0 bfq_del_bfqq_busy+0x28a/0x420 ? resched_curr+0x116/0x1d0 ? bfq_requeue_bfqq+0x70/0x70 ? check_preempt_wakeup+0x52b/0xbc0 __bfq_bfqq_expire+0x1a2/0x270 bfq_bfqq_expire+0xd16/0x2160 ? try_to_wake_up+0x4ee/0x1260 ? bfq_end_wr_async_queues+0xe0/0xe0 ? _raw_write_unlock_bh+0x60/0x60 ? _raw_spin_lock_irq+0x81/0xe0 bfq_idle_slice_timer+0x109/0x280 ? bfq_dispatch_request+0x4870/0x4870 __hrtimer_run_queues+0x37d/0x700 ? enqueue_hrtimer+0x1b0/0x1b0 ? kvm_clock_get_cycles+0xd/0x10 ? ktime_get_update_offsets_now+0x6f/0x280 hrtimer_interrupt+0x2c8/0x740 Fix the problem by checking that the parent of the two bfqqs we are merging in bfq_setup_merge() is the same. Link: https://lore.kernel.org/linux-block/20211125172809.GC19572@quack2.suse.cz/ CC: stable@vger.kernel.org Fixes: 430a67f9d616 ("block, bfq: merge bursts of newly-created queues") Signed-off-by: Jan Kara --- block/bfq-iosched.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 6d122c28086e..7d00b21ebe5d 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -2758,6 +2758,14 @@ bfq_setup_merge(struct bfq_queue *bfqq, struct bfq_queue *new_bfqq) if (process_refs == 0 || new_process_refs == 0) return NULL; + /* + * Make sure merged queues belong to the same parent. Parents could + * have changed since the time we decided the two queues are suitable + * for merging. + */ + if (new_bfqq->entity.parent != bfqq->entity.parent) + return NULL; + bfq_log_bfqq(bfqq->bfqd, bfqq, "scheduling merge with queue %d", new_bfqq->pid); From patchwork Wed Mar 30 12:42:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12795778 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39A28C433F5 for ; Wed, 30 Mar 2022 12:45:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345327AbiC3MrG (ORCPT ); Wed, 30 Mar 2022 08:47:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343840AbiC3Mqy (ORCPT ); Wed, 30 Mar 2022 08:46:54 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97A9B7DE0B; Wed, 30 Mar 2022 05:43:02 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 3702F1F86A; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uPmivVOPJBGDUdJhi3hnlYMRdFUbE2YZW82iLvFOdwY=; b=hc9hx9eGpAJNxnbA5HOoTm6CZwVwuhOGlVX8JgREGkR+c+iOWi274zDKOhzTQuAPbE8BZg NaB2seVsiS6p6I1TYsaNZSF1Km/o8S+Y8qGSbHSvYSA4j2lBwMAPvCiMC4fYHmPSrKcfLz r+5h0/CPJtJR++Vv1RWGryPItLTKEZE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uPmivVOPJBGDUdJhi3hnlYMRdFUbE2YZW82iLvFOdwY=; b=tSon/zF8llvLhu7ZRMxgAChT/DCRUH3+5TndANxfHPnLvaN0mlq/5risiy9IBeAPD6Vet2 zsyTG+ucRT5lmDAA== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 261D1A3B9A; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 736A3A0619; Wed, 30 Mar 2022 14:42:56 +0200 (CEST) From: Jan Kara To: Cc: Paolo Valente , Jens Axboe , "yukuai (C)" , Jan Kara , stable@vger.kernel.org Subject: [PATCH 3/9] bfq: Split shared queues on move between cgroups Date: Wed, 30 Mar 2022 14:42:46 +0200 Message-Id: <20220330124255.24581-3-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220330123438.32719-1-jack@suse.cz> References: <20220330123438.32719-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3501; h=from:subject; bh=wo8o0yMjT769m1S0iQKPUbqiFN7StyovImd+Ke6tEJM=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRFBGe4zgneYJoI7GeIBBbnV7ZRXSR3wDxmoiOVzL foKh/naJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkRQRgAKCRCcnaoHP2RA2YsiB/ 9CMf931CZelWEL44mTYfcON9A3tgpJoZhn5buQ9PXGhyiYCsljNHYSYaZThsmEMtiFhZjBeAXYC33O Vc7K1+2M8+WGt4wFxZoytbRmd8ODtwDt9UBhtpMP0fFby8x6hz3GppJg1MFLVE+3fgZEXKzSO+JKLl dOWci+UnfOmJjvMaJT0dAf08nNliMd6dQX2ETASRyDjeH6Mr7t9ek6Wg3seQ2wwXKoVD54J3RdEa8E zKWAAV1KqkyfizAczg5V+c5rkno3JAtpVlUFA2h9xKRzqS/oIeWS7unPglStAC4yucpfYvqR3bSWIH 5jTw1i5CmN8zkGkHF0Enoih9m3Ju/v X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org When bfqq is shared by multiple processes it can happen that one of the processes gets moved to a different cgroup (or just starts submitting IO for different cgroup). In case that happens we need to split the merged bfqq as otherwise we will have IO for multiple cgroups in one bfqq and we will just account IO time to wrong entities etc. Similarly if the bfqq is scheduled to merge with another bfqq but the merge didn't happen yet, cancel the merge as it need not be valid anymore. CC: stable@vger.kernel.org Fixes: e21b7a0b9887 ("block, bfq: add full hierarchical scheduling and cgroups support") Signed-off-by: Jan Kara --- block/bfq-cgroup.c | 36 +++++++++++++++++++++++++++++++++--- block/bfq-iosched.c | 2 +- block/bfq-iosched.h | 1 + 3 files changed, 35 insertions(+), 4 deletions(-) diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index 420eda2589c0..9352f3cc2377 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -743,9 +743,39 @@ static struct bfq_group *__bfq_bic_change_cgroup(struct bfq_data *bfqd, } if (sync_bfqq) { - entity = &sync_bfqq->entity; - if (entity->sched_data != &bfqg->sched_data) - bfq_bfqq_move(bfqd, sync_bfqq, bfqg); + if (!sync_bfqq->new_bfqq && !bfq_bfqq_coop(sync_bfqq)) { + /* We are the only user of this bfqq, just move it */ + if (sync_bfqq->entity.sched_data != &bfqg->sched_data) + bfq_bfqq_move(bfqd, sync_bfqq, bfqg); + } else { + struct bfq_queue *bfqq; + + /* + * The queue was merged to a different queue. Check + * that the merge chain still belongs to the same + * cgroup. + */ + for (bfqq = sync_bfqq; bfqq; bfqq = bfqq->new_bfqq) + if (bfqq->entity.sched_data != + &bfqg->sched_data) + break; + if (bfqq) { + /* + * Some queue changed cgroup so the merge is + * not valid anymore. We cannot easily just + * cancel the merge (by clearing new_bfqq) as + * there may be other processes using this + * queue and holding refs to all queues below + * sync_bfqq->new_bfqq. Similarly if the merge + * already happened, we need to detach from + * bfqq now so that we cannot merge bio to a + * request from the old cgroup. + */ + bfq_put_cooperator(sync_bfqq); + bfq_release_process_ref(bfqd, sync_bfqq); + bic_set_bfqq(bic, NULL, 1); + } + } } return bfqg; diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 7d00b21ebe5d..89fe3f85eb3c 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -5315,7 +5315,7 @@ static void bfq_put_stable_ref(struct bfq_queue *bfqq) bfq_put_queue(bfqq); } -static void bfq_put_cooperator(struct bfq_queue *bfqq) +void bfq_put_cooperator(struct bfq_queue *bfqq) { struct bfq_queue *__bfqq, *next; diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index 3b83e3d1c2e5..a56763045d19 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -979,6 +979,7 @@ void bfq_weights_tree_remove(struct bfq_data *bfqd, void bfq_bfqq_expire(struct bfq_data *bfqd, struct bfq_queue *bfqq, bool compensate, enum bfqq_expiration reason); void bfq_put_queue(struct bfq_queue *bfqq); +void bfq_put_cooperator(struct bfq_queue *bfqq); void bfq_end_wr_async_queues(struct bfq_data *bfqd, struct bfq_group *bfqg); void bfq_release_process_ref(struct bfq_data *bfqd, struct bfq_queue *bfqq); void bfq_schedule_dispatch(struct bfq_data *bfqd); From patchwork Wed Mar 30 12:42:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12795779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A413C433EF for ; Wed, 30 Mar 2022 12:45:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345728AbiC3MrI (ORCPT ); Wed, 30 Mar 2022 08:47:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46642 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345236AbiC3Mqy (ORCPT ); Wed, 30 Mar 2022 08:46:54 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97D5B7DE16; Wed, 30 Mar 2022 05:43:02 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 2E0C81F7AB; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RQpb3DlBpC+xUzJx/SvzpuHxoRpxIiDopbRVeNdSa9A=; b=yR5UWcrbn14XpWeWjX5H6HlNocJ16DztxIg7aPmwVLIq4VH4hY5gO9mr+pDHF0/5LMEHRL cVpcZ+6oRrfvj1fDHaousDrG2RYQApIkSIuzKkXefILHzX3xAn/teQuVYj/+j7J5YBcPzj +0qsftB9lTnGKin+iMojNJI6W5sXIM8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RQpb3DlBpC+xUzJx/SvzpuHxoRpxIiDopbRVeNdSa9A=; b=u6cKgGrGYXHqcfeLJb6mXH76Bf9vSh3/Z8nXsWCZIL3zBbP7NxAp/j/jUOehB6ZQ/Fe0hL Ak22qhc53arabJBA== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 19BA9A3B83; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 78D4CA061A; Wed, 30 Mar 2022 14:42:56 +0200 (CEST) From: Jan Kara To: Cc: Paolo Valente , Jens Axboe , "yukuai (C)" , Jan Kara , stable@vger.kernel.org Subject: [PATCH 4/9] bfq: Update cgroup information before merging bio Date: Wed, 30 Mar 2022 14:42:47 +0200 Message-Id: <20220330124255.24581-4-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220330123438.32719-1-jack@suse.cz> References: <20220330123438.32719-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1409; h=from:subject; bh=vBnlnDsUwY71kZB9dCskeOZQL4WP3DTzp9qFZwDR62M=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRFBH7s2MVgHfYyX7/j3Uz+L7Pwc2kwt7cjXogDoj nSmTnGWJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkRQRwAKCRCcnaoHP2RA2RSSB/ 0WB0b7X01Qo+1rx9nKW/zefbr92awfYlpwrSL8lo6045feevYqyavolAM8ZOa9k3I7GtKLYVHzYigp fYZuQrmyUn3ur6/sz58I126oHSGGjaAv902P4tt2JiDr7PN205xZ2AwEIebyJi9Q4gGoxJ+cE74ZcY PvqEBu5aNbHwug8UQxlxV4ZZnmvxOpdt0/3yJohDd7Jldyy4ze5rsj3mi5AvHllz9mWIsxw+cSwr/q joayjRM0nZU0KdoKgE4RU5qdIx3ZO82m6yWi7eZZxumNosImKmgoNv0ZsnFob9JRrSMlGB0hrui5uK FbalNNj69MP8f48mePNHAPO0NsLydj X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org When the process is migrated to a different cgroup (or in case of writeback just starts submitting bios associated with a different cgroup) bfq_merge_bio() can operate with stale cgroup information in bic. Thus the bio can be merged to a request from a different cgroup or it can result in merging of bfqqs for different cgroups or bfqqs of already dead cgroups and causing possible use-after-free issues. Fix the problem by updating cgroup information in bfq_merge_bio(). CC: stable@vger.kernel.org Fixes: e21b7a0b9887 ("block, bfq: add full hierarchical scheduling and cgroups support") Signed-off-by: Jan Kara --- block/bfq-iosched.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 89fe3f85eb3c..1fc4d4628fba 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -2457,10 +2457,17 @@ static bool bfq_bio_merge(struct request_queue *q, struct bio *bio, spin_lock_irq(&bfqd->lock); - if (bic) + if (bic) { + /* + * Make sure cgroup info is uptodate for current process before + * considering the merge. + */ + bfq_bic_update_cgroup(bic, bio); + bfqd->bio_bfqq = bic_to_bfqq(bic, op_is_sync(bio->bi_opf)); - else + } else { bfqd->bio_bfqq = NULL; + } bfqd->bio_bic = bic; ret = blk_mq_sched_try_merge(q, bio, nr_segs, &free); From patchwork Wed Mar 30 12:42:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12795782 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09347C433F5 for ; Wed, 30 Mar 2022 12:45:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242522AbiC3MrL (ORCPT ); Wed, 30 Mar 2022 08:47:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245207AbiC3Mqy (ORCPT ); Wed, 30 Mar 2022 08:46:54 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E50827DE22 for ; Wed, 30 Mar 2022 05:43:04 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 7094F1F86D; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9gbCrEQBuVqraNMaF2pon2hbUesKVBCAOTVxv1JIa08=; b=ETTyt026NxFyQ/KSPL/LEmBg2FGN8qMZwov+bwGztR0/F3BlFAV1rjXLgZzg6hzFuxUqNc FsrIN7jPuBV5vlkYiVF/vigQ6bKU+A7zRW7xem1CPFo4r4jkwikns8MO9FQjsIxpynYQu/ FXtK5sns/00QKmW4vP3b8im82FiVV5Q= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9gbCrEQBuVqraNMaF2pon2hbUesKVBCAOTVxv1JIa08=; b=hn+7XQcovYcOn6lrjYatXI5eBjXVnygl38mChB1ki4Hs6xkoXQua9poNfFaKTRwDiPYsUf Q7HofhfSO8fhDfBw== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 60FC8A3B9E; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 7E518A061B; Wed, 30 Mar 2022 14:42:56 +0200 (CEST) From: Jan Kara To: Cc: Paolo Valente , Jens Axboe , "yukuai (C)" , Jan Kara Subject: [PATCH 5/9] bfq: Drop pointless unlock-lock pair Date: Wed, 30 Mar 2022 14:42:48 +0200 Message-Id: <20220330124255.24581-5-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220330123438.32719-1-jack@suse.cz> References: <20220330123438.32719-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=834; h=from:subject; bh=ejFMnc6iu4HXPPNauOehah8NddMt8WVB+Bu5b9CadRo=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRFBI1ecBINF//KLV7KHnO8abV3gJKrd0rHHIYHxL H4iYTp+JATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkRQSAAKCRCcnaoHP2RA2T2MCA CaMaGTxUP0xKQIdFpG77UeFJcfQUAJTgZJjeajcCtWbDofVoNzNSvjkElFGEWd17FUqqDj0lctRK+Y 9xWEYSvZEjFwC8/wFmDvkI5ECWW770/d0KaVbx9S12DkPk357/FDSPFhG08cjxcSKLccvJPodYxndj 7hrtleLFkog7DtCZAW9AYPSjToPFPOHLw2De9PUkLzTWXDw07tfw3R69GZYEg9qnj3qoWdGFmXE/mY UmniuSrij9CACqCkrGnDCbZiwm2rNYCreUKnGVwJDFLko4P2NDixhYZemSKFs60NTLvUtmC3Gv1UWy 4CNAe3oCECIiwOECHR54u2X/n4djUS X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org In bfq_insert_request() we unlock bfqd->lock only to call trace_block_rq_insert() and then lock bfqd->lock again. This is really pointless since tracing is disabled if we really care about performance and even if the tracepoint is enabled, it is a quick call. Signed-off-by: Jan Kara --- block/bfq-iosched.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 1fc4d4628fba..19082e14f3c1 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -6150,11 +6150,8 @@ static void bfq_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, return; } - spin_unlock_irq(&bfqd->lock); - trace_block_rq_insert(rq); - spin_lock_irq(&bfqd->lock); bfqq = bfq_init_rq(rq); if (!bfqq || at_head) { if (at_head) From patchwork Wed Mar 30 12:42:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12795780 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D99DFC4332F for ; Wed, 30 Mar 2022 12:45:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345350AbiC3MrJ (ORCPT ); Wed, 30 Mar 2022 08:47:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48064 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344477AbiC3Mqy (ORCPT ); Wed, 30 Mar 2022 08:46:54 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E49667DE20 for ; Wed, 30 Mar 2022 05:43:04 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 6A9E71F86B; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D0gEdYTa/yJbd0ZyukRrmN7O+j86ffHtuBFY+SM0WwI=; b=GTHj6LWA63+xcxZwQbEr7WyYf16sdZSrelfX1nuieyACZmiigF9ky7PcBLaJMfeOLuyiWb NPeFM2yEJUipWWsVS9f2tRhdg2dxMkw5W0s7QjOaYlNIUbFBF/C3X6E6WtkOBNzH7p8H5r 66qu7NGXBVd1FgT+G4QKK0zjg/BW+7U= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D0gEdYTa/yJbd0ZyukRrmN7O+j86ffHtuBFY+SM0WwI=; b=xWDYqRvNceO3kCyl1p/PLywWkGOugdAlJq0yVDVwTmXkOl3kYZmK/GqTOanis5v7qeKLr3 zY7RZtu4t6W3WXDg== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 5DDABA3B9C; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 8405DA061C; Wed, 30 Mar 2022 14:42:56 +0200 (CEST) From: Jan Kara To: Cc: Paolo Valente , Jens Axboe , "yukuai (C)" , Jan Kara Subject: [PATCH 6/9] bfq: Remove pointless bfq_init_rq() calls Date: Wed, 30 Mar 2022 14:42:49 +0200 Message-Id: <20220330124255.24581-6-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220330123438.32719-1-jack@suse.cz> References: <20220330123438.32719-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2791; h=from:subject; bh=mjvmO64jJM0oJc8VwoSswmN8LpOszo38y6MW0wyqdfg=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRFBI/vYa3lEKQ4OeboocQVXJE+Fp+23HDiwaV1Qe IMcTThKJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkRQSAAKCRCcnaoHP2RA2dBKB/ 9mp8XB4oQmDnsC5abQSPglR6a6qHTl9FT91xItezHd4qKmljznJ8OSahGVdLh96chO4Td6WchmmTjZ 8fyqEesfjRc64as55V+fb3BBEZO2OU6b3OJqgWfQIuIhGAZNCqHiOutWzMAKFpwXYhGRkrV72RCznK zUxNwV3O3TaemKW6ArYtgAMWmOauvAo3ZjoVow4oMn3A3JTgvR4PnWNL0VvavkAimlrhckYsl4Cs23 o1kHgEtLlYlEpLBCH5/vk/sth5wr8XMEZVdpaBhzFoZ4VjkSGQMsP8Pi18oD7JgYVXfxQu8QAbMt79 wrfTsg3UwRBR0aDC7jp2wqnxa/Qzgu X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org We call bfq_init_rq() from request merging functions where requests we get should have already gone through bfq_init_rq() during insert and anyway we want to do anything only if the request is already tracked by BFQ. So replace calls to bfq_init_rq() with RQ_BFQQ() instead to simply skip requests untracked by BFQ. We move bfq_init_rq() call in bfq_insert_request() a bit earlier to cover request merging and thus can transfer FIFO position in case of a merge. Signed-off-by: Jan Kara --- block/bfq-iosched.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 19082e14f3c1..d7cf930b47bb 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -2497,8 +2497,6 @@ static int bfq_request_merge(struct request_queue *q, struct request **req, return ELEVATOR_NO_MERGE; } -static struct bfq_queue *bfq_init_rq(struct request *rq); - static void bfq_request_merged(struct request_queue *q, struct request *req, enum elv_merge type) { @@ -2507,7 +2505,7 @@ static void bfq_request_merged(struct request_queue *q, struct request *req, blk_rq_pos(req) < blk_rq_pos(container_of(rb_prev(&req->rb_node), struct request, rb_node))) { - struct bfq_queue *bfqq = bfq_init_rq(req); + struct bfq_queue *bfqq = RQ_BFQQ(req); struct bfq_data *bfqd; struct request *prev, *next_rq; @@ -2559,8 +2557,8 @@ static void bfq_request_merged(struct request_queue *q, struct request *req, static void bfq_requests_merged(struct request_queue *q, struct request *rq, struct request *next) { - struct bfq_queue *bfqq = bfq_init_rq(rq), - *next_bfqq = bfq_init_rq(next); + struct bfq_queue *bfqq = RQ_BFQQ(rq), + *next_bfqq = RQ_BFQQ(next); if (!bfqq) goto remove; @@ -6129,6 +6127,8 @@ static inline void bfq_update_insert_stats(struct request_queue *q, unsigned int cmd_flags) {} #endif /* CONFIG_BFQ_CGROUP_DEBUG */ +static struct bfq_queue *bfq_init_rq(struct request *rq); + static void bfq_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, bool at_head) { @@ -6144,6 +6144,7 @@ static void bfq_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, bfqg_stats_update_legacy_io(q, rq); #endif spin_lock_irq(&bfqd->lock); + bfqq = bfq_init_rq(rq); if (blk_mq_sched_try_insert_merge(q, rq, &free)) { spin_unlock_irq(&bfqd->lock); blk_mq_free_requests(&free); @@ -6152,7 +6153,6 @@ static void bfq_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, trace_block_rq_insert(rq); - bfqq = bfq_init_rq(rq); if (!bfqq || at_head) { if (at_head) list_add(&rq->queuelist, &bfqd->dispatch); From patchwork Wed Mar 30 12:42:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12795784 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90908C433F5 for ; Wed, 30 Mar 2022 12:47:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245730AbiC3MtN (ORCPT ); Wed, 30 Mar 2022 08:49:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34548 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245193AbiC3Mqy (ORCPT ); Wed, 30 Mar 2022 08:46:54 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E53237DE26 for ; Wed, 30 Mar 2022 05:43:04 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 6FBC71F86C; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kHU+Vg8Ryhg4yA0LaoIt150oR2BYg7vBbUFce/su5YI=; b=BKbYja+W3WcXgC+xRUCLbENx7rlJeGTpvvJ76Vjn9E9QBpckYOq+ar/XyOgz6kKlCsazuH 9qMB1e9OP85Loa25+ttv7X7sZY+ZqMnVGJVOWDkjhPbrGowNvi6S3TI3JYqqdIbSkk0kpe z5q286b86PX/t6KxsJjbxcfwjyYmy5U= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kHU+Vg8Ryhg4yA0LaoIt150oR2BYg7vBbUFce/su5YI=; b=hiR7y5QDlfGcmR0lgnTn9qKZ3pHTB+7Iku0fM24wkoEL/BDz3w/zJlYtiCvPu8wOBNlH16 RrUENKyy2ZTL7UDg== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 613D9A3B9F; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 89E74A061D; Wed, 30 Mar 2022 14:42:56 +0200 (CEST) From: Jan Kara To: Cc: Paolo Valente , Jens Axboe , "yukuai (C)" , Jan Kara Subject: [PATCH 7/9] bfq: Track whether bfq_group is still online Date: Wed, 30 Mar 2022 14:42:50 +0200 Message-Id: <20220330124255.24581-7-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220330123438.32719-1-jack@suse.cz> References: <20220330123438.32719-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1697; h=from:subject; bh=cd0xiQ5rstrE4vCdYR2bomPdo6Kc8upLVhtCRL+Ukpk=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRFBJkeiymotY/RfT315UBgzw5+Ib/fW3cQL89/sN AOy4g8uJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkRQSQAKCRCcnaoHP2RA2T3PB/ 9AJ0tCn2GBuVgxtSibGF3nwqMkxTE0ScwlhJ3GdLUyn0+3Xg9JJKbB3IQQWNJ9nE+93Xm+Mq4pXHF6 +p/WzK4Ztq/LNYvwJ+xvL+i5K3Iicb/2WkzXrIE51jjVCbkSTHOmx/KUyuFUojSNmM3jaOKxHGCVUr NZDoAiUUrquW0FcIlD5+UFaOUHHWUAP0KxqU54YXlrTys5JgHIf5+8gpmBdHJWEFrL2R6/pznq+JUL ixHquYeU86s6UWU2xWIAaAQseBl1NUcMeHSy9IjdNsXEyXOlY9riomcSDT/go8kAEvir6dtT5iWfH7 ZYSIXTP5kP3gdxDFTx3O5tW761D7ge X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Track whether bfq_group is still online. We cannot rely on blkcg_gq->online because that gets cleared only after all policies are offlined and we need something that gets updated already under bfqd->lock when we are cleaning up our bfq_group to be able to guarantee that when we see online bfq_group, it will stay online while we are holding bfqd->lock lock. Signed-off-by: Jan Kara --- block/bfq-cgroup.c | 3 ++- block/bfq-iosched.h | 2 ++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index 9352f3cc2377..879380c2bc7e 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -557,6 +557,7 @@ static void bfq_pd_init(struct blkg_policy_data *pd) */ bfqg->bfqd = bfqd; bfqg->active_entities = 0; + bfqg->online = true; bfqg->rq_pos_tree = RB_ROOT; } @@ -603,7 +604,6 @@ struct bfq_group *bfq_find_set_group(struct bfq_data *bfqd, struct bfq_entity *entity; bfqg = bfq_lookup_bfqg(bfqd, blkcg); - if (unlikely(!bfqg)) return NULL; @@ -979,6 +979,7 @@ static void bfq_pd_offline(struct blkg_policy_data *pd) put_async_queues: bfq_put_async_queues(bfqd, bfqg); + bfqg->online = false; spin_unlock_irqrestore(&bfqd->lock, flags); /* diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index a56763045d19..4664e2f3e828 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -928,6 +928,8 @@ struct bfq_group { /* reference counter (see comments in bfq_bic_update_cgroup) */ int ref; + /* Is bfq_group still online? */ + bool online; struct bfq_entity entity; struct bfq_sched_data sched_data; From patchwork Wed Mar 30 12:42:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12795783 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A679BC4332F for ; Wed, 30 Mar 2022 12:45:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345841AbiC3MrN (ORCPT ); Wed, 30 Mar 2022 08:47:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60844 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345488AbiC3Mqy (ORCPT ); Wed, 30 Mar 2022 08:46:54 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E76567DE2E for ; Wed, 30 Mar 2022 05:43:04 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 72B8D1F86E; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0wE+HnT1NzmRxFE6n4iI0AJWcg+fDvyQJY18TY/6yck=; b=Xen8v/79cJXaacu3rJt5uimlyD6fm9fjwyrx0X6XqW1OsTqHDQ+eM9nV4fa8D2uCThfOg/ T24p30bB/mGxL8iSd7rfaDWDnusB4jLOI1k6zTXan0dKfnBl0iIEaMT3M45IwOo2VKEZvi gl3cV9B59A8LgyayJa3H86nI3sqgSZ8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0wE+HnT1NzmRxFE6n4iI0AJWcg+fDvyQJY18TY/6yck=; b=YpDe6+uc7HYi+I+xtFYSEYy1hieSDwjpmMtmjHPhKrrbvarkZMYlli2XE8JQm6FYbJB595 3poqDS1A14QyquAQ== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 60E52A3B9D; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 904DCA061E; Wed, 30 Mar 2022 14:42:56 +0200 (CEST) From: Jan Kara To: Cc: Paolo Valente , Jens Axboe , "yukuai (C)" , Jan Kara Subject: [PATCH 8/9] bfq: Get rid of __bio_blkcg() usage Date: Wed, 30 Mar 2022 14:42:51 +0200 Message-Id: <20220330124255.24581-8-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220330123438.32719-1-jack@suse.cz> References: <20220330123438.32719-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6478; h=from:subject; bh=n+ahp18kHkoCcIYiZqMRynYyw9pppY5NnsV6SESkGwA=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRFBKbxDzYwSbJ7cK37Wq1qA6Qf2awFK/0kAByR1n oTGJbwGJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkRQSgAKCRCcnaoHP2RA2XpICA CTRfHmQhpAZ++D/uJUObjlomHl9uR0QTbNe4nafL2kDhrslv51TfbYXaDnPk6PZ0KQYvNAxKYKIG3H E4AHZnCF2a34xVjjUJmYc1KiYfMcokNwNEaHo/Mv6lW4u+uPfWSUQUpxlgnT4KN+sW051WihU8E6NL J/8g4cBiyVaicNBB+JjCFJgeUD1ebOq0beVAgLJlLrmB4ygBiezrhkofKj794vqxusFPk3h9cjDl9v 3HeMaHR8BAlRbS+wxOZDx7czfV1dtOKQ7OFk1uy0PdxmJTjjTlkVFThIe/jZyutAMZarMpcsq1deVt z3ZNN8B759+8o/aw2uBjT+BKYt2TCg X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org BFQ usage of __bio_blkcg() is a relict from the past. Furthermore if bio would not be associated with any blkcg, the usage of __bio_blkcg() in BFQ is prone to races with the task being migrated between cgroups as __bio_blkcg() calls at different places could return different blkcgs. Convert BFQ to the new situation where bio->bi_blkg is initialized in bio_set_dev() and thus practically always valid. This allows us to save blkcg_gq lookup and noticeably simplify the code. Signed-off-by: Jan Kara --- block/bfq-cgroup.c | 63 +++++++++++++++++---------------------------- block/bfq-iosched.c | 11 +------- block/bfq-iosched.h | 3 +-- 3 files changed, 25 insertions(+), 52 deletions(-) diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index 879380c2bc7e..32d2c2a47480 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -586,27 +586,11 @@ static void bfq_group_set_parent(struct bfq_group *bfqg, entity->sched_data = &parent->sched_data; } -static struct bfq_group *bfq_lookup_bfqg(struct bfq_data *bfqd, - struct blkcg *blkcg) +static void bfq_link_bfqg(struct bfq_data *bfqd, struct bfq_group *bfqg) { - struct blkcg_gq *blkg; - - blkg = blkg_lookup(blkcg, bfqd->queue); - if (likely(blkg)) - return blkg_to_bfqg(blkg); - return NULL; -} - -struct bfq_group *bfq_find_set_group(struct bfq_data *bfqd, - struct blkcg *blkcg) -{ - struct bfq_group *bfqg, *parent; + struct bfq_group *parent; struct bfq_entity *entity; - bfqg = bfq_lookup_bfqg(bfqd, blkcg); - if (unlikely(!bfqg)) - return NULL; - /* * Update chain of bfq_groups as we might be handling a leaf group * which, along with some of its relatives, has not been hooked yet @@ -623,8 +607,15 @@ struct bfq_group *bfq_find_set_group(struct bfq_data *bfqd, bfq_group_set_parent(curr_bfqg, parent); } } +} - return bfqg; +struct bfq_group *bfq_bio_bfqg(struct bfq_data *bfqd, struct bio *bio) +{ + struct blkcg_gq *blkg = bio->bi_blkg; + + if (!blkg) + return bfqd->root_group; + return blkg_to_bfqg(blkg); } /** @@ -714,25 +705,15 @@ void bfq_bfqq_move(struct bfq_data *bfqd, struct bfq_queue *bfqq, * Move bic to blkcg, assuming that bfqd->lock is held; which makes * sure that the reference to cgroup is valid across the call (see * comments in bfq_bic_update_cgroup on this issue) - * - * NOTE: an alternative approach might have been to store the current - * cgroup in bfqq and getting a reference to it, reducing the lookup - * time here, at the price of slightly more complex code. */ -static struct bfq_group *__bfq_bic_change_cgroup(struct bfq_data *bfqd, - struct bfq_io_cq *bic, - struct blkcg *blkcg) +static void *__bfq_bic_change_cgroup(struct bfq_data *bfqd, + struct bfq_io_cq *bic, + struct bfq_group *bfqg) { struct bfq_queue *async_bfqq = bic_to_bfqq(bic, 0); struct bfq_queue *sync_bfqq = bic_to_bfqq(bic, 1); - struct bfq_group *bfqg; struct bfq_entity *entity; - bfqg = bfq_find_set_group(bfqd, blkcg); - - if (unlikely(!bfqg)) - bfqg = bfqd->root_group; - if (async_bfqq) { entity = &async_bfqq->entity; @@ -784,20 +765,24 @@ static struct bfq_group *__bfq_bic_change_cgroup(struct bfq_data *bfqd, void bfq_bic_update_cgroup(struct bfq_io_cq *bic, struct bio *bio) { struct bfq_data *bfqd = bic_to_bfqd(bic); - struct bfq_group *bfqg = NULL; + struct bfq_group *bfqg = bfq_bio_bfqg(bfqd, bio); uint64_t serial_nr; - rcu_read_lock(); - serial_nr = __bio_blkcg(bio)->css.serial_nr; + serial_nr = bfqg_to_blkg(bfqg)->blkcg->css.serial_nr; /* * Check whether blkcg has changed. The condition may trigger * spuriously on a newly created cic but there's no harm. */ if (unlikely(!bfqd) || likely(bic->blkcg_serial_nr == serial_nr)) - goto out; + return; - bfqg = __bfq_bic_change_cgroup(bfqd, bic, __bio_blkcg(bio)); + /* + * New cgroup for this process. Make sure it is linked to bfq internal + * cgroup hierarchy. + */ + bfq_link_bfqg(bfqd, bfqg); + __bfq_bic_change_cgroup(bfqd, bic, bfqg); /* * Update blkg_path for bfq_log_* functions. We cache this * path, and update it here, for the following @@ -850,8 +835,6 @@ void bfq_bic_update_cgroup(struct bfq_io_cq *bic, struct bio *bio) */ blkg_path(bfqg_to_blkg(bfqg), bfqg->blkg_path, sizeof(bfqg->blkg_path)); bic->blkcg_serial_nr = serial_nr; -out: - rcu_read_unlock(); } /** @@ -1469,7 +1452,7 @@ void bfq_end_wr_async(struct bfq_data *bfqd) bfq_end_wr_async_queues(bfqd, bfqd->root_group); } -struct bfq_group *bfq_find_set_group(struct bfq_data *bfqd, struct blkcg *blkcg) +struct bfq_group *bfq_bio_bfqg(struct bfq_data *bfqd, struct bio *bio) { return bfqd->root_group; } diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index d7cf930b47bb..e47c75f1fa0f 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -5726,14 +5726,7 @@ static struct bfq_queue *bfq_get_queue(struct bfq_data *bfqd, struct bfq_queue *bfqq; struct bfq_group *bfqg; - rcu_read_lock(); - - bfqg = bfq_find_set_group(bfqd, __bio_blkcg(bio)); - if (!bfqg) { - bfqq = &bfqd->oom_bfqq; - goto out; - } - + bfqg = bfq_bio_bfqg(bfqd, bio); if (!is_sync) { async_bfqq = bfq_async_queue_prio(bfqd, bfqg, ioprio_class, ioprio); @@ -5779,8 +5772,6 @@ static struct bfq_queue *bfq_get_queue(struct bfq_data *bfqd, if (bfqq != &bfqd->oom_bfqq && is_sync && !respawn) bfqq = bfq_do_or_sched_stable_merge(bfqd, bfqq, bic); - - rcu_read_unlock(); return bfqq; } diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index 4664e2f3e828..978ef5d6fe6a 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -1009,8 +1009,7 @@ void bfq_bfqq_move(struct bfq_data *bfqd, struct bfq_queue *bfqq, void bfq_init_entity(struct bfq_entity *entity, struct bfq_group *bfqg); void bfq_bic_update_cgroup(struct bfq_io_cq *bic, struct bio *bio); void bfq_end_wr_async(struct bfq_data *bfqd); -struct bfq_group *bfq_find_set_group(struct bfq_data *bfqd, - struct blkcg *blkcg); +struct bfq_group *bfq_bio_bfqg(struct bfq_data *bfqd, struct bio *bio); struct blkcg_gq *bfqg_to_blkg(struct bfq_group *bfqg); struct bfq_group *bfqq_group(struct bfq_queue *bfqq); struct bfq_group *bfq_create_group_hierarchy(struct bfq_data *bfqd, int node); From patchwork Wed Mar 30 12:42:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12795781 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59E15C433F5 for ; Wed, 30 Mar 2022 12:45:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345778AbiC3MrK (ORCPT ); Wed, 30 Mar 2022 08:47:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58866 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345442AbiC3Mqy (ORCPT ); Wed, 30 Mar 2022 08:46:54 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0167255750; Wed, 30 Mar 2022 05:43:02 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 731AD212C6; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hRZ85jQoU7TvkqgM00hNxz5/k9iJcTYk7V8sofKyQmw=; b=1drrYO5eyLfp5ux6hC/3GacnJGXYFn//wlwdHrM9gp9VjuRCWRGttn+mK48mD2k5EB4TN1 U6uzbsFXQJtxmuauUMniJstpTCqcpM9iEMoBGOipGp1UCY/ddTbJSOqRYBObPExuwDbZk+ uiKt0SOaw3RNPQ4D5ZY1WZn9+zKxHxM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1648644181; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hRZ85jQoU7TvkqgM00hNxz5/k9iJcTYk7V8sofKyQmw=; b=qtXVwJeEhH0lSEWEd55HRZ1XWwPhK7EamdK2Y37aiASPy0QpR+LRE9Xa4KNhxfLodoiqPj EPKbHKJqQSQo1BDQ== Received: from quack3.suse.cz (unknown [10.163.28.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 66156A3BA0; Wed, 30 Mar 2022 12:43:01 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 963CFA061F; Wed, 30 Mar 2022 14:42:56 +0200 (CEST) From: Jan Kara To: Cc: Paolo Valente , Jens Axboe , "yukuai (C)" , Jan Kara , stable@vger.kernel.org Subject: [PATCH 9/9] bfq: Make sure bfqg for which we are queueing requests is online Date: Wed, 30 Mar 2022 14:42:52 +0200 Message-Id: <20220330124255.24581-9-jack@suse.cz> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220330123438.32719-1-jack@suse.cz> References: <20220330123438.32719-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1522; h=from:subject; bh=IG83mxZ/ZhVpcfY2IHN0K8dR/1UrFaCWqw7Cz0PSQXg=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBiRFBLFh/PSPExza98vsZtwLk6Rd2oChOwIwvG7Dk0 TN0o3ImJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYkRQSwAKCRCcnaoHP2RA2X7wB/ 0R1pDb6L3U/ExXLnsZiWZzWFa+fX4FBl0mHD6lOdrgiOForcDlHbYzI/PoKhXaYsJBbYxh8IlWsjWv 6acjRHUG3xm8ajWXu8pVoXVH0TXz3cAMYYWp24IxEggv8cX4E7G7IXBtYWAYmoGgm7D2LXBihKSUJT gj71z6W8jFUxl3zQbn4nm1CR7vttxv6/4+9321x1iDnJqbYyShC+iH+vs+BTZqdC88L0y/Ru2zIJTs JgklGBIHyUS8KxRfBy3RyNi+m6rni1pBZKowAuhZDDl50daPvx+mhC7UVGhGSh8TjALISxD3pFz8SO pDMLSzxOyNb9YZqkHK2Ta0hZKgsSW5 X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Bios queued into BFQ IO scheduler can be associated with a cgroup that was already offlined. This may then cause insertion of this bfq_group into a service tree. But this bfq_group will get freed as soon as last bio associated with it is completed leading to use after free issues for service tree users. Fix the problem by making sure we always operate on online bfq_group. If the bfq_group associated with the bio is not online, we pick the first online parent. CC: stable@vger.kernel.org Fixes: e21b7a0b9887 ("block, bfq: add full hierarchical scheduling and cgroups support") Signed-off-by: Jan Kara --- block/bfq-cgroup.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index 32d2c2a47480..09574af83566 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -612,10 +612,19 @@ static void bfq_link_bfqg(struct bfq_data *bfqd, struct bfq_group *bfqg) struct bfq_group *bfq_bio_bfqg(struct bfq_data *bfqd, struct bio *bio) { struct blkcg_gq *blkg = bio->bi_blkg; + struct bfq_group *bfqg; - if (!blkg) - return bfqd->root_group; - return blkg_to_bfqg(blkg); + while (blkg) { + bfqg = blkg_to_bfqg(blkg); + if (bfqg->online) { + bio_associate_blkg_from_css(bio, &blkg->blkcg->css); + return bfqg; + } + blkg = blkg->parent; + } + bio_associate_blkg_from_css(bio, + &bfqg_to_blkg(bfqd->root_group)->blkcg->css); + return bfqd->root_group; } /**