From patchwork Mon Jun 20 16:11:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12887884 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4B85C433EF for ; Mon, 20 Jun 2022 16:11:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243106AbiFTQL5 (ORCPT ); Mon, 20 Jun 2022 12:11:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242145AbiFTQL5 (ORCPT ); Mon, 20 Jun 2022 12:11:57 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6E9DC20BC7 for ; Mon, 20 Jun 2022 09:11:55 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 2FE8621B85; Mon, 20 Jun 2022 16:11:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655741514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=y2ws/QxBRb/M2rDqfKbqG9c7pW0RmRLlad4HiaJaQ2o=; b=MM8vOpUq6kNzxr402Z3QvaclRyMEtm5YEiVn6oW1cEHEVthIioNOAMAckKOzqpXygpquTo mzYY3JaMh3dPNH19RR/DiN0WoeigkSQRHHddVLjYY7qlSh7OlKTSF9qAHtHpgPob5K/ry7 CTR+r/aSB6mIXO+as8CCl9I9O2neRgE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655741514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=y2ws/QxBRb/M2rDqfKbqG9c7pW0RmRLlad4HiaJaQ2o=; b=KMgmpFONVktmrWp2pFvMgTH1Vuo6rCSX8cJ7Bdn4kSvIZZWjfdPv/CJqqdYl+vdtJ9+PY4 O9wixsKzeERXHYAA== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id A50432C142; Mon, 20 Jun 2022 16:11:53 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 39BFAA0637; Mon, 20 Jun 2022 18:11:53 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 1/8] block: fix default IO priority handling again Date: Mon, 20 Jun 2022 18:11:42 +0200 Message-Id: <20220620161153.11741-1-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220620160726.19798-1-jack@suse.cz> References: <20220620160726.19798-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3263; h=from:subject; bh=PJc+JSJchvfeOzZSCrbFJ2fBI9++bCpTYDxnDWkhbBI=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBisJw/ZmWK9FC6GkB36zwOPYVbpI8w1LLy0yy3ms1c L4WtZ7aJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrCcPwAKCRCcnaoHP2RA2c8sCA DRWzUveYUtr5xuIa3RDYS9zfHlHg2q3Ji1FgaW9fEq6IrzFGwEV7EDb6C/KGgBmbToivmjN+eG0gx7 XVIFkr4375nlrNblLUUmCQeNj0FUF4I2eK8LjjbeWMsJTF2zJ4LjICewAZXG/pCVD7UiqA683yRQND 9M4cZeXnjVkFqmX2KdsbJsp9VeuKOLX3D8uFnzLhEgUqysVja3krQRhAd54jemCqjokMFhy0y6SA5a sUXHwL0G/7M7SDTDKyXQufuw+2kPcRGGxzO4seLP+8lOMHKoucw7YTpkl7ZJY4xsd1JnnGymNaWuRh N8F2rZGqHc2yxCBQrWROKVhQPpuP8Z X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Commit e70344c05995 ("block: fix default IO priority handling") introduced an inconsistency in get_current_ioprio() that tasks without IO context return IOPRIO_DEFAULT priority while tasks with freshly allocated IO context will return 0 (IOPRIO_CLASS_NONE/0) IO priority. Tasks without IO context used to be rare before 5a9d041ba2f6 ("block: move io_context creation into where it's needed") but after this commit they became common because now only BFQ IO scheduler setups task's IO context. Similar inconsistency is there for get_task_ioprio() so this inconsistency is now exposed to userspace and userspace will see different IO priority for tasks operating on devices with BFQ compared to devices without BFQ. Furthemore the changes done by commit e70344c05995 change the behavior when no IO priority is set for BFQ IO scheduler which is also documented in ioprio_set(2) manpage: "If no I/O scheduler has been set for a thread, then by default the I/O priority will follow the CPU nice value (setpriority(2)). In Linux kernels before version 2.6.24, once an I/O priority had been set using ioprio_set(), there was no way to reset the I/O scheduling behavior to the default. Since Linux 2.6.24, specifying ioprio as 0 can be used to reset to the default I/O scheduling behavior." So make sure we default to IOPRIO_CLASS_NONE as used to be the case before commit e70344c05995. Also cleanup alloc_io_context() to explicitely set this IO priority for the allocated IO context to avoid future surprises. Note that we tweak ioprio_best() to maintain ioprio_get(2) behavior and make this commit easily backportable. Fixes: e70344c05995 ("block: fix default IO priority handling") Signed-off-by: Jan Kara Reviewed-by: Damien Le Moal --- block/blk-ioc.c | 2 ++ block/ioprio.c | 4 ++-- include/linux/ioprio.h | 2 +- 3 files changed, 5 insertions(+), 3 deletions(-) diff --git a/block/blk-ioc.c b/block/blk-ioc.c index df9cfe4ca532..63fc02042408 100644 --- a/block/blk-ioc.c +++ b/block/blk-ioc.c @@ -247,6 +247,8 @@ static struct io_context *alloc_io_context(gfp_t gfp_flags, int node) INIT_HLIST_HEAD(&ioc->icq_list); INIT_WORK(&ioc->release_work, ioc_release_fn); #endif + ioc->ioprio = IOPRIO_DEFAULT; + return ioc; } diff --git a/block/ioprio.c b/block/ioprio.c index 2fe068fcaad5..2a34cbca18ae 100644 --- a/block/ioprio.c +++ b/block/ioprio.c @@ -157,9 +157,9 @@ static int get_task_ioprio(struct task_struct *p) int ioprio_best(unsigned short aprio, unsigned short bprio) { if (!ioprio_valid(aprio)) - aprio = IOPRIO_DEFAULT; + aprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM); if (!ioprio_valid(bprio)) - bprio = IOPRIO_DEFAULT; + bprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM); return min(aprio, bprio); } diff --git a/include/linux/ioprio.h b/include/linux/ioprio.h index 3f53bc27a19b..3d088a88f832 100644 --- a/include/linux/ioprio.h +++ b/include/linux/ioprio.h @@ -11,7 +11,7 @@ /* * Default IO priority. */ -#define IOPRIO_DEFAULT IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM) +#define IOPRIO_DEFAULT IOPRIO_PRIO_VALUE(IOPRIO_CLASS_NONE, 0) /* * Check that a priority value has a valid class. From patchwork Mon Jun 20 16:11:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12887885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA067CCA479 for ; Mon, 20 Jun 2022 16:11:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243340AbiFTQL6 (ORCPT ); Mon, 20 Jun 2022 12:11:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242159AbiFTQL5 (ORCPT ); Mon, 20 Jun 2022 12:11:57 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A806205FF for ; Mon, 20 Jun 2022 09:11:55 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 2A81C1F74D; Mon, 20 Jun 2022 16:11:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655741514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WkQ5fnBmFcfHxnZx1ReNjXm4y069ZH0k8lDyjKcQQ1Y=; b=QjiO4PrXPxkLDIrsC6r89oEM4v0+SwNq7QsiSawWKRCKFolFCBbBpWGd8fW2rWXHmKNn8w 1ht9bAWHnYHT9L8/Ms8E4W9trtxtfYbm1/ucfp/w/VyTm2ceBo3oRbqtDJZ3O8VYQq8ovH 3q0js/2rP+jji+SMpNQzfTEGTEo5IIc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655741514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WkQ5fnBmFcfHxnZx1ReNjXm4y069ZH0k8lDyjKcQQ1Y=; b=DWXWj3cEP4mB4itMa/mkLvo1M0jo6d044sCv0cTC166IcudwaoWSlLyUa0TUQtWiwMNAAc KCovaufGjFzswyBQ== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id A4F3E2C141; Mon, 20 Jun 2022 16:11:53 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 3F878A0638; Mon, 20 Jun 2022 18:11:53 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 2/8] block: Return effective IO priority from get_current_ioprio() Date: Mon, 20 Jun 2022 18:11:43 +0200 Message-Id: <20220620161153.11741-2-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220620160726.19798-1-jack@suse.cz> References: <20220620160726.19798-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1110; h=from:subject; bh=D0lCQnDH8VrDiIGIyNdnu6/lqX72S5p5jWumi4VOzGk=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBisJw/gD/OntD7HMqvoCUG6Bp1gOuAFStnAO9I0u4L 1JaFXeuJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrCcPwAKCRCcnaoHP2RA2VFxB/ 9aBY3ZNGoFUEUvI+YpXl1F73mzewmdXg0B7zQVSwzAATH28oOCIr+LpfC6Vhni/Ssx/b6/B48RKklU /gsqdjmnrhJxAIcj0/O7aCs/FmZxGQzKIq53jyoDyxB4wAAzXGG8X3WkAbHFkJ6A0PpuZXTsykEsFQ epphzpOb40F9ah1xUC84KpqkP9Oa9cV0a/3uPGiR3xt/7kAcDC1HEHNEbQAGS6cE+phdMY9EfYREsa Mdi7kicOI6h8qg7MBdwuAuafvKtaH0H8NaTvk4ZqHfiLShf1ZtJ6cn5B7cLlP5HLyCMg5VnLdNPnzb LteryvbPyO1+CsJ4BjLteKvsc0RZFb X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org get_current_ioprio() is used to initialize IO priority of various requests. As such it should be returning the effective IO priority of the task (i.e., reflecting the fact that unset IO priority should get set based on task's CPU priority) so that the conversion is concentrated in one place. Signed-off-by: Jan Kara Reviewed-by: Damien Le Moal --- include/linux/ioprio.h | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/include/linux/ioprio.h b/include/linux/ioprio.h index 3d088a88f832..61ed6bb4998e 100644 --- a/include/linux/ioprio.h +++ b/include/linux/ioprio.h @@ -53,10 +53,17 @@ static inline int task_nice_ioclass(struct task_struct *task) static inline int get_current_ioprio(void) { struct io_context *ioc = current->io_context; + int prio; if (ioc) - return ioc->ioprio; - return IOPRIO_DEFAULT; + prio = ioc->ioprio; + else + prio = IOPRIO_DEFAULT; + + if (IOPRIO_PRIO_CLASS(prio) == IOPRIO_CLASS_NONE) + prio = IOPRIO_PRIO_VALUE(task_nice_ioclass(current), + task_nice_ioprio(current)); + return prio; } /* From patchwork Mon Jun 20 16:11:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12887888 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B6C6C43334 for ; Mon, 20 Jun 2022 16:12:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243497AbiFTQMA (ORCPT ); Mon, 20 Jun 2022 12:12:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37706 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242469AbiFTQL5 (ORCPT ); Mon, 20 Jun 2022 12:11:57 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7366520BCC for ; Mon, 20 Jun 2022 09:11:55 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 32B991F8A4; Mon, 20 Jun 2022 16:11:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655741514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=v7z7sMxKWqNuUnoONWOQLCm36kIZY/iXbJgtfkyS+es=; b=p3UMLOXLRC/7fUqNv8Kc+ui6hAEE77KnJUa96TRrJhl06HFZENQ5sBaUBv/UHh3xutr+Wi nBiRW6J6Psjt7exgKMjYAAPbVDkv8D9JHTns4ynsHm3HVTSyzuu9omqKBORhDs+AjNrHTn Mhi9wEaxOrV0rPZKv8oc+IukWlgz6JQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655741514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=v7z7sMxKWqNuUnoONWOQLCm36kIZY/iXbJgtfkyS+es=; b=fVgsDbhc8OajLHsl6qMrJF/XSZq0KwyizokqSucadfHP/eEy3L12sC7G7NfcRMxpaJe+ZB JXmEBvRFT0+mJXAg== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id AB5FC2C145; Mon, 20 Jun 2022 16:11:53 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 449EDA0639; Mon, 20 Jun 2022 18:11:53 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 3/8] block: Make ioprio_best() static Date: Mon, 20 Jun 2022 18:11:44 +0200 Message-Id: <20220620161153.11741-3-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220620160726.19798-1-jack@suse.cz> References: <20220620160726.19798-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1169; h=from:subject; bh=nzKwe/dsRzaRdBCCbEuxYw4XmmzvyvAPcFHj+sL8ObY=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBisJxAZ21nzu/gTPHMRUkAH9h8G3ab3BsI9KvWcDKd 9ZVbnNmJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrCcQAAKCRCcnaoHP2RA2ZCICA CSMsWE3MW3Z5Cea5uZregJPGHDY1+bQtjEAyazThE2B6z8ERgXKyV3b0YERt4dfoZ8cN8NN8UVtGgo oGlVfIotp0tZb5kQhHcZLpHvo7c5VE0cUGnkB9+3Gmh8ef/P6UjyaqVcEfi/ehP93xLEQ2PcYDMozC 1ScwhLRHFF4zPt8hGu/ooB+ufWCPAwTLG8Y+39/9ki3i8jKEE60U6ZfJkD1KejLj8OOWMIMBGFDxru xXLSAzhQIfNNbE5F1XiMC4Chul0X0J3h5cX3X0EdCEEWG8NoJfP+PubWBCDK/Bzd9h4gWJI7qaooME /J1eEfWQ6n0WPRRzKyCR3QxedJeGJe X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Nobody outside of block/ioprio.c uses it. Signed-off-by: Jan Kara Reviewed-by: Damien Le Moal --- block/ioprio.c | 2 +- include/linux/ioprio.h | 5 ----- 2 files changed, 1 insertion(+), 6 deletions(-) diff --git a/block/ioprio.c b/block/ioprio.c index 2a34cbca18ae..b5cf7339709b 100644 --- a/block/ioprio.c +++ b/block/ioprio.c @@ -154,7 +154,7 @@ static int get_task_ioprio(struct task_struct *p) return ret; } -int ioprio_best(unsigned short aprio, unsigned short bprio) +static int ioprio_best(unsigned short aprio, unsigned short bprio) { if (!ioprio_valid(aprio)) aprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM); diff --git a/include/linux/ioprio.h b/include/linux/ioprio.h index 61ed6bb4998e..519d51fc8902 100644 --- a/include/linux/ioprio.h +++ b/include/linux/ioprio.h @@ -66,11 +66,6 @@ static inline int get_current_ioprio(void) return prio; } -/* - * For inheritance, return the highest of the two given priorities - */ -extern int ioprio_best(unsigned short aprio, unsigned short bprio); - extern int set_task_ioprio(struct task_struct *task, int ioprio); #ifdef CONFIG_BLOCK From patchwork Mon Jun 20 16:11:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12887886 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C04C9C433EF for ; Mon, 20 Jun 2022 16:12:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242145AbiFTQL7 (ORCPT ); Mon, 20 Jun 2022 12:11:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242503AbiFTQL5 (ORCPT ); Mon, 20 Jun 2022 12:11:57 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DF37820BCD for ; Mon, 20 Jun 2022 09:11:55 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 3AFFE1F90B; Mon, 20 Jun 2022 16:11:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655741514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=O1blor4dVx/YleKP6DMl9anfpk9m5dHQbKtzreVHPY0=; b=FdyLjcHF4rQ552Arv1WN+Jm7DzbClqvR1DcHnmyXeo1RF82FAC0AX+CnRd8aNBjQrXgrAv FPxe6untZNQ3jHAcf6ogqItPVldVCOVh5s7z8MOS9rRlVt8xiaJeBfyaPnDpc6/1Vb6I/g g7fP9nDiMP25Xr9ujOjIhlN1TABxZo0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655741514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=O1blor4dVx/YleKP6DMl9anfpk9m5dHQbKtzreVHPY0=; b=soZYGLl77prtMloKS2OT+RhBEKTIFzYl7DtkX98fkJvXdB/xii0pLhJQ+GF+q5f8bLmyy3 +MRIxFrtL+Hsu2DQ== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id B16FB2C146; Mon, 20 Jun 2022 16:11:53 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 4B3A3A063A; Mon, 20 Jun 2022 18:11:53 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 4/8] block: Fix handling of tasks without ioprio in ioprio_get(2) Date: Mon, 20 Jun 2022 18:11:45 +0200 Message-Id: <20220620161153.11741-4-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220620160726.19798-1-jack@suse.cz> References: <20220620160726.19798-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4130; h=from:subject; bh=qs6TSsXEJH7St4+PyWGL2d7x3W3aJJBF3Q4+TOnvplc=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBisJxBeJ2e4V/j5ei30ZKfZZOVA8gx64pjb1wvenqc 6vPcqZKJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrCcQQAKCRCcnaoHP2RA2QFbCA CF+GhxKDrCpF+Ie4nsQ6V1wiuk6bAW0B2E+XxBkiy5jdWWyC+tdxX9oEDRr/zyr67M+9TLUnpKzTTU F9kbMWjhufrT8aj1vLhox5hUvg8ERA+kXfunChSjz/TT2JG/dGT4p8yoT8DvrOX5ubXMhFEfiGufqU 1g+d7nmEhkBbntjfOYiFCWQA1Aqd/j/enF5AYHDXpNwUX8EPmBgsgEGmahhuFs1shAXj5ZAKAKT76W rrITcTC+NovWN4SokH+S3wm6rxNSzQLVQ/4EE8Rc40gHTCvskxPjVppQskFnvgfzJwlfO1SwaSCH/U H6c6vkmOqBwHOcif846Ab1FNwiWdD2 X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org ioprio_get(2) can be asked to return the best IO priority from several tasks (IOPRIO_WHO_PGRP, IOPRIO_WHO_USER). Currently the call treats tasks without set IO priority as having priority IOPRIO_CLASS_BE/IOPRIO_BE_NORM however this does not really reflect the IO priority the task will get (which depends on task's nice value). Fix the code to use the real IO priority task's IO will use. For this we do some factoring out to share the code converting task's CPU priority to IO priority and we also have to modify code for ioprio_get(IOPRIO_WHO_PROCESS) to keep returning IOPRIO_CLASS_NONE priority for tasks without set IO priority as a special case to maintain userspace visible API. Signed-off-by: Jan Kara Reported-by: kernel test robot Reported-by: kernel test robot Reported-by: kernel test robot Reported-by: kernel test robot --- block/ioprio.c | 49 ++++++++++++++++++++++++++++++++++++------ include/linux/ioprio.h | 19 +++------------- 2 files changed, 45 insertions(+), 23 deletions(-) diff --git a/block/ioprio.c b/block/ioprio.c index b5cf7339709b..a4c19ce0de4c 100644 --- a/block/ioprio.c +++ b/block/ioprio.c @@ -138,6 +138,27 @@ SYSCALL_DEFINE3(ioprio_set, int, which, int, who, int, ioprio) return ret; } +/* + * If the task has set an I/O priority, use that. Otherwise, return + * the default I/O priority. + */ +int __get_task_ioprio(struct task_struct *p) +{ + struct io_context *ioc = p->io_context; + int prio; + + if (ioc) + prio = ioc->ioprio; + else + prio = IOPRIO_DEFAULT; + + if (IOPRIO_PRIO_CLASS(prio) == IOPRIO_CLASS_NONE) + prio = IOPRIO_PRIO_VALUE(task_nice_ioclass(p), + task_nice_ioprio(p)); + return prio; +} +EXPORT_SYMBOL_GPL(__get_task_ioprio); + static int get_task_ioprio(struct task_struct *p) { int ret; @@ -145,10 +166,29 @@ static int get_task_ioprio(struct task_struct *p) ret = security_task_getioprio(p); if (ret) goto out; - ret = IOPRIO_DEFAULT; + task_lock(p); + ret = __get_task_ioprio(p); + task_unlock(p); +out: + return ret; +} + +/* + * Return raw IO priority value as set by userspace. We use this for + * ioprio_get(pid, IOPRIO_WHO_PROCESS) so that we keep historical behavior and + * also so that userspace can distinguish unset IO priority (which just gets + * overriden based on task's nice value) from IO priority set to some value. + */ +static int get_task_raw_ioprio(struct task_struct *p) { int ret; + + ret = security_task_getioprio(p); + if (ret) + goto out; task_lock(p); if (p->io_context) ret = p->io_context->ioprio; + else + ret = IOPRIO_DEFAULT; task_unlock(p); out: return ret; @@ -156,11 +196,6 @@ static int get_task_ioprio(struct task_struct *p) static int ioprio_best(unsigned short aprio, unsigned short bprio) { - if (!ioprio_valid(aprio)) - aprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM); - if (!ioprio_valid(bprio)) - bprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM); - return min(aprio, bprio); } @@ -181,7 +216,7 @@ SYSCALL_DEFINE2(ioprio_get, int, which, int, who) else p = find_task_by_vpid(who); if (p) - ret = get_task_ioprio(p); + ret = get_task_raw_ioprio(p); break; case IOPRIO_WHO_PGRP: if (!who) diff --git a/include/linux/ioprio.h b/include/linux/ioprio.h index 519d51fc8902..24e648dc4fb3 100644 --- a/include/linux/ioprio.h +++ b/include/linux/ioprio.h @@ -46,24 +46,11 @@ static inline int task_nice_ioclass(struct task_struct *task) return IOPRIO_CLASS_BE; } -/* - * If the calling process has set an I/O priority, use that. Otherwise, return - * the default I/O priority. - */ +int __get_task_ioprio(struct task_struct *p); + static inline int get_current_ioprio(void) { - struct io_context *ioc = current->io_context; - int prio; - - if (ioc) - prio = ioc->ioprio; - else - prio = IOPRIO_DEFAULT; - - if (IOPRIO_PRIO_CLASS(prio) == IOPRIO_CLASS_NONE) - prio = IOPRIO_PRIO_VALUE(task_nice_ioclass(current), - task_nice_ioprio(current)); - return prio; + return __get_task_ioprio(current); } extern int set_task_ioprio(struct task_struct *task, int ioprio); From patchwork Mon Jun 20 16:11:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12887889 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAEBFC433EF for ; Mon, 20 Jun 2022 16:12:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242469AbiFTQMB (ORCPT ); Mon, 20 Jun 2022 12:12:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37728 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243047AbiFTQL5 (ORCPT ); Mon, 20 Jun 2022 12:11:57 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 181BA20BD1 for ; Mon, 20 Jun 2022 09:11:56 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id A791F1F924; Mon, 20 Jun 2022 16:11:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655741514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DqpItv11q9RFz4JOFsyAcCZ02pSGUPVwPnv55+p1+qw=; b=gurNjIr+YH+R1zUs7WmBIjLE+z32kTw6Ain8Jyk2kKj+Ocsg03vfMLphsGte1Pn7rPmZKs 1n0hcZdNoeGlWOq5kdq6vxyHmsc+ztZaEJG8tBQ0aqQ0DfFZB2pe057TvO90cPH4LGDjTY OAaWIJgo3JB2Gv0RLGPuvkr0eBc7138= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655741514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DqpItv11q9RFz4JOFsyAcCZ02pSGUPVwPnv55+p1+qw=; b=MjS8xhFQV83g01p4ClZa5QUbd4Zu8ux9xQnemCDbN5rZu3Xp3MYtJImjN+Gbcv6bmrfxL3 vJKJ3vS8hmioMjBw== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 9B2052C146; Mon, 20 Jun 2022 16:11:54 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 524D0A063B; Mon, 20 Jun 2022 18:11:53 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 5/8] blk-ioprio: Remove unneeded field Date: Mon, 20 Jun 2022 18:11:46 +0200 Message-Id: <20220620161153.11741-5-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220620160726.19798-1-jack@suse.cz> References: <20220620160726.19798-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1907; h=from:subject; bh=aeKHE3LvhJoWaFM7YpJcwiDk+5O0mTw/oPPCBn0n9AM=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBisJxCzGQLNJUmPklWz6HkDZPusSIiVep4XVNpKKnw dxcwcHeJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrCcQgAKCRCcnaoHP2RA2YLdB/ 97kmPnv4ixFwOZxukUu/5pRg7aKDOsjhz0gvG6F2l8FJ8wGDjRwWp3Va+dr70hKFRV02brD0XkYcXz cn9zz150B9Ea6rHJYXYjHTxyztidRZXpRpqBjFgj4y4GqnCbwPWTDuU9oenVRRh9TNImR6xMHkmfrw 3O9xNG7IyZIF887lFAfbva3sZVRpjOKSpEBFu86IpAEBEAQPBzO6ZyXAxf/P/O36LZH4KJzJOHBu7z tlP5wMhW5hdGQjFlg6pCPhfip23NN2Q2NdprK/G9MEzz8gVj9yCyr2nbvgpfYk4O00+ke5yMcdY7UE 81E9Tm12PiFTZ8AvPQMf9idYW9UHVt X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org blkcg->ioprio_set field is not really useful except for avoiding possibly more expensive checks inside blkcg_ioprio_track(). The check for blkcg->prio_policy being equal to POLICY_NO_CHANGE does the same service so just remove the ioprio_set field and replace the check. Signed-off-by: Jan Kara Reviewed-by: Damien Le Moal --- block/blk-ioprio.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/block/blk-ioprio.c b/block/blk-ioprio.c index 79e797f5d194..3f605583598b 100644 --- a/block/blk-ioprio.c +++ b/block/blk-ioprio.c @@ -62,7 +62,6 @@ struct ioprio_blkg { struct ioprio_blkcg { struct blkcg_policy_data cpd; enum prio_policy prio_policy; - bool prio_set; }; static inline struct ioprio_blkg *pd_to_ioprio(struct blkg_policy_data *pd) @@ -113,7 +112,6 @@ static ssize_t ioprio_set_prio_policy(struct kernfs_open_file *of, char *buf, if (ret < 0) return ret; blkcg->prio_policy = ret; - blkcg->prio_set = true; return nbytes; } @@ -193,16 +191,15 @@ static void blkcg_ioprio_track(struct rq_qos *rqos, struct request *rq, struct ioprio_blkcg *blkcg = ioprio_blkcg_from_bio(bio); u16 prio; - if (!blkcg->prio_set) + if (blkcg->prio_policy == POLICY_NO_CHANGE) return; /* * Except for IOPRIO_CLASS_NONE, higher I/O priority numbers * correspond to a lower priority. Hence, the max_t() below selects * the lower priority of bi_ioprio and the cgroup I/O priority class. - * If the cgroup policy has been set to POLICY_NO_CHANGE == 0, the - * bio I/O priority is not modified. If the bio I/O priority equals - * IOPRIO_CLASS_NONE, the cgroup I/O priority is assigned to the bio. + * If the bio I/O priority equals IOPRIO_CLASS_NONE, the cgroup I/O + * priority is assigned to the bio. */ prio = max_t(u16, bio->bi_ioprio, IOPRIO_PRIO_VALUE(blkcg->prio_policy, 0)); From patchwork Mon Jun 20 16:11:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12887891 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC07DCCA479 for ; Mon, 20 Jun 2022 16:12:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242487AbiFTQME (ORCPT ); Mon, 20 Jun 2022 12:12:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37748 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242047AbiFTQL7 (ORCPT ); Mon, 20 Jun 2022 12:11:59 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 52F53205FF for ; Mon, 20 Jun 2022 09:11:58 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id AC7A51F926; Mon, 20 Jun 2022 16:11:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655741514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gEBpPJCz8/bsKbGbLSHxdhAcO0Jh7iiRRT/PkSSqJ5Q=; b=Z5+g5HcEnfwfX3Ae9bNELWrjwj0IDJhydvGn3c75uMpKYqAO4IKEG/kPJxXwDwKXFDjy67 XLt/zxNgFfZ7L2bvyscZqVA5KP96hYU4wgdfDmkck2ilg/ydWx8Zx86k+wDTwn94Wb9gKD XQUzFfsnFVnZD/Pus2Pfonnjlrt5So4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655741514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gEBpPJCz8/bsKbGbLSHxdhAcO0Jh7iiRRT/PkSSqJ5Q=; b=3Dv+gy/NjkkJuqap0KDcPiOPsDXrziZHHtKYZiSfMNC9S/M3/Y0G7opmQ3kQAeWtEuhZ1T KePXr+b29GrVGxDw== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 9AF662C145; Mon, 20 Jun 2022 16:11:54 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 594C5A063C; Mon, 20 Jun 2022 18:11:53 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 6/8] blk-ioprio: Convert from rqos policy to direct call Date: Mon, 20 Jun 2022 18:11:47 +0200 Message-Id: <20220620161153.11741-6-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220620160726.19798-1-jack@suse.cz> References: <20220620160726.19798-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4547; h=from:subject; bh=ECoFLx4eh//cI4rGpxOBBAOY0ZfPr9sTKA2GSSz6C54=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBisJxDf6HOw5oGkoLu+DfDyVmCr1VTk4ceCf2I3wLl aQKq96qJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrCcQwAKCRCcnaoHP2RA2cXNB/ 4pbIr8/+zDCGa8uFiMbq8dnpnWZ6vVJiBEEteRgSjAwPHdkzrRGm0JD9EylII2u7SlWSWunTqOtnkC QZWz6vQQMMdKSc8RxGrmC5V8GxmAyMLsidGsBABbP1RQ8OdzpCORkS0hKdqX5IiwKJ9/fdbxosJumY WQa3vdGZWO0BEt13fVykKaeJcAbq2tXjiMIYcCWgt7VL86hcpTjd7fDy/gLScvrbdT8eZFMA5ZA87k 7q7IV6f2Dz7edmztH1Jx1+wo80DHJkbVPvzUfy6Hluikd8nuJ3PJSLAJSola2E5cCo6z/LO3axb15a AtwlZlXQj5SXqQmyGD2etOPg17komz X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Convert blk-ioprio handling from a rqos policy to a direct call from blk_mq_submit_bio(). Firstly, blk-ioprio is not a much of a rqos policy anyway, it just needs a hook in bio submission path to set the bio's IO priority. Secondly, the rqos .track hook gets actually called too late for blk-ioprio purposes and introducing a special rqos hook just for blk-ioprio looks even weirder. Signed-off-by: Jan Kara Reviewed-by: Damien Le Moal --- block/blk-cgroup.c | 1 + block/blk-ioprio.c | 50 +++++----------------------------------------- block/blk-ioprio.h | 9 +++++++++ block/blk-mq.c | 8 ++++++++ 4 files changed, 23 insertions(+), 45 deletions(-) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index 764e740b0c0f..6906981563f8 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -1299,6 +1299,7 @@ int blkcg_init_queue(struct request_queue *q) ret = blk_iolatency_init(q); if (ret) { blk_throtl_exit(q); + blk_ioprio_exit(q); goto err_destroy_all; } diff --git a/block/blk-ioprio.c b/block/blk-ioprio.c index 3f605583598b..c00060a02c6e 100644 --- a/block/blk-ioprio.c +++ b/block/blk-ioprio.c @@ -181,17 +181,12 @@ static struct blkcg_policy ioprio_policy = { .pd_free_fn = ioprio_free_pd, }; -struct blk_ioprio { - struct rq_qos rqos; -}; - -static void blkcg_ioprio_track(struct rq_qos *rqos, struct request *rq, - struct bio *bio) +void blkcg_set_ioprio(struct bio *bio) { struct ioprio_blkcg *blkcg = ioprio_blkcg_from_bio(bio); u16 prio; - if (blkcg->prio_policy == POLICY_NO_CHANGE) + if (!blkcg || blkcg->prio_policy == POLICY_NO_CHANGE) return; /* @@ -207,49 +202,14 @@ static void blkcg_ioprio_track(struct rq_qos *rqos, struct request *rq, bio->bi_ioprio = prio; } -static void blkcg_ioprio_exit(struct rq_qos *rqos) +void blk_ioprio_exit(struct request_queue *q) { - struct blk_ioprio *blkioprio_blkg = - container_of(rqos, typeof(*blkioprio_blkg), rqos); - - blkcg_deactivate_policy(rqos->q, &ioprio_policy); - kfree(blkioprio_blkg); + blkcg_deactivate_policy(q, &ioprio_policy); } -static struct rq_qos_ops blkcg_ioprio_ops = { - .track = blkcg_ioprio_track, - .exit = blkcg_ioprio_exit, -}; - int blk_ioprio_init(struct request_queue *q) { - struct blk_ioprio *blkioprio_blkg; - struct rq_qos *rqos; - int ret; - - blkioprio_blkg = kzalloc(sizeof(*blkioprio_blkg), GFP_KERNEL); - if (!blkioprio_blkg) - return -ENOMEM; - - ret = blkcg_activate_policy(q, &ioprio_policy); - if (ret) { - kfree(blkioprio_blkg); - return ret; - } - - rqos = &blkioprio_blkg->rqos; - rqos->id = RQ_QOS_IOPRIO; - rqos->ops = &blkcg_ioprio_ops; - rqos->q = q; - - /* - * Registering the rq-qos policy after activating the blk-cgroup - * policy guarantees that ioprio_blkcg_from_bio(bio) != NULL in the - * rq-qos callbacks. - */ - rq_qos_add(q, rqos); - - return 0; + return blkcg_activate_policy(q, &ioprio_policy); } static int __init ioprio_init(void) diff --git a/block/blk-ioprio.h b/block/blk-ioprio.h index a7785c2f1aea..5a1eb550e178 100644 --- a/block/blk-ioprio.h +++ b/block/blk-ioprio.h @@ -6,14 +6,23 @@ #include struct request_queue; +struct bio; #ifdef CONFIG_BLK_CGROUP_IOPRIO int blk_ioprio_init(struct request_queue *q); +void blk_ioprio_exit(struct request_queue *q); +void blkcg_set_ioprio(struct bio *bio); #else static inline int blk_ioprio_init(struct request_queue *q) { return 0; } +static inline void blk_ioprio_exit(struct request_queue *q) +{ +} +static inline void blkcg_set_ioprio(struct bio *bio) +{ +} #endif #endif /* _BLK_IOPRIO_H_ */ diff --git a/block/blk-mq.c b/block/blk-mq.c index e9bf950983c7..67a7bfa58b7c 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -42,6 +42,7 @@ #include "blk-stat.h" #include "blk-mq-sched.h" #include "blk-rq-qos.h" +#include "blk-ioprio.h" static DEFINE_PER_CPU(struct llist_head, blk_cpu_done); @@ -2790,6 +2791,11 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q, return rq; } +static void bio_set_ioprio(struct bio *bio) +{ + blkcg_set_ioprio(bio); +} + /** * blk_mq_submit_bio - Create and send a request to block device. * @bio: Bio pointer. @@ -2830,6 +2836,8 @@ void blk_mq_submit_bio(struct bio *bio) trace_block_getrq(bio); + bio_set_ioprio(bio); + rq_qos_track(q, rq, bio); blk_mq_bio_to_request(rq, bio, nr_segs); From patchwork Mon Jun 20 16:11:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12887887 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D74FDCCA479 for ; Mon, 20 Jun 2022 16:12:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243015AbiFTQL7 (ORCPT ); Mon, 20 Jun 2022 12:11:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37714 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242487AbiFTQL5 (ORCPT ); Mon, 20 Jun 2022 12:11:57 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EAD6E20BCE for ; Mon, 20 Jun 2022 09:11:55 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id A4F1B21B87; Mon, 20 Jun 2022 16:11:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655741514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=j+EeqhSguiL/f3al2niBNE1UPdRtw82hjLXb6sHZejg=; b=SXjRm6PhN7z8ogbFREcrkzm/MSYzIL3pxyD/NpsA8Fpde9TNMuH5HjYPpFcPNFqrGzD9bf +/IompQcumaqbNazN/47rxPyemyU0jPqaAiDb30jUk5skQrShYk1aLM6RLOLV+prxF337W 1nhlgEp+Joy9ouPt03POWf+sSPIiBtg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655741514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=j+EeqhSguiL/f3al2niBNE1UPdRtw82hjLXb6sHZejg=; b=JCnd9OmeGu9wx8gKMkJFvfLWZntoV+k+usU5RpU96bVjE9XVIVP56PO2ZZR0QqoEJFPCFM zIMMbi9ixZ/nzWDw== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 97FFA2C141; Mon, 20 Jun 2022 16:11:54 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 5F227A063D; Mon, 20 Jun 2022 18:11:53 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 7/8] block: Initialize bio priority earlier Date: Mon, 20 Jun 2022 18:11:48 +0200 Message-Id: <20220620161153.11741-7-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220620160726.19798-1-jack@suse.cz> References: <20220620160726.19798-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=902; h=from:subject; bh=rGyU+5zciSS3AScSd4SNeEGsxv246Ub1JBy28bN6W3M=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBisJxExwClY9U8vYvn2qq0rBlvaWqphh7dAHT9ZZtf PVXXROWJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrCcRAAKCRCcnaoHP2RA2VfcB/ 9j1S3iTl/cCZw6xS3rQUqDIcFwOg6RJ/w5XmmN34R63G/5myT3pof/aLTTHQlsvmms5VtzGza48kmb 5XF32xYvviYmEIPZS5D/2dhfFlw/4I9mGMh9BgpckoKRiUs6PfPeskydJedb94f7ojakkJSj2HMoYO YCCEAHZhMg+imC7rL43Zh+r0jLbxBJsEXswnoOLeiFQy2ds0tIaEfChFujxzXV6s5rQ/x2AjbU/cn6 vw4pIxyp1dnOTsxgLVBh+//HtfK2SZT7GatP3Ya5+/p08SUyl2OTxHARr6lghoFgkiTvrUxIux9IQ8 dM0KgTMRypNrEON0FGTjOotIp26LC0 X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Bio's IO priority needs to be initialized before we try to merge the bio with other bios. Otherwise we could merge bios which would otherwise receive different IO priorities leading to possible QoS issues. Signed-off-by: Jan Kara Reviewed-by: Damien Le Moal --- block/blk-mq.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 67a7bfa58b7c..e17d822e6051 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -2825,6 +2825,8 @@ void blk_mq_submit_bio(struct bio *bio) if (!bio_integrity_prep(bio)) return; + bio_set_ioprio(bio); + rq = blk_mq_get_cached_request(q, plug, &bio, nr_segs); if (!rq) { if (!bio) @@ -2836,8 +2838,6 @@ void blk_mq_submit_bio(struct bio *bio) trace_block_getrq(bio); - bio_set_ioprio(bio); - rq_qos_track(q, rq, bio); blk_mq_bio_to_request(rq, bio, nr_segs); From patchwork Mon Jun 20 16:11:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12887890 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39D3FC43334 for ; Mon, 20 Jun 2022 16:12:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242503AbiFTQMD (ORCPT ); Mon, 20 Jun 2022 12:12:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37726 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243051AbiFTQL5 (ORCPT ); Mon, 20 Jun 2022 12:11:57 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E10120BCF for ; Mon, 20 Jun 2022 09:11:56 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id A75141F913; Mon, 20 Jun 2022 16:11:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655741514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=weAAY5vzMZxLYO8eCbj69OrJkVAZGGP14yJXmPoHuTY=; b=h7sHo5cUGkH3v2m2Lpt5X2qpldXEa8GDwcz7GR3Agb5Dsx5zmU4Y2H/UtfTvi//+82CsvI /idTaccCWtunV5eAJjGFVeQ/WQbeY/RW8rVBOqdnCOUZAOdkvqMOCtLkIUxYQqEICaiDRv NnGYNUp/lx5WT07isxwWnVj/9iD42tM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655741514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=weAAY5vzMZxLYO8eCbj69OrJkVAZGGP14yJXmPoHuTY=; b=c8fVtA9aGYyobCe7fdPHMyX4Gl5YuX2TELnBMzgImY260ehTUK78lvDITCI7CZ5DbxG334 THy621XzSFJUJkDA== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 9A1922C143; Mon, 20 Jun 2022 16:11:54 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 658ACA063E; Mon, 20 Jun 2022 18:11:53 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 8/8] block: Always initialize bio IO priority on submit Date: Mon, 20 Jun 2022 18:11:49 +0200 Message-Id: <20220620161153.11741-8-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220620160726.19798-1-jack@suse.cz> References: <20220620160726.19798-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1013; h=from:subject; bh=qrJSja6cK1OVHuXY8zd4J5m/Hx2yD5nZJYW+nTm82wE=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBisJxF7tg/DFgZrptH3B9sVd5U2hBXDR2dXEahleS3 4cXvwIaJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrCcRQAKCRCcnaoHP2RA2SB5CA CH1bIOQf6wUi6goASTjrQqs5pxHum+YYRO6hIpHg2qWWeYbspu3rizMrer3xAlDrGq6TVton1Z40+E heBC206049MbOOpyse1K18hPWCGMFk4sC8JOGHPxGu1cqfqXpNmWLY+ZrdsSIL44QqXMgKFjGtV/v7 /l6hTj94WF6NV+i8GBalPVR5ouGQffNEaTs3Qte3Gv/wESu5MC7GrmcLF4Wk5IyJWM9NPTtaMis+zH GYp54NXBD5yVjF8vc8GgGzpUafJUiyfPMRJzT6Db7yFaVZPWi5FGKmUVVRYPe11VFg2nmBXte80JF+ jwg7cDvL33GgdyKj2fc6i3D4b5fOhx X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Currently, IO priority set in task's IO context is not reflected in the bio->bi_ioprio for most IO (only io_uring and direct IO set it). This results in odd results where process is submitting some bios with one priority and other bios with a different (unset) priority and due to differing priorities bios cannot be merged. Make sure bio->bi_ioprio is always set on bio submission. Signed-off-by: Jan Kara Reviewed-by: Damien Le Moal --- block/blk-mq.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/blk-mq.c b/block/blk-mq.c index e17d822e6051..7548f8aebea8 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -2793,6 +2793,9 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q, static void bio_set_ioprio(struct bio *bio) { + /* Nobody set ioprio so far? Initialize it based on task's nice value */ + if (IOPRIO_PRIO_CLASS(bio->bi_ioprio) == IOPRIO_CLASS_NONE) + bio->bi_ioprio = get_current_ioprio(); blkcg_set_ioprio(bio); }