From patchwork Tue Jun 21 10:24:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12888986 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82AA3C433EF for ; Tue, 21 Jun 2022 10:25:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241404AbiFUKZF (ORCPT ); Tue, 21 Jun 2022 06:25:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57604 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346086AbiFUKY7 (ORCPT ); Tue, 21 Jun 2022 06:24:59 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6859E2898F; Tue, 21 Jun 2022 03:24:58 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 287E021DE0; Tue, 21 Jun 2022 10:24:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+LL1dq+apRByD46C4qt8iyxS+9kz4FX5965o6Hrr9kI=; b=aHBePNtYp/dfR3sfYIr0qzyQOZiSnLsaIrrQLFFKP0++ROHz4OMWG02IvT8L4YF8DbTUJL 6oRN7cntcTpKeEPakIhoqxZGEqtpHOQbzhWpS/DR6/b/6A8+WivwRf/O4UDFG+wqVpVDSZ 9If7qhn3HvUa3wMvUyPBWyk11A8Nabw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+LL1dq+apRByD46C4qt8iyxS+9kz4FX5965o6Hrr9kI=; b=iwfaTRWJz4JJ7spVJ9vo0wxLmsYxO0e67XZiYuYKgmDAs1OcrF4lTVy94Ulxs5gAYZABAJ WSHvwYSllt6Ng0Bg== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id AEF5F2C141; Tue, 21 Jun 2022 10:24:56 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id CE5C3A03DF; Tue, 21 Jun 2022 12:24:55 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara , stable@vger.kernel.org Subject: [PATCH 1/9] block: fix default IO priority handling again Date: Tue, 21 Jun 2022 12:24:38 +0200 Message-Id: <20220621102455.13183-1-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220621102201.26337-1-jack@suse.cz> References: <20220621102201.26337-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3355; h=from:subject; bh=5Ogre5KU1EfoWt3GA/j6Xv+dEaBz2FnoJscfrVh0sr4=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBisZxmoyWjApEg1smue+DzBMpbA8NBsL+z2EMCkCzG i4BMSB6JATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrGcZgAKCRCcnaoHP2RA2UKJCA C7cQqOQK2/oXPwq5YrSXmvn19b794aXzfOSm35wb8GGxWtJudKYEumlMw1FNY0K/KAAYQXIUN17gw3 Q3uwqP5Qwfvk2cOLdU1J02G7qPDAOcHUAPIP3Lo2XJq2zBAT565Itfe/naUukv/K7Oxfk8hrc2unR8 UcXW8SrsvJ+FO1mNqqJ4gJ5VzPI+/Wr1mfxjSsq6ZEI3/p8x0kj766JyqRHvattwXazcSGPJOu8z2L /RZGoSzRz1HglD6Vs69RcFVZ0CeernhkFryTOqsrwn597jjAgBLlASn1cSmzcVAbm3SdCKzkPCoRbI ATf+4aMhl9vlM7BJLu+7KbqHXQOwU6 X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Commit e70344c05995 ("block: fix default IO priority handling") introduced an inconsistency in get_current_ioprio() that tasks without IO context return IOPRIO_DEFAULT priority while tasks with freshly allocated IO context will return 0 (IOPRIO_CLASS_NONE/0) IO priority. Tasks without IO context used to be rare before 5a9d041ba2f6 ("block: move io_context creation into where it's needed") but after this commit they became common because now only BFQ IO scheduler setups task's IO context. Similar inconsistency is there for get_task_ioprio() so this inconsistency is now exposed to userspace and userspace will see different IO priority for tasks operating on devices with BFQ compared to devices without BFQ. Furthemore the changes done by commit e70344c05995 change the behavior when no IO priority is set for BFQ IO scheduler which is also documented in ioprio_set(2) manpage: "If no I/O scheduler has been set for a thread, then by default the I/O priority will follow the CPU nice value (setpriority(2)). In Linux kernels before version 2.6.24, once an I/O priority had been set using ioprio_set(), there was no way to reset the I/O scheduling behavior to the default. Since Linux 2.6.24, specifying ioprio as 0 can be used to reset to the default I/O scheduling behavior." So make sure we default to IOPRIO_CLASS_NONE as used to be the case before commit e70344c05995. Also cleanup alloc_io_context() to explicitely set this IO priority for the allocated IO context to avoid future surprises. Note that we tweak ioprio_best() to maintain ioprio_get(2) behavior and make this commit easily backportable. CC: stable@vger.kernel.org Fixes: e70344c05995 ("block: fix default IO priority handling") Reviewed-by: Damien Le Moal Signed-off-by: Jan Kara --- block/blk-ioc.c | 2 ++ block/ioprio.c | 4 ++-- include/linux/ioprio.h | 2 +- 3 files changed, 5 insertions(+), 3 deletions(-) diff --git a/block/blk-ioc.c b/block/blk-ioc.c index df9cfe4ca532..63fc02042408 100644 --- a/block/blk-ioc.c +++ b/block/blk-ioc.c @@ -247,6 +247,8 @@ static struct io_context *alloc_io_context(gfp_t gfp_flags, int node) INIT_HLIST_HEAD(&ioc->icq_list); INIT_WORK(&ioc->release_work, ioc_release_fn); #endif + ioc->ioprio = IOPRIO_DEFAULT; + return ioc; } diff --git a/block/ioprio.c b/block/ioprio.c index 2fe068fcaad5..2a34cbca18ae 100644 --- a/block/ioprio.c +++ b/block/ioprio.c @@ -157,9 +157,9 @@ static int get_task_ioprio(struct task_struct *p) int ioprio_best(unsigned short aprio, unsigned short bprio) { if (!ioprio_valid(aprio)) - aprio = IOPRIO_DEFAULT; + aprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM); if (!ioprio_valid(bprio)) - bprio = IOPRIO_DEFAULT; + bprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM); return min(aprio, bprio); } diff --git a/include/linux/ioprio.h b/include/linux/ioprio.h index 3f53bc27a19b..3d088a88f832 100644 --- a/include/linux/ioprio.h +++ b/include/linux/ioprio.h @@ -11,7 +11,7 @@ /* * Default IO priority. */ -#define IOPRIO_DEFAULT IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM) +#define IOPRIO_DEFAULT IOPRIO_PRIO_VALUE(IOPRIO_CLASS_NONE, 0) /* * Check that a priority value has a valid class. From patchwork Tue Jun 21 10:24:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12888979 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2D90C43334 for ; Tue, 21 Jun 2022 10:25:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348483AbiFUKZB (ORCPT ); Tue, 21 Jun 2022 06:25:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57596 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231127AbiFUKY7 (ORCPT ); Tue, 21 Jun 2022 06:24:59 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 636C928726 for ; Tue, 21 Jun 2022 03:24:58 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 2281E21C38; Tue, 21 Jun 2022 10:24:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f113dIywoOVYzhnNFWbFdCjn+SuIZl6WdAtY+FhLYPI=; b=uo4+i2H4JVjMF4eXP1Kd06v8YqYyn5plHgRRl1phVQ/5REGIx5hX/MUqp3HZPAKZAIW5kH m08tNFvSf3iR12IRCa8czMbIWIqompf9KzaRykzN/0+gpQ9Ktre4ARjIpMR8e4h3gl+uaZ +jQi+k+TqgBHRKigxpseLtD6NjK6k2g= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f113dIywoOVYzhnNFWbFdCjn+SuIZl6WdAtY+FhLYPI=; b=nI9mPDh2wCikkmw40AUC8os3rLQoHBT1FUprOdbcsIEn8SDQYdvp3sDTnvYqyeIDPKfLH6 ujyl3TZbMjYm2OCw== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id AF10B2C142; Tue, 21 Jun 2022 10:24:56 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id D6471A0639; Tue, 21 Jun 2022 12:24:55 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 2/9] block: Return effective IO priority from get_current_ioprio() Date: Tue, 21 Jun 2022 12:24:39 +0200 Message-Id: <20220621102455.13183-2-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220621102201.26337-1-jack@suse.cz> References: <20220621102201.26337-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1174; h=from:subject; bh=LQysfpFiMQ3cqmtBoDvv++qBY9sUt5tIGAfQXZyR7vE=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBisZxnTn0JEceDPlbxY87QGISNPTJdYcb3ah4RzceG uqFigauJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrGcZwAKCRCcnaoHP2RA2WCTCA CjLphKtL1aVFh8h0Gz/2Za7iUY/tlID5FDMGW0g/xJLuXtPc48CvXOWaMbWfaIVCUWsYCwUOFFZ2xp m8mAEsqfyRxu0mcoIB9NqTYtCUSkZVzMfiElHE2yQylOkpU66HO9DA/74phCV/Hb6xWgLVEJcxeYG0 R9ss7BTuB1UkqtGshZ6Ux/aVl2CSTz+pDV08qtx8jDvovw5nrI8js5Ee7wNK0Cfh8W3Ss1WVgroht5 L0WOIc8z2A56kqHvAY6RODNkv+VX+epc+zHtHgbV/6J+upP2aypgF0w0HrwQGRFDPJDgVbxIuHl2Wq sJIIWiCaZhYfpk0BvfaJOFsdt904sv X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org get_current_ioprio() is used to initialize IO priority of various requests. As such it should be returning the effective IO priority of the task (i.e., reflecting the fact that unset IO priority should get set based on task's CPU priority) so that the conversion is concentrated in one place. Reviewed-by: Damien Le Moal Signed-off-by: Jan Kara --- include/linux/ioprio.h | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/include/linux/ioprio.h b/include/linux/ioprio.h index 3d088a88f832..61ed6bb4998e 100644 --- a/include/linux/ioprio.h +++ b/include/linux/ioprio.h @@ -53,10 +53,17 @@ static inline int task_nice_ioclass(struct task_struct *task) static inline int get_current_ioprio(void) { struct io_context *ioc = current->io_context; + int prio; if (ioc) - return ioc->ioprio; - return IOPRIO_DEFAULT; + prio = ioc->ioprio; + else + prio = IOPRIO_DEFAULT; + + if (IOPRIO_PRIO_CLASS(prio) == IOPRIO_CLASS_NONE) + prio = IOPRIO_PRIO_VALUE(task_nice_ioclass(current), + task_nice_ioprio(current)); + return prio; } /* From patchwork Tue Jun 21 10:24:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12888985 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99D10CCA480 for ; Tue, 21 Jun 2022 10:25:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346884AbiFUKZF (ORCPT ); Tue, 21 Jun 2022 06:25:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57606 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241404AbiFUKY7 (ORCPT ); Tue, 21 Jun 2022 06:24:59 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6ACFD28990 for ; Tue, 21 Jun 2022 03:24:58 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 2B89F1FB2A; Tue, 21 Jun 2022 10:24:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eEjbwCWVjPB/jpggmlLxEojbRYPZI/YTNH34t2fOtZ0=; b=ggDkfuQIob4HcgllOsZgxTko0EzW7ffQpxLVd8E32P4eS6v11Ve6DQcd9efR1vvDWsZiL5 wVebt4vvnH+qJGdc926kNBHa5xYda6etZHMPR6FlUEpcsy/LrRMNio99XWfOOnCAERZFS0 TdZt9zfkxlh2pU1RlJr64INiXu2auzQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eEjbwCWVjPB/jpggmlLxEojbRYPZI/YTNH34t2fOtZ0=; b=HsY4Cjp9JTMTBgytiVhGaYxxOOHvTxfeDaSryiDWvmRH1zDlpId9CIecUlkRZWHYYhMnVM mtFd/jbRDYRmKeDA== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id D2EE32C145; Tue, 21 Jun 2022 10:24:56 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id DD8B1A063A; Tue, 21 Jun 2022 12:24:55 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 3/9] block: Generalize get_current_ioprio() for any task Date: Tue, 21 Jun 2022 12:24:40 +0200 Message-Id: <20220621102455.13183-3-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220621102201.26337-1-jack@suse.cz> References: <20220621102201.26337-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2467; h=from:subject; bh=9440C2WTkt92Vhj6/OOVq67ucQ4XzuuTM1siEJfq8PQ=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBisZxoMiYKIwisSEU7NzSyNbINJH8//BHMjuVJWiPp oYv6MK6JATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrGcaAAKCRCcnaoHP2RA2WWSB/ wOAdNu13Djeoh/EwLx9qJlwGOw/qZ6e25XK5tTa7LW0okgK5n586yQq05cZwsmfZ56hnbD96zGcOiE qY9fjAUB5wp+XxfAWBf509ESFXUvwdReyqHVvjXp7Ccp5M2EnIdca533xAzEsb/gRauUWyTyQi79ZJ f9OiGAjWbNwx1j2lVJj224QXg0D4l9ThCoL2ZJ9ZWuRW/iAh61DvH++Vu5fTplQHt4+CrtzF/vFm8J JYxL7Fasdq0LFgDj9D9jWuMOUwawJaHGy+RoAfkxlPQGQQzgzrOdpO1hb4zatVlsz+uHOFS7NmrVuW 6JzAN1pd8hwYLrY0kW+i2Iw4VWae8K X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org get_current_ioprio() operates only on current task. We will need the same functionality for other tasks as well. Generalize get_current_ioprio() for that and also move the bulk out of the header file because it is large enough. Signed-off-by: Jan Kara Reviewed-by: Damien Le Moal Reported-by: kernel test robot Reported-by: kernel test robot Reported-by: kernel test robot Reported-by: kernel test robot --- block/ioprio.c | 26 ++++++++++++++++++++++++++ include/linux/ioprio.h | 26 ++++++++++---------------- 2 files changed, 36 insertions(+), 16 deletions(-) diff --git a/block/ioprio.c b/block/ioprio.c index 2a34cbca18ae..c4e3476155a1 100644 --- a/block/ioprio.c +++ b/block/ioprio.c @@ -138,6 +138,32 @@ SYSCALL_DEFINE3(ioprio_set, int, which, int, who, int, ioprio) return ret; } +/* + * If the task has set an I/O priority, use that. Otherwise, return + * the default I/O priority. + * + * Expected to be called for current task or with task_lock() held to keep + * io_context stable. + */ +int __get_task_ioprio(struct task_struct *p) +{ + struct io_context *ioc = p->io_context; + int prio; + + if (p != current) + lockdep_assert_held(&p->alloc_lock); + if (ioc) + prio = ioc->ioprio; + else + prio = IOPRIO_DEFAULT; + + if (IOPRIO_PRIO_CLASS(prio) == IOPRIO_CLASS_NONE) + prio = IOPRIO_PRIO_VALUE(task_nice_ioclass(p), + task_nice_ioprio(p)); + return prio; +} +EXPORT_SYMBOL_GPL(__get_task_ioprio); + static int get_task_ioprio(struct task_struct *p) { int ret; diff --git a/include/linux/ioprio.h b/include/linux/ioprio.h index 61ed6bb4998e..788a8ff57068 100644 --- a/include/linux/ioprio.h +++ b/include/linux/ioprio.h @@ -46,24 +46,18 @@ static inline int task_nice_ioclass(struct task_struct *task) return IOPRIO_CLASS_BE; } -/* - * If the calling process has set an I/O priority, use that. Otherwise, return - * the default I/O priority. - */ -static inline int get_current_ioprio(void) +#ifdef CONFIG_BLOCK +int __get_task_ioprio(struct task_struct *p); +#else +static inline int __get_task_ioprio(int ioprio) { - struct io_context *ioc = current->io_context; - int prio; - - if (ioc) - prio = ioc->ioprio; - else - prio = IOPRIO_DEFAULT; + return IOPRIO_DEFAULT; +} +#endif /* CONFIG_BLOCK */ - if (IOPRIO_PRIO_CLASS(prio) == IOPRIO_CLASS_NONE) - prio = IOPRIO_PRIO_VALUE(task_nice_ioclass(current), - task_nice_ioprio(current)); - return prio; +static inline int get_current_ioprio(void) +{ + return __get_task_ioprio(current); } /* From patchwork Tue Jun 21 10:24:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12888983 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4CA2C43334 for ; Tue, 21 Jun 2022 10:25:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345332AbiFUKZD (ORCPT ); Tue, 21 Jun 2022 06:25:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57602 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346796AbiFUKY7 (ORCPT ); Tue, 21 Jun 2022 06:24:59 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 65CD42898A for ; Tue, 21 Jun 2022 03:24:58 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 25C881FB02; Tue, 21 Jun 2022 10:24:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=neCImrPNIM49vhJQ2ShqPqogntJNWoMZW4WSJ3vzeOI=; b=BSEUQjLIauOjR0k5tzyOtsplLvwsMJf83r2UgmvnxWLMdVdsEuVVl3A44EJmAtnyMO7Dkt Ao/bJPwGHAvXJRU381eAWcrpWuoZqvk0WC6ZfVjmT+j20/ybNkj7ug8IUde02rxEHL0ZJ6 0jxNycRXzjua7xC2pXJlK+KYdll4OkI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=neCImrPNIM49vhJQ2ShqPqogntJNWoMZW4WSJ3vzeOI=; b=WFStd3IyG28jIxJLjmB0XBxoEl1wfXtEw/8+bCHiPOdRhYwhhPdP3Q8eYSvSmn6b2UV4r/ MVYX6M0lcgWylBCw== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id DDD982C146; Tue, 21 Jun 2022 10:24:56 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id E496CA063B; Tue, 21 Jun 2022 12:24:55 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 4/9] block: Make ioprio_best() static Date: Tue, 21 Jun 2022 12:24:41 +0200 Message-Id: <20220621102455.13183-4-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220621102201.26337-1-jack@suse.cz> References: <20220621102201.26337-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1255; h=from:subject; bh=ljYtpqMRICi8VhWJ1c8qfrBUQ6Hh1y2sIbE+Yv5+WBs=; b=owGbwMvMwME4Z+4qdvsUh5uMp9WSGJI2zsn0YesLUy4++b1n9s0rx/6d1GmRfniSsSXKwuX/ZsFY hyLuTkZjFgZGDgZZMUWW1ZEXta/NM+raGqohAzOIlQlkCgMXpwBMZA0TB8N5x5MnMv+4nVBb51W3In F5H5/5O6sDqSkPWW9O3KD09KVmZUNpe57HWm4pp8k3k00tuoNZsh58vmzZoz7XLIzj+oMuI4YTds/e T/fPWNn3buukX61Ktlc0fOzsjvv5r1CSXGEauFs4XvuBeLpyZFWcoluzp9SKihXcgcevxQc2xNiIX1 dp3fhCSf/8vPu8ixeba2lyLI2burnO/ZtVduNGTX5h2QjGnfOU4607QrNcRFoSi/auMVm/8mKCXNjz Dgv+Sy4HRdVnZL7NSF7ZdCh+w581zFM4X7bapnZvenqoUSZQ+1ooq0BL1fElVX4GJuYBGgYGwXbHuo vunlgkY5InbrV8JnOAnyHPxNsA X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Nobody outside of block/ioprio.c uses it. Reviewed-by: Damien Le Moal Signed-off-by: Jan Kara --- block/ioprio.c | 2 +- include/linux/ioprio.h | 5 ----- 2 files changed, 1 insertion(+), 6 deletions(-) diff --git a/block/ioprio.c b/block/ioprio.c index c4e3476155a1..8c46f672a0ba 100644 --- a/block/ioprio.c +++ b/block/ioprio.c @@ -180,7 +180,7 @@ static int get_task_ioprio(struct task_struct *p) return ret; } -int ioprio_best(unsigned short aprio, unsigned short bprio) +static int ioprio_best(unsigned short aprio, unsigned short bprio) { if (!ioprio_valid(aprio)) aprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM); diff --git a/include/linux/ioprio.h b/include/linux/ioprio.h index 788a8ff57068..3ba5804e5770 100644 --- a/include/linux/ioprio.h +++ b/include/linux/ioprio.h @@ -60,11 +60,6 @@ static inline int get_current_ioprio(void) return __get_task_ioprio(current); } -/* - * For inheritance, return the highest of the two given priorities - */ -extern int ioprio_best(unsigned short aprio, unsigned short bprio); - extern int set_task_ioprio(struct task_struct *task, int ioprio); #ifdef CONFIG_BLOCK From patchwork Tue Jun 21 10:24:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12888984 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32AB8CCA473 for ; Tue, 21 Jun 2022 10:25:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348536AbiFUKZE (ORCPT ); Tue, 21 Jun 2022 06:25:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57612 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348446AbiFUKZA (ORCPT ); Tue, 21 Jun 2022 06:25:00 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E1772899E for ; Tue, 21 Jun 2022 03:24:59 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id A1E8721E1F; Tue, 21 Jun 2022 10:24:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9EZYgGkXpA+F+aduRhJXYDAEAbUAiC1vagAru/4lOpU=; b=mfQlpI6wpk+U7jotXxa8JFnMoTBvHfTM9Xqg3J5EFB/EKS6Ckjv7K+rbbs4uAoPdwQAv0p rUcg750/s9vx1RVlBTC5X6/yTHUs/DTMT3o6wunhgLgg2lnEBm+N4WZ6rv9MQgHbfvPWhR 3kAAgEnJr+XYBBkqCj9vnJGenyhneik= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9EZYgGkXpA+F+aduRhJXYDAEAbUAiC1vagAru/4lOpU=; b=byMMXjTX42akqTRIlo5jQK/gWxdi7ZGKN6LNDtv1f7ivKIGC1LvKtjEuj/MA+P6v42t67z ht+vBAWBFsz34NCA== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 913462C14B; Tue, 21 Jun 2022 10:24:57 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id EB5D6A063C; Tue, 21 Jun 2022 12:24:55 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 5/9] block: Fix handling of tasks without ioprio in ioprio_get(2) Date: Tue, 21 Jun 2022 12:24:42 +0200 Message-Id: <20220621102455.13183-5-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220621102201.26337-1-jack@suse.cz> References: <20220621102201.26337-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2371; h=from:subject; bh=lIuUFnJjT8+7FS9MiTSu0lJmXk8YBvbq/vId5g6LMYA=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBisZxp2P/H7Idag4dVwNUJknDcdysxBcUVMt16DOvs jGGsn7eJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrGcaQAKCRCcnaoHP2RA2ZYfCA CXO2wQP8K+MevYWi072RZzf3F49MXYX8kGRTwvJ3WDW4ns5LE/JNHek9RRPLS5h4J7Fc3DJN0qti7h m85DL2ao/68hZ5oSLBE/4vMtbJXLXAL1T53SSKLOq8gcZZEn1/ehsJtAPO4nlVkC+bvW97ByuiEjgA 3RU1bGFZf8Pt4cVOoTKAY7NMaP4vPxN/wpCsgPGvC+t85mTUpcqLc0lvuYcOfjbaZvTtHo/8E5PolP IwDOT8m1g61ak6QvlJ9KDmKcP+R4PdArlr/UfLbV8cOg+IIyO4Dl7UPOR6xPl+Ea0KginRdvyeSoDu nZiIDLoReo+MqGkFt1eHVbyCKUUmN1 X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org ioprio_get(2) can be asked to return the best IO priority from several tasks (IOPRIO_WHO_PGRP, IOPRIO_WHO_USER). Currently the call treats tasks without set IO priority as having priority IOPRIO_CLASS_BE/IOPRIO_BE_NORM however this does not really reflect the IO priority the task will get (which depends on task's nice value). Fix the code to use the real IO priority task's IO will use. We have to modify code for ioprio_get(IOPRIO_WHO_PROCESS) to keep returning IOPRIO_CLASS_NONE priority for tasks without set IO priority as a special case to maintain userspace visible API. Signed-off-by: Jan Kara Reviewed-by: Damien Le Moal --- block/ioprio.c | 30 +++++++++++++++++++++++------- 1 file changed, 23 insertions(+), 7 deletions(-) diff --git a/block/ioprio.c b/block/ioprio.c index 8c46f672a0ba..32a456b45804 100644 --- a/block/ioprio.c +++ b/block/ioprio.c @@ -171,10 +171,31 @@ static int get_task_ioprio(struct task_struct *p) ret = security_task_getioprio(p); if (ret) goto out; - ret = IOPRIO_DEFAULT; + task_lock(p); + ret = __get_task_ioprio(p); + task_unlock(p); +out: + return ret; +} + +/* + * Return raw IO priority value as set by userspace. We use this for + * ioprio_get(pid, IOPRIO_WHO_PROCESS) so that we keep historical behavior and + * also so that userspace can distinguish unset IO priority (which just gets + * overriden based on task's nice value) from IO priority set to some value. + */ +static int get_task_raw_ioprio(struct task_struct *p) +{ + int ret; + + ret = security_task_getioprio(p); + if (ret) + goto out; task_lock(p); if (p->io_context) ret = p->io_context->ioprio; + else + ret = IOPRIO_DEFAULT; task_unlock(p); out: return ret; @@ -182,11 +203,6 @@ static int get_task_ioprio(struct task_struct *p) static int ioprio_best(unsigned short aprio, unsigned short bprio) { - if (!ioprio_valid(aprio)) - aprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM); - if (!ioprio_valid(bprio)) - bprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM); - return min(aprio, bprio); } @@ -207,7 +223,7 @@ SYSCALL_DEFINE2(ioprio_get, int, which, int, who) else p = find_task_by_vpid(who); if (p) - ret = get_task_ioprio(p); + ret = get_task_raw_ioprio(p); break; case IOPRIO_WHO_PGRP: if (!who) From patchwork Tue Jun 21 10:24:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12888981 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CD82CCA482 for ; Tue, 21 Jun 2022 10:25:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232502AbiFUKZC (ORCPT ); Tue, 21 Jun 2022 06:25:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345332AbiFUKZA (ORCPT ); Tue, 21 Jun 2022 06:25:00 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 04E1628992 for ; Tue, 21 Jun 2022 03:24:59 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id A60221FB2D; Tue, 21 Jun 2022 10:24:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ffB8Pj64vekbITtCYJ1AqOBYUgDlfoSBwpNPKYJ4wEY=; b=VFs6lHxPTf2zrrIYfdDyB/GVBlucsr74dYs/aPSzihJ2/zgjnCBPwK5CiScmlfluHP56IC xZih4E9BSjctL2mN633TXEcmhUQ+tR9OEsWP+cRM9k0BlDqESCntjqZ120BbOg/8XdfqyT LqVAz4MQAyTJuaEomPdgP5rDFMBT+sw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ffB8Pj64vekbITtCYJ1AqOBYUgDlfoSBwpNPKYJ4wEY=; b=mCw9tOH5XmbiQ0ysBfVq3L8RQkflVExmIRU/CAIojyskJQt1Y+tS8LGbZM7hfpI3heLgkZ 3QjiemkS8wPNpyAw== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 926342C14F; Tue, 21 Jun 2022 10:24:57 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id F24BFA063D; Tue, 21 Jun 2022 12:24:55 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 6/9] blk-ioprio: Remove unneeded field Date: Tue, 21 Jun 2022 12:24:43 +0200 Message-Id: <20220621102455.13183-6-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220621102201.26337-1-jack@suse.cz> References: <20220621102201.26337-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1971; h=from:subject; bh=BfCOkvsJnWQ3WtVgtAsGeIzxv1lGgzAZRrnzSnIYV4g=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBisZxqCitFMm4vfu5zWXI7jiJnSmNgLHVkIK2+HbMi 0X4+NdiJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrGcagAKCRCcnaoHP2RA2TPZB/ 0bv69/wsrMZDcKYp21+bwdiamWi+gm1Cvn05wBKxbyGFMgeHCzf0azjaYZNIijeSFiy871SEV5269/ qV/Om+1qsgjzZfAB8Volke19mpY3EN5hfOQz06y5fS3iPYrY0uqj0YtSMl8dKs6sm7aCejNIwjr5E6 AV399JlX24jcUhOmzJ7dafS8TtSrsKfLsYjEllr3SLd+P3DI9xlBXZlD/6oM5+4HrvGbGhS/Gw8fxP Y308lmXQF3WjhGa/edGSOeDTAx+SGM9sj2c0iXGyGm7CWXaFh/ymx8hUVd7wvKz82kQzwxEku2sZol hiyLDv1IBX4zMElPkhZnPmFjZQb9Sw X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org blkcg->ioprio_set field is not really useful except for avoiding possibly more expensive checks inside blkcg_ioprio_track(). The check for blkcg->prio_policy being equal to POLICY_NO_CHANGE does the same service so just remove the ioprio_set field and replace the check. Reviewed-by: Damien Le Moal Signed-off-by: Jan Kara --- block/blk-ioprio.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/block/blk-ioprio.c b/block/blk-ioprio.c index 79e797f5d194..3f605583598b 100644 --- a/block/blk-ioprio.c +++ b/block/blk-ioprio.c @@ -62,7 +62,6 @@ struct ioprio_blkg { struct ioprio_blkcg { struct blkcg_policy_data cpd; enum prio_policy prio_policy; - bool prio_set; }; static inline struct ioprio_blkg *pd_to_ioprio(struct blkg_policy_data *pd) @@ -113,7 +112,6 @@ static ssize_t ioprio_set_prio_policy(struct kernfs_open_file *of, char *buf, if (ret < 0) return ret; blkcg->prio_policy = ret; - blkcg->prio_set = true; return nbytes; } @@ -193,16 +191,15 @@ static void blkcg_ioprio_track(struct rq_qos *rqos, struct request *rq, struct ioprio_blkcg *blkcg = ioprio_blkcg_from_bio(bio); u16 prio; - if (!blkcg->prio_set) + if (blkcg->prio_policy == POLICY_NO_CHANGE) return; /* * Except for IOPRIO_CLASS_NONE, higher I/O priority numbers * correspond to a lower priority. Hence, the max_t() below selects * the lower priority of bi_ioprio and the cgroup I/O priority class. - * If the cgroup policy has been set to POLICY_NO_CHANGE == 0, the - * bio I/O priority is not modified. If the bio I/O priority equals - * IOPRIO_CLASS_NONE, the cgroup I/O priority is assigned to the bio. + * If the bio I/O priority equals IOPRIO_CLASS_NONE, the cgroup I/O + * priority is assigned to the bio. */ prio = max_t(u16, bio->bi_ioprio, IOPRIO_PRIO_VALUE(blkcg->prio_policy, 0)); From patchwork Tue Jun 21 10:24:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12888987 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85366CCA480 for ; Tue, 21 Jun 2022 10:25:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348446AbiFUKZG (ORCPT ); Tue, 21 Jun 2022 06:25:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57630 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231127AbiFUKZB (ORCPT ); Tue, 21 Jun 2022 06:25:01 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7FDF28983 for ; Tue, 21 Jun 2022 03:25:00 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id BBFE321E79; Tue, 21 Jun 2022 10:24:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=O8WBv5yuDjiKNHctbcjKS+vyT91fXz9vb1YSfzMnH9k=; b=omvFTfNQf6s9aeK6yrFyvHbjuDQMzMqT9fE3ISSRGAz4NAt8cB24ymHxWgHkTKAImkNXfF YOx18+NiPEiX9R3o905OmajtTe/2UU0StmuizwFed0jmPRrnA84/5wvhDC8UYcxunqYTB+ AwxtERIcrdkqwizkeUj70mwKetMZf34= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=O8WBv5yuDjiKNHctbcjKS+vyT91fXz9vb1YSfzMnH9k=; b=U8i4CvwRhdTwU0lYTdBLCKAIgAVzLBovoFAOpyjqdQ6OHjG8mVo5oafwJhFAS2oBZvQBIe MtRWyhA5Mdy6U7Bw== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 923562C14E; Tue, 21 Jun 2022 10:24:57 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 0570DA063E; Tue, 21 Jun 2022 12:24:56 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 7/9] blk-ioprio: Convert from rqos policy to direct call Date: Tue, 21 Jun 2022 12:24:44 +0200 Message-Id: <20220621102455.13183-7-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220621102201.26337-1-jack@suse.cz> References: <20220621102201.26337-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4609; h=from:subject; bh=CHODQwYjVhwO17QUBAYr8MHIT3js+gfR1lh220Jg6pY=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBisZxr5J9xAdkQpv13TJYJ9zqx8/y7TqtxH+rB7KpT inazx5KJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrGcawAKCRCcnaoHP2RA2Y3sCA DUuCuTN82k+2yvG/TPF8LtlU4o0PI3B8Atr8TXF5s5ZEmHBcnEn7cWYbZj86GZPopONDBgRjMRupCD Oa3NGNEgn1Nx7kkX06m1NCMaY2B9XRr0TejW5QdcEG6W3pFQ432vIde8admIWLir1Lalx4RfumpCVV mPV++A5ZJgcQ/vZzuWWavToR/Mfc2EEIxc7AeMpfW7s58+qu3ClXib2rRZXG6bjqdE6mJwv6lbBRnb 1haWLPZeAC72i7pMQHGShEJYVpnzdMAP7lVE90pe1Hx+ZXP3Kjylcow0VC7rnVe+u1P2LqgNat/INR I9HI0LI25kHH+0A78eKG/26c1NFkG0 X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Convert blk-ioprio handling from a rqos policy to a direct call from blk_mq_submit_bio(). Firstly, blk-ioprio is not much of a rqos policy anyway, it just needs a hook in bio submission path to set the bio's IO priority. Secondly, the rqos .track hook gets actually called too late for blk-ioprio purposes and introducing a special rqos hook just for blk-ioprio looks even weirder. Reviewed-by: Damien Le Moal Signed-off-by: Jan Kara --- block/blk-cgroup.c | 1 + block/blk-ioprio.c | 50 +++++----------------------------------------- block/blk-ioprio.h | 9 +++++++++ block/blk-mq.c | 8 ++++++++ 4 files changed, 23 insertions(+), 45 deletions(-) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index 764e740b0c0f..6906981563f8 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -1299,6 +1299,7 @@ int blkcg_init_queue(struct request_queue *q) ret = blk_iolatency_init(q); if (ret) { blk_throtl_exit(q); + blk_ioprio_exit(q); goto err_destroy_all; } diff --git a/block/blk-ioprio.c b/block/blk-ioprio.c index 3f605583598b..c00060a02c6e 100644 --- a/block/blk-ioprio.c +++ b/block/blk-ioprio.c @@ -181,17 +181,12 @@ static struct blkcg_policy ioprio_policy = { .pd_free_fn = ioprio_free_pd, }; -struct blk_ioprio { - struct rq_qos rqos; -}; - -static void blkcg_ioprio_track(struct rq_qos *rqos, struct request *rq, - struct bio *bio) +void blkcg_set_ioprio(struct bio *bio) { struct ioprio_blkcg *blkcg = ioprio_blkcg_from_bio(bio); u16 prio; - if (blkcg->prio_policy == POLICY_NO_CHANGE) + if (!blkcg || blkcg->prio_policy == POLICY_NO_CHANGE) return; /* @@ -207,49 +202,14 @@ static void blkcg_ioprio_track(struct rq_qos *rqos, struct request *rq, bio->bi_ioprio = prio; } -static void blkcg_ioprio_exit(struct rq_qos *rqos) +void blk_ioprio_exit(struct request_queue *q) { - struct blk_ioprio *blkioprio_blkg = - container_of(rqos, typeof(*blkioprio_blkg), rqos); - - blkcg_deactivate_policy(rqos->q, &ioprio_policy); - kfree(blkioprio_blkg); + blkcg_deactivate_policy(q, &ioprio_policy); } -static struct rq_qos_ops blkcg_ioprio_ops = { - .track = blkcg_ioprio_track, - .exit = blkcg_ioprio_exit, -}; - int blk_ioprio_init(struct request_queue *q) { - struct blk_ioprio *blkioprio_blkg; - struct rq_qos *rqos; - int ret; - - blkioprio_blkg = kzalloc(sizeof(*blkioprio_blkg), GFP_KERNEL); - if (!blkioprio_blkg) - return -ENOMEM; - - ret = blkcg_activate_policy(q, &ioprio_policy); - if (ret) { - kfree(blkioprio_blkg); - return ret; - } - - rqos = &blkioprio_blkg->rqos; - rqos->id = RQ_QOS_IOPRIO; - rqos->ops = &blkcg_ioprio_ops; - rqos->q = q; - - /* - * Registering the rq-qos policy after activating the blk-cgroup - * policy guarantees that ioprio_blkcg_from_bio(bio) != NULL in the - * rq-qos callbacks. - */ - rq_qos_add(q, rqos); - - return 0; + return blkcg_activate_policy(q, &ioprio_policy); } static int __init ioprio_init(void) diff --git a/block/blk-ioprio.h b/block/blk-ioprio.h index a7785c2f1aea..5a1eb550e178 100644 --- a/block/blk-ioprio.h +++ b/block/blk-ioprio.h @@ -6,14 +6,23 @@ #include struct request_queue; +struct bio; #ifdef CONFIG_BLK_CGROUP_IOPRIO int blk_ioprio_init(struct request_queue *q); +void blk_ioprio_exit(struct request_queue *q); +void blkcg_set_ioprio(struct bio *bio); #else static inline int blk_ioprio_init(struct request_queue *q) { return 0; } +static inline void blk_ioprio_exit(struct request_queue *q) +{ +} +static inline void blkcg_set_ioprio(struct bio *bio) +{ +} #endif #endif /* _BLK_IOPRIO_H_ */ diff --git a/block/blk-mq.c b/block/blk-mq.c index e9bf950983c7..67a7bfa58b7c 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -42,6 +42,7 @@ #include "blk-stat.h" #include "blk-mq-sched.h" #include "blk-rq-qos.h" +#include "blk-ioprio.h" static DEFINE_PER_CPU(struct llist_head, blk_cpu_done); @@ -2790,6 +2791,11 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q, return rq; } +static void bio_set_ioprio(struct bio *bio) +{ + blkcg_set_ioprio(bio); +} + /** * blk_mq_submit_bio - Create and send a request to block device. * @bio: Bio pointer. @@ -2830,6 +2836,8 @@ void blk_mq_submit_bio(struct bio *bio) trace_block_getrq(bio); + bio_set_ioprio(bio); + rq_qos_track(q, rq, bio); blk_mq_bio_to_request(rq, bio, nr_segs); From patchwork Tue Jun 21 10:24:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12888982 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F3C0C433EF for ; Tue, 21 Jun 2022 10:25:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346796AbiFUKZD (ORCPT ); Tue, 21 Jun 2022 06:25:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348456AbiFUKZA (ORCPT ); Tue, 21 Jun 2022 06:25:00 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E14028997 for ; Tue, 21 Jun 2022 03:24:59 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 9F4D221E18; Tue, 21 Jun 2022 10:24:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TU+VB9FLVByEd3dua4vGm7KA60JO5fn9ySddRdhTlZk=; b=LHrflqPe7IuETS0m/lpSmvNz7clvubUxcObJQvKT5KPdbxcJxowJUjDi4VXUyvhukKb+rz WFi5uipZCo1GKu0mvuawVHKa+3vEjOZ4aijMCiw1cWYEHCyJbz1Rz/IN/K0xGGnGsi4wqO dh1MNtKFzy1QkKWqqS6q0ZOJvt/I2DE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TU+VB9FLVByEd3dua4vGm7KA60JO5fn9ySddRdhTlZk=; b=pnqbrQ3r+sYE+GbYGNmOdCLHj8Gq4a49u/GK2Klu01mxnPDTq2uVVmJy/bWPZs5nfzoLm0 5x8T8KSpig7QC4AQ== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 907FB2C149; Tue, 21 Jun 2022 10:24:57 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 0D1EDA063F; Tue, 21 Jun 2022 12:24:56 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 8/9] block: Initialize bio priority earlier Date: Tue, 21 Jun 2022 12:24:45 +0200 Message-Id: <20220621102455.13183-8-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220621102201.26337-1-jack@suse.cz> References: <20220621102201.26337-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=966; h=from:subject; bh=ToM3nKfji8UyBGmYo0J4IX46p7CgkI9nwQg5TxlG+ms=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBisZxs0nH7AJLmGTjHh/hmBGnZkV5SUQeHqiNpZWRS BTqzHr6JATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrGcbAAKCRCcnaoHP2RA2cE9B/ 4+LdMvrHkEVisL4mrugY+Cl0R+llEMg4OwwImzSlT3IsNQSGcSIgWXYzwuy/bBVQtNd4+M4z/DpDHl II1COAKK75lRfWdejT7pJcygGkSUSXunyCBLoDM0WL90pBw9Aj96bKD90o0E+kas6o4Q7SplQ61ucT ebKuOGHn+bv4EoPV20uQAOX/BDnoXJSGL80dDbGj6X5Jo9p9p92KDOOZLTrAdq6aQMKMyAv/zxQ05O ELKItcNG4Fgy4e7W/qM4Vbv/dHN/XMDtwuCITIoLMlg0nXV7+6N43p9jsHf2bg4B/XQi6TUQf7x2w6 uya0fYUyTKYTVxqnOCI+uc3c18WLfm X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Bio's IO priority needs to be initialized before we try to merge the bio with other bios. Otherwise we could merge bios which would otherwise receive different IO priorities leading to possible QoS issues. Reviewed-by: Damien Le Moal Signed-off-by: Jan Kara --- block/blk-mq.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 67a7bfa58b7c..e17d822e6051 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -2825,6 +2825,8 @@ void blk_mq_submit_bio(struct bio *bio) if (!bio_integrity_prep(bio)) return; + bio_set_ioprio(bio); + rq = blk_mq_get_cached_request(q, plug, &bio, nr_segs); if (!rq) { if (!bio) @@ -2836,8 +2838,6 @@ void blk_mq_submit_bio(struct bio *bio) trace_block_getrq(bio); - bio_set_ioprio(bio); - rq_qos_track(q, rq, bio); blk_mq_bio_to_request(rq, bio, nr_segs); From patchwork Tue Jun 21 10:24:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12888988 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08364CCA473 for ; Tue, 21 Jun 2022 10:25:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348556AbiFUKZH (ORCPT ); Tue, 21 Jun 2022 06:25:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57628 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348528AbiFUKZB (ORCPT ); Tue, 21 Jun 2022 06:25:01 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7D7328726 for ; Tue, 21 Jun 2022 03:25:00 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id B095521E2A; Tue, 21 Jun 2022 10:24:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4kC6rjFDlsoZbHR5CLGsg9tbqVMXxLLOhXzA+rqhzIU=; b=AhFVhZPxenMh9p2/aqGmnVh7NyyLjRjh6+nWtJdeaz88iNgryCf9pinRLmvwbwddnAb5JX HIQjNX5oqhBV6gefnXcGVfcrG7UQWcvVe6ZccbIPZKi9v13JYQuyyXEV+HdQtGPjuM/ifm 5vXFjz1M32jiziFH59eN3DvQnharQp0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655807097; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4kC6rjFDlsoZbHR5CLGsg9tbqVMXxLLOhXzA+rqhzIU=; b=5Is+O+V3z2Uio1pV5sTJ+VTEZU0D6pE762g+NW58JApAjHjKxWes4r0ptkvHni1yjJpE1w yBqylhDz+/e+xoDQ== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 9B1112C152; Tue, 21 Jun 2022 10:24:57 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id 14303A0640; Tue, 21 Jun 2022 12:24:56 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 9/9] block: Always initialize bio IO priority on submit Date: Tue, 21 Jun 2022 12:24:46 +0200 Message-Id: <20220621102455.13183-9-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220621102201.26337-1-jack@suse.cz> References: <20220621102201.26337-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1077; h=from:subject; bh=LJzShUwYAe4N1tAr4q1jrBbjZR2RONUIkCyA8INg9kM=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBisZxtKkrEjuXUIhwRb0SFwOaiQQsBJGvIZqjb7Gez IEYeUoKJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrGcbQAKCRCcnaoHP2RA2ecGB/ 9K3Kgp1h6Bu8oSzveKILh6N61qDhuateivPoOOESKFXKd/LpdqFYRqSOJAsJ3/cx6CjjhDZxAd3sb4 qIm+JfUsyxKkHdn4TtxKqcw5uszl7nV4ZyNnDI/M0KJ7RvPW4o9mWadWkPnVSaIColADS7ENOiAPIJ 9AQwaf8/OGppYAPZN1qaYtKWLvB5NamAG4MxNncVc8vKpk99h7UGos+yhryXBqz6/chpT4nkO0WeF6 MKSJI7Am5gxfnfXXxgOcfUHCp4Ew++h/Ixnq09p+r08RUYRY32MHlp4VGVozS5b0Ih9O6f3E0XPdul MarnD4ii2lry12Od8e5pjr0J+QWDNU X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Currently, IO priority set in task's IO context is not reflected in the bio->bi_ioprio for most IO (only io_uring and direct IO set it). This results in odd results where process is submitting some bios with one priority and other bios with a different (unset) priority and due to differing priorities bios cannot be merged. Make sure bio->bi_ioprio is always set on bio submission. Reviewed-by: Damien Le Moal Signed-off-by: Jan Kara --- block/blk-mq.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/blk-mq.c b/block/blk-mq.c index e17d822e6051..7548f8aebea8 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -2793,6 +2793,9 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q, static void bio_set_ioprio(struct bio *bio) { + /* Nobody set ioprio so far? Initialize it based on task's nice value */ + if (IOPRIO_PRIO_CLASS(bio->bi_ioprio) == IOPRIO_CLASS_NONE) + bio->bi_ioprio = get_current_ioprio(); blkcg_set_ioprio(bio); }