From patchwork Thu Jun 23 07:48:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12891887 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA64EC43334 for ; Thu, 23 Jun 2022 07:48:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229866AbiFWHsv (ORCPT ); Thu, 23 Jun 2022 03:48:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34968 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231213AbiFWHso (ORCPT ); Thu, 23 Jun 2022 03:48:44 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5940A2BB3D; Thu, 23 Jun 2022 00:48:42 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 171DD1FD39; Thu, 23 Jun 2022 07:48:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GtdTaFjqWCSCe1yaSCiZGssqCXgG5Q79Oqlpz7qVDg4=; b=z3sftmOXa8+4Wp6CcqcekDvq/mXI/k1vhvU697LNBTVyCxrPjC5X1Fo/CAzCcJXCaal8WW EPx2hzPlbwi6EP0vm1mu9bDOXqk+ShpRTXyBzhgbYF3dvW0BdAXe8OoWCYHgLLOQvuA1XK VjkJLVXC3Fcuza7AMfsgF7j5C1drxg4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GtdTaFjqWCSCe1yaSCiZGssqCXgG5Q79Oqlpz7qVDg4=; b=UgRVohwe42EARuwopRiapyQJVfTf/D0LVxYurSvMiDews+z4b+oxS4ySV/HJgr+50TT0B4 HSSP+zODo1YTU3Bw== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 68EA22C14F; Thu, 23 Jun 2022 07:48:35 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id B0DE9A0638; Thu, 23 Jun 2022 09:48:40 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara , stable@vger.kernel.org Subject: [PATCH 1/9] block: fix default IO priority handling again Date: Thu, 23 Jun 2022 09:48:26 +0200 Message-Id: <20220623074840.5960-1-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220623074450.30550-1-jack@suse.cz> References: <20220623074450.30550-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3417; h=from:subject; bh=r9iLhnVzNhOEfGHjVQ85hsfk7zhW/kHTRg4ZxFaWAvY=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBitBrK+weyWxyVqCcz47tvDyJwY53WZgb4OadrTp/U nYQmP1aJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrQaygAKCRCcnaoHP2RA2ZZ/B/ 9Q0tU999QCIlBcqvLmWCGHSZt/sryoccdj9h9ZPNCtK9su6kNTpHpUrB4/2iIZhiM4U4G0ZvvCw5n/ LhROVhnm4RGm2gaausInYd5XM6AmKfTnGIX2Z8w4lRsYbV3HTUx+EDQzzArjw39m09G2H3ql3kF+/S Y6o4wwsesLtrR/2hFvTugrO54ikmUgqEboc2dKJN5XxCfyPz3O7ka8R4HZ7Hs5mQH84tBQLnQTYDlY uwPs4sLDMRb3ug4Qctu32or9+2ZZ/9VsTEzfJ8npQjHByCm3rd5vD7krSix9MOeHK1MBWRN5IKeq10 5q2Pc+dKvr5VV74ZdVYMa6XUVZ9QwD X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Commit e70344c05995 ("block: fix default IO priority handling") introduced an inconsistency in get_current_ioprio() that tasks without IO context return IOPRIO_DEFAULT priority while tasks with freshly allocated IO context will return 0 (IOPRIO_CLASS_NONE/0) IO priority. Tasks without IO context used to be rare before 5a9d041ba2f6 ("block: move io_context creation into where it's needed") but after this commit they became common because now only BFQ IO scheduler setups task's IO context. Similar inconsistency is there for get_task_ioprio() so this inconsistency is now exposed to userspace and userspace will see different IO priority for tasks operating on devices with BFQ compared to devices without BFQ. Furthemore the changes done by commit e70344c05995 change the behavior when no IO priority is set for BFQ IO scheduler which is also documented in ioprio_set(2) manpage: "If no I/O scheduler has been set for a thread, then by default the I/O priority will follow the CPU nice value (setpriority(2)). In Linux kernels before version 2.6.24, once an I/O priority had been set using ioprio_set(), there was no way to reset the I/O scheduling behavior to the default. Since Linux 2.6.24, specifying ioprio as 0 can be used to reset to the default I/O scheduling behavior." So make sure we default to IOPRIO_CLASS_NONE as used to be the case before commit e70344c05995. Also cleanup alloc_io_context() to explicitely set this IO priority for the allocated IO context to avoid future surprises. Note that we tweak ioprio_best() to maintain ioprio_get(2) behavior and make this commit easily backportable. CC: stable@vger.kernel.org Fixes: e70344c05995 ("block: fix default IO priority handling") Reviewed-by: Damien Le Moal Tested-by: Damien Le Moal Signed-off-by: Jan Kara Reviewed-by: Christoph Hellwig --- block/blk-ioc.c | 2 ++ block/ioprio.c | 4 ++-- include/linux/ioprio.h | 2 +- 3 files changed, 5 insertions(+), 3 deletions(-) diff --git a/block/blk-ioc.c b/block/blk-ioc.c index df9cfe4ca532..63fc02042408 100644 --- a/block/blk-ioc.c +++ b/block/blk-ioc.c @@ -247,6 +247,8 @@ static struct io_context *alloc_io_context(gfp_t gfp_flags, int node) INIT_HLIST_HEAD(&ioc->icq_list); INIT_WORK(&ioc->release_work, ioc_release_fn); #endif + ioc->ioprio = IOPRIO_DEFAULT; + return ioc; } diff --git a/block/ioprio.c b/block/ioprio.c index 2fe068fcaad5..2a34cbca18ae 100644 --- a/block/ioprio.c +++ b/block/ioprio.c @@ -157,9 +157,9 @@ static int get_task_ioprio(struct task_struct *p) int ioprio_best(unsigned short aprio, unsigned short bprio) { if (!ioprio_valid(aprio)) - aprio = IOPRIO_DEFAULT; + aprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM); if (!ioprio_valid(bprio)) - bprio = IOPRIO_DEFAULT; + bprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM); return min(aprio, bprio); } diff --git a/include/linux/ioprio.h b/include/linux/ioprio.h index 3f53bc27a19b..3d088a88f832 100644 --- a/include/linux/ioprio.h +++ b/include/linux/ioprio.h @@ -11,7 +11,7 @@ /* * Default IO priority. */ -#define IOPRIO_DEFAULT IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM) +#define IOPRIO_DEFAULT IOPRIO_PRIO_VALUE(IOPRIO_CLASS_NONE, 0) /* * Check that a priority value has a valid class. From patchwork Thu Jun 23 07:48:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12891888 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 135B7CCA481 for ; Thu, 23 Jun 2022 07:48:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229940AbiFWHsx (ORCPT ); Thu, 23 Jun 2022 03:48:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34966 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231215AbiFWHso (ORCPT ); Thu, 23 Jun 2022 03:48:44 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B1B12C640 for ; Thu, 23 Jun 2022 00:48:42 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 17D3A21D3D; Thu, 23 Jun 2022 07:48:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FDV83hsVfY2ZNIyLWdNXdAzjSRvhop8S8h3xh9/rThg=; b=WbyaFSgTRHXQaWtFeacmi7INA2KFP5125Wf4JBdkbjq7JzH9DG91A2QTJ/zdIk6eOR+Zbh q3POb9rY8mxjYle2SnK+QDY75pqQx5r9n6ljNiMJ1pO40/E3tZ+GTdm0rH9+gD1fN/0G16 2vXh0Gnu2iNFzlIxshOt9W6uG2iPVj8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FDV83hsVfY2ZNIyLWdNXdAzjSRvhop8S8h3xh9/rThg=; b=aRX8U2z/YXMb671/z/uVuYlVCo7b8MzVIVvZmC308NPrFRO60LMBZxdtidT9PcWmowxneR dqqf9I2CRqOdcbAQ== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 6E8DA2C15C; Thu, 23 Jun 2022 07:48:35 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id B77A2A0639; Thu, 23 Jun 2022 09:48:40 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 2/9] block: Return effective IO priority from get_current_ioprio() Date: Thu, 23 Jun 2022 09:48:27 +0200 Message-Id: <20220623074840.5960-2-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220623074450.30550-1-jack@suse.cz> References: <20220623074450.30550-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1236; h=from:subject; bh=+1Tzs3ltee4pP0iXIHD+6UM/ZVl4By/ycGmxzX3amRQ=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBitBrL8ZtxsTisTWjL9RIxw38tHqKfJEe+WtHgcdns guTTFumJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrQaywAKCRCcnaoHP2RA2YBSB/ 97prQjVmo/UalKzhkgv5/5J7f5dTX3mku66ZOJwH4mUFKdnVvJk9XroHFxSWt8iCIiHrJveU3tA3/g BMKwOXCK8Cmxs+tH41j3xjSy5HIMiZuVUdgJ5iW5FJGum53WPQbq35oY8fAXwqcSpB3OXq0Lbpq10R OUsFXX/lGzmEPPZeDQlQLgkEluVmKLasr2wlg2rMFnzW8xdfBa0gyTZHFX97HfgtgrQA1dSW1IY0k5 O7hMCHbm9zlUy/u51APbK1qnFzGfGbATHlyZnr5wS5hX4mhvUzDmK1dgYY0Ehu9sGt9/xTkIh1IPL1 qX59xkYG76R1OcUMNoNjA9wf3v2FVH X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org get_current_ioprio() is used to initialize IO priority of various requests. As such it should be returning the effective IO priority of the task (i.e., reflecting the fact that unset IO priority should get set based on task's CPU priority) so that the conversion is concentrated in one place. Reviewed-by: Damien Le Moal Tested-by: Damien Le Moal Signed-off-by: Jan Kara Reviewed-by: Christoph Hellwig --- include/linux/ioprio.h | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/include/linux/ioprio.h b/include/linux/ioprio.h index 3d088a88f832..61ed6bb4998e 100644 --- a/include/linux/ioprio.h +++ b/include/linux/ioprio.h @@ -53,10 +53,17 @@ static inline int task_nice_ioclass(struct task_struct *task) static inline int get_current_ioprio(void) { struct io_context *ioc = current->io_context; + int prio; if (ioc) - return ioc->ioprio; - return IOPRIO_DEFAULT; + prio = ioc->ioprio; + else + prio = IOPRIO_DEFAULT; + + if (IOPRIO_PRIO_CLASS(prio) == IOPRIO_CLASS_NONE) + prio = IOPRIO_PRIO_VALUE(task_nice_ioclass(current), + task_nice_ioprio(current)); + return prio; } /* From patchwork Thu Jun 23 07:48:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12891895 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A02FC43334 for ; Thu, 23 Jun 2022 07:49:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229686AbiFWHtA (ORCPT ); Thu, 23 Jun 2022 03:49:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34964 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231208AbiFWHso (ORCPT ); Thu, 23 Jun 2022 03:48:44 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 58E452BB13 for ; Thu, 23 Jun 2022 00:48:42 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 191261FD3D; Thu, 23 Jun 2022 07:48:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4Fe7KrmHSZpLPz9yJPwVaEgjCjeMQ+t/dg6ntxma6pE=; b=rrJQ5vVkRFmA2g+Vak7spCALqHy3PMyYpj/KCufIT3JaNE9rtdi6bShFJut5ptIG1E89OM IfbXqpZzIkiXZxHWokJ+EoMgdnped2ItUI8S/0n4D4U+RpGvYL4WzB7qGEXhYKPaV32xGE MKhmCyrjSmsh9TxSg7OloNlSuzH5lxc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4Fe7KrmHSZpLPz9yJPwVaEgjCjeMQ+t/dg6ntxma6pE=; b=mSYPJrtzMHLyCwGBAHDcqGkHNz2n+WcSoTnXQblD5a3ZZfndIvNCEE1X4T0rvJ6lQYMt2p Ww2UILlfIFHBjECw== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 6FD7B2C16B; Thu, 23 Jun 2022 07:48:35 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id BE02DA063A; Thu, 23 Jun 2022 09:48:40 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 3/9] block: Generalize get_current_ioprio() for any task Date: Thu, 23 Jun 2022 09:48:28 +0200 Message-Id: <20220623074840.5960-3-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220623074450.30550-1-jack@suse.cz> References: <20220623074450.30550-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2604; h=from:subject; bh=nnEJuXFVsjIzIcp2wWmkBloBi4x4LTLaOeRvPsp51v8=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBitBrMr/1fdXsdLendb3WGbhiZPjMfKbhCF/13DeyJ +BUIk6WJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrQazAAKCRCcnaoHP2RA2RPRB/ 4kNNFm3u44aGb0Dqb5GvWfUjJWJgzZ3D0fBi4cwHZjb2BWHHVN2rfr484rL0UazdCJ8GnvBFMY/2IF qbbrq4X5XSMlUqWWfjZbcBpE1knQUyYIzZOubeicTu5LO9Jc2UR1NF2/Li3XOepethsg/hcfJx9dGe xl0Yx0O1DGqAt5BLItQRelft+A+JvvtdnzYwZP09C53GgWkJEB/RJfAPe3p6L8ZU+zzhZ0ZtZVsYjA wit0Ka+/Px9QvIjqLMa3vwj7kMkmoyAyNKgMz4Kt7sALetw1XnkqhuGJhhML2BO5/ORPj7NEs/P2ev up/8dAjFBkqTmFN8cf1Ghik7GGAisH X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org get_current_ioprio() operates only on current task. We will need the same functionality for other tasks as well. Generalize get_current_ioprio() for that and also move the bulk out of the header file because it is large enough. Reviewed-by: Damien Le Moal Tested-by: Damien Le Moal Signed-off-by: Jan Kara Reviewed-by: Christoph Hellwig --- block/ioprio.c | 26 ++++++++++++++++++++++++++ include/linux/ioprio.h | 26 ++++++++++---------------- 2 files changed, 36 insertions(+), 16 deletions(-) diff --git a/block/ioprio.c b/block/ioprio.c index 2a34cbca18ae..c4e3476155a1 100644 --- a/block/ioprio.c +++ b/block/ioprio.c @@ -138,6 +138,32 @@ SYSCALL_DEFINE3(ioprio_set, int, which, int, who, int, ioprio) return ret; } +/* + * If the task has set an I/O priority, use that. Otherwise, return + * the default I/O priority. + * + * Expected to be called for current task or with task_lock() held to keep + * io_context stable. + */ +int __get_task_ioprio(struct task_struct *p) +{ + struct io_context *ioc = p->io_context; + int prio; + + if (p != current) + lockdep_assert_held(&p->alloc_lock); + if (ioc) + prio = ioc->ioprio; + else + prio = IOPRIO_DEFAULT; + + if (IOPRIO_PRIO_CLASS(prio) == IOPRIO_CLASS_NONE) + prio = IOPRIO_PRIO_VALUE(task_nice_ioclass(p), + task_nice_ioprio(p)); + return prio; +} +EXPORT_SYMBOL_GPL(__get_task_ioprio); + static int get_task_ioprio(struct task_struct *p) { int ret; diff --git a/include/linux/ioprio.h b/include/linux/ioprio.h index 61ed6bb4998e..9752cf4a9c7c 100644 --- a/include/linux/ioprio.h +++ b/include/linux/ioprio.h @@ -46,24 +46,18 @@ static inline int task_nice_ioclass(struct task_struct *task) return IOPRIO_CLASS_BE; } -/* - * If the calling process has set an I/O priority, use that. Otherwise, return - * the default I/O priority. - */ -static inline int get_current_ioprio(void) +#ifdef CONFIG_BLOCK +int __get_task_ioprio(struct task_struct *p); +#else +static inline int __get_task_ioprio(struct task_struct *p) { - struct io_context *ioc = current->io_context; - int prio; - - if (ioc) - prio = ioc->ioprio; - else - prio = IOPRIO_DEFAULT; + return IOPRIO_DEFAULT; +} +#endif /* CONFIG_BLOCK */ - if (IOPRIO_PRIO_CLASS(prio) == IOPRIO_CLASS_NONE) - prio = IOPRIO_PRIO_VALUE(task_nice_ioclass(current), - task_nice_ioprio(current)); - return prio; +static inline int get_current_ioprio(void) +{ + return __get_task_ioprio(current); } /* From patchwork Thu Jun 23 07:48:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12891896 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C31BCCA47C for ; Thu, 23 Jun 2022 07:51:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230003AbiFWHvU (ORCPT ); Thu, 23 Jun 2022 03:51:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34970 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231210AbiFWHso (ORCPT ); Thu, 23 Jun 2022 03:48:44 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6C77A2C643 for ; Thu, 23 Jun 2022 00:48:42 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 2BB381FD42; Thu, 23 Jun 2022 07:48:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XShOVy+XtTozsCdsQOdsSaoQ9uJ1HILWluwpZp3vhJw=; b=J3kZj9X7xOhjp8bKn+Bx7gZssjvi54W35C6iUwMtPG362m/evCNXlIO2OXv5tNMi6AJyRf HcQHLuxZR2cXfdtkhfr/QW0AfgBsIN/akCvE3GC1bOz7XJt4Jy2OVcnsQ0qYY9VRxlA3u0 MNcEylGC4Zin3pXyN5Aq/SrUGAn1ZNs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XShOVy+XtTozsCdsQOdsSaoQ9uJ1HILWluwpZp3vhJw=; b=b/oqteHPoDStGH/ewpf5UB0wDp3PwP7QoNozVoaknmxYei/SX1Eacs9Oc2qWZfeIHkYPeM PDflX/tfn6bPkGDA== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 7F6782C16E; Thu, 23 Jun 2022 07:48:35 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id C538DA063B; Thu, 23 Jun 2022 09:48:40 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 4/9] block: Make ioprio_best() static Date: Thu, 23 Jun 2022 09:48:29 +0200 Message-Id: <20220623074840.5960-4-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220623074450.30550-1-jack@suse.cz> References: <20220623074450.30550-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1317; h=from:subject; bh=Y2FBsRgMq5JsaN1zwMeKVun8U5TAAsYgfzfvXCApjQY=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBitBrMtdOK2pJ+9C1oyvZHl0+uQkWDvGzKY/1cBJgs eQq1n0CJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrQazAAKCRCcnaoHP2RA2cTyB/ 0dFyI9r/nCa5zybam6dnIfz9Y8UIc/znD79w/uop+iKCEly2memfaDjaFdKAomTWzMyPP8nWgcjP/S bw6QvoFFrsZQ7gTXx9BK+8GeBVEXHJq/FhIuHXYczRxLuuDseVJgkCM545p3oxQM5R2FRzk+f8B8o/ 9CZcAV4iuOzGNV3RblRTgHD5KoZgkadoWuRm5JAeDhXQy2inpwyWGjZYgGciLCwc2Bcj1LLLx1gPsN Zf6OXdyzexuSMcEPzPfhL3Cb3wa8hfCyHzIGilouTdVYLqtziO7mxTutF3JSLpgvbszH/ZfgLakc7E KPS2ib6KRIzA0dd+CWLwiZEQJXISeP X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Nobody outside of block/ioprio.c uses it. Reviewed-by: Damien Le Moal Tested-by: Damien Le Moal Signed-off-by: Jan Kara Reviewed-by: Christoph Hellwig --- block/ioprio.c | 2 +- include/linux/ioprio.h | 5 ----- 2 files changed, 1 insertion(+), 6 deletions(-) diff --git a/block/ioprio.c b/block/ioprio.c index c4e3476155a1..8c46f672a0ba 100644 --- a/block/ioprio.c +++ b/block/ioprio.c @@ -180,7 +180,7 @@ static int get_task_ioprio(struct task_struct *p) return ret; } -int ioprio_best(unsigned short aprio, unsigned short bprio) +static int ioprio_best(unsigned short aprio, unsigned short bprio) { if (!ioprio_valid(aprio)) aprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM); diff --git a/include/linux/ioprio.h b/include/linux/ioprio.h index 9752cf4a9c7c..7578d4f6a969 100644 --- a/include/linux/ioprio.h +++ b/include/linux/ioprio.h @@ -60,11 +60,6 @@ static inline int get_current_ioprio(void) return __get_task_ioprio(current); } -/* - * For inheritance, return the highest of the two given priorities - */ -extern int ioprio_best(unsigned short aprio, unsigned short bprio); - extern int set_task_ioprio(struct task_struct *task, int ioprio); #ifdef CONFIG_BLOCK From patchwork Thu Jun 23 07:48:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12891891 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 283EECCA47C for ; Thu, 23 Jun 2022 07:48:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230033AbiFWHsz (ORCPT ); Thu, 23 Jun 2022 03:48:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231229AbiFWHsp (ORCPT ); Thu, 23 Jun 2022 03:48:45 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8812E2CE20 for ; Thu, 23 Jun 2022 00:48:42 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 52BB621D3F; Thu, 23 Jun 2022 07:48:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Sai0gSNhCOOCwT4p71673N36tYVieClulJBCa4VuXMw=; b=k01duLq6QjRpaxN3IFdcHRkKxnIrmnTaXfbyNa4Ntr/x7MF5RObkpWABIewNsZPQsCqqP9 uvKERR7gcRjxB86Hk5xfvO/WFwZtDUjuaePCN9h8EY9+/2IeRHJLr9++AJ1oVvooeENerZ yFTjbt4LsyMbNhtE16A97ux+rrAb9yw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Sai0gSNhCOOCwT4p71673N36tYVieClulJBCa4VuXMw=; b=oPGZJdD7M2vjPEBl/1+JpDJjzcB3a8cwZgp3G/yxDpLPaZfe21n4+NVux70Qw2kuUbmv6z lv9jcKIf2wP8I8Aw== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id A77232C15C; Thu, 23 Jun 2022 07:48:35 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id CBC51A063C; Thu, 23 Jun 2022 09:48:40 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 5/9] block: Fix handling of tasks without ioprio in ioprio_get(2) Date: Thu, 23 Jun 2022 09:48:30 +0200 Message-Id: <20220623074840.5960-5-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220623074450.30550-1-jack@suse.cz> References: <20220623074450.30550-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2497; h=from:subject; bh=bkmCruvw4Mbu2TMSHVoHVwTNJClRHpULvxS2rdsetms=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBitBrN640MSQDobxOglOTA9H26GCW8L5RH6J6gVOKF b6K7ckeJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrQazQAKCRCcnaoHP2RA2bQbCA C6qBgQBva9K4iAF+uB/TDNW67QPkrnaxPzU8E0tdEdg6rRGouyhwuwXf50GAsyMHiSHC2BMpnDMARL wH4Yv0T6kwEGRx7VRTpfrFjlWPaawSLCzoDAMkVXL3hDZSlPw0fP0Ldj46JqTWp4rZAEGB8ZOJ4qKp Ven/kdO7nugKJxk7jguXA/GwyUlcvh9i2ej/uQ3dK3KvE2LsD8NOyzjtxVacC1r5C/O6Zin5sFWJrC gOph1adLh8fPcJRpEHogHbKPXuLxbyfDCWvwZCIq5MmwA4esuYFG6F3o12UXvDd2/Rfn0l/oT9ay10 tqIt8ovHXpE1kvdkKZhwpNJAGoAlc7 X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org ioprio_get(2) can be asked to return the best IO priority from several tasks (IOPRIO_WHO_PGRP, IOPRIO_WHO_USER). Currently the call treats tasks without set IO priority as having priority IOPRIO_CLASS_BE/IOPRIO_BE_NORM however this does not really reflect the IO priority the task will get (which depends on task's nice value). Fix the code to use the real IO priority task's IO will use. We have to modify code for ioprio_get(IOPRIO_WHO_PROCESS) to keep returning IOPRIO_CLASS_NONE priority for tasks without set IO priority as a special case to maintain userspace visible API. Reviewed-by: Damien Le Moal Tested-by: Damien Le Moal Signed-off-by: Jan Kara --- block/ioprio.c | 30 +++++++++++++++++++++++------- 1 file changed, 23 insertions(+), 7 deletions(-) diff --git a/block/ioprio.c b/block/ioprio.c index 8c46f672a0ba..32a456b45804 100644 --- a/block/ioprio.c +++ b/block/ioprio.c @@ -171,10 +171,31 @@ static int get_task_ioprio(struct task_struct *p) ret = security_task_getioprio(p); if (ret) goto out; - ret = IOPRIO_DEFAULT; + task_lock(p); + ret = __get_task_ioprio(p); + task_unlock(p); +out: + return ret; +} + +/* + * Return raw IO priority value as set by userspace. We use this for + * ioprio_get(pid, IOPRIO_WHO_PROCESS) so that we keep historical behavior and + * also so that userspace can distinguish unset IO priority (which just gets + * overriden based on task's nice value) from IO priority set to some value. + */ +static int get_task_raw_ioprio(struct task_struct *p) +{ + int ret; + + ret = security_task_getioprio(p); + if (ret) + goto out; task_lock(p); if (p->io_context) ret = p->io_context->ioprio; + else + ret = IOPRIO_DEFAULT; task_unlock(p); out: return ret; @@ -182,11 +203,6 @@ static int get_task_ioprio(struct task_struct *p) static int ioprio_best(unsigned short aprio, unsigned short bprio) { - if (!ioprio_valid(aprio)) - aprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM); - if (!ioprio_valid(bprio)) - bprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_BE_NORM); - return min(aprio, bprio); } @@ -207,7 +223,7 @@ SYSCALL_DEFINE2(ioprio_get, int, which, int, who) else p = find_task_by_vpid(who); if (p) - ret = get_task_ioprio(p); + ret = get_task_raw_ioprio(p); break; case IOPRIO_WHO_PGRP: if (!who) From patchwork Thu Jun 23 07:48:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12891892 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1CDB8C433EF for ; Thu, 23 Jun 2022 07:48:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229983AbiFWHs5 (ORCPT ); Thu, 23 Jun 2022 03:48:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34520 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231223AbiFWHsp (ORCPT ); Thu, 23 Jun 2022 03:48:45 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 87DBF2CE0E for ; Thu, 23 Jun 2022 00:48:42 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 4DE0121D3E; Thu, 23 Jun 2022 07:48:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=02jsHGCYECcrORsmJgplcrBdxZZ3tsQTZklav0bRcY8=; b=gvbLHXnVJuttfvRVnDmySixyTqnIDwqSVarjdUJDNZBDdh8sBs+8s1ZoZfv5KP1Awva41k aRF1NzO8jD/nkOfeOrU98B73EKaY4Ajv9ZBgrJYdrvQU/pLdZCbvYK5LlEgfjP3LFEhDTq rZgZ8x1tWN50QZqAFJiXYncNCdfo4os= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=02jsHGCYECcrORsmJgplcrBdxZZ3tsQTZklav0bRcY8=; b=6+v0bFlKWqCgQqoblYsS4sX6d7KOqUopCiKPrw5PN1kbSe9TZbMTEE3POIEuWydDxzOvKj Th5JnDZP52bfPNCg== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id A5EEE2C145; Thu, 23 Jun 2022 07:48:35 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id D2BA3A063D; Thu, 23 Jun 2022 09:48:40 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 6/9] blk-ioprio: Remove unneeded field Date: Thu, 23 Jun 2022 09:48:31 +0200 Message-Id: <20220623074840.5960-6-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220623074450.30550-1-jack@suse.cz> References: <20220623074450.30550-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2033; h=from:subject; bh=TvQ37LVU1nkBVVpWfKskESMfETYuFiKG2oEvfs86ANI=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBitBrOHVYqsam6Fzw+rQ9gi4PbcqT1FMajDyGOaYFO R0tBLkCJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrQazgAKCRCcnaoHP2RA2bEZB/ 0XTHjIp9dJsjw5+THTv1StASlcGA/z6WyIM6BUdBHRLvZ4Ukex91v2GeYoZvH6bebHSlubO6oP2wzx cjwcSD6F2+h+1JMeGOHvSaDwyW0YSssOgd9qzuaQY0eHsHNb32m2Cic70SMVAvBbMwkjn6EeWrnKL+ J4c8B+ueicIGQT8B/eiMRFw0q0EJzlTrKA3aull1o7GgUN+cu6QQBsqCZtKdjn/lPDCyPF2drxX7Ip OTberSIRKOdYqsH0z6OXC/ZwYe8YD9tKy+NsIxfta42bqLbzCoswEByotvryxI2OPvxmT8KACN+8AL IZh8w4gw4RVaK8l9breVSvvzUbM+eL X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org blkcg->ioprio_set field is not really useful except for avoiding possibly more expensive checks inside blkcg_ioprio_track(). The check for blkcg->prio_policy being equal to POLICY_NO_CHANGE does the same service so just remove the ioprio_set field and replace the check. Reviewed-by: Damien Le Moal Tested-by: Damien Le Moal Signed-off-by: Jan Kara Reviewed-by: Christoph Hellwig --- block/blk-ioprio.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/block/blk-ioprio.c b/block/blk-ioprio.c index 79e797f5d194..3f605583598b 100644 --- a/block/blk-ioprio.c +++ b/block/blk-ioprio.c @@ -62,7 +62,6 @@ struct ioprio_blkg { struct ioprio_blkcg { struct blkcg_policy_data cpd; enum prio_policy prio_policy; - bool prio_set; }; static inline struct ioprio_blkg *pd_to_ioprio(struct blkg_policy_data *pd) @@ -113,7 +112,6 @@ static ssize_t ioprio_set_prio_policy(struct kernfs_open_file *of, char *buf, if (ret < 0) return ret; blkcg->prio_policy = ret; - blkcg->prio_set = true; return nbytes; } @@ -193,16 +191,15 @@ static void blkcg_ioprio_track(struct rq_qos *rqos, struct request *rq, struct ioprio_blkcg *blkcg = ioprio_blkcg_from_bio(bio); u16 prio; - if (!blkcg->prio_set) + if (blkcg->prio_policy == POLICY_NO_CHANGE) return; /* * Except for IOPRIO_CLASS_NONE, higher I/O priority numbers * correspond to a lower priority. Hence, the max_t() below selects * the lower priority of bi_ioprio and the cgroup I/O priority class. - * If the cgroup policy has been set to POLICY_NO_CHANGE == 0, the - * bio I/O priority is not modified. If the bio I/O priority equals - * IOPRIO_CLASS_NONE, the cgroup I/O priority is assigned to the bio. + * If the bio I/O priority equals IOPRIO_CLASS_NONE, the cgroup I/O + * priority is assigned to the bio. */ prio = max_t(u16, bio->bi_ioprio, IOPRIO_PRIO_VALUE(blkcg->prio_policy, 0)); From patchwork Thu Jun 23 07:48:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12891894 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81B99C433EF for ; Thu, 23 Jun 2022 07:49:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229730AbiFWHs6 (ORCPT ); Thu, 23 Jun 2022 03:48:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34896 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231225AbiFWHsp (ORCPT ); Thu, 23 Jun 2022 03:48:45 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 87EF12CE1E for ; Thu, 23 Jun 2022 00:48:42 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 7075F1FD46; Thu, 23 Jun 2022 07:48:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jVIcdW2PgX9kFZqpfLo1/VhqYOEzcn49NbnMlqS0quQ=; b=xg+M8BljLFOZZb7yrSuymz4qh2EhGIZNiCAL+ozTmhuT3I9geRGIR57CtX0MMrrZJrhrEQ kiu1VzHH7grppqlC51J8bcmhYXOnVNMeJQNhIgUVTe1iZqT4RfDUOnxDY/kv3zE52fwKq7 FtXXSrNCG9dldLSco4miuYo6BLx+kP0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jVIcdW2PgX9kFZqpfLo1/VhqYOEzcn49NbnMlqS0quQ=; b=fOgLO2dkR16t4y9mSilot4IKNTI6NyqeYO1IsBg6ZRlwN2Y4BFWWcCCvXhB05ABSBwLT7u cR2eyn7nlObobPAg== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id B69122C16B; Thu, 23 Jun 2022 07:48:35 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id DA3E7A063E; Thu, 23 Jun 2022 09:48:40 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 7/9] blk-ioprio: Convert from rqos policy to direct call Date: Thu, 23 Jun 2022 09:48:32 +0200 Message-Id: <20220623074840.5960-7-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220623074450.30550-1-jack@suse.cz> References: <20220623074450.30550-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4671; h=from:subject; bh=afKJr3K/2dP9j/YLwXorGsPJY4Vd7kLwgJMBSi3SAcg=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBitBrPArWTrlmfbi5gA9WW6HELzSuHMzAvpjf7634w wN9LO4SJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrQazwAKCRCcnaoHP2RA2YAxCA C9uyeXrmbeh6RXCW/h9FtXcLrd0NscBBvDgekndaWHudW6LUX0NsWFitKMgb0ck5Vdb/Oux5jWvxCl 9yOH3Uy94AwgoCWcM+q4eU/lGOoTAbobOn0EgwnUZrtzM6TtOimtadEzHgPH14z2iBcjGEwKIWQRyv ScPU2xo9NAP4FhFUImCi6mVBnWrwBNNZcFfzAnvAQ9HcJPU/uz+pcU2wYhCqpwNpxBLSJb1fLa8nxl 4loPby3Uyx81uKPIk0efnaIFsv+NqPEnfSTkLG80/gCilLZ5XynlFJqdXLfQi0M5hpHlXKeVqPORAE kcnqNsUdSp80vR8pBuHzDLaaQTCUxU X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Convert blk-ioprio handling from a rqos policy to a direct call from blk_mq_submit_bio(). Firstly, blk-ioprio is not much of a rqos policy anyway, it just needs a hook in bio submission path to set the bio's IO priority. Secondly, the rqos .track hook gets actually called too late for blk-ioprio purposes and introducing a special rqos hook just for blk-ioprio looks even weirder. Reviewed-by: Damien Le Moal Tested-by: Damien Le Moal Signed-off-by: Jan Kara Reviewed-by: Christoph Hellwig --- block/blk-cgroup.c | 1 + block/blk-ioprio.c | 50 +++++----------------------------------------- block/blk-ioprio.h | 9 +++++++++ block/blk-mq.c | 8 ++++++++ 4 files changed, 23 insertions(+), 45 deletions(-) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index 764e740b0c0f..6906981563f8 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -1299,6 +1299,7 @@ int blkcg_init_queue(struct request_queue *q) ret = blk_iolatency_init(q); if (ret) { blk_throtl_exit(q); + blk_ioprio_exit(q); goto err_destroy_all; } diff --git a/block/blk-ioprio.c b/block/blk-ioprio.c index 3f605583598b..c00060a02c6e 100644 --- a/block/blk-ioprio.c +++ b/block/blk-ioprio.c @@ -181,17 +181,12 @@ static struct blkcg_policy ioprio_policy = { .pd_free_fn = ioprio_free_pd, }; -struct blk_ioprio { - struct rq_qos rqos; -}; - -static void blkcg_ioprio_track(struct rq_qos *rqos, struct request *rq, - struct bio *bio) +void blkcg_set_ioprio(struct bio *bio) { struct ioprio_blkcg *blkcg = ioprio_blkcg_from_bio(bio); u16 prio; - if (blkcg->prio_policy == POLICY_NO_CHANGE) + if (!blkcg || blkcg->prio_policy == POLICY_NO_CHANGE) return; /* @@ -207,49 +202,14 @@ static void blkcg_ioprio_track(struct rq_qos *rqos, struct request *rq, bio->bi_ioprio = prio; } -static void blkcg_ioprio_exit(struct rq_qos *rqos) +void blk_ioprio_exit(struct request_queue *q) { - struct blk_ioprio *blkioprio_blkg = - container_of(rqos, typeof(*blkioprio_blkg), rqos); - - blkcg_deactivate_policy(rqos->q, &ioprio_policy); - kfree(blkioprio_blkg); + blkcg_deactivate_policy(q, &ioprio_policy); } -static struct rq_qos_ops blkcg_ioprio_ops = { - .track = blkcg_ioprio_track, - .exit = blkcg_ioprio_exit, -}; - int blk_ioprio_init(struct request_queue *q) { - struct blk_ioprio *blkioprio_blkg; - struct rq_qos *rqos; - int ret; - - blkioprio_blkg = kzalloc(sizeof(*blkioprio_blkg), GFP_KERNEL); - if (!blkioprio_blkg) - return -ENOMEM; - - ret = blkcg_activate_policy(q, &ioprio_policy); - if (ret) { - kfree(blkioprio_blkg); - return ret; - } - - rqos = &blkioprio_blkg->rqos; - rqos->id = RQ_QOS_IOPRIO; - rqos->ops = &blkcg_ioprio_ops; - rqos->q = q; - - /* - * Registering the rq-qos policy after activating the blk-cgroup - * policy guarantees that ioprio_blkcg_from_bio(bio) != NULL in the - * rq-qos callbacks. - */ - rq_qos_add(q, rqos); - - return 0; + return blkcg_activate_policy(q, &ioprio_policy); } static int __init ioprio_init(void) diff --git a/block/blk-ioprio.h b/block/blk-ioprio.h index a7785c2f1aea..5a1eb550e178 100644 --- a/block/blk-ioprio.h +++ b/block/blk-ioprio.h @@ -6,14 +6,23 @@ #include struct request_queue; +struct bio; #ifdef CONFIG_BLK_CGROUP_IOPRIO int blk_ioprio_init(struct request_queue *q); +void blk_ioprio_exit(struct request_queue *q); +void blkcg_set_ioprio(struct bio *bio); #else static inline int blk_ioprio_init(struct request_queue *q) { return 0; } +static inline void blk_ioprio_exit(struct request_queue *q) +{ +} +static inline void blkcg_set_ioprio(struct bio *bio) +{ +} #endif #endif /* _BLK_IOPRIO_H_ */ diff --git a/block/blk-mq.c b/block/blk-mq.c index e9bf950983c7..67a7bfa58b7c 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -42,6 +42,7 @@ #include "blk-stat.h" #include "blk-mq-sched.h" #include "blk-rq-qos.h" +#include "blk-ioprio.h" static DEFINE_PER_CPU(struct llist_head, blk_cpu_done); @@ -2790,6 +2791,11 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q, return rq; } +static void bio_set_ioprio(struct bio *bio) +{ + blkcg_set_ioprio(bio); +} + /** * blk_mq_submit_bio - Create and send a request to block device. * @bio: Bio pointer. @@ -2830,6 +2836,8 @@ void blk_mq_submit_bio(struct bio *bio) trace_block_getrq(bio); + bio_set_ioprio(bio); + rq_qos_track(q, rq, bio); blk_mq_bio_to_request(rq, bio, nr_segs); From patchwork Thu Jun 23 07:48:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12891889 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DE5CCCA47F for ; Thu, 23 Jun 2022 07:48:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229717AbiFWHsw (ORCPT ); Thu, 23 Jun 2022 03:48:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35002 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231228AbiFWHsp (ORCPT ); Thu, 23 Jun 2022 03:48:45 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 882AF2CE23 for ; Thu, 23 Jun 2022 00:48:42 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 62B081FD45; Thu, 23 Jun 2022 07:48:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=b2Sf7t6tr6gpY3pUYOUhrmZUuO5cAJPXkgwkxm4e2cc=; b=nodMkB9cY8JUQ+V7ikiIC6X/XhXT1emA2uptBTOWFdUmA172OyZlyRzlAoDiyPO3XDUfz6 xDg4ktHVYuYJ3CleuvxgO8k3SlZP7cBL11SLV7PSBKdka8zDUCze/Ta7Not0E+iJ1xWmHC k2DogR2p9qxC1Zle7gW1mU8tRXd0P2w= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=b2Sf7t6tr6gpY3pUYOUhrmZUuO5cAJPXkgwkxm4e2cc=; b=7ZaMRAUlcHDhdFRYt/H3IxW20+dCRWFiYOHBvgLryOAaqBfT5Ji9RsnET5C67nryRraTyY YpQYe2+r1g08GxAA== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id BB4F32C171; Thu, 23 Jun 2022 07:48:35 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id E1632A063F; Thu, 23 Jun 2022 09:48:40 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 8/9] block: Initialize bio priority earlier Date: Thu, 23 Jun 2022 09:48:33 +0200 Message-Id: <20220623074840.5960-8-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220623074450.30550-1-jack@suse.cz> References: <20220623074450.30550-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1028; h=from:subject; bh=tx/JE6SidyE6A6hUsnIpuHZJkxSzADvNTRDi+mddbwk=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBitBrQ8LHuUj4Ws0qvjsTLv9XDhybQ958TCOnJKMmm ah32u1iJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrQa0AAKCRCcnaoHP2RA2RpGB/ 4rvTp7yOtuxUJLXFwk8+drTwKMWv/dtHf14E7zeFNiwwS+5jRDAVUiyu/vpbIfq4nqF8jQ/v9KAPqT VvwTHzGDAE15GcPsR23JX+y5sTshFW6XDeOHd4hG6gWac4leTuvTQpSZzGk9w+rDQfXvto6OeCG/SB Kucm0MrmxI9pzUOIADC++NA1QRo83SugCGBNszidLeMAAffpVrAlmtbIubSyJ6nCYrQGEzzTUwU7GC nfd/QrmlyM1dcKkbfpNaR/ssuiRNZW7nsGibkzsYrgV6TbOq+7OUmmK624ytcchCsPEILhi+2R5C7F HYv/Sx76xuHeOFuLuw4J99Oy761XeQ X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Bio's IO priority needs to be initialized before we try to merge the bio with other bios. Otherwise we could merge bios which would otherwise receive different IO priorities leading to possible QoS issues. Reviewed-by: Damien Le Moal Tested-by: Damien Le Moal Signed-off-by: Jan Kara Reviewed-by: Christoph Hellwig --- block/blk-mq.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 67a7bfa58b7c..e17d822e6051 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -2825,6 +2825,8 @@ void blk_mq_submit_bio(struct bio *bio) if (!bio_integrity_prep(bio)) return; + bio_set_ioprio(bio); + rq = blk_mq_get_cached_request(q, plug, &bio, nr_segs); if (!rq) { if (!bio) @@ -2836,8 +2838,6 @@ void blk_mq_submit_bio(struct bio *bio) trace_block_getrq(bio); - bio_set_ioprio(bio); - rq_qos_track(q, rq, bio); blk_mq_bio_to_request(rq, bio, nr_segs); From patchwork Thu Jun 23 07:48:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 12891890 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42D15C43334 for ; Thu, 23 Jun 2022 07:48:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230156AbiFWHsy (ORCPT ); Thu, 23 Jun 2022 03:48:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35006 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231230AbiFWHsp (ORCPT ); Thu, 23 Jun 2022 03:48:45 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 884642CE2E for ; Thu, 23 Jun 2022 00:48:42 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 60FA621D40; Thu, 23 Jun 2022 07:48:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wvipttuSl4M+V7NYqqLGj6wiGpPYypJfA0Q3a/puHs8=; b=FurYDeENjsTWXxKZhTE+42dRDWSd+e3tuw2Z8IX3bRSbOLFQH4IMFCh+JwetwCR4t+y5/H WND4XT6F5ZvWliITgThp+Eal203n8Rt0s2hB7OC6oBgj3lx36zeKgz8Klb+5is0PZ+TdcV C1x7HOI9Xpahy3mGVRcUGdm0p9nPZQM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1655970521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wvipttuSl4M+V7NYqqLGj6wiGpPYypJfA0Q3a/puHs8=; b=pbSBJr2cvM14nqXQVzqH7wVZ9wdERBQ1vm6I0kLfurNAoTlWSWNpFMkE00u8ypsqzOWyjV lKXVZfVQtZzy0LBw== Received: from quack3.suse.cz (unknown [10.100.224.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id BA5042C16E; Thu, 23 Jun 2022 07:48:35 +0000 (UTC) Received: by quack3.suse.cz (Postfix, from userid 1000) id E8ABBA0640; Thu, 23 Jun 2022 09:48:40 +0200 (CEST) From: Jan Kara To: Jens Axboe Cc: , Damien Le Moal , Bart Van Assche , Niklas Cassel , Jan Kara Subject: [PATCH 9/9] block: Always initialize bio IO priority on submit Date: Thu, 23 Jun 2022 09:48:34 +0200 Message-Id: <20220623074840.5960-9-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220623074450.30550-1-jack@suse.cz> References: <20220623074450.30550-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1139; h=from:subject; bh=Ug+rrkhsTkSRexpYAARxGdaSKp+1EVBfwusw2kUXiE8=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBitBrRepLkaXYqNWpW1kK+9fKw6BgoW+v5MzIBEaM6 DJuLwtKJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYrQa0QAKCRCcnaoHP2RA2XQuB/ 9LB3o/NhnOHgl80BRG64Cy8eFb4IrGgZR5MTMBhv5DdfvXwgrqq4uMFKpYLR56ufvHq1vBPzzN8f/C rDjd+DTkH8KHJNMgByxagZvRlDTMPFITOl/1uvmdqHz/gZNYl/hf2ezaFghJIhwRgYxeic3A61Tre3 d591rWmvmXCdBk9Bgj85TB96A9a5pMrGD11xzhNXdXJ44xPVu0XreBiUbGyvhxzunRGN8QFaWRc+FB GlYgjaXohO6qlCjlP+jAcnswQ7tGB8f1KkDDETAOrPam0b+7Tlmvl+QaJrEj+OUrHK9ZWMbtKLTyQu lxMwv8YlrX6qKWBLIVpghC9L9bFJuw X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Currently, IO priority set in task's IO context is not reflected in the bio->bi_ioprio for most IO (only io_uring and direct IO set it). This results in odd results where process is submitting some bios with one priority and other bios with a different (unset) priority and due to differing priorities bios cannot be merged. Make sure bio->bi_ioprio is always set on bio submission. Reviewed-by: Damien Le Moal Tested-by: Damien Le Moal Signed-off-by: Jan Kara Reviewed-by: Christoph Hellwig --- block/blk-mq.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/blk-mq.c b/block/blk-mq.c index e17d822e6051..7548f8aebea8 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -2793,6 +2793,9 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q, static void bio_set_ioprio(struct bio *bio) { + /* Nobody set ioprio so far? Initialize it based on task's nice value */ + if (IOPRIO_PRIO_CLASS(bio->bi_ioprio) == IOPRIO_CLASS_NONE) + bio->bi_ioprio = get_current_ioprio(); blkcg_set_ioprio(bio); }