From patchwork Fri Jun 21 06:26:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Sun X-Patchwork-Id: 13706889 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.sourceforge.net (lists.sourceforge.net [216.105.38.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 08F90C27C4F for ; Fri, 21 Jun 2024 06:28:55 +0000 (UTC) Received: from [127.0.0.1] (helo=sfs-ml-4.v29.lw.sourceforge.com) by sfs-ml-4.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1sKXlV-00065U-RO; Fri, 21 Jun 2024 06:28:54 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-4.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sKXlU-00065O-Ff for linux-f2fs-devel@lists.sourceforge.net; Fri, 21 Jun 2024 06:28:52 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Type:Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-ID:Date:Subject:CC:To:From:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=ubp4rSItjAZBCkY7SbCuk+EN2RlkpACkKXKQ/j5iRL8=; b=HxAUCQ5FXzUXpRlT4TChEfWHjO rvFiYsqcVfVpbrO3V41HtDmuy8V201J/MSweeuLdtX3pmY7UI+DnErHt+amud/shSoijzT4unx/Ld cHAOBOXn6mccm4y8OReAQXbn5YSWAknj5wjbga/gXc2EldhZtjNxwEv2jYT79UzH3VEI=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Type:Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:CC:To:From:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=ubp4rSItjAZBCkY7SbCuk+EN2RlkpACkKXKQ/j5iRL8=; b=T58AwZ4dLmkAFddhJqa4vPS5jz SL97zduRSDqL2diH+NwMOCdFLGnyAC83wP/bRiZGSwTT71RB9xkhkRJqCU75Mh35ZENoHL8f+NGjh E/Jiprt5n9u7Cwacqt7L1A8PhTRTpN53BUFh/J1yZarf9iV46T3QA3zCNEWzHBA3a8aU=; Received: from mx1.unisoc.com ([222.66.158.135] helo=SHSQR01.spreadtrum.com) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1sKXlU-0005hF-1y for linux-f2fs-devel@lists.sourceforge.net; Fri, 21 Jun 2024 06:28:52 +0000 Received: from dlp.unisoc.com ([10.29.3.86]) by SHSQR01.spreadtrum.com with ESMTP id 45L6QVZj041339; Fri, 21 Jun 2024 14:26:31 +0800 (+08) (envelope-from Yi.Sun@unisoc.com) Received: from SHDLP.spreadtrum.com (bjmbx02.spreadtrum.com [10.0.64.8]) by dlp.unisoc.com (SkyGuard) with ESMTPS id 4W56hR1r47z2SFWL2; Fri, 21 Jun 2024 14:21:59 +0800 (CST) Received: from tj10379pcu.spreadtrum.com (10.5.32.15) by BJMBX02.spreadtrum.com (10.0.64.8) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Fri, 21 Jun 2024 14:26:28 +0800 From: Yi Sun To: , , , Date: Fri, 21 Jun 2024 14:26:16 +0800 Message-ID: <20240621062617.595007-2-yi.sun@unisoc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240621062617.595007-1-yi.sun@unisoc.com> References: <20240621062617.595007-1-yi.sun@unisoc.com> MIME-Version: 1.0 X-Originating-IP: [10.5.32.15] X-ClientProxiedBy: SHCAS03.spreadtrum.com (10.0.1.207) To BJMBX02.spreadtrum.com (10.0.64.8) X-MAIL: SHSQR01.spreadtrum.com 45L6QVZj041339 X-Headers-End: 1sKXlU-0005hF-1y Subject: [f2fs-dev] [PATCH 1/2] workqueue: add io priority to work_struct X-BeenThere: linux-f2fs-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yunlongxing23@gmail.com, ebiggers@google.com, kent.overstreet@linux.dev, jiangshanlai@gmail.com, linux-kernel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net, Hao_hao.Wang@unisoc.com Errors-To: linux-f2fs-devel-bounces@lists.sourceforge.net Many works will go to submit_bio(), and in many cases the io priority of kworker cannot meet the real-time requirements of this work. So add the basic attribute ioprio to work_struct, and kworker can adjust its io priority according to this attribute. Add function set_work_ioprio() to set the io priority of this work. Add function may_adjust_work_task_ioprio() to adjust kworker's io priority. Add function restore_work_task_ioprio() to restore kworker's io priority. Signed-off-by: Yi Sun --- include/linux/workqueue.h | 9 ++++++ include/linux/workqueue_types.h | 6 ++++ kernel/workqueue.c | 51 +++++++++++++++++++++++++++++++++ 3 files changed, 66 insertions(+) diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h index fb3993894536..f6191774b730 100644 --- a/include/linux/workqueue.h +++ b/include/linux/workqueue.h @@ -286,6 +286,9 @@ static inline unsigned int work_static(struct work_struct *work) { return 0; } lockdep_init_map(&(_work)->lockdep_map, "(work_completion)"#_work, (_key), 0); \ INIT_LIST_HEAD(&(_work)->entry); \ (_work)->func = (_func); \ + (_work)->ioprio = 0; \ + (_work)->ori_ioprio = 0; \ + (_work)->ioprio_flag = 0; \ } while (0) #else #define __INIT_WORK_KEY(_work, _func, _onstack, _key) \ @@ -294,6 +297,9 @@ static inline unsigned int work_static(struct work_struct *work) { return 0; } (_work)->data = (atomic_long_t) WORK_DATA_INIT(); \ INIT_LIST_HEAD(&(_work)->entry); \ (_work)->func = (_func); \ + (_work)->ioprio = 0; \ + (_work)->ori_ioprio = 0; \ + (_work)->ioprio_flag = 0; \ } while (0) #endif @@ -585,6 +591,9 @@ extern struct work_struct *current_work(void); extern bool current_is_workqueue_rescuer(void); extern bool workqueue_congested(int cpu, struct workqueue_struct *wq); extern unsigned int work_busy(struct work_struct *work); +extern void set_work_ioprio(struct work_struct *work, unsigned short ioprio); +extern void may_adjust_work_task_ioprio(struct work_struct *work); +extern void restore_work_task_ioprio(struct work_struct *work); extern __printf(1, 2) void set_worker_desc(const char *fmt, ...); extern void print_worker_info(const char *log_lvl, struct task_struct *task); extern void show_all_workqueues(void); diff --git a/include/linux/workqueue_types.h b/include/linux/workqueue_types.h index 4c38824f3ab4..d9969596bbc3 100644 --- a/include/linux/workqueue_types.h +++ b/include/linux/workqueue_types.h @@ -17,6 +17,12 @@ struct work_struct { atomic_long_t data; struct list_head entry; work_func_t func; + /* If the work does submit_bio, io priority may be needed. */ + unsigned short ioprio; + /* Record kworker's original io priority. */ + unsigned short ori_ioprio; + /* Whether the work has set io priority? */ + long ioprio_flag; #ifdef CONFIG_LOCKDEP struct lockdep_map lockdep_map; #endif diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 003474c9a77d..a44a8f92eec2 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -55,6 +55,7 @@ #include #include #include +#include #include "workqueue_internal.h" @@ -6025,6 +6026,56 @@ unsigned int work_busy(struct work_struct *work) } EXPORT_SYMBOL_GPL(work_busy); +/** + * set_work_ioprio - set io priority for the current work + * @work: the work to be set + * @ioprio: desired io priority + * + * This function can be called after INIT_WORK if the io priority + * of the work needs to adjust. And it is recommended to use this + * function together with may_adjust_work_task_ioprio() and + * restore_work_task_ioprio(). + */ +void set_work_ioprio(struct work_struct *work, unsigned short ioprio) +{ + work->ioprio = ioprio; + work->ioprio_flag = 1; +} +EXPORT_SYMBOL_GPL(set_work_ioprio); + +/** + * may_adjust_work_task_ioprio - adjust the io priority of kworker + * @work: the work that kworker will do + * + * It is recommended to use this function together with set_work_ioprio() + * and restore_work_task_ioprio(). + */ +void may_adjust_work_task_ioprio(struct work_struct *work) +{ + if (work->ioprio_flag) { + work->ori_ioprio = get_current_ioprio(); + set_task_ioprio(current, work->ioprio); + } +} +EXPORT_SYMBOL_GPL(may_adjust_work_task_ioprio); + +/** + * restore_work_task_ioprio - restore the io priority of kworker + * @work: the work that kworker just did + * + * When kworker finishes the work, the original io priority of + * kworker should be restored. It is recommended to use this function + * together with set_work_ioprio() and may_adjust_work_task_ioprio(). + */ +void restore_work_task_ioprio(struct work_struct *work) +{ + if (work->ioprio_flag) { + set_task_ioprio(current, work->ori_ioprio); + work->ioprio_flag = 0; + } +} +EXPORT_SYMBOL_GPL(restore_work_task_ioprio); + /** * set_worker_desc - set description for the current work item * @fmt: printf-style format string From patchwork Fri Jun 21 06:26:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Sun X-Patchwork-Id: 13706888 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.sourceforge.net (lists.sourceforge.net [216.105.38.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 652DBC2BB85 for ; Fri, 21 Jun 2024 06:27:56 +0000 (UTC) Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1sKXkY-0007A3-Sb; Fri, 21 Jun 2024 06:27:55 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sKXkX-00079w-Qj for linux-f2fs-devel@lists.sourceforge.net; Fri, 21 Jun 2024 06:27:54 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Type:Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-ID:Date:Subject:CC:To:From:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=ZmPGPlcByAYd2NWqNYdveqrwqnNx26k1juQa+7BnkR0=; b=M4ZI5oOTotSS1AN48ChsSN06qx m9++wACTPAACMUzjZlBv65KACGl/Ql4iWfnlkDYgmLIg95brvPke0KTj3+bUvnDTgDmiRHTkjxqZ0 9M1XDFoitL/zeDwplAX3uDwxF7/muP2xi5hJFRL0XeGLrG1piU2Nj9sQE7Ir+P2Qk4gU=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Type:Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:CC:To:From:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=ZmPGPlcByAYd2NWqNYdveqrwqnNx26k1juQa+7BnkR0=; b=ZGwkPReO94BNiWh0rJPreIquyj 5YqnITvmFpsmvMSwtl6o7jMViLXYr6zXBY7bBSzkgQ4USMciQz+xNcYw+56b4MIYiqYbxBOc8GbP8 6Ljwqh91M6fqp7OPfnJPB/krY2Sgb44x9ZGrrEV8jBGLDKKrMVUwCvLu2GtOuYjBkPNs=; Received: from mx1.unisoc.com ([222.66.158.135] helo=SHSQR01.spreadtrum.com) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1sKXkW-0005d5-UY for linux-f2fs-devel@lists.sourceforge.net; Fri, 21 Jun 2024 06:27:54 +0000 Received: from dlp.unisoc.com ([10.29.3.86]) by SHSQR01.spreadtrum.com with ESMTP id 45L6QWPL041450; Fri, 21 Jun 2024 14:26:32 +0800 (+08) (envelope-from Yi.Sun@unisoc.com) Received: from SHDLP.spreadtrum.com (bjmbx02.spreadtrum.com [10.0.64.8]) by dlp.unisoc.com (SkyGuard) with ESMTPS id 4W56hS1zfhz2SFCkm; Fri, 21 Jun 2024 14:22:00 +0800 (CST) Received: from tj10379pcu.spreadtrum.com (10.5.32.15) by BJMBX02.spreadtrum.com (10.0.64.8) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Fri, 21 Jun 2024 14:26:30 +0800 From: Yi Sun To: , , , Date: Fri, 21 Jun 2024 14:26:17 +0800 Message-ID: <20240621062617.595007-3-yi.sun@unisoc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240621062617.595007-1-yi.sun@unisoc.com> References: <20240621062617.595007-1-yi.sun@unisoc.com> MIME-Version: 1.0 X-Originating-IP: [10.5.32.15] X-ClientProxiedBy: SHCAS03.spreadtrum.com (10.0.1.207) To BJMBX02.spreadtrum.com (10.0.64.8) X-MAIL: SHSQR01.spreadtrum.com 45L6QWPL041450 X-Headers-End: 1sKXkW-0005d5-UY Subject: [f2fs-dev] [PATCH 2/2] f2fs: set f2fs verify work io priority X-BeenThere: linux-f2fs-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yunlongxing23@gmail.com, ebiggers@google.com, kent.overstreet@linux.dev, jiangshanlai@gmail.com, linux-kernel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net, Hao_hao.Wang@unisoc.com Errors-To: linux-f2fs-devel-bounces@lists.sourceforge.net The io priority of reading the original data of the verity-file and reading hash blocks during verification should be consistent. If the io priority is not set, when a high-task-priority process reads the verity-file, the kworker's io priority may be too low, resulting in delayed reading of hash blocks, which eventually causes the high-task-priority process to response slowly. Signed-off-by: Yi Sun --- fs/f2fs/data.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index b9b0debc6b3d..705a2bdf0848 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -172,6 +172,8 @@ static void f2fs_verify_bio(struct work_struct *work) struct bio *bio = ctx->bio; bool may_have_compressed_pages = (ctx->enabled_steps & STEP_DECOMPRESS); + may_adjust_work_task_ioprio(work); + /* * fsverity_verify_bio() may call readahead() again, and while verity * will be disabled for this, decryption and/or decompression may still @@ -204,6 +206,8 @@ static void f2fs_verify_bio(struct work_struct *work) } f2fs_finish_read_bio(bio, true); + + restore_work_task_ioprio(work); } /* @@ -221,6 +225,7 @@ static void f2fs_verify_and_finish_bio(struct bio *bio, bool in_task) if (ctx && (ctx->enabled_steps & STEP_VERITY)) { INIT_WORK(&ctx->work, f2fs_verify_bio); + set_work_ioprio(&ctx->work, bio->bi_ioprio); fsverity_enqueue_verify_work(&ctx->work); } else { f2fs_finish_read_bio(bio, in_task);