From patchwork Wed May 15 01:23:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "zhaoyang.huang" X-Patchwork-Id: 13664519 Received: from SHSQR01.spreadtrum.com (mx1.unisoc.com [222.66.158.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BBCC25CB0 for ; Wed, 15 May 2024 01:24:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=222.66.158.135 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715736268; cv=none; b=JdZvL1cvFrvNwNqgXyyceVxid8pashBUI84G+tPQmdDeOWwpPpCCuPNqvoKsWFN4eJfWug1TyuM7nuGmyX0FgLgSfE9QNeUvaxZo7IK3e2iypk7Xse5vY2hZIJNQZVmSuuk9qoNupaITGma9pk10wusYVf8VqrT4zr0AYtPL1Jg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715736268; c=relaxed/simple; bh=cxIwpQSHcfuYZZ9GvvSMLZ3wG9q7rFvAlf9yitR+PVo=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Bkj79r9ndFOwiAxPQKs4+6KNgQIfsUDCT5X7+zgECCekECAW05cFY2kSaQAhuzITO9QTNvBLkuHtSvIzgJKYbstPsCkW6g9JS0sSorwi9I1J2LziRxMEzsBVPgy2z5VaziSE9orxxE/mTFmpwnTD8bRE1FzYJ6sZB3hiv/RbcT4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=unisoc.com; spf=pass smtp.mailfrom=unisoc.com; arc=none smtp.client-ip=222.66.158.135 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=unisoc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=unisoc.com Received: from dlp.unisoc.com ([10.29.3.86]) by SHSQR01.spreadtrum.com with ESMTP id 44F1O5Om051360; Wed, 15 May 2024 09:24:05 +0800 (+08) (envelope-from zhaoyang.huang@unisoc.com) Received: from SHDLP.spreadtrum.com (bjmbx01.spreadtrum.com [10.0.64.7]) by dlp.unisoc.com (SkyGuard) with ESMTPS id 4VfFly3fXQz2QDRDk; Wed, 15 May 2024 09:20:46 +0800 (CST) Received: from bj03382pcu01.spreadtrum.com (10.0.73.40) by BJMBX01.spreadtrum.com (10.0.64.7) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Wed, 15 May 2024 09:24:03 +0800 From: "zhaoyang.huang" To: Andrew Morton , Matthew Wilcox , Jens Axboe , Tejun Heo , Josef Bacik , Baolin Wang , , , , , Zhaoyang Huang , Subject: [RFC PATCH 1/2] block: introduce helper function to calculate bps budgt Date: Wed, 15 May 2024 09:23:49 +0800 Message-ID: <20240515012350.1166350-2-zhaoyang.huang@unisoc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240515012350.1166350-1-zhaoyang.huang@unisoc.com> References: <20240515012350.1166350-1-zhaoyang.huang@unisoc.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SHCAS03.spreadtrum.com (10.0.1.207) To BJMBX01.spreadtrum.com (10.0.64.7) X-MAIL: SHSQR01.spreadtrum.com 44F1O5Om051360 From: Zhaoyang Huang The 'over-sized' bio under blk throttle control will be delayed to launch which breaks original IO timing and have the immediate BW be not within the bps limit. Introduce a helper function to calculate block device's budgt which provide the allowed bytes for current bio. Signed-off-by: Zhaoyang Huang --- block/blk-throttle.c | 44 ++++++++++++++++++++++++++++++++++++++ include/linux/blk-cgroup.h | 10 +++++++++ 2 files changed, 54 insertions(+) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index f4850a6f860b..41c75258183d 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -10,6 +10,7 @@ #include #include #include +#include #include "blk.h" #include "blk-cgroup-rwstat.h" #include "blk-stat.h" @@ -2365,6 +2366,49 @@ void blk_throtl_bio_endio(struct bio *bio) } #endif +unsigned long blk_throttle_budgt(struct block_device *bdev) +{ + struct request_queue *q = bdev_get_queue(bdev); + struct blkcg *blkcg; + struct blkcg_gq *blkg; + struct throtl_grp *tg; + long long bytes_allowed = 0; + unsigned long jiffy_elapsed, jiffy_elapsed_rnd; + u64 bps_limit; + + if (!q) + return U64_MAX; + + rcu_read_lock(); + spin_lock_irq(&q->queue_lock); + blkcg = css_to_blkcg(task_css(current, io_cgrp_id)); + if (!blkcg) + goto out; + + blkg = blkg_lookup(blkcg, q); + if (!blkg || !blkg_tryget(blkg)) + goto out; + + tg = blkg_to_tg(blkg); + bps_limit = tg_bps_limit(tg, READ); + if (bps_limit == U64_MAX) + goto out; + + jiffy_elapsed = jiffy_elapsed_rnd = jiffies - tg->slice_start[READ]; + if (!jiffy_elapsed) + jiffy_elapsed_rnd = tg->td->throtl_slice; + + jiffy_elapsed_rnd = roundup(jiffy_elapsed_rnd, tg->td->throtl_slice); + bytes_allowed = calculate_bytes_allowed(bps_limit, jiffy_elapsed_rnd) + + tg->carryover_bytes[READ]; + blkg_put(blkg); +out: + spin_unlock_irq(&q->queue_lock); + rcu_read_unlock(); + return bytes_allowed; +} + + int blk_throtl_init(struct gendisk *disk) { struct request_queue *q = disk->queue; diff --git a/include/linux/blk-cgroup.h b/include/linux/blk-cgroup.h index dd5841a42c33..ba79fa464e0a 100644 --- a/include/linux/blk-cgroup.h +++ b/include/linux/blk-cgroup.h @@ -15,10 +15,12 @@ */ #include +#include struct bio; struct cgroup_subsys_state; struct gendisk; +struct block_device; #define FC_APPID_LEN 129 @@ -45,6 +47,14 @@ static inline struct cgroup_subsys_state *bio_blkcg_css(struct bio *bio) } #endif /* CONFIG_BLK_CGROUP */ +#ifdef CONFIG_BLK_DEV_THROTTLING +unsigned long blk_throttle_budgt(struct block_device *bdev); +#else +static inline unsigned long blk_throttle_budgt(struct block_device *bdev) +{ + return U64_MAX; +} +#endif int blkcg_set_fc_appid(char *app_id, u64 cgrp_id, size_t app_id_len); char *blkcg_get_fc_appid(struct bio *bio); From patchwork Wed May 15 01:23:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "zhaoyang.huang" X-Patchwork-Id: 13664518 Received: from SHSQR01.spreadtrum.com (mx1.unisoc.com [222.66.158.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB3805CAC for ; Wed, 15 May 2024 01:24:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=222.66.158.135 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715736266; cv=none; b=FH1bQQilWSsc3pmappvV2W5VgBElVL6dXYHUkhjMw/xzeAnD7BeF1gfg2UEkyj7fVpL5SKgQCPW0+WZNV8kvS2W//ox7ij1eaHKvccizMzRLECDiqBkIwiihJOAjD3HKgnhsh2g+CJ14KP++a8kNF5LwZtaXivvrY6x0diTBXM8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715736266; c=relaxed/simple; bh=yMMIUz7DubXFQXYpZ3HCVv1OQrz7GVWdZJEPrbTdrZU=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=hEv8u0dE90Wsu1mVwLJocuYNpojbQidtnBxILkGCCgVg8Nn38XLGutNlNcbTxvV+ij6ujyhlnD+nSpV7LdWshlibGKDE+WF710AFhsPt2+MPlgDevxANzlonmRN+P53IPHvoTL//9/Vr+NyI9wAt7OOjrCzgj6CkPSF8nQuSgpk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=unisoc.com; spf=pass smtp.mailfrom=unisoc.com; arc=none smtp.client-ip=222.66.158.135 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=unisoc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=unisoc.com Received: from dlp.unisoc.com ([10.29.3.86]) by SHSQR01.spreadtrum.com with ESMTP id 44F1O7LZ051706; Wed, 15 May 2024 09:24:07 +0800 (+08) (envelope-from zhaoyang.huang@unisoc.com) Received: from SHDLP.spreadtrum.com (bjmbx01.spreadtrum.com [10.0.64.7]) by dlp.unisoc.com (SkyGuard) with ESMTPS id 4VfFm04QmWz2PDtZH; Wed, 15 May 2024 09:20:48 +0800 (CST) Received: from bj03382pcu01.spreadtrum.com (10.0.73.40) by BJMBX01.spreadtrum.com (10.0.64.7) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Wed, 15 May 2024 09:24:05 +0800 From: "zhaoyang.huang" To: Andrew Morton , Matthew Wilcox , Jens Axboe , Tejun Heo , Josef Bacik , Baolin Wang , , , , , Zhaoyang Huang , Subject: [RFC PATCH 2/2] mm: introduce budgt control in readahead Date: Wed, 15 May 2024 09:23:50 +0800 Message-ID: <20240515012350.1166350-3-zhaoyang.huang@unisoc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240515012350.1166350-1-zhaoyang.huang@unisoc.com> References: <20240515012350.1166350-1-zhaoyang.huang@unisoc.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SHCAS03.spreadtrum.com (10.0.1.207) To BJMBX01.spreadtrum.com (10.0.64.7) X-MAIL: SHSQR01.spreadtrum.com 44F1O7LZ051706 From: Zhaoyang Huang Currently, readahead's size is decided mainly by page cache's status like hit/miss or hole size which could lead to suspension of following bio which is over the size of blk-throttle allowed size when BLK_THROTTLING is on. Introduce the budgt value here to have the bio's size be within the legal size. Signed-off-by: Zhaoyang Huang --- mm/readahead.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/mm/readahead.c b/mm/readahead.c index 130c0e7df99f..2b6120ced6f9 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -128,6 +128,7 @@ #include #include #include +#include #include "internal.h" @@ -358,16 +359,23 @@ static unsigned long get_init_ra_size(unsigned long size, unsigned long max) * Get the previous window size, ramp it up, and * return it as the new window size. */ -static unsigned long get_next_ra_size(struct file_ra_state *ra, +static unsigned long get_next_ra_size(struct readahead_control *ractl, unsigned long max) { - unsigned long cur = ra->size; + unsigned long cur = ractl->ra->size; + struct inode *inode = ractl->mapping->host; + unsigned long budgt = inode->i_sb->s_bdev ? + blk_throttle_budgt(inode->i_sb->s_bdev) : 0; + unsigned long val = max; if (cur < max / 16) - return 4 * cur; + val = 4 * cur; if (cur <= max / 2) - return 2 * cur; - return max; + val = 2 * cur; + + val = budgt ? min(budgt / PAGE_SIZE, val) : val; + + return val; } /* @@ -437,6 +445,8 @@ static int try_context_readahead(struct address_space *mapping, unsigned long max) { pgoff_t size; + unsigned long budgt = mapping->host->i_sb->s_bdev ? + blk_throttle_budgt(mapping->host->i_sb->s_bdev) : 0; size = count_history_pages(mapping, index, max); @@ -455,7 +465,7 @@ static int try_context_readahead(struct address_space *mapping, size *= 2; ra->start = index; - ra->size = min(size + req_size, max); + ra->size = min3(budgt / PAGE_SIZE, size + req_size, max); ra->async_size = 1; return 1; @@ -552,6 +562,8 @@ static void ondemand_readahead(struct readahead_control *ractl, pgoff_t index = readahead_index(ractl); pgoff_t expected, prev_index; unsigned int order = folio ? folio_order(folio) : 0; + unsigned long budgt = ractl->mapping->host->i_sb->s_bdev ? + blk_throttle_budgt(ractl->mapping->host->i_sb->s_bdev) : 0; /* * If the request exceeds the readahead window, allow the read to @@ -574,7 +586,7 @@ static void ondemand_readahead(struct readahead_control *ractl, 1UL << order); if (index == expected || index == (ra->start + ra->size)) { ra->start += ra->size; - ra->size = get_next_ra_size(ra, max_pages); + ra->size = get_next_ra_size(ractl, max_pages); ra->async_size = ra->size; goto readit; } @@ -599,7 +611,7 @@ static void ondemand_readahead(struct readahead_control *ractl, ra->start = start; ra->size = start - index; /* old async_size */ ra->size += req_size; - ra->size = get_next_ra_size(ra, max_pages); + ra->size = get_next_ra_size(ractl, max_pages); ra->async_size = ra->size; goto readit; } @@ -631,6 +643,9 @@ static void ondemand_readahead(struct readahead_control *ractl, * standalone, small random read * Read as is, and do not pollute the readahead state. */ + if (budgt) + req_size = min(budgt / PAGE_SIZE, req_size); + do_page_cache_ra(ractl, req_size, 0); return; @@ -647,7 +662,7 @@ static void ondemand_readahead(struct readahead_control *ractl, * Take care of maximum IO pages as above. */ if (index == ra->start && ra->size == ra->async_size) { - add_pages = get_next_ra_size(ra, max_pages); + add_pages = get_next_ra_size(ractl, max_pages); if (ra->size + add_pages <= max_pages) { ra->async_size = add_pages; ra->size += add_pages;