From patchwork Wed Apr 20 04:27:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12819734 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CE71C433F5 for ; Wed, 20 Apr 2022 04:27:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 145456B0082; Wed, 20 Apr 2022 00:27:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0CE3A6B0083; Wed, 20 Apr 2022 00:27:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E89346B0085; Wed, 20 Apr 2022 00:27:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.26]) by kanga.kvack.org (Postfix) with ESMTP id D10556B0082 for ; Wed, 20 Apr 2022 00:27:51 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id A78F821459 for ; Wed, 20 Apr 2022 04:27:51 +0000 (UTC) X-FDA: 79375974342.21.DE14636 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf03.hostedemail.com (Postfix) with ESMTP id 00F4720016 for ; Wed, 20 Apr 2022 04:27:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=bBRlU0MB4o3o14sfrZ9nIu2HZBK9cfvY6qzPJHQbEBY=; b=Cdrft2Wr3XmwlNrAG7Z6/WRhE5 tn4Fp6Z9JPcCxW4Q8f+a7x31QxxIT/HNDIwzl7Ba/BNs+5bveZ7fwZW7D1WyKDkZDnn2fl2Gtoady ZdKlF3apGB08LNiUGT9Pry6N1VslmyF1DESC5me5YZ+5oBtwaLl31RCRxof4DeS0u3iSJ/o594Ga0 G1bpFiV96jNTYB3RwUS0v3eDmzUlWxRrg1xCKTQVoZ0S2ABPjg3GNjPqn962cwEqx8QdoAoVHqy8G GQevrH3xLliY9LU0wAKJj17y9BHnGt2eSLk+7kMRFQ3CfruCONaUBoiRnaDlUuxm4H469t8qLOLDI /12zTITw==; Received: from 089144220023.atnat0029.highway.webapn.at ([89.144.220.23] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nh1wQ-007FK2-EX; Wed, 20 Apr 2022 04:27:47 +0000 From: Christoph Hellwig To: Jens Axboe Cc: Paolo Valente , Tejun Heo , James Smart , Dick Kennedy , linux-block@vger.kernel.org, cgroups@vger.kernel.org, linux-nvme@lists.infradead.org, linux-mm@kvack.org Subject: [PATCH 06/15] blk-cgroup: move blkcg_{pin,unpin}_online out of line Date: Wed, 20 Apr 2022 06:27:14 +0200 Message-Id: <20220420042723.1010598-7-hch@lst.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220420042723.1010598-1-hch@lst.de> References: <20220420042723.1010598-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 00F4720016 X-Rspam-User: Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=Cdrft2Wr; dmarc=none; spf=none (imf03.hostedemail.com: domain of BATV+4b42fcde86a6810335df+6814+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+4b42fcde86a6810335df+6814+infradead.org+hch@bombadil.srs.infradead.org X-Stat-Signature: tyruk6pdqwkfpxn91smzpbjfqo5to5hp X-HE-Tag: 1650428869-630730 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Move these two functions out of line as there is no good reason to inline them. Also switch to passing a cgroup_subsys_state instead of doing the conversion in the caller to prepare for making the blkcg structure private to blk-cgroup. Signed-off-by: Christoph Hellwig --- block/blk-cgroup.c | 88 ++++++++++++++++++++++++++++---------- include/linux/blk-cgroup.h | 46 +------------------- mm/backing-dev.c | 5 +-- 3 files changed, 69 insertions(+), 70 deletions(-) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index 97266ebde975b..3af0d4a619550 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -155,6 +155,17 @@ static void blkg_async_bio_workfn(struct work_struct *work) blk_finish_plug(&plug); } +/** + * blkcg_parent - get the parent of a blkcg + * @blkcg: blkcg of interest + * + * Return the parent blkcg of @blkcg. Can be called anytime. + */ +static inline struct blkcg *blkcg_parent(struct blkcg *blkcg) +{ + return css_to_blkcg(blkcg->css.parent); +} + /** * blkg_alloc - allocate a blkg * @blkcg: block cgroup the new blkg is associated with @@ -1015,25 +1026,6 @@ static struct cftype blkcg_legacy_files[] = { * This finally frees the blkcg. */ -/** - * blkcg_css_offline - cgroup css_offline callback - * @css: css of interest - * - * This function is called when @css is about to go away. Here the cgwbs are - * offlined first and only once writeback associated with the blkcg has - * finished do we start step 2 (see above). - */ -static void blkcg_css_offline(struct cgroup_subsys_state *css) -{ - struct blkcg *blkcg = css_to_blkcg(css); - - /* this prevents anyone from attaching or migrating to this blkcg */ - wb_blkcg_offline(blkcg); - - /* put the base online pin allowing step 2 to be triggered */ - blkcg_unpin_online(blkcg); -} - /** * blkcg_destroy_blkgs - responsible for shooting down blkgs * @blkcg: blkcg of interest @@ -1045,7 +1037,7 @@ static void blkcg_css_offline(struct cgroup_subsys_state *css) * * This is the blkcg counterpart of ioc_release_fn(). */ -void blkcg_destroy_blkgs(struct blkcg *blkcg) +static void blkcg_destroy_blkgs(struct blkcg *blkcg) { might_sleep(); @@ -1075,6 +1067,57 @@ void blkcg_destroy_blkgs(struct blkcg *blkcg) spin_unlock_irq(&blkcg->lock); } +/** + * blkcg_pin_online - pin online state + * @blkcg_css: blkcg of interest + * + * While pinned, a blkcg is kept online. This is primarily used to + * impedance-match blkg and cgwb lifetimes so that blkg doesn't go offline + * while an associated cgwb is still active. + */ +void blkcg_pin_online(struct cgroup_subsys_state *blkcg_css) +{ + refcount_inc(&css_to_blkcg(blkcg_css)->online_pin); +} + +/** + * blkcg_unpin_online - unpin online state + * @blkcg_css: blkcg of interest + * + * This is primarily used to impedance-match blkg and cgwb lifetimes so + * that blkg doesn't go offline while an associated cgwb is still active. + * When this count goes to zero, all active cgwbs have finished so the + * blkcg can continue destruction by calling blkcg_destroy_blkgs(). + */ +void blkcg_unpin_online(struct cgroup_subsys_state *blkcg_css) +{ + struct blkcg *blkcg = css_to_blkcg(blkcg_css); + + do { + if (!refcount_dec_and_test(&blkcg->online_pin)) + break; + blkcg_destroy_blkgs(blkcg); + blkcg = blkcg_parent(blkcg); + } while (blkcg); +} + +/** + * blkcg_css_offline - cgroup css_offline callback + * @css: css of interest + * + * This function is called when @css is about to go away. Here the cgwbs are + * offlined first and only once writeback associated with the blkcg has + * finished do we start step 2 (see above). + */ +static void blkcg_css_offline(struct cgroup_subsys_state *css) +{ + /* this prevents anyone from attaching or migrating to this blkcg */ + wb_blkcg_offline(css_to_blkcg(css)); + + /* put the base online pin allowing step 2 to be triggered */ + blkcg_unpin_online(css); +} + static void blkcg_css_free(struct cgroup_subsys_state *css) { struct blkcg *blkcg = css_to_blkcg(css); @@ -1163,8 +1206,7 @@ blkcg_css_alloc(struct cgroup_subsys_state *parent_css) static int blkcg_css_online(struct cgroup_subsys_state *css) { - struct blkcg *blkcg = css_to_blkcg(css); - struct blkcg *parent = blkcg_parent(blkcg); + struct blkcg *parent = blkcg_parent(css_to_blkcg(css)); /* * blkcg_pin_online() is used to delay blkcg offline so that blkgs @@ -1172,7 +1214,7 @@ static int blkcg_css_online(struct cgroup_subsys_state *css) * parent so that offline always happens towards the root. */ if (parent) - blkcg_pin_online(parent); + blkcg_pin_online(css); return 0; } diff --git a/include/linux/blk-cgroup.h b/include/linux/blk-cgroup.h index 988965c1c27b8..0fb7459096e9f 100644 --- a/include/linux/blk-cgroup.h +++ b/include/linux/blk-cgroup.h @@ -111,7 +111,6 @@ struct blkcg_gq { extern struct cgroup_subsys_state * const blkcg_root_css; -void blkcg_destroy_blkgs(struct blkcg *blkcg); void blkcg_schedule_throttle(struct request_queue *q, bool use_memdelay); void blkcg_maybe_throttle_current(void); @@ -136,49 +135,8 @@ static inline struct blkcg *bio_blkcg(struct bio *bio) } bool blk_cgroup_congested(void); - -/** - * blkcg_parent - get the parent of a blkcg - * @blkcg: blkcg of interest - * - * Return the parent blkcg of @blkcg. Can be called anytime. - */ -static inline struct blkcg *blkcg_parent(struct blkcg *blkcg) -{ - return css_to_blkcg(blkcg->css.parent); -} - -/** - * blkcg_pin_online - pin online state - * @blkcg: blkcg of interest - * - * While pinned, a blkcg is kept online. This is primarily used to - * impedance-match blkg and cgwb lifetimes so that blkg doesn't go offline - * while an associated cgwb is still active. - */ -static inline void blkcg_pin_online(struct blkcg *blkcg) -{ - refcount_inc(&blkcg->online_pin); -} - -/** - * blkcg_unpin_online - unpin online state - * @blkcg: blkcg of interest - * - * This is primarily used to impedance-match blkg and cgwb lifetimes so - * that blkg doesn't go offline while an associated cgwb is still active. - * When this count goes to zero, all active cgwbs have finished so the - * blkcg can continue destruction by calling blkcg_destroy_blkgs(). - */ -static inline void blkcg_unpin_online(struct blkcg *blkcg) -{ - do { - if (!refcount_dec_and_test(&blkcg->online_pin)) - break; - blkcg_destroy_blkgs(blkcg); - blkcg = blkcg_parent(blkcg); - } while (blkcg); -} +void blkcg_pin_online(struct cgroup_subsys_state *blkcg_css); +void blkcg_unpin_online(struct cgroup_subsys_state *blkcg_css); #else /* CONFIG_BLK_CGROUP */ diff --git a/mm/backing-dev.c b/mm/backing-dev.c index 7176af65b103a..93cddbcd4eb83 100644 --- a/mm/backing-dev.c +++ b/mm/backing-dev.c @@ -390,7 +390,6 @@ static void cgwb_release_workfn(struct work_struct *work) { struct bdi_writeback *wb = container_of(work, struct bdi_writeback, release_work); - struct blkcg *blkcg = css_to_blkcg(wb->blkcg_css); struct backing_dev_info *bdi = wb->bdi; mutex_lock(&wb->bdi->cgwb_release_mutex); @@ -401,7 +400,7 @@ static void cgwb_release_workfn(struct work_struct *work) mutex_unlock(&wb->bdi->cgwb_release_mutex); /* triggers blkg destruction if no online users left */ - blkcg_unpin_online(blkcg); + blkcg_unpin_online(wb->blkcg_css); fprop_local_destroy_percpu(&wb->memcg_completions); @@ -511,7 +510,7 @@ static int cgwb_create(struct backing_dev_info *bdi, list_add_tail_rcu(&wb->bdi_node, &bdi->wb_list); list_add(&wb->memcg_node, memcg_cgwb_list); list_add(&wb->blkcg_node, blkcg_cgwb_list); - blkcg_pin_online(blkcg); + blkcg_pin_online(blkcg_css); css_get(memcg_css); css_get(blkcg_css); }