From patchwork Tue May 29 21:17:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10437063 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5AE6B601E9 for ; Tue, 29 May 2018 21:17:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4BDAC28994 for ; Tue, 29 May 2018 21:17:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 402A62899D; Tue, 29 May 2018 21:17:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7582128994 for ; Tue, 29 May 2018 21:17:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 97FAD6B0008; Tue, 29 May 2018 17:17:30 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8AE9E6B000A; Tue, 29 May 2018 17:17:30 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 729036B000C; Tue, 29 May 2018 17:17:30 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f198.google.com (mail-qt0-f198.google.com [209.85.216.198]) by kanga.kvack.org (Postfix) with ESMTP id 44FEA6B0008 for ; Tue, 29 May 2018 17:17:30 -0400 (EDT) Received: by mail-qt0-f198.google.com with SMTP id q13-v6so14262583qtk.8 for ; Tue, 29 May 2018 14:17:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=PK9SM6Or93iN9H7VsfeL6v/TgtFB9uw6ZcSS0L5F/OA=; b=Nr2YhtgQGnW/vhZFOFJqsqnG0XfvBejgqCmcoguN+mrxvd1Lo6TqO4KVJNPFayJCt9 uMtYmhmp7NFMm1y2bMVax0Pk2x9kSU/cGZloLfgMQRAsg/TlgyEybcB1MIXzwVT9+zrO oGUZofjDDOTxNwyp3ChfHxussNYOhUjdr6+sozAN9MP1oXbZhgTGSVMdUu5fXsDaz6k0 a0QZ1gQo9Uy72MHxYzWRWrK7kkmyduro8t/AG9wiggBu9a5X3AoWhGlHqCC+4h7Jd5at i0DLfI5bbiVCGK+fgTwsBni23VG0caQFGuA0lU6OoOL2NhdAjlnhEdAahl4ycM/gTEea t7Pg== X-Gm-Message-State: APt69E2ro9Kj+KS6wSnzFhITqv9TSi2EANsdDou3qa91DenvboZq1n7z lAg5rkVqoRE6n+RessYnHCpEJc5CW7putp7IHG+FIhUNQL5NJo0BqYCJ5uEFaqfrgphsULlObmp 6KkX6rqwxHkboulHdVGjVbayThBJH+B9P8cX4AiDj/wDrCQjkdXzRLv0sHb4pYp16NRVeUg8we2 D+9WX79Xu4LxRFHhcXD+PW7ylKZRVQVsdv2YdPYA1b4a0XaPaP8OLwJs4hhilMwSV23GFmAqDQ0 S4FToJRbdjThaVVeElFwWrMbOksY1pqzuntkCNanErZ3RcDlgLpAFmE3nsJu6Yin0AuFvZfTJ51 KfIe3rlp+z6E8fSODQmX4J6aTRtLcOa4VtLtUlxSio7W6HZ7At6Pp//TJrrYQTUlRPADyI1DSe7 i X-Received: by 2002:ac8:7355:: with SMTP id q21-v6mr70714qtp.27.1527628650030; Tue, 29 May 2018 14:17:30 -0700 (PDT) X-Received: by 2002:ac8:7355:: with SMTP id q21-v6mr70677qtp.27.1527628649334; Tue, 29 May 2018 14:17:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527628649; cv=none; d=google.com; s=arc-20160816; b=uYP/FrhWQNFHsubTAJY3bf84qet3fGBjDyHKxspBxQyDXixL8t88jaqNxY0ILWMqIO usQFYHprbGkaaovXP8F13WMkJK2C5wvXgJsV3C31+rprAvs4Q2OjlygtQEN8E9YwRtBj ypi3+lCQAFBJT5/w3y9/siQswbRJnvcepiudfvC67jCGjnMHFSBj6A5VO7PImT/fW3Ff RcmR2lbUtLlaZUFWLYJScjs6kuAsNOH9ZR9Fh1pvPeZ18swJ5suNQ5IDZay6ql/G5+qR Tu0kcbIIkxmX4ML+xeJ/FaVK+IZkhjwCtqLlQEfqLYgAVNCxnWgcTUkIMm4T1+1bHRFi 1O+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=PK9SM6Or93iN9H7VsfeL6v/TgtFB9uw6ZcSS0L5F/OA=; b=NVdA+NRRhjsS8GhcIRor0a3tia9iN6b6EpYXA1EMEf4kdwaft4w7tbKPzTRR1Cn1kA z4/Z36EbIZQCw65XhmOv2R3MAgYbqX0el2e1e2H/JcPVHTd+jy8q0YDG+KMln5uQB6fa S9MqUYox+2RCdWNBHmjFrSbjU7bOrAh4YcM15jCR1x70n955fj1dpVf4EdjTjB4sN87z OLss/kinxutQNDlJ/P0jXappIf8ubZZYbGx5acMcHae8GFZVs7uqHCCnfNffY72LA8VI x0aV9BO/fCxVN6Ym1DfeTUxagIWuhUguFvlLddeqYc9Vhc/HFQlPPpzdB9quVnyiYDIY ofiA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@toxicpanda-com.20150623.gappssmtp.com header.s=20150623 header.b=Q2eXIVJs; spf=neutral (google.com: 209.85.220.65 is neither permitted nor denied by best guess record for domain of josef@toxicpanda.com) smtp.mailfrom=josef@toxicpanda.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id g17-v6sor9919893qkm.17.2018.05.29.14.17.29 for (Google Transport Security); Tue, 29 May 2018 14:17:29 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.220.65 is neither permitted nor denied by best guess record for domain of josef@toxicpanda.com) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@toxicpanda-com.20150623.gappssmtp.com header.s=20150623 header.b=Q2eXIVJs; spf=neutral (google.com: 209.85.220.65 is neither permitted nor denied by best guess record for domain of josef@toxicpanda.com) smtp.mailfrom=josef@toxicpanda.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=PK9SM6Or93iN9H7VsfeL6v/TgtFB9uw6ZcSS0L5F/OA=; b=Q2eXIVJsZfqY/e3e9Z68Wc6g8uTnwGkptwm4e0Xa8K1DN2IOzcNmABsQKyCa0FUE/k +V/fRQyszCDzaDy+PlKLsJ2ac/IptAt4CTBrGPjwS2w9VKD09Wz8TMKf7B+4/4eqMDdf PvmQmJ+6eqkpN6wmey3a7KU5OpInAiTjsu6FHUHuRAFXFbjkesvgaiiGsBKjUDciQ2jq ZTxYuMS8DdWIQcSZKm42jMyFrsiruXU53jBQeFC6i8Rct7kk1Xuis6QuVCN9694UQ0ip S4R5ncFmyr0orZrPxpzdUS39A8hk7i55CoSp3T5bfqRD7buxoEcMTUQARLaqSHAHyC7M cotQ== X-Google-Smtp-Source: ADUXVKLij/YbfWwoXlhTIsq3l6kEOu1tE3bFXIUvrF/sX9h2a+NB9G5T5MkxlQV53k6hqZUK29nKUw== X-Received: by 2002:a37:b943:: with SMTP id j64-v6mr76003qkf.362.1527628649081; Tue, 29 May 2018 14:17:29 -0700 (PDT) Received: from localhost (cpe-2606-A000-4381-1201-225-22FF-FEB3-E51A.dyn6.twc.com. [2606:a000:4381:1201:225:22ff:feb3:e51a]) by smtp.gmail.com with ESMTPSA id e5-v6sm22430568qkf.35.2018.05.29.14.17.28 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 29 May 2018 14:17:28 -0700 (PDT) From: Josef Bacik To: axboe@kernel.dk, kernel-team@fb.com, linux-block@vger.kernel.org, akpm@linux-foundation.org, linux-mm@kvack.org, hannes@cmpxchg.org, linux-kernel@vger.kernel.org, tj@kernel.org, linux-fsdevel@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 01/13] block: add bi_blkg to the bio for cgroups Date: Tue, 29 May 2018 17:17:12 -0400 Message-Id: <20180529211724.4531-2-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180529211724.4531-1-josef@toxicpanda.com> References: <20180529211724.4531-1-josef@toxicpanda.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik Currently io.low uses a bi_cg_private to stash its private data for the blkg, however other blkcg policies may want to use this as well. Since we can get the private data out of the blkg, move this to bi_blkg in the bio and make it generic, then we can use bio_associate_blkg() to attach the blkg to the bio. Theoretically we could simply replace the bi_css with this since we can get to all the same information from the blkg, however you have to lookup the blkg, so for example wbc_init_bio() would have to lookup and possibly allocate the blkg for the css it was trying to attach to the bio. This could be problematic and result in us either not attaching the css at all to the bio, or falling back to the root blkcg if we are unable to allocate the corresponding blkg. So for now do this, and in the future if possible we could just replace the bi_css with bi_blkg and update the helpers to do the correct translation. Signed-off-by: Josef Bacik Acked-by: Tejun Heo --- block/bio.c | 23 +++++++++++++++++++++++ block/blk-throttle.c | 21 +++++++-------------- include/linux/bio.h | 1 + include/linux/blk_types.h | 2 +- 4 files changed, 32 insertions(+), 15 deletions(-) diff --git a/block/bio.c b/block/bio.c index 0a4df92cd689..57c4b1986e76 100644 --- a/block/bio.c +++ b/block/bio.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include "blk.h" @@ -2026,6 +2027,24 @@ int bio_associate_blkcg(struct bio *bio, struct cgroup_subsys_state *blkcg_css) } EXPORT_SYMBOL_GPL(bio_associate_blkcg); +/** + * bio_associate_blkg - associate a bio with the specified blkg + * @bio: target bio + * @blkg: the blkg to associate + * + * Associate @bio with the blkg specified by @blkg. This is the queue specific + * blkcg information associated with the @bio, a reference will be taken on the + * @blkg and will be freed when the bio is freed. + */ +int bio_associate_blkg(struct bio *bio, struct blkcg_gq *blkg) +{ + if (unlikely(bio->bi_blkg)) + return -EBUSY; + blkg_get(blkg); + bio->bi_blkg = blkg; + return 0; +} + /** * bio_disassociate_task - undo bio_associate_current() * @bio: target bio @@ -2040,6 +2059,10 @@ void bio_disassociate_task(struct bio *bio) css_put(bio->bi_css); bio->bi_css = NULL; } + if (bio->bi_blkg) { + blkg_put(bio->bi_blkg); + bio->bi_blkg = NULL; + } } /** diff --git a/block/blk-throttle.c b/block/blk-throttle.c index f63d88c92c3a..5112cef3166b 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -2131,12 +2131,8 @@ static inline void throtl_update_latency_buckets(struct throtl_data *td) static void blk_throtl_assoc_bio(struct throtl_grp *tg, struct bio *bio) { #ifdef CONFIG_BLK_DEV_THROTTLING_LOW - if (bio->bi_css) { - if (bio->bi_cg_private) - blkg_put(tg_to_blkg(bio->bi_cg_private)); - bio->bi_cg_private = tg; - blkg_get(tg_to_blkg(tg)); - } + if (bio->bi_css) + bio_associate_blkg(bio, tg_to_blkg(tg)); bio_issue_init(&bio->bi_issue, bio_sectors(bio)); #endif } @@ -2284,6 +2280,7 @@ void blk_throtl_stat_add(struct request *rq, u64 time_ns) void blk_throtl_bio_endio(struct bio *bio) { + struct blkcg_gq *blkg; struct throtl_grp *tg; u64 finish_time_ns; unsigned long finish_time; @@ -2291,20 +2288,18 @@ void blk_throtl_bio_endio(struct bio *bio) unsigned long lat; int rw = bio_data_dir(bio); - tg = bio->bi_cg_private; - if (!tg) + blkg = bio->bi_blkg; + if (!blkg) return; - bio->bi_cg_private = NULL; + tg = blkg_to_tg(blkg); finish_time_ns = ktime_get_ns(); tg->last_finish_time = finish_time_ns >> 10; start_time = bio_issue_time(&bio->bi_issue) >> 10; finish_time = __bio_issue_time(finish_time_ns) >> 10; - if (!start_time || finish_time <= start_time) { - blkg_put(tg_to_blkg(tg)); + if (!start_time || finish_time <= start_time) return; - } lat = finish_time - start_time; /* this is only for bio based driver */ @@ -2333,8 +2328,6 @@ void blk_throtl_bio_endio(struct bio *bio) tg->bio_cnt /= 2; tg->bad_bio_cnt /= 2; } - - blkg_put(tg_to_blkg(tg)); } #endif diff --git a/include/linux/bio.h b/include/linux/bio.h index 98b175cc00d5..f2f3f1428e81 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -547,6 +547,7 @@ do { \ #ifdef CONFIG_BLK_CGROUP int bio_associate_blkcg(struct bio *bio, struct cgroup_subsys_state *blkcg_css); +int bio_associate_blkg(struct bio *bio, struct blkcg_gq *blkg); void bio_disassociate_task(struct bio *bio); void bio_clone_blkcg_association(struct bio *dst, struct bio *src); #else /* CONFIG_BLK_CGROUP */ diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 4cb970cdcd11..ff181df8c195 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -179,8 +179,8 @@ struct bio { */ struct io_context *bi_ioc; struct cgroup_subsys_state *bi_css; + struct blkcg_gq *bi_blkg; #ifdef CONFIG_BLK_DEV_THROTTLING_LOW - void *bi_cg_private; struct bio_issue bi_issue; #endif #endif