From patchwork Mon Sep 17 20:15:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10603373 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 434B317D5 for ; Mon, 17 Sep 2018 20:16:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 373062A313 for ; Mon, 17 Sep 2018 20:16:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2B81E2A319; Mon, 17 Sep 2018 20:16:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 343582A35C for ; Mon, 17 Sep 2018 20:16:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728451AbeIRBox (ORCPT ); Mon, 17 Sep 2018 21:44:53 -0400 Received: from out002.mailprotect.be ([83.217.72.86]:46689 "EHLO out002.mailprotect.be" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728190AbeIRBow (ORCPT ); Mon, 17 Sep 2018 21:44:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mailprotect.be; s=mail; h=Content-Transfer-Encoding:MIME-Version:References :In-Reply-To:Message-Id:Date:Subject:Cc:To:From:reply-to:sender:bcc: content-type; bh=uGTqZ/I1DERDWhbXQNeTyhkA3T3szIIRKzXeRmsBqIU=; b=Cl+0rT6oc2n8 r0+zW3Bf0Lo3N1BWK+zMjGZ1V6GqMby2bHIQWzmxhyAcrKqlXIh2bzdiEaSNj4LktZFcLmlfGjrI3 1nYh0R6jHtAHuSPxxbvzVRUWjDIFqQ2Rm0rsoy61uC1F40elS+rVxFH5pDGV1fqKdPx1xVPy64EAo OooI4eXtg9vw588jYKyOmNOWdxldus2twAxpnZt7x5YnCFZSPttikO9uwaTk+vLgO/W7195iBGxFV p+X5fLQzOd2CT0/HvR+iwVpMvEVBs1gaVCJ4lOX3Vy+r4bs6wj21jjpy0lmFm+XeNc2AUB9v3mLPC 3Ax99rYKdDltmFILywyQmw==; Received: from smtp-auth.mailprotect.be ([178.208.39.155]) by com-mpt-out002.mailprotect.be with esmtp (Exim 4.89) (envelope-from ) id 1g1zvx-000AbJ-GJ; Mon, 17 Sep 2018 22:15:49 +0200 Received: from asus.hsd1.ca.comcast.net (c-174-62-111-89.hsd1.ca.comcast.net [174.62.111.89]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-auth.mailprotect.be (Postfix) with ESMTPSA id 7F05AC05E8; Mon, 17 Sep 2018 22:15:47 +0200 (CEST) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Ming Lei , Jianchao Wang , Hannes Reinecke , Johannes Thumshirn , Alan Stern Subject: [PATCH v7 5/6] block: Make blk_get_request() block for non-PM requests while suspended Date: Mon, 17 Sep 2018 13:15:24 -0700 Message-Id: <20180917201525.26570-6-bvanassche@acm.org> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180917201525.26570-1-bvanassche@acm.org> References: <20180917201525.26570-1-bvanassche@acm.org> MIME-Version: 1.0 X-Originating-IP: 178.208.39.155 X-SpamExperts-Domain: mailprotect.be X-SpamExperts-Username: 178.208.39.128/27 Authentication-Results: mailprotect.be; auth=pass smtp.auth=178.208.39.128/27@mailprotect.be X-SpamExperts-Outgoing-Class: ham X-SpamExperts-Outgoing-Evidence: SB/global_tokens (0.000475936851056) X-Recommended-Action: accept X-Filter-ID: EX5BVjFpneJeBchSMxfU5is4jj6pL6YMflByL4LVCsB602E9L7XzfQH6nu9C/Fh9KJzpNe6xgvOx q3u0UDjvO1tLifGj39bI0bcPyaJsYTaOqZpfO3PVJjdazu3l6Zm3CrxbKqqxVb1b/D8J7mjn9ilV cTJ2PzGYt6C/dLANFNC9eBrzwetNxEiSKDRqnfKMchy245qaCGq3TeWvoSybAt5ZBZ2f8Z/psVOj OL3ZCmydclC0ZTlj5mu9SqtPNLdvD5ZcU14fn9LDWdfT12La6fnkh42tAYDRw9CusfrzVmK0WsJ7 KGS+qYB0uWxgiY7CnWgbDYoAoppqKkXjYJMeqS7vMPoASQBHF7NsWwEU7vVcSv/qU9FFqqWjPxOZ 4rS9q8bJZbS7LYC0Zu1jKFR3I4SYjNUoBCz9aTu56OA1tEt5/6v7c9QHckE3f/XVZktwTvADW575 0b8F3/kmeu/2zgCZ3lN0dg693VokPoLlNvu19yPULOCRMJQ2BgM/Wut6cO9MNpwkpncXgB6bffaX 7xYrrIBbnoZxxFp2ILDar2JNQvoC5ZFW9jc7Gmccf4fCAIyNSTWVq4dZvieuPRMUTLlOMYNNn/Xj 9WX1Vxj8fmy0GPm9ewau6K38/Gs14AgnMdRatk7KHHjyULplJGegbdxbXpCgbiKBsA+Ddi6mawer tqWJnTnoN4VUNd4gjQeVvLN728IxxZDbDiI7+RnRY0757KiUYziXyuXG4Z+FT7f2Z5FmZiYh5zrn 6SPoZlSUVqZvWGb8FLl4ADZ/Txs2SsRyUiMV0bJ59aqbfue3i8/41P+vcbHOYMtZCUK3cTvB5CC0 B0N4GDXwtimutwTDmjDptKOiJtD3jqCFxjwevNzM4oqJungo5Nyb7xtfruqYRZ1RoScDOsq2Parb hQOhvUFGnvKVJnFQQaAnDicXOB3EjzrwzpOE5mwgEdfdYo4D+vH10Kef7/oaji7+V0umCbBezpWT CcpgMNUt3YH99/LRMrw5DdwH1AXXXLvYNFIcFNl57WdwhmTsAKzUogeC9MstwPY3IpV4nahE1DKZ QsJSOYE66SFhT6NVrQT/pWDn674K8PUeEVvDdxLV5QoWYCUJDd78sizi2LOFEV5aLJmxmF3d2RZv qUM/cwNhuECOygEwwwWe3xXNBQ8NJdm4Puz0ZaEH4EagAPQe7incqLSp9Bs6vFf0/xkS2e/oeNZr pAEntM8yKElR+Y84QVlpY8ekY5wl4Q3nPDb6eyxRMZLQiWYpjzKs/f2li8jAd+iCpKNMVj7VMvuF NvyQYijYug== X-Report-Abuse-To: spam@com-mpt-mgt001.mailprotect.be Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Instead of allowing requests that are not power management requests to enter the queue in runtime suspended status (RPM_SUSPENDED), make the blk_get_request() caller block. This change fixes a starvation issue: it is now guaranteed that power management requests will be executed no matter how many blk_get_request() callers are waiting. Instead of maintaining the q->nr_pending counter, rely on q->q_usage_counter. Call pm_runtime_mark_last_busy() every time a request finishes instead of only if the queue depth drops to zero. Signed-off-by: Bart Van Assche Cc: Christoph Hellwig Cc: Ming Lei Cc: Jianchao Wang Cc: Hannes Reinecke Cc: Johannes Thumshirn Cc: Alan Stern --- block/blk-core.c | 37 ++++++------------------- block/blk-pm.c | 72 ++++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 75 insertions(+), 34 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 18b874d5c9c9..ae092ca121d5 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -2747,30 +2747,6 @@ void blk_account_io_done(struct request *req, u64 now) } } -#ifdef CONFIG_PM -/* - * Don't process normal requests when queue is suspended - * or in the process of suspending/resuming - */ -static bool blk_pm_allow_request(struct request *rq) -{ - switch (rq->q->rpm_status) { - case RPM_RESUMING: - case RPM_SUSPENDING: - return rq->rq_flags & RQF_PM; - case RPM_SUSPENDED: - return false; - default: - return true; - } -} -#else -static bool blk_pm_allow_request(struct request *rq) -{ - return true; -} -#endif - void blk_account_io_start(struct request *rq, bool new_io) { struct hd_struct *part; @@ -2816,11 +2792,14 @@ static struct request *elv_next_request(struct request_queue *q) while (1) { list_for_each_entry(rq, &q->queue_head, queuelist) { - if (blk_pm_allow_request(rq)) - return rq; - - if (rq->rq_flags & RQF_SOFTBARRIER) - break; +#ifdef CONFIG_PM + /* + * If a request gets queued in state RPM_SUSPENDED + * then that's a kernel bug. + */ + WARN_ON_ONCE(q->rpm_status == RPM_SUSPENDED); +#endif + return rq; } /* diff --git a/block/blk-pm.c b/block/blk-pm.c index 9b636960d285..5f21bedcb307 100644 --- a/block/blk-pm.c +++ b/block/blk-pm.c @@ -1,8 +1,11 @@ // SPDX-License-Identifier: GPL-2.0 +#include #include #include #include +#include "blk-mq.h" +#include "blk-mq-tag.h" /** * blk_pm_runtime_init - Block layer runtime PM initialization routine @@ -40,6 +43,36 @@ void blk_pm_runtime_init(struct request_queue *q, struct device *dev) } EXPORT_SYMBOL(blk_pm_runtime_init); +struct in_flight_data { + struct request_queue *q; + int in_flight; +}; + +static void blk_count_in_flight(struct blk_mq_hw_ctx *hctx, struct request *rq, + void *priv, bool reserved) +{ + struct in_flight_data *in_flight = priv; + + if (rq->q == in_flight->q) + in_flight->in_flight++; +} + +/* + * Count the number of requests that are in flight for request queue @q. Use + * @q->nr_pending for legacy queues. Iterate over the tag set for blk-mq + * queues. Use blk_mq_queue_tag_busy_iter() instead of + * blk_mq_tagset_busy_iter() because the latter only considers requests that + * have already been started. + */ +static int blk_requests_in_flight(struct request_queue *q) +{ + struct in_flight_data in_flight = { .q = q }; + + if (q->mq_ops) + blk_mq_queue_tag_busy_iter(q, blk_count_in_flight, &in_flight); + return q->nr_pending + in_flight.in_flight; +} + /** * blk_pre_runtime_suspend - Pre runtime suspend check * @q: the queue of the device @@ -68,14 +101,38 @@ int blk_pre_runtime_suspend(struct request_queue *q) if (!q->dev) return ret; + WARN_ON_ONCE(q->rpm_status != RPM_ACTIVE); + + blk_set_pm_only(q); + /* + * This function only gets called if the most recent + * pm_request_resume() call occurred at least autosuspend_delay_ms + * ago. Since blk_queue_enter() is called by the request allocation + * code before pm_request_resume(), if no requests have a tag assigned + * it is safe to suspend the device. + */ + ret = -EBUSY; + if (blk_requests_in_flight(q) == 0) { + /* + * Call synchronize_rcu() such that later blk_queue_enter() + * calls see the pm-only state. See also + * http://lwn.net/Articles/573497/. + */ + synchronize_rcu(); + if (blk_requests_in_flight(q) == 0) + ret = 0; + } + spin_lock_irq(q->queue_lock); - if (q->nr_pending) { - ret = -EBUSY; + if (ret < 0) pm_runtime_mark_last_busy(q->dev); - } else { + else q->rpm_status = RPM_SUSPENDING; - } spin_unlock_irq(q->queue_lock); + + if (ret) + blk_clear_pm_only(q); + return ret; } EXPORT_SYMBOL(blk_pre_runtime_suspend); @@ -106,6 +163,9 @@ void blk_post_runtime_suspend(struct request_queue *q, int err) pm_runtime_mark_last_busy(q->dev); } spin_unlock_irq(q->queue_lock); + + if (err) + blk_clear_pm_only(q); } EXPORT_SYMBOL(blk_post_runtime_suspend); @@ -153,13 +213,15 @@ void blk_post_runtime_resume(struct request_queue *q, int err) spin_lock_irq(q->queue_lock); if (!err) { q->rpm_status = RPM_ACTIVE; - __blk_run_queue(q); pm_runtime_mark_last_busy(q->dev); pm_request_autosuspend(q->dev); } else { q->rpm_status = RPM_SUSPENDED; } spin_unlock_irq(q->queue_lock); + + if (!err) + blk_clear_pm_only(q); } EXPORT_SYMBOL(blk_post_runtime_resume);