From patchwork Thu Aug 9 19:41:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10561875 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 17DEA13B4 for ; Thu, 9 Aug 2018 19:42:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 06CE82B979 for ; Thu, 9 Aug 2018 19:42:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EEC6B2B97F; Thu, 9 Aug 2018 19:42:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6EBFC2B979 for ; Thu, 9 Aug 2018 19:42:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727141AbeHIWIf (ORCPT ); Thu, 9 Aug 2018 18:08:35 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:41000 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727139AbeHIWIf (ORCPT ); Thu, 9 Aug 2018 18:08:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1533843738; x=1565379738; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=WGINhsKce1VX+PJBG+XVK6Bprksx1tcX3kRRsIWfRzM=; b=rRyQSSzl71upW9wOQRYGyyqorLYt1McFRfWUzqAFtNAfdU4vY/YI0LGH BIQvuVq4n0cVFt/8hFkMghmQrCtdTfIFKRHaDP7h2K9qkOdJh3HZGsY6W DCB1weG3rMX8uiRzmy9w/AfJK6r0SoxdilcRxmhCfae9QEDIjs5JvLU/V R1Bm1Lvf0/T5GBAyZZhzIvPaShPEBP3zMREfdzktP0fIwylGuJwQyhe6G onZpQBAjI8GaHvwVX6LRWNXY4xBTsJQSXO2DPNXb2h32PMI21hfXi1S3J D3xxgSz9Eahxjo4VVA7PHWMOKgarwho+0tR21eUhThilO+X16xokFv2N2 w==; X-IronPort-AV: E=Sophos;i="5.53,215,1531756800"; d="scan'208";a="90927380" Received: from mail-sn1nam01lp0111.outbound.protection.outlook.com (HELO NAM01-SN1-obe.outbound.protection.outlook.com) ([207.46.163.111]) by ob1.hgst.iphmx.com with ESMTP; 10 Aug 2018 03:42:17 +0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector1-wdc-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=30Gm0FC2H6EtXIhKstBXcVr8j83+VOpuhB+08kwHXfg=; b=C9puPwPO36MOSX+R2LiQ0I+zuPTa24FuA3Rn9mOHv7ryYYoP7tRAhbgco/GDCg4aDPcPzLcz54q+4bOo12kvj3YWkxcU6cv5j6cr83FDIGWcRZ6U8x7k155rivSNYWGx+Wupu/J33p1HmB34np1BS4539rI3XsD2C+3GR1lqD4E= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Bart.VanAssche@wdc.com; Received: from asus.hsd1.ca.comcast.net (174.62.111.89) by MWHPR04MB1198.namprd04.prod.outlook.com (2603:10b6:300:71::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1038.23; Thu, 9 Aug 2018 19:42:11 +0000 From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Ming Lei , Jianchao Wang , Hannes Reinecke , Johannes Thumshirn , Alan Stern Subject: [PATCH v6 11/12] block: Change the runtime power management approach (2/2) Date: Thu, 9 Aug 2018 12:41:48 -0700 Message-Id: <20180809194149.15285-12-bart.vanassche@wdc.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180809194149.15285-1-bart.vanassche@wdc.com> References: <20180809194149.15285-1-bart.vanassche@wdc.com> MIME-Version: 1.0 X-Originating-IP: [174.62.111.89] X-ClientProxiedBy: MWHPR2201CA0066.namprd22.prod.outlook.com (2603:10b6:301:5e::19) To MWHPR04MB1198.namprd04.prod.outlook.com (2603:10b6:300:71::23) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1f9544cf-5402-4ed6-01e6-08d5fe3033cc X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(7020095)(4652040)(8989117)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020);SRVR:MWHPR04MB1198; X-Microsoft-Exchange-Diagnostics: 1;MWHPR04MB1198;3:nbufttFaBWzWjSBY8dZFqSz/AsaMDKZJ/KDMUTWpw12VLpjpO3vEoKDimyJmi5lGNC6VbNPKrJG/jOJeIKYruKmNIkOH75y4B1YJGSu33Id9LXwBHl/29Gy4pvwHCxaSGvikJIOguVb40TlRsxrDHsMd1wxE0pKMhqeuEdk2cXmU+DDjjXKrGMovHkU+qmtK+Hl9lDp4S2Z1yhM5bNUTocSP2AVmevL49bJ14FTCl8V+YcG6qbCw14M0VHEDgH1b;25:1kO8LHfelVqUNvGD9V6Cvm8mousY279Qkua8zIj0+5OanIs9EMYjY5NsfyOl/vkPU/vXL3JIenI/nckgJR3H329/452XowIE9bLPDWzuXNchsw7vichTH7qMJ1PUtiHO4AiTmXBSrt3uO0Cn6klqs9LYezFqnAwP1YHj36B6Xjv9fH1xg96k0kSn7EWUMEDspc0k5jhQHKvJNKRrk/VOkLyKiUVUBdW2QE6FSnUb3lsBwqfvc3m3ipahvBI0jWt7v9JXM2DwSN1IT+G8sQZrywmqtYzu/d0aQDXWMsDUcG5GVVzcUYy8pfgV0OSrW76DjZJgbLJI6TSZys/aF3QJpA==;31:HahrEkoQimp64yFsNJMBbQE+hZCMnd+K5tAhIyreQsRXhGFkEnDmPekynyirAQyHhJSjqGjOR3UlTHpsCixEv69r5WIbfj1k78ALOYTOwjCaBsQtoZg6c3gBKfaHVIInW+uOMkqUJNxwao0kPRzAu8CP4pLZiAybIErh+NhGfD9gpdd5weNVv4z431scDhC28Sq4Qyq96gb/ZRRM2/SPIMT7d+Muh9XCh6nw9cGdyWo= X-MS-TrafficTypeDiagnostic: MWHPR04MB1198: WDCIPOUTBOUND: EOP-TRUE X-Microsoft-Exchange-Diagnostics: 1;MWHPR04MB1198;20:Rtr3zXd68uqtW8YLWfHuWNRlu8xAYNo4N0g2kyJODVwTupc+N/ToFxfrBivPYOaQ0p2ILtJbcKZjgB+NvjaL1JsUQ4Onx/2fLQZbWVGnNZ9ulYoLyVE9tjvTT8zwsagpckC9DtfcWNQHwO86HAq6vy3o/A+nV01ggWmhcVfqGBLZLGEyd+9W4wj+eE6Utrmjd8GgX+Bpa/08aU3gP/XRTruetmWrAt11SLklzKgk7HhoUFyooqzmcNZljD2VkW6u88FsuxqzZnNdoZmkXs+UYkqt17BgoAo88Myt2cgHjr7fQHeBM29Os/N6K6xCkHePtXh70PNZAk9FHLXFlsld8vJVhrIHz/YcndQLWnUQWnDTr5PO1NRRgpXhitJ2XRfQDHNonw7wFzNC4T/yE466DjhUj0QLDA/gmoP0roeVqs3WB5oJFOJARCay6aHpkJzy5ea0TsKv3U3U2z6ZJGsapqkLfK/LyZspMJz2uAbtUgYMZtbhV2PMVGe0C4M6Ku9k;4:NAqgjwFKPeLM78WyA0NjEu4YUsqeUfsxxld4qxQRnOUnGLQJKIVaYqnTTXiXWHqOOrRKMfL+0MaNTKezRw2P2mP8bblmIOdd2eyHRKRd2ARnUgc7PIyGgFg289NWlQUksAKLs30an+frUnM8bUL4esqQ7N3MmynHU6helX0LgU8aBt6vi8ou39eFTh/SlDOqR7SlI2GWGZCBXZWZ8iBCoS97SWxHU+biN9T/OhFWGFaqPB1p4qs58lgd+K/24eEA/CAhOAwJg/zRlAvwC/DHg+1UWjfxJpP7OReLxuVgwtZj9x2xrf8mKGKhzTTDqcWjJYITYMMigfblrSydkhDxEtyKn2t4OwY9sK6eHzh8aV8= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(35762410373642)(146099531331640); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040522)(2401047)(8121501046)(5005006)(93006095)(93001095)(3231311)(944501410)(52105095)(3002001)(10201501046)(6055026)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123558120)(20161123564045)(20161123560045)(6072148)(201708071742011)(7699016);SRVR:MWHPR04MB1198;BCL:0;PCL:0;RULEID:;SRVR:MWHPR04MB1198; X-Forefront-PRVS: 0759F7A50A X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(396003)(39860400002)(136003)(366004)(346002)(376002)(199004)(189003)(966005)(76176011)(50466002)(476003)(51416003)(4326008)(478600001)(52116002)(6486002)(105586002)(72206003)(48376002)(5660300001)(47776003)(305945005)(2616005)(956004)(53376002)(7736002)(6916009)(6666003)(486006)(8936002)(2906002)(68736007)(36756003)(66066001)(575784001)(186003)(446003)(81166006)(50226002)(8676002)(81156014)(316002)(6506007)(386003)(106356001)(97736004)(25786009)(11346002)(16586007)(54906003)(14444005)(6116002)(1076002)(3846002)(86362001)(26005)(16526019)(53936002)(6512007)(6306002);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR04MB1198;H:asus.hsd1.ca.comcast.net;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; X-Microsoft-Exchange-Diagnostics: 1;MWHPR04MB1198;23:2hD2eBkzvCqcGC+Z4g2T5UzkyXGl+BW2tFE6j+pzrGIBSNKCBJ1nN94me5WgTEFDRKgqxIvBwCoRB+jcXFAoDmQSI2WytebdY7vxRFMYVMYqo07FlS4vlYET94ocqP/IUorjUNQyFLfXPgYSMLGan2AbIUerAypexPz3WIvdIVbH8D5BnlIxSAw8nMiVXRQRjn1d/5ewgGK0wSXFPwbVbk9wYcntqvKhc1+zYfK/xCKgrdc5gT+ugkHr82TXk+q7eM5ZoX16EWLFnWXVoiUUiGTJnKVsTFecRw06Ddjd9B0SePtXqJyGlxxlmsHWd4hKibq7NAdZPQmu1z4TI8Go55kPu0RM0kCfpRn4Rkt01yBbd7yQf9X6hpjzZ+0MHS9xW7vyjTOJrynevdz7RwH+e0Jzvd603GCf8rdVSVVSIQ2ASjhFfFg00ShnvlG7rJwF/gdNW6QnMATbvEo99KgqMMRQqE7BTlxyFnu61nlyKSAoKJUcksyQYr3hyyHd4bc1WfRzgj0fcU993khrmV/MSFaLmw8lq7VtbLovEPQiuUwWhF8SkPUA3U8RK2NmM4GXsALpXvswDx4D8KyXlKx/AawZ+Vi/+oOH5daUbHgFiQY3snsv+i5xBiT+jrQ4GCVf9dhp4ZlPwPd06QNu7hR+CdDeGRGKzGVwwxgkhrmpAdnLsF0T6RysfYiibH5X7T7SYvBpi7+O5bKCIG42Ay83i6rSq1NtvPmjRUY/rRrqUbWEks5i6WffXwau72Ovw4cMJi4XLrBP6tf2I7UfPvuWfE/iOpV+pmc/SdLYdVfZqa/RCK+qgtsGf62j0WGYtBYxLHxEHP1wwJEDrwcW8Q8RBaddkGgKzqL3rCFiucLT7ykeX0cNZU+thXJ9v7O02OBkKpuMnIsxbc0rzYJ9ePnsL3oiIi+2qLrW1Tn6FPWaCiZvqmSJNpQuvid/qAWMiF8TBMHB9hVHx3vAEgkcOVpfOHMCbPhcUyXDLmsFPEUYpK5UFadHiopfcHhtpi4Ilx9onNlwgCS3UbRtkuBsqyBpniXd3Xpn3neW0U5nuBPHnjA5VHoa6lEc6Q/j8h/9ZwejzxRTjhWctdg7rrMcwE0TzMVsSm1KxapTqlMBNi92BvC7SxNkyqBKOBlmlcWqhNFto2WE5WxZN2nHDamDsMt+oTmyHwPwtXaWr57EEmfQP3+2NeGHPZft4l99c8hNhxbFZdKks+3wk1lpbCiOh4F+Uw9F/mCMrmHibPZCaA8O1gIJVXVeI/r6/i+ae+SjXPh5UOPzr+ITvy7lZonQEkc4ULW5ZluVbomM8AUG0KAifig+jRZsTlNHEEyIal7M1zQC X-Microsoft-Antispam-Message-Info: JwqcRv2LWWH0aofGlkaLCM6N7WvpjmruzuI/0jMogqvceDnwZhctfwgt6kLvUC/aduf0F30nQv8X5f2UHLILHyqwIX3YoGV2vV7H9WwMRb2QKkwManAJB9a6QPj41CmW+/bEd5SjqxQqT5g0RbI9pqvM4e78kk+FHI0f+mcRImXZyGTQGVGgFkN+vfrpTg9OiwRwRcMp4+VoByyGEVjyjlyifod0TkGpvcJQOVzTMC1QWi0K7swy3OH+F8sclJflalPbvFTp5/kKP3PS8Fe6c7Jvp1gl5RPQp0odzRxdhjdyWsju+KPo9nfxTcb2YMjqe/ooG8qppTAylHIwwafB/PCUvujhymrLG+t9bY+mMpE= X-Microsoft-Exchange-Diagnostics: 1;MWHPR04MB1198;6:CR4kXQ4svJ8J3M/jSz0n+M1ljct7l/9vQC9t+t6EkRlWcO/fmFuJoUopK76Vuwiu0xvuBnsuBdLyZcx2p2CusZ2jQpY9oVr5wZjFbJdoEUXxe0cI2f3WAZlnq0aDpbh022cyNRCAR6KOyJJDm/twdn0I15RGGvHfSRkxo18Rogka6wMK/jVGpqeBZHd+ZHhMGIRX7oDBdMuYbHRXvy1ekoSgnPOMBMUYJYCOXRZ0XCbptOVrW/aB5oqGBsK3j017OFptJZRkZi7rzVtQAMlaeLujgup+GjluEu+gaHlbq23H94DJD9OTxvg8KmPRplzfs9pj/fMnb/SRZ/rv+1t6Ylxm9s2gtFqXxezoud+BSdjnQW6pwSxpCexFVGXWumuUwWyrVeAT3p086x56UstgGRGWSIOPGp9Qriq+i08ltJwmJSWfRrF7z/MP5pEEVlwp+PjOwMruxzOEuKBTskI2RQ==;5:ltHhLe2JH/ATMbIXxR5+zu3szhoVDFl8S7lyuHydALWDnXVIiSR6UPQJhyStycj260ste1rZWkw3axVNqQk8QEQ8x7iBV71+JKjRa1XqFOUikHMwBPb6Qqw+IiW5m8v/zafG7oQ2DweHIr82w0EqK8tv82RCCLpcpDvHJYpIFw4=;7:hMU7ujAMvzydVMD8eK9x7ZLWKyAWm35aWWlCpuDxQeigkBQT05NU3po+jqbanW0fA8yug5gwm6n6x8MFbVcN+pIcVGg2aCEJeLbv5PiqDiixRpFkbNFNR6vKHjlW8APp3pYRCgoT6fCViStdosHq/oj4DcMYqDlott+YQlctMltyliyxugBd7OeK0ggOMfg8fGTLq2MmDYaGCobpuT5g8sl1DDYR3z33b5oc2UFRJCh91BOiP1FKbJzWBDKrlOba SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;MWHPR04MB1198;20:GGMEosi5vGYnrWsNmoey6n+yOZw2E0qSXhLl4Je/eRd0dvBNykCUyoDL00rCPEyqX4jAndh1A0VB5iRQGUZANkEu92F0faWsZauI/RIbDa+GZqUTeNzGEgO5nxFr2CtV2Dy7gxTkLR6A79lTY3sm0yF6tGLApTuXxKA6kGtjm80= X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2018 19:42:11.9993 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1f9544cf-5402-4ed6-01e6-08d5fe3033cc X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR04MB1198 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Instead of allowing requests that are not power management requests to enter the queue in runtime suspended status (RPM_SUSPENDED), make the blk_get_request() caller block. This change fixes a starvation issue: it is now guaranteed that power management requests will be executed no matter how many blk_get_request() callers are waiting. Instead of maintaining the q->nr_pending counter, rely on q->q_usage_counter. Call pm_runtime_mark_last_busy() every time a request finishes instead of only if the queue depth drops to zero. Signed-off-by: Bart Van Assche Cc: Christoph Hellwig Cc: Ming Lei Cc: Jianchao Wang Cc: Hannes Reinecke Cc: Johannes Thumshirn Cc: Alan Stern --- block/blk-core.c | 37 ++++++------------------- block/blk-pm.c | 72 ++++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 75 insertions(+), 34 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index f30545fb2de2..b0bb6b5320fe 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -2772,30 +2772,6 @@ void blk_account_io_done(struct request *req, u64 now) } } -#ifdef CONFIG_PM -/* - * Don't process normal requests when queue is suspended - * or in the process of suspending/resuming - */ -static bool blk_pm_allow_request(struct request *rq) -{ - switch (rq->q->rpm_status) { - case RPM_RESUMING: - case RPM_SUSPENDING: - return rq->rq_flags & RQF_PM; - case RPM_SUSPENDED: - return false; - default: - return true; - } -} -#else -static bool blk_pm_allow_request(struct request *rq) -{ - return true; -} -#endif - void blk_account_io_start(struct request *rq, bool new_io) { struct hd_struct *part; @@ -2841,11 +2817,14 @@ static struct request *elv_next_request(struct request_queue *q) while (1) { list_for_each_entry(rq, &q->queue_head, queuelist) { - if (blk_pm_allow_request(rq)) - return rq; - - if (rq->rq_flags & RQF_SOFTBARRIER) - break; +#ifdef CONFIG_PM + /* + * If a request gets queued in state RPM_SUSPENDED + * then that's a kernel bug. + */ + WARN_ON_ONCE(q->rpm_status == RPM_SUSPENDED); +#endif + return rq; } /* diff --git a/block/blk-pm.c b/block/blk-pm.c index bf8532da952d..977beffdccd2 100644 --- a/block/blk-pm.c +++ b/block/blk-pm.c @@ -1,8 +1,11 @@ // SPDX-License-Identifier: GPL-2.0 +#include #include #include #include +#include "blk-mq.h" +#include "blk-mq-tag.h" /** * blk_pm_runtime_init - Block layer runtime PM initialization routine @@ -58,6 +61,36 @@ void blk_pm_runtime_exit(struct request_queue *q) } EXPORT_SYMBOL(blk_pm_runtime_exit); +struct in_flight_data { + struct request_queue *q; + int in_flight; +}; + +static void blk_count_in_flight(struct blk_mq_hw_ctx *hctx, struct request *rq, + void *priv, bool reserved) +{ + struct in_flight_data *in_flight = priv; + + if (rq->q == in_flight->q) + in_flight->in_flight++; +} + +/* + * Count the number of requests that are in flight for request queue @q. Use + * @q->nr_pending for legacy queues. Iterate over the tag set for blk-mq + * queues. Use blk_mq_queue_tag_busy_iter() instead of + * blk_mq_tagset_busy_iter() because the latter only considers requests that + * have already been started. + */ +static int blk_requests_in_flight(struct request_queue *q) +{ + struct in_flight_data in_flight = { .q = q }; + + if (q->mq_ops) + blk_mq_queue_tag_busy_iter(q, blk_count_in_flight, &in_flight); + return q->nr_pending + in_flight.in_flight; +} + /** * blk_pre_runtime_suspend - Pre runtime suspend check * @q: the queue of the device @@ -86,14 +119,38 @@ int blk_pre_runtime_suspend(struct request_queue *q) if (!q->dev) return ret; + WARN_ON_ONCE(q->rpm_status != RPM_ACTIVE); + + blk_set_pm_only(q); + /* + * This function only gets called if the most recent + * pm_request_resume() call occurred at least autosuspend_delay_ms + * ago. Since blk_queue_enter() is called by the request allocation + * code before pm_request_resume(), if no requests have a tag assigned + * it is safe to suspend the device. + */ + ret = -EBUSY; + if (blk_requests_in_flight(q) == 0) { + /* + * Call synchronize_rcu() such that later blk_queue_enter() + * calls see the preempt-only state. See also + * http://lwn.net/Articles/573497/. + */ + synchronize_rcu(); + if (blk_requests_in_flight(q) == 0) + ret = 0; + } + spin_lock_irq(q->queue_lock); - if (q->nr_pending) { - ret = -EBUSY; + if (ret < 0) pm_runtime_mark_last_busy(q->dev); - } else { + else q->rpm_status = RPM_SUSPENDING; - } spin_unlock_irq(q->queue_lock); + + if (ret) + blk_clear_pm_only(q); + return ret; } EXPORT_SYMBOL(blk_pre_runtime_suspend); @@ -124,6 +181,9 @@ void blk_post_runtime_suspend(struct request_queue *q, int err) pm_runtime_mark_last_busy(q->dev); } spin_unlock_irq(q->queue_lock); + + if (err) + blk_clear_pm_only(q); } EXPORT_SYMBOL(blk_post_runtime_suspend); @@ -171,13 +231,15 @@ void blk_post_runtime_resume(struct request_queue *q, int err) spin_lock_irq(q->queue_lock); if (!err) { q->rpm_status = RPM_ACTIVE; - __blk_run_queue(q); pm_runtime_mark_last_busy(q->dev); pm_request_autosuspend(q->dev); } else { q->rpm_status = RPM_SUSPENDED; } spin_unlock_irq(q->queue_lock); + + if (!err) + blk_clear_pm_only(q); } EXPORT_SYMBOL(blk_post_runtime_resume);