From patchwork Sat Aug 4 00:03:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10555557 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 903E41390 for ; Sat, 4 Aug 2018 00:03:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 80FAF2C913 for ; Sat, 4 Aug 2018 00:03:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 74EEC2C971; Sat, 4 Aug 2018 00:03:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9BEAE2C91B for ; Sat, 4 Aug 2018 00:03:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732072AbeHDCB4 (ORCPT ); Fri, 3 Aug 2018 22:01:56 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:34551 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732001AbeHDCBz (ORCPT ); Fri, 3 Aug 2018 22:01:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1533341008; x=1564877008; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=L6z1y9HvN3xiuToMQx3QAk41wd6VwloclwNtW8urHlw=; b=KkB3aEiYP+Ulk3L8PsE4Q2cnW9StU1tWp7rVorN8LzdxQUNrVTZerGdN 0sRGKuokVRz4CtZxf031vl5XtDcDmJkQkZw/JOD/d4/JwEXg1bXdpzCX9 tCShSuuH73cny5/YzukyMXlPZT7/WmbKa/OrjX5butcGdljnb5zNataEs KSFemG3djWHJfmFrD4hcQJ28sO5A9WBzY6WnRiufl137hHpxZTQIXaGhw +pNcs3Qqg3CrYPbSGa7pUIC23tQ9wD9u3LsOBK/v7Ml+zjLtlcs0BIard nNfnR3DYQdWmd5qAnL77ply/RUaeT5NqhVhAFhP6Qazf+JRBwszOCRx/H g==; X-IronPort-AV: E=Sophos;i="5.51,440,1526313600"; d="scan'208";a="90059254" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 04 Aug 2018 08:03:26 +0800 Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP; 03 Aug 2018 16:51:11 -0700 Received: from thinkpad-bart.sdcorp.global.sandisk.com ([10.111.67.248]) by uls-op-cesaip01.wdc.com with ESMTP; 03 Aug 2018 17:03:25 -0700 From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Jianchao Wang , Ming Lei , Johannes Thumshirn , Alan Stern Subject: [PATCH v4 01/10] block: Change the preempt-only flag into a counter Date: Fri, 3 Aug 2018 17:03:16 -0700 Message-Id: <20180804000325.3610-2-bart.vanassche@wdc.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180804000325.3610-1-bart.vanassche@wdc.com> References: <20180804000325.3610-1-bart.vanassche@wdc.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Rename "preempt-only" into "pm-only" because the primary purpose of this mode is power management. Since the power management core may but does not have to resume a runtime suspended device before performing system-wide suspend and since a later patch will set "pm-only" mode as long as a block device is runtime suspended, make it possible to set "pm-only" mode from more than one context. Signed-off-by: Bart Van Assche Cc: Christoph Hellwig Cc: Jianchao Wang Cc: Ming Lei Cc: Johannes Thumshirn Cc: Alan Stern Reviewed-by: Hannes Reinecke Reviewed-by: Christoph Hellwig --- block/blk-core.c | 35 ++++++++++++++++++----------------- block/blk-mq-debugfs.c | 10 +++++++++- drivers/scsi/scsi_lib.c | 8 ++++---- include/linux/blkdev.h | 14 +++++++++----- 4 files changed, 40 insertions(+), 27 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index f9ad73d8573c..b83798c03ca8 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -421,24 +421,25 @@ void blk_sync_queue(struct request_queue *q) EXPORT_SYMBOL(blk_sync_queue); /** - * blk_set_preempt_only - set QUEUE_FLAG_PREEMPT_ONLY + * blk_set_pm_only - increment pm_only counter * @q: request queue pointer - * - * Returns the previous value of the PREEMPT_ONLY flag - 0 if the flag was not - * set and 1 if the flag was already set. */ -int blk_set_preempt_only(struct request_queue *q) +void blk_set_pm_only(struct request_queue *q) { - return blk_queue_flag_test_and_set(QUEUE_FLAG_PREEMPT_ONLY, q); + atomic_inc(&q->pm_only); } -EXPORT_SYMBOL_GPL(blk_set_preempt_only); +EXPORT_SYMBOL_GPL(blk_set_pm_only); -void blk_clear_preempt_only(struct request_queue *q) +void blk_clear_pm_only(struct request_queue *q) { - blk_queue_flag_clear(QUEUE_FLAG_PREEMPT_ONLY, q); - wake_up_all(&q->mq_freeze_wq); + int pm_only; + + pm_only = atomic_dec_return(&q->pm_only); + WARN_ON_ONCE(pm_only < 0); + if (pm_only == 0) + wake_up_all(&q->mq_freeze_wq); } -EXPORT_SYMBOL_GPL(blk_clear_preempt_only); +EXPORT_SYMBOL_GPL(blk_clear_pm_only); /** * __blk_run_queue_uncond - run a queue whether or not it has been stopped @@ -911,7 +912,7 @@ EXPORT_SYMBOL(blk_alloc_queue); */ int blk_queue_enter(struct request_queue *q, blk_mq_req_flags_t flags) { - const bool preempt = flags & BLK_MQ_REQ_PREEMPT; + const bool pm = flags & BLK_MQ_REQ_PREEMPT; while (true) { bool success = false; @@ -919,11 +920,11 @@ int blk_queue_enter(struct request_queue *q, blk_mq_req_flags_t flags) rcu_read_lock(); if (percpu_ref_tryget_live(&q->q_usage_counter)) { /* - * The code that sets the PREEMPT_ONLY flag is - * responsible for ensuring that that flag is globally - * visible before the queue is unfrozen. + * The code that increments the pm_only counter is + * responsible for ensuring that that counter is + * globally visible before the queue is unfrozen. */ - if (preempt || !blk_queue_preempt_only(q)) { + if (pm || !blk_queue_pm_only(q)) { success = true; } else { percpu_ref_put(&q->q_usage_counter); @@ -948,7 +949,7 @@ int blk_queue_enter(struct request_queue *q, blk_mq_req_flags_t flags) wait_event(q->mq_freeze_wq, (atomic_read(&q->mq_freeze_depth) == 0 && - (preempt || !blk_queue_preempt_only(q))) || + (pm || !blk_queue_pm_only(q))) || blk_queue_dying(q)); if (blk_queue_dying(q)) return -ENODEV; diff --git a/block/blk-mq-debugfs.c b/block/blk-mq-debugfs.c index cb1e6cf7ac48..a5ea86835fcb 100644 --- a/block/blk-mq-debugfs.c +++ b/block/blk-mq-debugfs.c @@ -102,6 +102,14 @@ static int blk_flags_show(struct seq_file *m, const unsigned long flags, return 0; } +static int queue_pm_only_show(void *data, struct seq_file *m) +{ + struct request_queue *q = data; + + seq_printf(m, "%d\n", atomic_read(&q->pm_only)); + return 0; +} + #define QUEUE_FLAG_NAME(name) [QUEUE_FLAG_##name] = #name static const char *const blk_queue_flag_name[] = { QUEUE_FLAG_NAME(QUEUED), @@ -132,7 +140,6 @@ static const char *const blk_queue_flag_name[] = { QUEUE_FLAG_NAME(REGISTERED), QUEUE_FLAG_NAME(SCSI_PASSTHROUGH), QUEUE_FLAG_NAME(QUIESCED), - QUEUE_FLAG_NAME(PREEMPT_ONLY), }; #undef QUEUE_FLAG_NAME @@ -209,6 +216,7 @@ static ssize_t queue_write_hint_store(void *data, const char __user *buf, static const struct blk_mq_debugfs_attr blk_mq_debugfs_queue_attrs[] = { { "poll_stat", 0400, queue_poll_stat_show }, { "requeue_list", 0400, .seq_ops = &queue_requeue_list_seq_ops }, + { "pm_only", 0600, queue_pm_only_show, NULL }, { "state", 0600, queue_state_show, queue_state_write }, { "write_hints", 0600, queue_write_hint_show, queue_write_hint_store }, { "zone_wlock", 0400, queue_zone_wlock_show, NULL }, diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index 9cb9a166fa0c..007bb2bc817d 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -2996,11 +2996,11 @@ scsi_device_quiesce(struct scsi_device *sdev) */ WARN_ON_ONCE(sdev->quiesced_by && sdev->quiesced_by != current); - blk_set_preempt_only(q); + blk_set_pm_only(q); blk_mq_freeze_queue(q); /* - * Ensure that the effect of blk_set_preempt_only() will be visible + * Ensure that the effect of blk_set_pm_only() will be visible * for percpu_ref_tryget() callers that occur after the queue * unfreeze even if the queue was already frozen before this function * was called. See also https://lwn.net/Articles/573497/. @@ -3013,7 +3013,7 @@ scsi_device_quiesce(struct scsi_device *sdev) if (err == 0) sdev->quiesced_by = current; else - blk_clear_preempt_only(q); + blk_clear_pm_only(q); mutex_unlock(&sdev->state_mutex); return err; @@ -3038,7 +3038,7 @@ void scsi_device_resume(struct scsi_device *sdev) mutex_lock(&sdev->state_mutex); WARN_ON_ONCE(!sdev->quiesced_by); sdev->quiesced_by = NULL; - blk_clear_preempt_only(sdev->request_queue); + blk_clear_pm_only(sdev->request_queue); if (sdev->sdev_state == SDEV_QUIESCE) scsi_device_set_state(sdev, SDEV_RUNNING); mutex_unlock(&sdev->state_mutex); diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 050d599f5ea9..1a08edbddf9f 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -506,6 +506,12 @@ struct request_queue { * various queue flags, see QUEUE_* below */ unsigned long queue_flags; + /* + * Number of contexts that have called blk_set_pm_only(). If this + * counter is above zero then only RQF_PM and RQF_PREEMPT requests are + * processed. + */ + atomic_t pm_only; /* * ida allocated id for this queue. Used to index queues from @@ -700,7 +706,6 @@ struct request_queue { #define QUEUE_FLAG_REGISTERED 26 /* queue has been registered to a disk */ #define QUEUE_FLAG_SCSI_PASSTHROUGH 27 /* queue supports SCSI commands */ #define QUEUE_FLAG_QUIESCED 28 /* queue has been quiesced */ -#define QUEUE_FLAG_PREEMPT_ONLY 29 /* only process REQ_PREEMPT requests */ #define QUEUE_FLAG_DEFAULT ((1 << QUEUE_FLAG_IO_STAT) | \ (1 << QUEUE_FLAG_SAME_COMP) | \ @@ -738,12 +743,11 @@ bool blk_queue_flag_test_and_clear(unsigned int flag, struct request_queue *q); ((rq)->cmd_flags & (REQ_FAILFAST_DEV|REQ_FAILFAST_TRANSPORT| \ REQ_FAILFAST_DRIVER)) #define blk_queue_quiesced(q) test_bit(QUEUE_FLAG_QUIESCED, &(q)->queue_flags) -#define blk_queue_preempt_only(q) \ - test_bit(QUEUE_FLAG_PREEMPT_ONLY, &(q)->queue_flags) +#define blk_queue_pm_only(q) atomic_read(&(q)->pm_only) #define blk_queue_fua(q) test_bit(QUEUE_FLAG_FUA, &(q)->queue_flags) -extern int blk_set_preempt_only(struct request_queue *q); -extern void blk_clear_preempt_only(struct request_queue *q); +extern void blk_set_pm_only(struct request_queue *q); +extern void blk_clear_pm_only(struct request_queue *q); static inline int queue_in_flight(struct request_queue *q) { From patchwork Sat Aug 4 00:03:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10555559 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A6263A637 for ; Sat, 4 Aug 2018 00:03:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 966302C913 for ; Sat, 4 Aug 2018 00:03:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8AC9A2C9D8; Sat, 4 Aug 2018 00:03:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C70952C91D for ; Sat, 4 Aug 2018 00:03:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732001AbeHDCB4 (ORCPT ); Fri, 3 Aug 2018 22:01:56 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:34549 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732034AbeHDCBz (ORCPT ); Fri, 3 Aug 2018 22:01:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1533341008; x=1564877008; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=hxHc6lUfwT9eME+EJ6NJNER28iMmJRU0r6f7tc4cbao=; b=qOxw9DZ240qj5Kt5c0vNEZnD3e2QkXFt+rNE2zij8aWDphBcZDWoC3ZR lWgc8FSVpW2dzBREVy9jFZRkXRrQREd3AgOjGnov43iZNyYdClvF+u4vO c6zzWbAFxRlmXOHjiZ/0nyn1ol3uu7SJsOvXc8zoeEpdocnHY/kDrY9XQ 0FIEIFlhAcyDoYd51J90VMHPJo+qOGY5DzS3cH/yAi7bjs3IAeWf9JZ/j UPNl2aG8ygXYDI5at2LfHjBgMu8maGlZRGJAvbG6C4WHsax1sCqShTmIU tzsEtFuvs6x0re36+uxWgTw6RrA1YOa40X4Utppa//aEHZpaoaOqU+2o3 A==; X-IronPort-AV: E=Sophos;i="5.51,440,1526313600"; d="scan'208";a="90059256" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 04 Aug 2018 08:03:27 +0800 Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP; 03 Aug 2018 16:51:11 -0700 Received: from thinkpad-bart.sdcorp.global.sandisk.com ([10.111.67.248]) by uls-op-cesaip01.wdc.com with ESMTP; 03 Aug 2018 17:03:25 -0700 From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , "Martin K . Petersen" , Jianchao Wang , Ming Lei , Alan Stern , Johannes Thumshirn Subject: [PATCH v4 02/10] block, scsi: Give RQF_PREEMPT back its original meaning Date: Fri, 3 Aug 2018 17:03:17 -0700 Message-Id: <20180804000325.3610-3-bart.vanassche@wdc.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180804000325.3610-1-bart.vanassche@wdc.com> References: <20180804000325.3610-1-bart.vanassche@wdc.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In kernel v2.6.18 RQF_PREEMPT was only set for IDE preempt requests. Later on the SCSI core was modified such that RQF_PREEMPT requests was set for all requests submitted by __scsi_execute(), including power management requests. RQF_PREEMPT requests are the only requests processed in the SDEV_QUIESCE state. Instead of setting RQF_PREEMPT for all requests submitted by __scsi_execute(), only set RQF_PM for power management requests. Modify blk_get_request() such that it blocks in pm-only mode on non-RQF_PM requests. Leave the code out from scsi_prep_state_check() that is no longer reachable. Signed-off-by: Bart Van Assche Cc: Martin K. Petersen Cc: Christoph Hellwig Cc: Jianchao Wang Cc: Ming Lei Cc: Alan Stern Cc: Johannes Thumshirn --- block/blk-core.c | 9 +++++---- drivers/scsi/scsi_lib.c | 28 ++++++++++++---------------- include/linux/blk-mq.h | 2 ++ include/linux/blkdev.h | 6 ++---- 4 files changed, 21 insertions(+), 24 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index b83798c03ca8..3378fe478e67 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -908,11 +908,11 @@ EXPORT_SYMBOL(blk_alloc_queue); /** * blk_queue_enter() - try to increase q->q_usage_counter * @q: request queue pointer - * @flags: BLK_MQ_REQ_NOWAIT and/or BLK_MQ_REQ_PREEMPT + * @flags: BLK_MQ_REQ_NOWAIT, BLK_MQ_REQ_PREEMPT and/or BLK_MQ_REQ_PM */ int blk_queue_enter(struct request_queue *q, blk_mq_req_flags_t flags) { - const bool pm = flags & BLK_MQ_REQ_PREEMPT; + const bool pm = flags & BLK_MQ_REQ_PM; while (true) { bool success = false; @@ -1570,7 +1570,7 @@ static struct request *get_request(struct request_queue *q, unsigned int op, goto retry; } -/* flags: BLK_MQ_REQ_PREEMPT and/or BLK_MQ_REQ_NOWAIT. */ +/* flags: BLK_MQ_REQ_PREEMPT, BLK_MQ_REQ_PM and/or BLK_MQ_REQ_NOWAIT. */ static struct request *blk_old_get_request(struct request_queue *q, unsigned int op, blk_mq_req_flags_t flags) { @@ -1613,7 +1613,8 @@ struct request *blk_get_request(struct request_queue *q, unsigned int op, struct request *req; WARN_ON_ONCE(op & REQ_NOWAIT); - WARN_ON_ONCE(flags & ~(BLK_MQ_REQ_NOWAIT | BLK_MQ_REQ_PREEMPT)); + WARN_ON_ONCE(flags & ~(BLK_MQ_REQ_NOWAIT | BLK_MQ_REQ_PREEMPT | + BLK_MQ_REQ_PM)); if (q->mq_ops) { req = blk_mq_alloc_request(q, op, flags); diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index 007bb2bc817d..481158ab5081 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -248,8 +248,8 @@ void scsi_queue_insert(struct scsi_cmnd *cmd, int reason) * @sshdr: optional decoded sense header * @timeout: request timeout in seconds * @retries: number of times to retry request - * @flags: flags for ->cmd_flags - * @rq_flags: flags for ->rq_flags + * @flags: REQ_* flags for ->cmd_flags + * @rq_flags: RQF_* flags for ->rq_flags * @resid: optional residual length * * Returns the scsi_cmnd result field if a command was executed, or a negative @@ -267,7 +267,8 @@ int __scsi_execute(struct scsi_device *sdev, const unsigned char *cmd, req = blk_get_request(sdev->request_queue, data_direction == DMA_TO_DEVICE ? - REQ_OP_SCSI_OUT : REQ_OP_SCSI_IN, BLK_MQ_REQ_PREEMPT); + REQ_OP_SCSI_OUT : REQ_OP_SCSI_IN, + rq_flags & RQF_PM ? BLK_MQ_REQ_PM : 0); if (IS_ERR(req)) return ret; rq = scsi_req(req); @@ -1353,20 +1354,15 @@ scsi_prep_state_check(struct scsi_device *sdev, struct request *req) ret = BLKPREP_DEFER; break; case SDEV_QUIESCE: - /* - * If the devices is blocked we defer normal commands. - */ - if (req && !(req->rq_flags & RQF_PREEMPT)) - ret = BLKPREP_DEFER; + WARN_ON_ONCE(!(req->rq_flags & RQF_PM)); break; - default: - /* - * For any other not fully online state we only allow - * special commands. In particular any user initiated - * command is not allowed. - */ - if (req && !(req->rq_flags & RQF_PREEMPT)) - ret = BLKPREP_KILL; + case SDEV_CANCEL: + ret = BLKPREP_KILL; + break; + case SDEV_CREATED: + case SDEV_RUNNING: + /* This code is never reached */ + WARN_ONCE(true, "sdev state = %d\n", sdev->sdev_state); break; } } diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 1da59c16f637..f15c1de51f5e 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -223,6 +223,8 @@ enum { BLK_MQ_REQ_INTERNAL = (__force blk_mq_req_flags_t)(1 << 2), /* set RQF_PREEMPT */ BLK_MQ_REQ_PREEMPT = (__force blk_mq_req_flags_t)(1 << 3), + /* for power management requests */ + BLK_MQ_REQ_PM = (__force blk_mq_req_flags_t)(1 << 4), }; struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op, diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 1a08edbddf9f..1b3fe78872dc 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -99,8 +99,7 @@ typedef __u32 __bitwise req_flags_t; #define RQF_MQ_INFLIGHT ((__force req_flags_t)(1 << 6)) /* don't call prep for this one */ #define RQF_DONTPREP ((__force req_flags_t)(1 << 7)) -/* set for "ide_preempt" requests and also for requests for which the SCSI - "quiesce" state must be ignored. */ +/* set for "ide_preempt" requests */ #define RQF_PREEMPT ((__force req_flags_t)(1 << 8)) /* contains copies of user pages */ #define RQF_COPY_USER ((__force req_flags_t)(1 << 9)) @@ -508,8 +507,7 @@ struct request_queue { unsigned long queue_flags; /* * Number of contexts that have called blk_set_pm_only(). If this - * counter is above zero then only RQF_PM and RQF_PREEMPT requests are - * processed. + * counter is above zero then only RQF_PM requests are processed. */ atomic_t pm_only; From patchwork Sat Aug 4 00:03:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10555561 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DDCBC157D for ; Sat, 4 Aug 2018 00:03:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D092C2C913 for ; Sat, 4 Aug 2018 00:03:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C4DA62C91D; Sat, 4 Aug 2018 00:03:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 12E922C9CD for ; Sat, 4 Aug 2018 00:03:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732073AbeHDCB4 (ORCPT ); Fri, 3 Aug 2018 22:01:56 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:34551 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731959AbeHDCB4 (ORCPT ); Fri, 3 Aug 2018 22:01:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1533341008; x=1564877008; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=CMm+QZdcd7NFpQmX8hg665eQvS4gE7t3ks7clca/ey8=; b=Xyob/EORuCTkf4zP/ZanjnUXtwaRLSuzalqbUJnjhfope7vT0sSv87h3 bFoeqHhiel9Ky4KnZKqifuH8HRh5XKDsPFlTth4jpa87/z4SLM7QVpGqo jggumG42WSIGjHaTVKAMGtPeJUBRD3s+XC+JWHYwfzz29C53I8Ba2Kvic ysuN5zJRe7zoMZB62uZugZlZcXc21Y/x8J21Tl9PMJhZ0UDSCBGlkfRdr +Ns7k9Kckc4egLYjX3OqXnz8AIaePNR8DA5z1ohVF3nYazggBUmOoyhsV 12gcTfHAGsM9Zg3Z3IHJKO6Vn8ci5C0KERRqx/+B05y9XvGAtNSVh8FhL g==; X-IronPort-AV: E=Sophos;i="5.51,440,1526313600"; d="scan'208";a="90059258" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 04 Aug 2018 08:03:27 +0800 Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP; 03 Aug 2018 16:51:11 -0700 Received: from thinkpad-bart.sdcorp.global.sandisk.com ([10.111.67.248]) by uls-op-cesaip01.wdc.com with ESMTP; 03 Aug 2018 17:03:25 -0700 From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , "David S . Miller" , Jianchao Wang , Ming Lei , Alan Stern , Johannes Thumshirn Subject: [PATCH v4 03/10] block, ide: Remove flag BLK_MQ_REQ_PREEMPT Date: Fri, 3 Aug 2018 17:03:18 -0700 Message-Id: <20180804000325.3610-4-bart.vanassche@wdc.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180804000325.3610-1-bart.vanassche@wdc.com> References: <20180804000325.3610-1-bart.vanassche@wdc.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Since it is no longer necessary that blk_get_request() knowns whether or not RQF_PREEMPT will be set, remove flag BLK_MQ_REQ_PREEMPT. This patch does not change any functionality. See also 039c635f4e66 ("ide, scsi: Tell the block layer at request allocation time about preempt requests"). Signed-off-by: Bart Van Assche Cc: David S. Miller Cc: Christoph Hellwig Cc: Jianchao Wang Cc: Ming Lei Cc: Alan Stern Cc: Johannes Thumshirn --- block/blk-core.c | 9 +++------ block/blk-mq.c | 2 -- drivers/ide/ide-pm.c | 3 ++- include/linux/blk-mq.h | 4 +--- 4 files changed, 6 insertions(+), 12 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 3378fe478e67..3a5aa56eab9d 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -908,7 +908,7 @@ EXPORT_SYMBOL(blk_alloc_queue); /** * blk_queue_enter() - try to increase q->q_usage_counter * @q: request queue pointer - * @flags: BLK_MQ_REQ_NOWAIT, BLK_MQ_REQ_PREEMPT and/or BLK_MQ_REQ_PM + * @flags: BLK_MQ_REQ_NOWAIT and/or BLK_MQ_REQ_PM */ int blk_queue_enter(struct request_queue *q, blk_mq_req_flags_t flags) { @@ -1431,8 +1431,6 @@ static struct request *__get_request(struct request_list *rl, unsigned int op, blk_rq_set_rl(rq, rl); rq->cmd_flags = op; rq->rq_flags = rq_flags; - if (flags & BLK_MQ_REQ_PREEMPT) - rq->rq_flags |= RQF_PREEMPT; /* init elvpriv */ if (rq_flags & RQF_ELVPRIV) { @@ -1570,7 +1568,7 @@ static struct request *get_request(struct request_queue *q, unsigned int op, goto retry; } -/* flags: BLK_MQ_REQ_PREEMPT, BLK_MQ_REQ_PM and/or BLK_MQ_REQ_NOWAIT. */ +/* flags: BLK_MQ_REQ_PM and/or BLK_MQ_REQ_NOWAIT. */ static struct request *blk_old_get_request(struct request_queue *q, unsigned int op, blk_mq_req_flags_t flags) { @@ -1613,8 +1611,7 @@ struct request *blk_get_request(struct request_queue *q, unsigned int op, struct request *req; WARN_ON_ONCE(op & REQ_NOWAIT); - WARN_ON_ONCE(flags & ~(BLK_MQ_REQ_NOWAIT | BLK_MQ_REQ_PREEMPT | - BLK_MQ_REQ_PM)); + WARN_ON_ONCE(flags & ~(BLK_MQ_REQ_NOWAIT | BLK_MQ_REQ_PM)); if (q->mq_ops) { req = blk_mq_alloc_request(q, op, flags); diff --git a/block/blk-mq.c b/block/blk-mq.c index c92ce06fd565..8b23ae34d949 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -300,8 +300,6 @@ static struct request *blk_mq_rq_ctx_init(struct blk_mq_alloc_data *data, rq->rq_flags = rq_flags; rq->cpu = -1; rq->cmd_flags = op; - if (data->flags & BLK_MQ_REQ_PREEMPT) - rq->rq_flags |= RQF_PREEMPT; if (blk_queue_io_stat(data->q)) rq->rq_flags |= RQF_IO_STAT; INIT_LIST_HEAD(&rq->queuelist); diff --git a/drivers/ide/ide-pm.c b/drivers/ide/ide-pm.c index 59217aa1d1fb..10d10e01c0f4 100644 --- a/drivers/ide/ide-pm.c +++ b/drivers/ide/ide-pm.c @@ -90,8 +90,9 @@ int generic_ide_resume(struct device *dev) } memset(&rqpm, 0, sizeof(rqpm)); - rq = blk_get_request(drive->queue, REQ_OP_DRV_IN, BLK_MQ_REQ_PREEMPT); + rq = blk_get_request(drive->queue, REQ_OP_DRV_IN, 0); ide_req(rq)->type = ATA_PRIV_PM_RESUME; + rq->rq_flags |= RQF_PREEMPT; rq->special = &rqpm; rqpm.pm_step = IDE_PM_START_RESUME; rqpm.pm_state = PM_EVENT_ON; diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index f15c1de51f5e..59f99b0019bb 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -221,10 +221,8 @@ enum { BLK_MQ_REQ_RESERVED = (__force blk_mq_req_flags_t)(1 << 1), /* allocate internal/sched tag */ BLK_MQ_REQ_INTERNAL = (__force blk_mq_req_flags_t)(1 << 2), - /* set RQF_PREEMPT */ - BLK_MQ_REQ_PREEMPT = (__force blk_mq_req_flags_t)(1 << 3), /* for power management requests */ - BLK_MQ_REQ_PM = (__force blk_mq_req_flags_t)(1 << 4), + BLK_MQ_REQ_PM = (__force blk_mq_req_flags_t)(1 << 3), }; struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op, From patchwork Sat Aug 4 00:03:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10555569 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BA2DF1390 for ; Sat, 4 Aug 2018 00:03:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A45F12C913 for ; Sat, 4 Aug 2018 00:03:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 96A052C91B; Sat, 4 Aug 2018 00:03:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 447392C91D for ; Sat, 4 Aug 2018 00:03:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732034AbeHDCB5 (ORCPT ); Fri, 3 Aug 2018 22:01:57 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:34549 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732049AbeHDCB5 (ORCPT ); Fri, 3 Aug 2018 22:01:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1533341008; x=1564877008; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=4nYhpe75/k6buxSInv4mlzCxtZpp48Ba14D3G6PKQ4I=; b=bNDF8ewwAxRbkvgXjvxigCIFYr7dSL0MpXKrweE3Yfjuaj1sPdLxu++q LLw5ABg2z1K19GSPzFkSHInqOOebnfGIilCfVmn6JBIW4ow8rtk+DRfGO bo0KpbqmKnigy9HHJzEfec5nK2chUrfNmYd3etRkkgqtk8pcs64OoxpT0 Qq8JpCN+q8DOcUswx2LSELPACi9bzMe+bt3y2QdldzRoBJBFvj54zqWMU SLKhA1SWFBqIbNQwzNFC7UaDMIraegOXMegOlc9XcGzUp0jj4VZBXGvug lcf3z6FJMi5bku3sJomweiR6hNmPFhPvRdhIvA1gwA7DwLe5S7z2jHuEo A==; X-IronPort-AV: E=Sophos;i="5.51,440,1526313600"; d="scan'208";a="90059261" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 04 Aug 2018 08:03:27 +0800 Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP; 03 Aug 2018 16:51:11 -0700 Received: from thinkpad-bart.sdcorp.global.sandisk.com ([10.111.67.248]) by uls-op-cesaip01.wdc.com with ESMTP; 03 Aug 2018 17:03:26 -0700 From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Jianchao Wang , Ming Lei , Johannes Thumshirn , Alan Stern Subject: [PATCH v4 04/10] block: Move power management code into a new source file Date: Fri, 3 Aug 2018 17:03:19 -0700 Message-Id: <20180804000325.3610-5-bart.vanassche@wdc.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180804000325.3610-1-bart.vanassche@wdc.com> References: <20180804000325.3610-1-bart.vanassche@wdc.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Move the code for runtime power management from blk-core.c into the new source file blk-pm.c. Move the corresponding declarations from into . For CONFIG_PM=n, leave out the declarations of the functions that are not used in that mode. This patch not only reduces the number of #ifdefs in the block layer core code but also reduces the size of header file and hence should help to reduce the build time of the Linux kernel if CONFIG_PM is not defined. Signed-off-by: Bart Van Assche Cc: Christoph Hellwig Cc: Jianchao Wang Cc: Ming Lei Cc: Johannes Thumshirn Cc: Alan Stern --- block/Kconfig | 5 ++ block/Makefile | 1 + block/blk-core.c | 196 +---------------------------------------- block/blk-pm.c | 188 +++++++++++++++++++++++++++++++++++++++ block/blk-pm.h | 43 +++++++++ block/elevator.c | 22 +---- drivers/scsi/scsi_pm.c | 1 + drivers/scsi/sd.c | 1 + drivers/scsi/sr.c | 1 + include/linux/blk-pm.h | 24 +++++ include/linux/blkdev.h | 23 ----- 11 files changed, 266 insertions(+), 239 deletions(-) create mode 100644 block/blk-pm.c create mode 100644 block/blk-pm.h create mode 100644 include/linux/blk-pm.h diff --git a/block/Kconfig b/block/Kconfig index 1f2469a0123c..e213d90a5e64 100644 --- a/block/Kconfig +++ b/block/Kconfig @@ -228,4 +228,9 @@ config BLK_MQ_RDMA depends on BLOCK && INFINIBAND default y +config BLK_PM + bool + depends on BLOCK && PM + default y + source block/Kconfig.iosched diff --git a/block/Makefile b/block/Makefile index 572b33f32c07..27eac600474f 100644 --- a/block/Makefile +++ b/block/Makefile @@ -37,3 +37,4 @@ obj-$(CONFIG_BLK_WBT) += blk-wbt.o obj-$(CONFIG_BLK_DEBUG_FS) += blk-mq-debugfs.o obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o obj-$(CONFIG_BLK_SED_OPAL) += sed-opal.o +obj-$(CONFIG_BLK_PM) += blk-pm.o diff --git a/block/blk-core.c b/block/blk-core.c index 3a5aa56eab9d..03cff7445dee 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -42,6 +42,7 @@ #include "blk.h" #include "blk-mq.h" #include "blk-mq-sched.h" +#include "blk-pm.h" #include "blk-rq-qos.h" #ifdef CONFIG_DEBUG_FS @@ -1720,16 +1721,6 @@ void part_round_stats(struct request_queue *q, int cpu, struct hd_struct *part) } EXPORT_SYMBOL_GPL(part_round_stats); -#ifdef CONFIG_PM -static void blk_pm_put_request(struct request *rq) -{ - if (rq->q->dev && !(rq->rq_flags & RQF_PM) && !--rq->q->nr_pending) - pm_runtime_mark_last_busy(rq->q->dev); -} -#else -static inline void blk_pm_put_request(struct request *rq) {} -#endif - void __blk_put_request(struct request_queue *q, struct request *req) { req_flags_t rq_flags = req->rq_flags; @@ -3745,191 +3736,6 @@ void blk_finish_plug(struct blk_plug *plug) } EXPORT_SYMBOL(blk_finish_plug); -#ifdef CONFIG_PM -/** - * blk_pm_runtime_init - Block layer runtime PM initialization routine - * @q: the queue of the device - * @dev: the device the queue belongs to - * - * Description: - * Initialize runtime-PM-related fields for @q and start auto suspend for - * @dev. Drivers that want to take advantage of request-based runtime PM - * should call this function after @dev has been initialized, and its - * request queue @q has been allocated, and runtime PM for it can not happen - * yet(either due to disabled/forbidden or its usage_count > 0). In most - * cases, driver should call this function before any I/O has taken place. - * - * This function takes care of setting up using auto suspend for the device, - * the autosuspend delay is set to -1 to make runtime suspend impossible - * until an updated value is either set by user or by driver. Drivers do - * not need to touch other autosuspend settings. - * - * The block layer runtime PM is request based, so only works for drivers - * that use request as their IO unit instead of those directly use bio's. - */ -void blk_pm_runtime_init(struct request_queue *q, struct device *dev) -{ - /* Don't enable runtime PM for blk-mq until it is ready */ - if (q->mq_ops) { - pm_runtime_disable(dev); - return; - } - - q->dev = dev; - q->rpm_status = RPM_ACTIVE; - pm_runtime_set_autosuspend_delay(q->dev, -1); - pm_runtime_use_autosuspend(q->dev); -} -EXPORT_SYMBOL(blk_pm_runtime_init); - -/** - * blk_pre_runtime_suspend - Pre runtime suspend check - * @q: the queue of the device - * - * Description: - * This function will check if runtime suspend is allowed for the device - * by examining if there are any requests pending in the queue. If there - * are requests pending, the device can not be runtime suspended; otherwise, - * the queue's status will be updated to SUSPENDING and the driver can - * proceed to suspend the device. - * - * For the not allowed case, we mark last busy for the device so that - * runtime PM core will try to autosuspend it some time later. - * - * This function should be called near the start of the device's - * runtime_suspend callback. - * - * Return: - * 0 - OK to runtime suspend the device - * -EBUSY - Device should not be runtime suspended - */ -int blk_pre_runtime_suspend(struct request_queue *q) -{ - int ret = 0; - - if (!q->dev) - return ret; - - spin_lock_irq(q->queue_lock); - if (q->nr_pending) { - ret = -EBUSY; - pm_runtime_mark_last_busy(q->dev); - } else { - q->rpm_status = RPM_SUSPENDING; - } - spin_unlock_irq(q->queue_lock); - return ret; -} -EXPORT_SYMBOL(blk_pre_runtime_suspend); - -/** - * blk_post_runtime_suspend - Post runtime suspend processing - * @q: the queue of the device - * @err: return value of the device's runtime_suspend function - * - * Description: - * Update the queue's runtime status according to the return value of the - * device's runtime suspend function and mark last busy for the device so - * that PM core will try to auto suspend the device at a later time. - * - * This function should be called near the end of the device's - * runtime_suspend callback. - */ -void blk_post_runtime_suspend(struct request_queue *q, int err) -{ - if (!q->dev) - return; - - spin_lock_irq(q->queue_lock); - if (!err) { - q->rpm_status = RPM_SUSPENDED; - } else { - q->rpm_status = RPM_ACTIVE; - pm_runtime_mark_last_busy(q->dev); - } - spin_unlock_irq(q->queue_lock); -} -EXPORT_SYMBOL(blk_post_runtime_suspend); - -/** - * blk_pre_runtime_resume - Pre runtime resume processing - * @q: the queue of the device - * - * Description: - * Update the queue's runtime status to RESUMING in preparation for the - * runtime resume of the device. - * - * This function should be called near the start of the device's - * runtime_resume callback. - */ -void blk_pre_runtime_resume(struct request_queue *q) -{ - if (!q->dev) - return; - - spin_lock_irq(q->queue_lock); - q->rpm_status = RPM_RESUMING; - spin_unlock_irq(q->queue_lock); -} -EXPORT_SYMBOL(blk_pre_runtime_resume); - -/** - * blk_post_runtime_resume - Post runtime resume processing - * @q: the queue of the device - * @err: return value of the device's runtime_resume function - * - * Description: - * Update the queue's runtime status according to the return value of the - * device's runtime_resume function. If it is successfully resumed, process - * the requests that are queued into the device's queue when it is resuming - * and then mark last busy and initiate autosuspend for it. - * - * This function should be called near the end of the device's - * runtime_resume callback. - */ -void blk_post_runtime_resume(struct request_queue *q, int err) -{ - if (!q->dev) - return; - - spin_lock_irq(q->queue_lock); - if (!err) { - q->rpm_status = RPM_ACTIVE; - __blk_run_queue(q); - pm_runtime_mark_last_busy(q->dev); - pm_request_autosuspend(q->dev); - } else { - q->rpm_status = RPM_SUSPENDED; - } - spin_unlock_irq(q->queue_lock); -} -EXPORT_SYMBOL(blk_post_runtime_resume); - -/** - * blk_set_runtime_active - Force runtime status of the queue to be active - * @q: the queue of the device - * - * If the device is left runtime suspended during system suspend the resume - * hook typically resumes the device and corrects runtime status - * accordingly. However, that does not affect the queue runtime PM status - * which is still "suspended". This prevents processing requests from the - * queue. - * - * This function can be used in driver's resume hook to correct queue - * runtime PM status and re-enable peeking requests from the queue. It - * should be called before first request is added to the queue. - */ -void blk_set_runtime_active(struct request_queue *q) -{ - spin_lock_irq(q->queue_lock); - q->rpm_status = RPM_ACTIVE; - pm_runtime_mark_last_busy(q->dev); - pm_request_autosuspend(q->dev); - spin_unlock_irq(q->queue_lock); -} -EXPORT_SYMBOL(blk_set_runtime_active); -#endif - int __init blk_dev_init(void) { BUILD_BUG_ON(REQ_OP_LAST >= (1 << REQ_OP_BITS)); diff --git a/block/blk-pm.c b/block/blk-pm.c new file mode 100644 index 000000000000..9b636960d285 --- /dev/null +++ b/block/blk-pm.c @@ -0,0 +1,188 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include + +/** + * blk_pm_runtime_init - Block layer runtime PM initialization routine + * @q: the queue of the device + * @dev: the device the queue belongs to + * + * Description: + * Initialize runtime-PM-related fields for @q and start auto suspend for + * @dev. Drivers that want to take advantage of request-based runtime PM + * should call this function after @dev has been initialized, and its + * request queue @q has been allocated, and runtime PM for it can not happen + * yet(either due to disabled/forbidden or its usage_count > 0). In most + * cases, driver should call this function before any I/O has taken place. + * + * This function takes care of setting up using auto suspend for the device, + * the autosuspend delay is set to -1 to make runtime suspend impossible + * until an updated value is either set by user or by driver. Drivers do + * not need to touch other autosuspend settings. + * + * The block layer runtime PM is request based, so only works for drivers + * that use request as their IO unit instead of those directly use bio's. + */ +void blk_pm_runtime_init(struct request_queue *q, struct device *dev) +{ + /* Don't enable runtime PM for blk-mq until it is ready */ + if (q->mq_ops) { + pm_runtime_disable(dev); + return; + } + + q->dev = dev; + q->rpm_status = RPM_ACTIVE; + pm_runtime_set_autosuspend_delay(q->dev, -1); + pm_runtime_use_autosuspend(q->dev); +} +EXPORT_SYMBOL(blk_pm_runtime_init); + +/** + * blk_pre_runtime_suspend - Pre runtime suspend check + * @q: the queue of the device + * + * Description: + * This function will check if runtime suspend is allowed for the device + * by examining if there are any requests pending in the queue. If there + * are requests pending, the device can not be runtime suspended; otherwise, + * the queue's status will be updated to SUSPENDING and the driver can + * proceed to suspend the device. + * + * For the not allowed case, we mark last busy for the device so that + * runtime PM core will try to autosuspend it some time later. + * + * This function should be called near the start of the device's + * runtime_suspend callback. + * + * Return: + * 0 - OK to runtime suspend the device + * -EBUSY - Device should not be runtime suspended + */ +int blk_pre_runtime_suspend(struct request_queue *q) +{ + int ret = 0; + + if (!q->dev) + return ret; + + spin_lock_irq(q->queue_lock); + if (q->nr_pending) { + ret = -EBUSY; + pm_runtime_mark_last_busy(q->dev); + } else { + q->rpm_status = RPM_SUSPENDING; + } + spin_unlock_irq(q->queue_lock); + return ret; +} +EXPORT_SYMBOL(blk_pre_runtime_suspend); + +/** + * blk_post_runtime_suspend - Post runtime suspend processing + * @q: the queue of the device + * @err: return value of the device's runtime_suspend function + * + * Description: + * Update the queue's runtime status according to the return value of the + * device's runtime suspend function and mark last busy for the device so + * that PM core will try to auto suspend the device at a later time. + * + * This function should be called near the end of the device's + * runtime_suspend callback. + */ +void blk_post_runtime_suspend(struct request_queue *q, int err) +{ + if (!q->dev) + return; + + spin_lock_irq(q->queue_lock); + if (!err) { + q->rpm_status = RPM_SUSPENDED; + } else { + q->rpm_status = RPM_ACTIVE; + pm_runtime_mark_last_busy(q->dev); + } + spin_unlock_irq(q->queue_lock); +} +EXPORT_SYMBOL(blk_post_runtime_suspend); + +/** + * blk_pre_runtime_resume - Pre runtime resume processing + * @q: the queue of the device + * + * Description: + * Update the queue's runtime status to RESUMING in preparation for the + * runtime resume of the device. + * + * This function should be called near the start of the device's + * runtime_resume callback. + */ +void blk_pre_runtime_resume(struct request_queue *q) +{ + if (!q->dev) + return; + + spin_lock_irq(q->queue_lock); + q->rpm_status = RPM_RESUMING; + spin_unlock_irq(q->queue_lock); +} +EXPORT_SYMBOL(blk_pre_runtime_resume); + +/** + * blk_post_runtime_resume - Post runtime resume processing + * @q: the queue of the device + * @err: return value of the device's runtime_resume function + * + * Description: + * Update the queue's runtime status according to the return value of the + * device's runtime_resume function. If it is successfully resumed, process + * the requests that are queued into the device's queue when it is resuming + * and then mark last busy and initiate autosuspend for it. + * + * This function should be called near the end of the device's + * runtime_resume callback. + */ +void blk_post_runtime_resume(struct request_queue *q, int err) +{ + if (!q->dev) + return; + + spin_lock_irq(q->queue_lock); + if (!err) { + q->rpm_status = RPM_ACTIVE; + __blk_run_queue(q); + pm_runtime_mark_last_busy(q->dev); + pm_request_autosuspend(q->dev); + } else { + q->rpm_status = RPM_SUSPENDED; + } + spin_unlock_irq(q->queue_lock); +} +EXPORT_SYMBOL(blk_post_runtime_resume); + +/** + * blk_set_runtime_active - Force runtime status of the queue to be active + * @q: the queue of the device + * + * If the device is left runtime suspended during system suspend the resume + * hook typically resumes the device and corrects runtime status + * accordingly. However, that does not affect the queue runtime PM status + * which is still "suspended". This prevents processing requests from the + * queue. + * + * This function can be used in driver's resume hook to correct queue + * runtime PM status and re-enable peeking requests from the queue. It + * should be called before first request is added to the queue. + */ +void blk_set_runtime_active(struct request_queue *q) +{ + spin_lock_irq(q->queue_lock); + q->rpm_status = RPM_ACTIVE; + pm_runtime_mark_last_busy(q->dev); + pm_request_autosuspend(q->dev); + spin_unlock_irq(q->queue_lock); +} +EXPORT_SYMBOL(blk_set_runtime_active); diff --git a/block/blk-pm.h b/block/blk-pm.h new file mode 100644 index 000000000000..1ffc8ef203ec --- /dev/null +++ b/block/blk-pm.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _BLOCK_BLK_PM_H_ +#define _BLOCK_BLK_PM_H_ + +#include + +#ifdef CONFIG_PM +static inline void blk_pm_requeue_request(struct request *rq) +{ + if (rq->q->dev && !(rq->rq_flags & RQF_PM)) + rq->q->nr_pending--; +} + +static inline void blk_pm_add_request(struct request_queue *q, + struct request *rq) +{ + if (q->dev && !(rq->rq_flags & RQF_PM) && q->nr_pending++ == 0 && + (q->rpm_status == RPM_SUSPENDED || q->rpm_status == RPM_SUSPENDING)) + pm_request_resume(q->dev); +} + +static inline void blk_pm_put_request(struct request *rq) +{ + if (rq->q->dev && !(rq->rq_flags & RQF_PM) && !--rq->q->nr_pending) + pm_runtime_mark_last_busy(rq->q->dev); +} +#else +static inline void blk_pm_requeue_request(struct request *rq) +{ +} + +static inline void blk_pm_add_request(struct request_queue *q, + struct request *rq) +{ +} + +static inline void blk_pm_put_request(struct request *rq) +{ +} +#endif + +#endif /* _BLOCK_BLK_PM_H_ */ diff --git a/block/elevator.c b/block/elevator.c index fa828b5bfd4b..4c15f0240c99 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -41,6 +41,7 @@ #include "blk.h" #include "blk-mq-sched.h" +#include "blk-pm.h" #include "blk-wbt.h" static DEFINE_SPINLOCK(elv_list_lock); @@ -557,27 +558,6 @@ void elv_bio_merged(struct request_queue *q, struct request *rq, e->type->ops.sq.elevator_bio_merged_fn(q, rq, bio); } -#ifdef CONFIG_PM -static void blk_pm_requeue_request(struct request *rq) -{ - if (rq->q->dev && !(rq->rq_flags & RQF_PM)) - rq->q->nr_pending--; -} - -static void blk_pm_add_request(struct request_queue *q, struct request *rq) -{ - if (q->dev && !(rq->rq_flags & RQF_PM) && q->nr_pending++ == 0 && - (q->rpm_status == RPM_SUSPENDED || q->rpm_status == RPM_SUSPENDING)) - pm_request_resume(q->dev); -} -#else -static inline void blk_pm_requeue_request(struct request *rq) {} -static inline void blk_pm_add_request(struct request_queue *q, - struct request *rq) -{ -} -#endif - void elv_requeue_request(struct request_queue *q, struct request *rq) { /* diff --git a/drivers/scsi/scsi_pm.c b/drivers/scsi/scsi_pm.c index b44c1bb687a2..a2b4179bfdf7 100644 --- a/drivers/scsi/scsi_pm.c +++ b/drivers/scsi/scsi_pm.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c index bbebdc3769b0..69ab459abb98 100644 --- a/drivers/scsi/sd.c +++ b/drivers/scsi/sd.c @@ -45,6 +45,7 @@ #include #include #include +#include #include #include #include diff --git a/drivers/scsi/sr.c b/drivers/scsi/sr.c index 3f3cb72e0c0c..de4413e66eca 100644 --- a/drivers/scsi/sr.c +++ b/drivers/scsi/sr.c @@ -43,6 +43,7 @@ #include #include #include +#include #include #include #include diff --git a/include/linux/blk-pm.h b/include/linux/blk-pm.h new file mode 100644 index 000000000000..b80c65aba249 --- /dev/null +++ b/include/linux/blk-pm.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _BLK_PM_H_ +#define _BLK_PM_H_ + +struct device; +struct request_queue; + +/* + * block layer runtime pm functions + */ +#ifdef CONFIG_PM +extern void blk_pm_runtime_init(struct request_queue *q, struct device *dev); +extern int blk_pre_runtime_suspend(struct request_queue *q); +extern void blk_post_runtime_suspend(struct request_queue *q, int err); +extern void blk_pre_runtime_resume(struct request_queue *q); +extern void blk_post_runtime_resume(struct request_queue *q, int err); +extern void blk_set_runtime_active(struct request_queue *q); +#else +static inline void blk_pm_runtime_init(struct request_queue *q, + struct device *dev) {} +#endif + +#endif /* _BLK_PM_H_ */ diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 1b3fe78872dc..2ef38739d645 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1284,29 +1284,6 @@ struct request_queue *blk_alloc_queue_node(gfp_t gfp_mask, int node_id, extern void blk_put_queue(struct request_queue *); extern void blk_set_queue_dying(struct request_queue *); -/* - * block layer runtime pm functions - */ -#ifdef CONFIG_PM -extern void blk_pm_runtime_init(struct request_queue *q, struct device *dev); -extern int blk_pre_runtime_suspend(struct request_queue *q); -extern void blk_post_runtime_suspend(struct request_queue *q, int err); -extern void blk_pre_runtime_resume(struct request_queue *q); -extern void blk_post_runtime_resume(struct request_queue *q, int err); -extern void blk_set_runtime_active(struct request_queue *q); -#else -static inline void blk_pm_runtime_init(struct request_queue *q, - struct device *dev) {} -static inline int blk_pre_runtime_suspend(struct request_queue *q) -{ - return -ENOSYS; -} -static inline void blk_post_runtime_suspend(struct request_queue *q, int err) {} -static inline void blk_pre_runtime_resume(struct request_queue *q) {} -static inline void blk_post_runtime_resume(struct request_queue *q, int err) {} -static inline void blk_set_runtime_active(struct request_queue *q) {} -#endif - /* * blk_plug permits building a queue of related requests by holding the I/O * fragments for a short period. This allows merging of sequential requests From patchwork Sat Aug 4 00:03:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10555563 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 41B03A748 for ; Sat, 4 Aug 2018 00:03:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 339EE2C91B for ; Sat, 4 Aug 2018 00:03:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 276262C933; Sat, 4 Aug 2018 00:03:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 924BC2C91B for ; Sat, 4 Aug 2018 00:03:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732079AbeHDCB4 (ORCPT ); Fri, 3 Aug 2018 22:01:56 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:34551 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732034AbeHDCB4 (ORCPT ); Fri, 3 Aug 2018 22:01:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1533341008; x=1564877008; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=8XI7MifHDMuZW1bDCr56gI9HbP8znkzB3erwSeAgvG8=; b=bXXpZppSFxZ24VxUFk9ERLFXLHh5mjW3PFQy2108VbmPSLyxaEaHPaP2 JxV1l0XYJVWnHytpR5myjEpwUjOIucUk5r6YuRsM1dKT76cNS7bYsv5q/ xpL/r7UEgTU0wO4mTCrRbd68AAVaIa5h/FZsQxpWbVxBOt91AzcpD2U5/ EB1WgJ0BDqXuUhw1ZvX2RghNlT730p7Wc7JeVHgsx7umpsAaWslOipbbi s9QCXsdgN2HXYDA/E7F150oF5AIp8fZy56bJwWo8adbbCJnQ/KDJRrY5V 4hLnksaHmUCUYPKgmWGEK7EBfzPOyqZ+R6KHqyLgpvSB29b095hZkcgq1 g==; X-IronPort-AV: E=Sophos;i="5.51,440,1526313600"; d="scan'208";a="90059263" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 04 Aug 2018 08:03:27 +0800 Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP; 03 Aug 2018 16:51:11 -0700 Received: from thinkpad-bart.sdcorp.global.sandisk.com ([10.111.67.248]) by uls-op-cesaip01.wdc.com with ESMTP; 03 Aug 2018 17:03:26 -0700 From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Jianchao Wang , Ming Lei , Johannes Thumshirn , Alan Stern Subject: [PATCH v4 05/10] block: Serialize queue freezing and blk_pre_runtime_suspend() Date: Fri, 3 Aug 2018 17:03:20 -0700 Message-Id: <20180804000325.3610-6-bart.vanassche@wdc.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180804000325.3610-1-bart.vanassche@wdc.com> References: <20180804000325.3610-1-bart.vanassche@wdc.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Serialize these operations because a later patch will add code into blk_pre_runtime_suspend() that should not run concurrently with queue freezing nor unfreezing. Signed-off-by: Bart Van Assche Cc: Christoph Hellwig Cc: Jianchao Wang Cc: Ming Lei Cc: Johannes Thumshirn Cc: Alan Stern --- block/blk-core.c | 5 +++++ block/blk-mq.c | 3 +++ block/blk-pm.c | 44 ++++++++++++++++++++++++++++++++++++++++++ include/linux/blk-pm.h | 6 ++++++ include/linux/blkdev.h | 5 +++++ 5 files changed, 63 insertions(+) diff --git a/block/blk-core.c b/block/blk-core.c index 03cff7445dee..59382c758155 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -696,6 +697,7 @@ void blk_set_queue_dying(struct request_queue *q) * prevent I/O from crossing blk_queue_enter(). */ blk_freeze_queue_start(q); + blk_pm_runtime_unlock(q); if (q->mq_ops) blk_mq_wake_waiters(q); @@ -756,6 +758,7 @@ void blk_cleanup_queue(struct request_queue *q) * prevent that q->request_fn() gets invoked after draining finished. */ blk_freeze_queue(q); + blk_pm_runtime_unlock(q); spin_lock_irq(lock); queue_flag_set(QUEUE_FLAG_DEAD, q); spin_unlock_irq(lock); @@ -1045,6 +1048,8 @@ struct request_queue *blk_alloc_queue_node(gfp_t gfp_mask, int node_id, #ifdef CONFIG_BLK_DEV_IO_TRACE mutex_init(&q->blk_trace_mutex); #endif + blk_pm_init(q); + mutex_init(&q->sysfs_lock); spin_lock_init(&q->__queue_lock); diff --git a/block/blk-mq.c b/block/blk-mq.c index 8b23ae34d949..b1882a3a5216 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -138,6 +139,7 @@ void blk_freeze_queue_start(struct request_queue *q) { int freeze_depth; + blk_pm_runtime_lock(q); freeze_depth = atomic_inc_return(&q->mq_freeze_depth); if (freeze_depth == 1) { percpu_ref_kill(&q->q_usage_counter); @@ -201,6 +203,7 @@ void blk_mq_unfreeze_queue(struct request_queue *q) percpu_ref_reinit(&q->q_usage_counter); wake_up_all(&q->mq_freeze_wq); } + blk_pm_runtime_unlock(q); } EXPORT_SYMBOL_GPL(blk_mq_unfreeze_queue); diff --git a/block/blk-pm.c b/block/blk-pm.c index 9b636960d285..2a4632d0be4b 100644 --- a/block/blk-pm.c +++ b/block/blk-pm.c @@ -3,6 +3,45 @@ #include #include #include +#include +#include + +/* + * Initialize the request queue members used by blk_pm_runtime_lock() and + * blk_pm_runtime_unlock(). + */ +void blk_pm_init(struct request_queue *q) +{ + spin_lock_init(&q->rpm_lock); + init_waitqueue_head(&q->rpm_wq); + q->rpm_owner = NULL; + q->rpm_nesting_level = 0; +} + +void blk_pm_runtime_lock(struct request_queue *q) +{ + might_sleep(); + + spin_lock(&q->rpm_lock); + wait_event_exclusive_cmd(q->rpm_wq, + q->rpm_owner == NULL || q->rpm_owner == current, + spin_unlock(&q->rpm_lock), spin_lock(&q->rpm_lock)); + if (q->rpm_owner == NULL) + q->rpm_owner = current; + q->rpm_nesting_level++; + spin_unlock(&q->rpm_lock); +} + +void blk_pm_runtime_unlock(struct request_queue *q) +{ + spin_lock(&q->rpm_lock); + WARN_ON_ONCE(q->rpm_nesting_level <= 0); + if (--q->rpm_nesting_level == 0) { + q->rpm_owner = NULL; + wake_up(&q->rpm_wq); + } + spin_unlock(&q->rpm_lock); +} /** * blk_pm_runtime_init - Block layer runtime PM initialization routine @@ -68,6 +107,8 @@ int blk_pre_runtime_suspend(struct request_queue *q) if (!q->dev) return ret; + blk_pm_runtime_lock(q); + spin_lock_irq(q->queue_lock); if (q->nr_pending) { ret = -EBUSY; @@ -76,6 +117,9 @@ int blk_pre_runtime_suspend(struct request_queue *q) q->rpm_status = RPM_SUSPENDING; } spin_unlock_irq(q->queue_lock); + + blk_pm_runtime_unlock(q); + return ret; } EXPORT_SYMBOL(blk_pre_runtime_suspend); diff --git a/include/linux/blk-pm.h b/include/linux/blk-pm.h index b80c65aba249..aafcc7877e53 100644 --- a/include/linux/blk-pm.h +++ b/include/linux/blk-pm.h @@ -10,6 +10,9 @@ struct request_queue; * block layer runtime pm functions */ #ifdef CONFIG_PM +extern void blk_pm_init(struct request_queue *q); +extern void blk_pm_runtime_lock(struct request_queue *q); +extern void blk_pm_runtime_unlock(struct request_queue *q); extern void blk_pm_runtime_init(struct request_queue *q, struct device *dev); extern int blk_pre_runtime_suspend(struct request_queue *q); extern void blk_post_runtime_suspend(struct request_queue *q, int err); @@ -17,6 +20,9 @@ extern void blk_pre_runtime_resume(struct request_queue *q); extern void blk_post_runtime_resume(struct request_queue *q, int err); extern void blk_set_runtime_active(struct request_queue *q); #else +static inline void blk_pm_init(struct request_queue *q) {} +static inline void blk_pm_runtime_lock(struct request_queue *q) {} +static inline void blk_pm_runtime_unlock(struct request_queue *q) {} static inline void blk_pm_runtime_init(struct request_queue *q, struct device *dev) {} #endif diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 2ef38739d645..72d569218231 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -548,6 +548,11 @@ struct request_queue { struct device *dev; int rpm_status; unsigned int nr_pending; + wait_queue_head_t rpm_wq; + /* rpm_lock protects rpm_owner and rpm_nesting_level */ + spinlock_t rpm_lock; + struct task_struct *rpm_owner; + int rpm_nesting_level; #endif /* From patchwork Sat Aug 4 00:03:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10555567 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 32C7B157D for ; Sat, 4 Aug 2018 00:03:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 257E12C913 for ; Sat, 4 Aug 2018 00:03:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 19ACC2C933; Sat, 4 Aug 2018 00:03:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B1D062C913 for ; Sat, 4 Aug 2018 00:03:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732087AbeHDCB5 (ORCPT ); Fri, 3 Aug 2018 22:01:57 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:34551 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731959AbeHDCB5 (ORCPT ); Fri, 3 Aug 2018 22:01:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1533341009; x=1564877009; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=3AkKkLX9BVTxaoiVppvD1s5JwJRlC1KOncgOkwY9TKQ=; b=AcXzXcHbTIU1K75ONebr5b785b0x0lXr3BThEDnYggIFMyfjZ/L/yXL8 3G5au1qf2ebusu9puIWNfTMyiei8Ba7eEg03FffJ4Mx3KFSbbxNQ0RRsj ncUnPxv2mKluD2ECi68VG3R33VO6YSpx01P1TxanSoj9hGlLciX5zOMaW fKi2GpZJbWBBD8f2JwwbZ5+vDe88G/VYBIO8JjDGMVjdNGk5N1wBsIia5 Oj8XpLm15WX2U7cDKk4X6GjEiKx22bOTBLbH3ltL8JOOTaF7eSVTwQutP ROhravOz5nFpNKtTBPO3zXVnMO+ThsC/+5O2uiU4vBVHCOqTxcCXDFIos A==; X-IronPort-AV: E=Sophos;i="5.51,440,1526313600"; d="scan'208";a="90059266" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 04 Aug 2018 08:03:28 +0800 Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP; 03 Aug 2018 16:51:12 -0700 Received: from thinkpad-bart.sdcorp.global.sandisk.com ([10.111.67.248]) by uls-op-cesaip01.wdc.com with ESMTP; 03 Aug 2018 17:03:26 -0700 From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Tejun Heo , Jianchao Wang , Ming Lei , Alan Stern , Johannes Thumshirn Subject: [PATCH v4 06/10] percpu-refcount: Introduce percpu_ref_read() Date: Fri, 3 Aug 2018 17:03:21 -0700 Message-Id: <20180804000325.3610-7-bart.vanassche@wdc.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180804000325.3610-1-bart.vanassche@wdc.com> References: <20180804000325.3610-1-bart.vanassche@wdc.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Introduce a function that allows to read the value of a per-cpu counter. This function will be used in the next patch to check whether a per-cpu counter has the value one. Signed-off-by: Bart Van Assche Cc: Tejun Heo Cc: Christoph Hellwig Cc: Jianchao Wang Cc: Ming Lei Cc: Alan Stern Cc: Johannes Thumshirn --- include/linux/percpu-refcount.h | 2 ++ lib/percpu-refcount.c | 29 +++++++++++++++++++++++++++++ 2 files changed, 31 insertions(+) diff --git a/include/linux/percpu-refcount.h b/include/linux/percpu-refcount.h index 009cdf3d65b6..5707289ba828 100644 --- a/include/linux/percpu-refcount.h +++ b/include/linux/percpu-refcount.h @@ -331,4 +331,6 @@ static inline bool percpu_ref_is_zero(struct percpu_ref *ref) return !atomic_long_read(&ref->count); } +unsigned long percpu_ref_read(struct percpu_ref *ref); + #endif diff --git a/lib/percpu-refcount.c b/lib/percpu-refcount.c index 9f96fa7bc000..c0b9fc8efa6b 100644 --- a/lib/percpu-refcount.c +++ b/lib/percpu-refcount.c @@ -369,3 +369,32 @@ void percpu_ref_reinit(struct percpu_ref *ref) spin_unlock_irqrestore(&percpu_ref_switch_lock, flags); } EXPORT_SYMBOL_GPL(percpu_ref_reinit); + +/** + * percpu_ref_read - read a percpu refcount + * @ref: percpu_ref to test + * + * This function is safe to call as long as @ref is between init and exit. It + * is the responsibility of the caller to handle changes of @ref concurrently + * with this function. If this function is called while @ref is in per-cpu + * mode the returned value may be incorrect if e.g. percpu_ref_get() is called + * from one CPU and percpu_ref_put() is called from another CPU. + */ +unsigned long percpu_ref_read(struct percpu_ref *ref) +{ + unsigned long __percpu *percpu_count; + unsigned long sum = 0; + int cpu; + + rcu_read_lock_sched(); + if (__ref_is_percpu(ref, &percpu_count)) { + for_each_possible_cpu(cpu) + sum += *per_cpu_ptr(percpu_count, cpu); + } + rcu_read_unlock_sched(); + sum += atomic_long_read(&ref->count); + sum &= ~PERCPU_COUNT_BIAS; + + return sum; +} +EXPORT_SYMBOL_GPL(percpu_ref_read); From patchwork Sat Aug 4 00:03:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10555565 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C6FFA13AC for ; Sat, 4 Aug 2018 00:03:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BAAE32C91B for ; Sat, 4 Aug 2018 00:03:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AEA9C2C933; Sat, 4 Aug 2018 00:03:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 236B42C913 for ; Sat, 4 Aug 2018 00:03:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732049AbeHDCB5 (ORCPT ); Fri, 3 Aug 2018 22:01:57 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:34555 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732087AbeHDCB5 (ORCPT ); Fri, 3 Aug 2018 22:01:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1533341009; x=1564877009; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=aW0Dbclesq37GRBT6ontBax5fvr2ckfE8pTN8FPZEk4=; b=LyoHecjp0WuFBGHAgLmtN6mi9NvEW9v7tmHlkLwrhaSZIPsnx3VkTGJ6 dEZ21nBx89IOy7lCvZmSJBRKKH7TIg80notVwckhJPp0tVBv4EQAnd1jd RKPKQqn8s9SRvMLl5PSY+re0Outzv9SnRgrsUA3o7/sLr8P3YSwkpo7h8 ihDm/946RmeBI5NJtOjg6oJYXJyOzzcRrWoej7F14dD48vUpgMy6uTXIo opTjHyYbBETuevvsaERFEq8LRdOAHQwygPYwBiiAfN6WIBX/qIU6cOkVF x1vGlfuA6gxU7Cjx2dCgy0XGqTyQXjZ3s5lNWzQPqBGXp6S4/AxYyX0UB w==; X-IronPort-AV: E=Sophos;i="5.51,440,1526313600"; d="scan'208";a="90059267" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 04 Aug 2018 08:03:28 +0800 Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP; 03 Aug 2018 16:51:12 -0700 Received: from thinkpad-bart.sdcorp.global.sandisk.com ([10.111.67.248]) by uls-op-cesaip01.wdc.com with ESMTP; 03 Aug 2018 17:03:26 -0700 From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , "Martin K . Petersen" , Jianchao Wang , Ming Lei , Alan Stern , Johannes Thumshirn Subject: [PATCH v4 07/10] block, scsi: Rework runtime power management Date: Fri, 3 Aug 2018 17:03:22 -0700 Message-Id: <20180804000325.3610-8-bart.vanassche@wdc.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180804000325.3610-1-bart.vanassche@wdc.com> References: <20180804000325.3610-1-bart.vanassche@wdc.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Instead of allowing requests that are not power management requests to enter the queue in runtime suspended status (RPM_SUSPENDED), make the blk_get_request() caller block. This change fixes a starvation issue: it is now guaranteed that power management requests will be executed no matter how many blk_get_request() callers are waiting. Instead of maintaining the q->nr_pending counter, rely on q->q_usage_counter. Call pm_runtime_mark_last_busy() every time a request finishes instead of only if the queue depth drops to zero. Signed-off-by: Bart Van Assche Cc: Martin K. Petersen Cc: Christoph Hellwig Cc: Jianchao Wang Cc: Ming Lei Cc: Alan Stern Cc: Johannes Thumshirn --- block/blk-core.c | 37 ++++++++----------------------------- block/blk-mq-debugfs.c | 1 - block/blk-pm.c | 29 ++++++++++++++++++++++++----- block/blk-pm.h | 14 ++++++++------ include/linux/blkdev.h | 1 - 5 files changed, 40 insertions(+), 42 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 59382c758155..1e208e134ca9 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -2736,30 +2736,6 @@ void blk_account_io_done(struct request *req, u64 now) } } -#ifdef CONFIG_PM -/* - * Don't process normal requests when queue is suspended - * or in the process of suspending/resuming - */ -static bool blk_pm_allow_request(struct request *rq) -{ - switch (rq->q->rpm_status) { - case RPM_RESUMING: - case RPM_SUSPENDING: - return rq->rq_flags & RQF_PM; - case RPM_SUSPENDED: - return false; - default: - return true; - } -} -#else -static bool blk_pm_allow_request(struct request *rq) -{ - return true; -} -#endif - void blk_account_io_start(struct request *rq, bool new_io) { struct hd_struct *part; @@ -2805,11 +2781,14 @@ static struct request *elv_next_request(struct request_queue *q) while (1) { list_for_each_entry(rq, &q->queue_head, queuelist) { - if (blk_pm_allow_request(rq)) - return rq; - - if (rq->rq_flags & RQF_SOFTBARRIER) - break; +#ifdef CONFIG_PM + /* + * If a request gets queued in state RPM_SUSPENDED + * then that's a kernel bug. + */ + WARN_ON_ONCE(q->rpm_status == RPM_SUSPENDED); +#endif + return rq; } /* diff --git a/block/blk-mq-debugfs.c b/block/blk-mq-debugfs.c index a5ea86835fcb..7d74d53dc098 100644 --- a/block/blk-mq-debugfs.c +++ b/block/blk-mq-debugfs.c @@ -332,7 +332,6 @@ static const char *const rqf_name[] = { RQF_NAME(ELVPRIV), RQF_NAME(IO_STAT), RQF_NAME(ALLOCED), - RQF_NAME(PM), RQF_NAME(HASHED), RQF_NAME(STATS), RQF_NAME(SPECIAL_PAYLOAD), diff --git a/block/blk-pm.c b/block/blk-pm.c index 2a4632d0be4b..0708185ead61 100644 --- a/block/blk-pm.c +++ b/block/blk-pm.c @@ -107,17 +107,31 @@ int blk_pre_runtime_suspend(struct request_queue *q) if (!q->dev) return ret; + WARN_ON_ONCE(q->rpm_status != RPM_ACTIVE); + blk_pm_runtime_lock(q); + /* + * Switch to preempt-only mode before calling synchronize_rcu() such + * that later blk_queue_enter() calls see the preempt-only state. See + * also http://lwn.net/Articles/573497/. + */ + blk_set_pm_only(q); + ret = -EBUSY; + if (percpu_ref_read(&q->q_usage_counter) == 1) { + synchronize_rcu(); + if (percpu_ref_read(&q->q_usage_counter) == 1) + ret = 0; + } spin_lock_irq(q->queue_lock); - if (q->nr_pending) { - ret = -EBUSY; + if (ret < 0) pm_runtime_mark_last_busy(q->dev); - } else { + else q->rpm_status = RPM_SUSPENDING; - } spin_unlock_irq(q->queue_lock); + if (ret) + blk_clear_pm_only(q); blk_pm_runtime_unlock(q); return ret; @@ -150,6 +164,9 @@ void blk_post_runtime_suspend(struct request_queue *q, int err) pm_runtime_mark_last_busy(q->dev); } spin_unlock_irq(q->queue_lock); + + if (err) + blk_clear_pm_only(q); } EXPORT_SYMBOL(blk_post_runtime_suspend); @@ -197,13 +214,15 @@ void blk_post_runtime_resume(struct request_queue *q, int err) spin_lock_irq(q->queue_lock); if (!err) { q->rpm_status = RPM_ACTIVE; - __blk_run_queue(q); pm_runtime_mark_last_busy(q->dev); pm_request_autosuspend(q->dev); } else { q->rpm_status = RPM_SUSPENDED; } spin_unlock_irq(q->queue_lock); + + if (!err) + blk_clear_pm_only(q); } EXPORT_SYMBOL(blk_post_runtime_resume); diff --git a/block/blk-pm.h b/block/blk-pm.h index 1ffc8ef203ec..8d250e545e42 100644 --- a/block/blk-pm.h +++ b/block/blk-pm.h @@ -8,21 +8,23 @@ #ifdef CONFIG_PM static inline void blk_pm_requeue_request(struct request *rq) { - if (rq->q->dev && !(rq->rq_flags & RQF_PM)) - rq->q->nr_pending--; } static inline void blk_pm_add_request(struct request_queue *q, struct request *rq) { - if (q->dev && !(rq->rq_flags & RQF_PM) && q->nr_pending++ == 0 && - (q->rpm_status == RPM_SUSPENDED || q->rpm_status == RPM_SUSPENDING)) - pm_request_resume(q->dev); + if (!q->dev || (rq->rq_flags & RQF_PREEMPT)) + return; + /* + * If the below warning is triggered then that means that the caller + * did not use pm_runtime_get*(). + */ + WARN_ON_ONCE(atomic_read(&q->dev->power.usage_count) == 0); } static inline void blk_pm_put_request(struct request *rq) { - if (rq->q->dev && !(rq->rq_flags & RQF_PM) && !--rq->q->nr_pending) + if (rq->q->dev && !(rq->rq_flags & RQF_PREEMPT)) pm_runtime_mark_last_busy(rq->q->dev); } #else diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 72d569218231..9f859f32757c 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -547,7 +547,6 @@ struct request_queue { #ifdef CONFIG_PM struct device *dev; int rpm_status; - unsigned int nr_pending; wait_queue_head_t rpm_wq; /* rpm_lock protects rpm_owner and rpm_nesting_level */ spinlock_t rpm_lock; From patchwork Sat Aug 4 00:03:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10555571 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1ACBA1390 for ; Sat, 4 Aug 2018 00:03:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0CBF12C913 for ; Sat, 4 Aug 2018 00:03:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 00FF62C91D; Sat, 4 Aug 2018 00:03:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A067C2C913 for ; Sat, 4 Aug 2018 00:03:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732155AbeHDCCA (ORCPT ); Fri, 3 Aug 2018 22:02:00 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:34555 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731959AbeHDCCA (ORCPT ); Fri, 3 Aug 2018 22:02:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1533341011; x=1564877011; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=Sg0QanZa8tIvNjDkvf3zNRUhzz8CIG9GLmvDJgc39a8=; b=O/PlLLHizqNQV+95fSsqqx22msHQHJG30fnw5OAKMriaRQkIxvqPWSB4 hGQ+6E048Rc06sN8TmsCW1vmg17pj8cRktNhIfETH5qMeuo9CWOQ1QSXW 3zdUYz9tZ4d260czMa0jZjLRImJ0I+l09/Rp5jnZgV9BEuGArPAHYXX4L Gq5RjAf+2behJi8VtCSD5oOE4nVuK/9f4GIzJKJn/UUI79DGCPrn9m/xD ktjadAropAHAqHw6ipwXDtRcPhxNuoHDV+hTXBG5jE6fhH3eHIGvXFbre SYoaISuZx8Iqg/SK35Yp/sSxYqaU8CzMiPMdk94jxD7MzXv9Lvsbd1PSR w==; X-IronPort-AV: E=Sophos;i="5.51,440,1526313600"; d="scan'208";a="90059269" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 04 Aug 2018 08:03:28 +0800 Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP; 03 Aug 2018 16:51:12 -0700 Received: from thinkpad-bart.sdcorp.global.sandisk.com ([10.111.67.248]) by uls-op-cesaip01.wdc.com with ESMTP; 03 Aug 2018 17:03:27 -0700 From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Jianchao Wang , Ming Lei , Alan Stern , Johannes Thumshirn Subject: [PATCH v4 08/10] block: Remove blk_pm_requeue_request() Date: Fri, 3 Aug 2018 17:03:23 -0700 Message-Id: <20180804000325.3610-9-bart.vanassche@wdc.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180804000325.3610-1-bart.vanassche@wdc.com> References: <20180804000325.3610-1-bart.vanassche@wdc.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Since this function is now empty, remove it. Signed-off-by: Bart Van Assche Cc: Christoph Hellwig Cc: Jianchao Wang Cc: Ming Lei Cc: Alan Stern Cc: Johannes Thumshirn --- block/blk-pm.h | 8 -------- block/elevator.c | 2 -- 2 files changed, 10 deletions(-) diff --git a/block/blk-pm.h b/block/blk-pm.h index 8d250e545e42..0bfd0d3b5356 100644 --- a/block/blk-pm.h +++ b/block/blk-pm.h @@ -6,10 +6,6 @@ #include #ifdef CONFIG_PM -static inline void blk_pm_requeue_request(struct request *rq) -{ -} - static inline void blk_pm_add_request(struct request_queue *q, struct request *rq) { @@ -28,10 +24,6 @@ static inline void blk_pm_put_request(struct request *rq) pm_runtime_mark_last_busy(rq->q->dev); } #else -static inline void blk_pm_requeue_request(struct request *rq) -{ -} - static inline void blk_pm_add_request(struct request_queue *q, struct request *rq) { diff --git a/block/elevator.c b/block/elevator.c index 4c15f0240c99..fd7b12bb32a3 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -572,8 +572,6 @@ void elv_requeue_request(struct request_queue *q, struct request *rq) rq->rq_flags &= ~RQF_STARTED; - blk_pm_requeue_request(rq); - __elv_add_request(q, rq, ELEVATOR_INSERT_REQUEUE); } From patchwork Sat Aug 4 00:03:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10555573 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 389481390 for ; Sat, 4 Aug 2018 00:03:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2AFB62C91D for ; Sat, 4 Aug 2018 00:03:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1F13A2C933; Sat, 4 Aug 2018 00:03:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A60352C913 for ; Sat, 4 Aug 2018 00:03:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731959AbeHDCCB (ORCPT ); Fri, 3 Aug 2018 22:02:01 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:34565 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732090AbeHDCCA (ORCPT ); Fri, 3 Aug 2018 22:02:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1533341012; x=1564877012; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=p9d8yQ2cIAtH58G7Wae8cgzhiAFsZwlKB+UZa4aE/D4=; b=kl0/ceqihawBI/kmxqi7j3jKldX80dKslBIJnHOXW8l07m22YmBekGre lOxprmvSf14N0NLGmMCu9IpDeE/82BSG9nm6DfZy52GKaNljDVyvrw0w2 9EG1Zq1ecGLYn212hYf85XHFyxSMGk9pOhxhoA9G8UWSwE9fnGKPVznVU KQjMDtRX9CKuWpF3tus3/RxwQTHbtTHJjFcwTye681DuNphG+dmsOw+S6 t+WKJrNY2ftlYNs0RAS8dRnP5wIyyUGrHP/sBXP8P+qJEsqQUvYYR46db RYVgRf63Nkf91AeI870POh2y7XVlVVsZ5EtxB5zLphsPZx71zl1wGcoUa A==; X-IronPort-AV: E=Sophos;i="5.51,440,1526313600"; d="scan'208";a="90059280" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 04 Aug 2018 08:03:31 +0800 Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP; 03 Aug 2018 16:51:15 -0700 Received: from thinkpad-bart.sdcorp.global.sandisk.com ([10.111.67.248]) by uls-op-cesaip01.wdc.com with ESMTP; 03 Aug 2018 17:03:27 -0700 From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Jianchao Wang , Ming Lei , Alan Stern , Johannes Thumshirn Subject: [PATCH v4 09/10] blk-mq: Insert blk_pm_{add,put}_request() calls Date: Fri, 3 Aug 2018 17:03:24 -0700 Message-Id: <20180804000325.3610-10-bart.vanassche@wdc.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180804000325.3610-1-bart.vanassche@wdc.com> References: <20180804000325.3610-1-bart.vanassche@wdc.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Make sure that blk_pm_add_request() is called exactly once before a request is added to a software queue, to the scheduler and also before .queue_rq() is called directly. Call blk_pm_put_request() after a request has finished. Signed-off-by: Bart Van Assche Cc: Christoph Hellwig Cc: Jianchao Wang Cc: Ming Lei Cc: Alan Stern Cc: Johannes Thumshirn --- block/blk-mq-sched.c | 13 +++++++++++-- block/blk-mq.c | 8 ++++++++ 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index cf9c66c6d35a..d87839b31d56 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -14,6 +14,7 @@ #include "blk-mq-debugfs.h" #include "blk-mq-sched.h" #include "blk-mq-tag.h" +#include "blk-pm.h" #include "blk-wbt.h" void blk_mq_sched_free_hctx_data(struct request_queue *q, @@ -349,6 +350,8 @@ static bool blk_mq_sched_bypass_insert(struct blk_mq_hw_ctx *hctx, { /* dispatch flush rq directly */ if (rq->rq_flags & RQF_FLUSH_SEQ) { + blk_pm_add_request(rq->q, rq); + spin_lock(&hctx->lock); list_add(&rq->queuelist, &hctx->dispatch); spin_unlock(&hctx->lock); @@ -380,6 +383,8 @@ void blk_mq_sched_insert_request(struct request *rq, bool at_head, if (blk_mq_sched_bypass_insert(hctx, !!e, rq)) goto run; + blk_pm_add_request(q, rq); + if (e && e->type->ops.mq.insert_requests) { LIST_HEAD(list); @@ -402,10 +407,14 @@ void blk_mq_sched_insert_requests(struct request_queue *q, { struct blk_mq_hw_ctx *hctx = blk_mq_map_queue(q, ctx->cpu); struct elevator_queue *e = hctx->queue->elevator; + struct request *rq; + + if (e && e->type->ops.mq.insert_requests) { + list_for_each_entry(rq, list, queuelist) + blk_pm_add_request(q, rq); - if (e && e->type->ops.mq.insert_requests) e->type->ops.mq.insert_requests(hctx, list, false); - else { + } else { /* * try to issue requests directly if the hw queue isn't * busy in case of 'none' scheduler, and this way may save diff --git a/block/blk-mq.c b/block/blk-mq.c index b1882a3a5216..74a575a32dda 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -36,6 +36,7 @@ #include "blk-mq-tag.h" #include "blk-stat.h" #include "blk-mq-sched.h" +#include "blk-pm.h" #include "blk-rq-qos.h" static bool blk_mq_poll(struct request_queue *q, blk_qc_t cookie); @@ -474,6 +475,8 @@ static void __blk_mq_free_request(struct request *rq) struct blk_mq_hw_ctx *hctx = blk_mq_map_queue(q, ctx->cpu); const int sched_tag = rq->internal_tag; + blk_pm_put_request(rq); + if (rq->tag != -1) blk_mq_put_tag(hctx, hctx->tags, ctx, rq->tag); if (sched_tag != -1) @@ -1563,6 +1566,8 @@ void blk_mq_request_bypass_insert(struct request *rq, bool run_queue) struct blk_mq_ctx *ctx = rq->mq_ctx; struct blk_mq_hw_ctx *hctx = blk_mq_map_queue(rq->q, ctx->cpu); + blk_pm_add_request(rq->q, rq); + spin_lock(&hctx->lock); list_add_tail(&rq->queuelist, &hctx->dispatch); spin_unlock(&hctx->lock); @@ -1584,6 +1589,7 @@ void blk_mq_insert_requests(struct blk_mq_hw_ctx *hctx, struct blk_mq_ctx *ctx, list_for_each_entry(rq, list, queuelist) { BUG_ON(rq->mq_ctx != ctx); trace_block_rq_insert(hctx->queue, rq); + blk_pm_add_request(rq->q, rq); } spin_lock(&ctx->lock); @@ -1680,6 +1686,8 @@ static blk_status_t __blk_mq_issue_directly(struct blk_mq_hw_ctx *hctx, blk_qc_t new_cookie; blk_status_t ret; + blk_pm_add_request(q, rq); + new_cookie = request_to_qc_t(hctx, rq); /* From patchwork Sat Aug 4 00:03:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10555575 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A18DC13AC for ; Sat, 4 Aug 2018 00:03:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E0ED2C913 for ; Sat, 4 Aug 2018 00:03:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 327AA2C971; Sat, 4 Aug 2018 00:03:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D42332C91B for ; Sat, 4 Aug 2018 00:03:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732090AbeHDCCB (ORCPT ); Fri, 3 Aug 2018 22:02:01 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:34555 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732150AbeHDCCA (ORCPT ); Fri, 3 Aug 2018 22:02:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1533341012; x=1564877012; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=5WJT3BJpez8UmTLcpNtNBO+pixyn/LNc6PmBgnCL7YA=; b=gNGH4nyYZwM6HO979yh0crOvvVit7UOwkjPeo6fcqs3jSeHNW9dgq065 xWHkHEzSbjG5YQW3uIoH+Z4oN/Ib45G6tPzdC7BCwbLcp/+9uER5vH+OO B72C0T+YfuMiBMj0D8XOVEToJzGzj6GSQam3PalL0otRBmKKZ2ExcVZO4 3uvnSHUJ8KLbtPqHWMw6khJ2g6C71GHxOAOauHr31I8I4jd6GFm9bKePu 9PaAlOZEN/ga0bZpRb/F18MGV4WU7JYJxyxyWP/NlcAvl5F1FPg3qS6x9 Njg0G0UmI7pB/2b7drJFzb8uildXO7uzGs5Qf6fRtn1/4tBph9xCX1KXT A==; X-IronPort-AV: E=Sophos;i="5.51,440,1526313600"; d="scan'208";a="90059282" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 04 Aug 2018 08:03:32 +0800 Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP; 03 Aug 2018 16:51:15 -0700 Received: from thinkpad-bart.sdcorp.global.sandisk.com ([10.111.67.248]) by uls-op-cesaip01.wdc.com with ESMTP; 03 Aug 2018 17:03:27 -0700 From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Jianchao Wang , Ming Lei , Alan Stern , Johannes Thumshirn Subject: [PATCH v4 10/10] blk-mq: Enable support for runtime power management Date: Fri, 3 Aug 2018 17:03:25 -0700 Message-Id: <20180804000325.3610-11-bart.vanassche@wdc.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180804000325.3610-1-bart.vanassche@wdc.com> References: <20180804000325.3610-1-bart.vanassche@wdc.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Now that the blk-mq core processes power management requests (marked with RQF_PREEMPT) in other states than RPM_ACTIVE, enable runtime power management for blk-mq. Signed-off-by: Bart Van Assche Cc: Christoph Hellwig Cc: Jianchao Wang Cc: Ming Lei Cc: Alan Stern Cc: Johannes Thumshirn --- block/blk-pm.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/block/blk-pm.c b/block/blk-pm.c index 0708185ead61..583cfc04e869 100644 --- a/block/blk-pm.c +++ b/block/blk-pm.c @@ -66,12 +66,6 @@ void blk_pm_runtime_unlock(struct request_queue *q) */ void blk_pm_runtime_init(struct request_queue *q, struct device *dev) { - /* Don't enable runtime PM for blk-mq until it is ready */ - if (q->mq_ops) { - pm_runtime_disable(dev); - return; - } - q->dev = dev; q->rpm_status = RPM_ACTIVE; pm_runtime_set_autosuspend_delay(q->dev, -1);