From patchwork Tue Oct 3 14:04:05 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 9982959 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 436526029B for ; Tue, 3 Oct 2017 14:06:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 364B02880C for ; Tue, 3 Oct 2017 14:06:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2B41428883; Tue, 3 Oct 2017 14:06:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C70372880C for ; Tue, 3 Oct 2017 14:06:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752172AbdJCOFj (ORCPT ); Tue, 3 Oct 2017 10:05:39 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57712 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751156AbdJCOFi (ORCPT ); Tue, 3 Oct 2017 10:05:38 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 827FA7AE91; Tue, 3 Oct 2017 14:05:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 827FA7AE91 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=ming.lei@redhat.com Received: from localhost (ovpn-12-19.pek2.redhat.com [10.72.12.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7E81462684; Tue, 3 Oct 2017 14:05:30 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org, Christoph Hellwig , linux-scsi@vger.kernel.org, "Martin K . Petersen" , "James E . J . Bottomley" Cc: Bart Van Assche , Oleksandr Natalenko , Johannes Thumshirn , Cathy Avery , Martin Steigerwald , linux-kernel@vger.kernel.org, Hannes Reinecke , Ming Lei , Bart Van Assche Subject: [PATCH V8 7/8] block: support PREEMPT_ONLY Date: Tue, 3 Oct 2017 22:04:05 +0800 Message-Id: <20171003140406.26060-8-ming.lei@redhat.com> In-Reply-To: <20171003140406.26060-1-ming.lei@redhat.com> References: <20171003140406.26060-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 03 Oct 2017 14:05:38 +0000 (UTC) Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When queue is in PREEMPT_ONLY mode, only REQ_PREEMPT request can be allocated and dispatched, other requests won't be allowed to enter I/O path. This is useful for supporting safe SCSI quiesce. Part of this patch is from Bart's '[PATCH v4 4∕7] block: Add the QUEUE_FLAG_PREEMPT_ONLY request queue flag'. Tested-by: Oleksandr Natalenko Tested-by: Martin Steigerwald Cc: Bart Van Assche Signed-off-by: Ming Lei --- block/blk-core.c | 44 +++++++++++++++++++++++++++++++++++++++++--- include/linux/blkdev.h | 5 +++++ 2 files changed, 46 insertions(+), 3 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 1bb566245d37..7849cc1687bc 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -346,6 +346,34 @@ void blk_sync_queue(struct request_queue *q) } EXPORT_SYMBOL(blk_sync_queue); +void blk_set_preempt_only(struct request_queue *q, bool preempt_only) +{ + unsigned long flags; + + spin_lock_irqsave(q->queue_lock, flags); + if (preempt_only) + queue_flag_set(QUEUE_FLAG_PREEMPT_ONLY, q); + else + queue_flag_clear(QUEUE_FLAG_PREEMPT_ONLY, q); + spin_unlock_irqrestore(q->queue_lock, flags); + + /* + * The synchronize_rcu() implicied in blk_mq_freeze_queue() + * or the explicit one will make sure the above write on + * PREEMPT_ONLY is observed in blk_queue_enter() before + * running blk_mq_unfreeze_queue(). + * + * blk_mq_freeze_queue() also drains up any request in queue, + * so blk_queue_enter() will see the above updated value of + * PREEMPT flag before any new allocation. + */ + if (!blk_mq_freeze_queue(q)) + synchronize_rcu(); + + blk_mq_unfreeze_queue(q); +} +EXPORT_SYMBOL(blk_set_preempt_only); + /** * __blk_run_queue_uncond - run a queue whether or not it has been stopped * @q: The queue to run @@ -771,8 +799,16 @@ int blk_queue_enter(struct request_queue *q, unsigned int op) while (true) { int ret; - if (percpu_ref_tryget_live(&q->q_usage_counter)) - return 0; + rcu_read_lock_sched(); + if (__percpu_ref_tryget_live(&q->q_usage_counter)) { + if (likely((op & REQ_PREEMPT) || + !blk_queue_preempt_only(q))) { + rcu_read_unlock_sched(); + return 0; + } else + percpu_ref_put(&q->q_usage_counter); + } + rcu_read_unlock_sched(); if (op & REQ_NOWAIT) return -EBUSY; @@ -787,7 +823,9 @@ int blk_queue_enter(struct request_queue *q, unsigned int op) smp_rmb(); ret = wait_event_interruptible(q->mq_freeze_wq, - !atomic_read(&q->mq_freeze_depth) || + (!atomic_read(&q->mq_freeze_depth) && + ((op & REQ_PREEMPT) || + !blk_queue_preempt_only(q))) || blk_queue_dying(q)); if (blk_queue_dying(q)) return -ENODEV; diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 4c688385d866..66d46d9eac29 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -628,6 +628,7 @@ struct request_queue { #define QUEUE_FLAG_REGISTERED 26 /* queue has been registered to a disk */ #define QUEUE_FLAG_SCSI_PASSTHROUGH 27 /* queue supports SCSI commands */ #define QUEUE_FLAG_QUIESCED 28 /* queue has been quiesced */ +#define QUEUE_FLAG_PREEMPT_ONLY 29 /* only process REQ_PREEMPT requests */ #define QUEUE_FLAG_DEFAULT ((1 << QUEUE_FLAG_IO_STAT) | \ (1 << QUEUE_FLAG_STACKABLE) | \ @@ -732,6 +733,10 @@ static inline void queue_flag_clear(unsigned int flag, struct request_queue *q) ((rq)->cmd_flags & (REQ_FAILFAST_DEV|REQ_FAILFAST_TRANSPORT| \ REQ_FAILFAST_DRIVER)) #define blk_queue_quiesced(q) test_bit(QUEUE_FLAG_QUIESCED, &(q)->queue_flags) +#define blk_queue_preempt_only(q) \ + test_bit(QUEUE_FLAG_PREEMPT_ONLY, &(q)->queue_flags) + +extern void blk_set_preempt_only(struct request_queue *q, bool preempt_only); static inline bool blk_account_rq(struct request *rq) {