From patchwork Mon Apr 27 17:36:23 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 6282041 Return-Path: X-Original-To: patchwork-linux-fsdevel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 91756BEEE1 for ; Mon, 27 Apr 2015 17:36:30 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id D194C202B4 for ; Mon, 27 Apr 2015 17:36:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 20408202AE for ; Mon, 27 Apr 2015 17:36:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932990AbbD0Rg1 (ORCPT ); Mon, 27 Apr 2015 13:36:27 -0400 Received: from mail-ie0-f175.google.com ([209.85.223.175]:36140 "EHLO mail-ie0-f175.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932203AbbD0Rg1 (ORCPT ); Mon, 27 Apr 2015 13:36:27 -0400 Received: by iebrs15 with SMTP id rs15so132065555ieb.3 for ; Mon, 27 Apr 2015 10:36:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type; bh=uhwQ0+otu74yAJKvwDTEIJ2MGEdUsLg/37WBA1l+UBA=; b=fFMuX5zQADki56Nuac3/wh4B5Q8C8sdJStzzI74D3xlxwkKNE+X9n3Bxz5pCkwCS2O uytJ+OiiIPivIUirtTvIKbgSZBi2rXp5m/UNhr8bCmSK4zWE03SsE+aFS78DWtp/lmxZ Kw+bak7g+lM3O7HbuUvHWlYPdQXuzFVDONblRETbWMY4qBdgSggtRUUpchJG1S5SzNkj BGawW+/i8n0cAqzSdGMTUtGGIhGEi0YuKkV3EXMcZRaqBlmgAdIqfqeph6FRXKZbUT5v crF+K8za4cpe+HckGnsAH56Gmwe1FDiSrNljnr9vgeWwlF/QF/d2rVQyPf6d5f3bgOPc t9kA== X-Gm-Message-State: ALoCoQn2UVMkxTeXfnrmCgPu5wkYZgawp9ob9Ue+XGwUOeP9aofEov5mQ73CrJJRTcZkW1lQY/gR X-Received: by 10.43.110.198 with SMTP id el6mr13665495icc.69.1430156186234; Mon, 27 Apr 2015 10:36:26 -0700 (PDT) Received: from [192.168.1.118] ([216.160.245.98]) by mx.google.com with ESMTPSA id 16sm13067044ion.20.2015.04.27.10.36.24 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Apr 2015 10:36:24 -0700 (PDT) Message-ID: <553E7397.2010208@kernel.dk> Date: Mon, 27 Apr 2015 11:36:23 -0600 From: Jens Axboe User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Dmitry Krivenok , linux-fsdevel@vger.kernel.org Subject: Re: [PATCH 1/1] null_blk: fix handling of BLKPREP_DEFER case References: <553E70FA.1080302@kernel.dk> In-Reply-To: <553E70FA.1080302@kernel.dk> Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, T_TVD_MIME_EPI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 04/27/2015 11:25 AM, Jens Axboe wrote: > On 04/19/2015 12:19 PM, Dmitry Krivenok wrote: >> When we fail to allocate new cmd in null_rq_prep_fn we return >> BLKPREP_DEFER >> which is not handled properly. In single-queue mode of null_blk the >> following >> command hangs forever in io_schedule(): >> $ dd if=/dev/nullb0 of=/dev/null bs=8M count=5000 iflag=direct >> >> The reason is that when 64 commands have been allocated, the 65th command >> allocation will fail due to missing free tag. The request, however, >> will be >> kept in the queue which will never be started again (unless you run >> another >> command that does I/O to /dev/nullb0). >> >> This small patch tries to solve the issue by stopping the queue when we >> detect that all tags were exhausted and starting it again when we free >> the tag. >> >> I've verified that the command mentioned above doesn't hang anymore >> and also >> made sure that null_blk with my change survives fio-based stress tests. > > You are right, legacy request_fn mode has a bug there. I'd get rid of > the no_cmds bool, though. If we fail allocating a command in alloc_cmd() > and we're in NULL_Q_RQ mode, stop the queue. In free_cmd(), again if > we're in NULL_Q_RQ_MODE, check the stopped flag and start the queue if > it is set. Something like the attached, totally untested. diff --git a/drivers/block/null_blk.c b/drivers/block/null_blk.c index 65cd61a4145e..d7141d5ef4ff 100644 --- a/drivers/block/null_blk.c +++ b/drivers/block/null_blk.c @@ -24,6 +24,7 @@ struct nullb_queue { wait_queue_head_t wait; unsigned int queue_depth; + struct nullb *dev; struct nullb_cmd *cmds; }; @@ -153,6 +154,13 @@ static void put_tag(struct nullb_queue *nq, unsigned int tag) if (waitqueue_active(&nq->wait)) wake_up(&nq->wait); + + if (queue_mode == NULL_Q_RQ) { + struct nullb *nullb = nq->dev; + + if (blk_queue_stopped(nullb->q)) + blk_start_queue(nullb->q); + } } static unsigned int get_tag(struct nullb_queue *nq) @@ -196,7 +204,7 @@ static struct nullb_cmd *alloc_cmd(struct nullb_queue *nq, int can_wait) cmd = __alloc_cmd(nq); if (cmd || !can_wait) - return cmd; + goto out; do { prepare_to_wait(&nq->wait, &wait, TASK_UNINTERRUPTIBLE); @@ -208,6 +216,11 @@ static struct nullb_cmd *alloc_cmd(struct nullb_queue *nq, int can_wait) } while (1); finish_wait(&nq->wait, &wait); +out: + if (!cmd) { + BUG_ON(queue_mode != NULL_Q_RQ); + blk_stop_queue(nq->dev->q); + } return cmd; } @@ -372,6 +385,7 @@ static void null_init_queue(struct nullb *nullb, struct nullb_queue *nq) init_waitqueue_head(&nq->wait); nq->queue_depth = nullb->queue_depth; + nq->dev = nullb; } static int null_init_hctx(struct blk_mq_hw_ctx *hctx, void *data,