From patchwork Wed Apr 26 18:22:43 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 9701795 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 4EA6260245 for ; Wed, 26 Apr 2017 18:22:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E3702846C for ; Wed, 26 Apr 2017 18:22:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2F3072860C; Wed, 26 Apr 2017 18:22:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9B7FE2846C for ; Wed, 26 Apr 2017 18:22:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933185AbdDZSWq (ORCPT ); Wed, 26 Apr 2017 14:22:46 -0400 Received: from mail-pg0-f50.google.com ([74.125.83.50]:33677 "EHLO mail-pg0-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933178AbdDZSWp (ORCPT ); Wed, 26 Apr 2017 14:22:45 -0400 Received: by mail-pg0-f50.google.com with SMTP id 63so3976813pgh.0 for ; Wed, 26 Apr 2017 11:22:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=R9VeneaROHpqduWmMbVEjaRHCRgzwqTrQn/TP4jFV2g=; b=Jnrj4S0LGzbI1VD5M8ZQTg552cw3QDJGDyERYDfXY3m+mx5+QzfDdCNEklBhgBBo8N 7Ip273VNQYwObdV9yfKFmQOtGTnlGMwPHrYg4WxCxgLRGBVqEH/ilDhmibPRedptvK4q kyb36s/r8wqLvtolFP1zJLB4MbrOVVtp2+rOuC7Mtijto0Vl7vKvMNTfleh1ptjhHb5P esirN3ByxV9lsCYLlSe1LKVT6Jnn3p79ap2jNTO4LlVMgTLjujWgMj+tsTbsAdrJA3zz TmtCswoczKNLOFw//W8XqcBEcybkK5me85T1W2ph5WM5zO9uHn6zA4kIAcYQzOnDn8tX aD5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=R9VeneaROHpqduWmMbVEjaRHCRgzwqTrQn/TP4jFV2g=; b=E6kNVH4PvqRlQIOUbpzSHdzpcCaDjdcoraB1YwJUQk+Bk6YVLBsL5NJWr8dyl5gMC9 gtkZ58GMY3uMpI+uR1VxZQrsQlhvmAg7HpA7JTN44b4ynGOeSPWfApHQlis7/5dKCEPz EuPEkGhT218cUrpYZ7Hdl4imQukWt39+AplEAu7Q0bqJR7/YE7fe9oKp8L5N6oWP77ca wuvxtAa+A/h6/97/gMa2dC3+NutkAcTgrp2Ui+GeQ3CtvIPcWfTf1PkD2Ie71Soe1GDy jyM95POBSLMB60J8PW7e/d/izQfNoXlaadSUjufCZmwjUNAzj7bX7EVR2Ow1z9EP/zXY PDlQ== X-Gm-Message-State: AN3rC/5YxELI4m94jnZDjGoHB5BLyyIw6Y9nwhiiJYUwhoD0HfKk4LX7 QdNs4Gpg4OjFOw== X-Received: by 10.84.135.34 with SMTP id 31mr1453801pli.99.1493230964446; Wed, 26 Apr 2017 11:22:44 -0700 (PDT) Received: from [10.55.53.86] ([216.240.19.104]) by smtp.gmail.com with ESMTPSA id g89sm27248pfk.25.2017.04.26.11.22.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Apr 2017 11:22:43 -0700 (PDT) Subject: Re: [PATCH 0/4] blk-mq-sched: allow to use hw tag for sched To: Ming Lei , Christoph Hellwig References: <20170415123825.32716-1-ming.lei@redhat.com> <20170420004410.GA16917@ming.t460p> <90e249d6-996c-a8d9-f54d-e8142082bfa5@fb.com> <20170420010346.GC16917@ming.t460p> <20170420045408.GB2235@infradead.org> <20170420083041.GA1637@ming.t460p> <20170426104842.GB28062@ming.t460p> <38bc590d-8ab4-5c88-d7e7-10cd2ec7d900@kernel.dk> Cc: linux-block@vger.kernel.org, Omar Sandoval , Jozef Mikovic From: Jens Axboe Message-ID: Date: Wed, 26 Apr 2017 11:22:43 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <38bc590d-8ab4-5c88-d7e7-10cd2ec7d900@kernel.dk> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 04/26/2017 11:15 AM, Jens Axboe wrote: > On 04/26/2017 03:48 AM, Ming Lei wrote: >> Hi Christoph, >> >> On Thu, Apr 20, 2017 at 04:30:42PM +0800, Ming Lei wrote: >>> Hi Christoph, >>> >>> On Wed, Apr 19, 2017 at 09:54:08PM -0700, Christoph Hellwig wrote: >>>> On Thu, Apr 20, 2017 at 09:03:47AM +0800, Ming Lei wrote: >>>>> If we don't need to reserve tag for internal command, I am happy >>>>> to fix it in the interim. However, the mtip32xx driver has to >>>>> ask blk-mq to reserve the tag zero for internal command. Then >>>>> if we can't allow the driver to use that reserved tag, it looks >>>>> quite awkward, doesn't it? >>>> >>>> It doesn't. Just offset the hardware tags by 1 from the blk-mq tags. >>>> >>>> But I'm pretty sure the hardware wouldn't require a specific tag >>>> for the internal ops. >>> >>> From mtip32xx code, the tag 0 is used to send internal command only, and >>> not used for submitting normal request. >>> >>> From code of mtip_issue_non_ncq_command() and mtip_issue_ncq_command(), >>> even same hardware address is writen to when issuing non-ncq and ncq >>> commands. So I think the internal ops and normal requests share one same >>> tag space on mtip32xx. >>> >>> Could you explain it a bit why you think the hw doesn't need the tag >>> for internal ops in this case? >> >> Gentle ping... >> >> Looks there are some choices for this issue: >> >> 1) if internal ops uses independent tag space >> - we need to clean up the mtip32xx driver >> >> 2) if internal ops shares tag space with normal request >> - export blk_mq_get_driver_tag() for mtip32xx >> >> or >> >> - use private way to send internal ops, and the drawback >> is that we still need to reserve tag and the reserved tag >> can't be used at all. >> >> From mtip32xx source code, I can see both share same tag space. >> When I try to search hw manual via google, not succeed yet. >> >> Christoph, could you share us your findings about if internal ops >> uses independent tag space or not? > > Why aren't we just bypassing for RESERVED? If someone is asking > for an reserved tag, ensure that we use the right tag pool (sched > vs normal) and sub pool (reserved vs normal). Then insert just > bypasses the scheduler, like it already does if we have a driver > tag assigned already. > > I've got a few of these cards, but traveling. I'll wire up something > and test it end this week. Something like this. Totally untested. Tested-by: Ming Lei diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index c974a1bbf4cb..11109b5b64b6 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -119,7 +119,11 @@ struct request *blk_mq_sched_get_request(struct request_queue *q, if (likely(!data->hctx)) data->hctx = blk_mq_map_queue(q, data->ctx->cpu); - if (e) { + /* + * For a reserved tag, allocate a normal request since we might + * have driver dependencies on the value of the internal tag. + */ + if (e && !(data->flags & BLK_MQ_REQ_RESERVED)) { data->flags |= BLK_MQ_REQ_INTERNAL; /*