From patchwork Mon Oct 31 17:24:02 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 9406093 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 32A3C601C0 for ; Mon, 31 Oct 2016 17:24:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 254DF2932B for ; Mon, 31 Oct 2016 17:24:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 19E7E2935F; Mon, 31 Oct 2016 17:24:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5221A29355 for ; Mon, 31 Oct 2016 17:24:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S944920AbcJaRYG (ORCPT ); Mon, 31 Oct 2016 13:24:06 -0400 Received: from mail-yb0-f170.google.com ([209.85.213.170]:35689 "EHLO mail-yb0-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934479AbcJaRYF (ORCPT ); Mon, 31 Oct 2016 13:24:05 -0400 Received: by mail-yb0-f170.google.com with SMTP id d128so63790854ybh.2 for ; Mon, 31 Oct 2016 10:24:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=tpKUaiC0CBZx9ZQCJXqTeverLwO3IFfnedtNDNSIW38=; b=BeCknCDu1YQ/BhwfNsJ4b6MoBoXyFQxmvLwSmW2SKoq8qG/g8v2YTS1Cf9KC/a8V8L C1nxwhbZ4ZK+TiV+wPoHiypF0wMLgpnz05STc0vfrviFpUlSk6gPcZX+DN+Ou2FNvJut CYeJK17G0fM7nIGIrUUwia8p79+TsKAvIstpk3vwcRT8i/fVZBG+TTi7h8tMf0Oz/SLx 4e6FRxc8pRq+vOPCQ7lEbyUELDLzEJLgrC7adjEJQAY9gPF7Zf2XX3FtACTrZwTf70Ny 6uZ7SryRXc+IeSLW3zDmf93DJMuKN6Zz9Fck2FmOBzcZIDB48zbZWA4XmqijSOKjKmME Lszw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=tpKUaiC0CBZx9ZQCJXqTeverLwO3IFfnedtNDNSIW38=; b=DrmhkEUaSi6wX5vXrcw0+AIjs9qCJG4l4FcNKJQAjfb3NIB4JKZZy5jw7NureS65zY 3rqIwTDOOOwMKqXrGEpAPP571CTkxvyVmDdggij+krPga6xmsLKt5Cyl2jHidB+uwPjH aovtyOod7JuoypKIv8jJIWpyHKwseJzSpDlfR8/j0Bn6LBef7l1zQrDcJZTph4Zmmw3Z VnYt5lb98Io/YNaX+rmvEiJiUVtLgvS7WkGxqJfeF5/wnlJ6u4BEO35jgO4HNLQ60lOJ AEnC6rfVpU8oy9JKscd/awKb2lXlsxlmEdiaoDl3+qtbPer+VCu5dys+JgMH6kJUy38L XMGA== X-Gm-Message-State: ABUngvdzn9UPd5FMefu/tOT8ykjzx7g58xKexQ/4UrQUSybqePvxDYxIkftgzWKoCBjg8w== X-Received: by 10.36.95.138 with SMTP id r132mr9091037itb.95.1477934644167; Mon, 31 Oct 2016 10:24:04 -0700 (PDT) Received: from [10.141.86.225] (70-90-202-86-Albuquerque.hfc.comcastbusiness.net. [70.90.202.86]) by smtp.gmail.com with ESMTPSA id l63sm324067ioe.8.2016.10.31.10.24.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 31 Oct 2016 10:24:03 -0700 (PDT) Subject: Re: Device or HBA level QD throttling creates randomness in sequetial workload To: Kashyap Desai , Omar Sandoval References: <2d656e9c9fbde7206e40a635c61a6084@mail.gmail.com> Cc: linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, Christoph Hellwig , paolo.valente@linaro.org From: Jens Axboe Message-ID: <298b6ff6-9feb-4b70-ec4c-d1295a0e1f41@kernel.dk> Date: Mon, 31 Oct 2016 11:24:02 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <2d656e9c9fbde7206e40a635c61a6084@mail.gmail.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hi, One guess would be that this isn't around a requeue condition, but rather the fact that we don't really guarantee any sort of hard FIFO behavior between the software queues. Can you try this test patch to see if it changes the behavior for you? Warning: untested... * Note that this function currently has various problems around ordering @@ -812,6 +820,14 @@ static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx) } /* + * If the device is rotational, sort the list sanely to avoid + * unecessary seeks. The software queues are roughly FIFO, but + * only roughly, there are no hard guarantees. + */ + if (!blk_queue_nonrot(q)) + list_sort(NULL, &rq_list, rq_pos_cmp); + + /* * Start off with dptr being NULL, so we start the first request * immediately, even if we have more pending. */ diff --git a/block/blk-mq.c b/block/blk-mq.c index f3d27a6dee09..5404ca9c71b2 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -772,6 +772,14 @@ static inline unsigned int queued_to_index(unsigned int queued) return min(BLK_MQ_MAX_DISPATCH_ORDER - 1, ilog2(queued) + 1); } +static int rq_pos_cmp(void *priv, struct list_head *a, struct list_head *b) +{ + struct request *rqa = container_of(a, struct request, queuelist); + struct request *rqb = container_of(b, struct request, queuelist); + + return blk_rq_pos(rqa) < blk_rq_pos(rqb); +} + /* * Run this hardware queue, pulling any software queues mapped to it in.