From patchwork Thu May 30 11:28:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 10968627 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 966481398 for ; Thu, 30 May 2019 11:28:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 864BF28B03 for ; Thu, 30 May 2019 11:28:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7A3AA28B0F; Thu, 30 May 2019 11:28:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E1F8928B03 for ; Thu, 30 May 2019 11:28:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726849AbfE3L2R (ORCPT ); Thu, 30 May 2019 07:28:17 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:52679 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726693AbfE3L2Q (ORCPT ); Thu, 30 May 2019 07:28:16 -0400 Received: by mail-wm1-f68.google.com with SMTP id y3so3739755wmm.2; Thu, 30 May 2019 04:28:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SX81W3B+rOFD9q++bpOTVE7YZYqatJqgvbGp4A8Bpd4=; b=qtfA2WUU97N7gZfyk7DWfoQPdw8Kpd87wEcrAaWdIVYQ5JZ/5X78DZYI7MJRxT8L7N 0Up3Ksnc/XWVO+nKcLk3vR2sxLNo+cp9Fzw0/sH+pOnwjfsNa6QNvbJWpAw3/ICY/K4q J9/fH2X4mA59nLQp6NYv81p0QcLuViSBUtCcyiuvO+8MjNvSuXywaGxjtruYXYneLg5z 0QPmkteRQ3NUVrLqtRUz0epkgEQBJQ3muJ9rV2oLbucSppFNwA1LAtFV323HFCDLzwd5 Ul0rlQLkjagDorhpJRKxGhdnnH2ITXaIXqR4jClV+NStLbhqT2HDMeVSVho4UKvSy7we R8kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=SX81W3B+rOFD9q++bpOTVE7YZYqatJqgvbGp4A8Bpd4=; b=OLia7Aj3GSuUGs2F4nylQ/O8wSGm0sXhlUCwG0JshPCgkUF+fMk+VvJp3Q6l7BlAr7 8mxvpOYj/Hm7c2lAKY/yWhxJnzWjfDvF9xlrvsC0QiQ/3wOOEzfHYgt4cYSmUpJ9rO9T mo2HOmc8fvqhrdPpFugdFdMuIXUEZzK8xwI5a9PsV8AZFdeGCi4moaF7f6QPupis4CZi 82NDr5JI/Y8pJpZvF0OKD4rqqGHi4NQw8XeeZ49RZVC4CGdva9/T9idRvvXPRnu+/5sm I1SbEftcUvSbOq5Rk+5TG6wgyydDbHYVcXECsLfybhAaakFWu7XPPBNaO6o1N3RCm8po 7o2g== X-Gm-Message-State: APjAAAUmJwq2OWCnoNWXPImEoHpH6JaSPXYnncPw7E6FjZ2U+OXk4qU9 ZyaNSjjzeJCJtBC+f5Zi5PAQ2KV6 X-Google-Smtp-Source: APXvYqwuawoqoaxsaAi5dfLXs/kRgdB7egTSB3YbtyStW8FUYG8b0Z7tqygOGZGRB8BZjVD9A/viEg== X-Received: by 2002:a05:600c:2187:: with SMTP id e7mr1972772wme.16.1559215694070; Thu, 30 May 2019 04:28:14 -0700 (PDT) Received: from donizetti.redhat.com ([2001:b07:6468:f312:f91e:ffe0:9205:3b26]) by smtp.gmail.com with ESMTPSA id o14sm2601855wrp.77.2019.05.30.04.28.13 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 30 May 2019 04:28:13 -0700 (PDT) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jejb@linux.ibm.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, stefanha@redhat.com Subject: [PATCH 1/2] scsi_host: add support for request batching Date: Thu, 30 May 2019 13:28:10 +0200 Message-Id: <20190530112811.3066-2-pbonzini@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190530112811.3066-1-pbonzini@redhat.com> References: <20190530112811.3066-1-pbonzini@redhat.com> MIME-Version: 1.0 Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This allows a list of requests to be issued, with the LLD only writing the hardware doorbell when necessary, after the last request was prepared. This is more efficient if we have lists of requests to issue, particularly on virtualized hardware, where writing the doorbell is more expensive than on real hardware. The use case for this is plugged IO, where blk-mq flushes a batch of requests all at once. The API is the same as for blk-mq, just with blk-mq concepts tweaked to fit the SCSI subsystem API: the "last" flag in blk_mq_queue_data becomes a flag in scsi_cmnd, while the queue_num in the commit_rqs callback is extracted from the hctx and passed as a parameter. The only complication is that blk-mq uses different plugging heuristics depending on whether commit_rqs is present or not. So we have two different sets of blk_mq_ops and pick one depending on whether the scsi_host template uses commit_rqs or not. Signed-off-by: Paolo Bonzini Reviewed-by: Bart Van Assche Reviewed-by: Ming Lei Reviewed-by: Hannes Reinecke --- drivers/scsi/scsi_lib.c | 37 ++++++++++++++++++++++++++++++++++--- include/scsi/scsi_cmnd.h | 1 + include/scsi/scsi_host.h | 16 ++++++++++++++-- 3 files changed, 49 insertions(+), 5 deletions(-) diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index 601b9f1de267..eb4e67d02bfe 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -1673,10 +1673,11 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx, blk_mq_start_request(req); } + cmd->flags &= SCMD_PRESERVED_FLAGS; if (sdev->simple_tags) cmd->flags |= SCMD_TAGGED; - else - cmd->flags &= ~SCMD_TAGGED; + if (bd->last) + cmd->flags |= SCMD_LAST; scsi_init_cmd_errh(cmd); cmd->scsi_done = scsi_mq_done; @@ -1807,10 +1808,37 @@ void __scsi_init_queue(struct Scsi_Host *shost, struct request_queue *q) } EXPORT_SYMBOL_GPL(__scsi_init_queue); +static const struct blk_mq_ops scsi_mq_ops_no_commit = { + .get_budget = scsi_mq_get_budget, + .put_budget = scsi_mq_put_budget, + .queue_rq = scsi_queue_rq, + .complete = scsi_softirq_done, + .timeout = scsi_timeout, +#ifdef CONFIG_BLK_DEBUG_FS + .show_rq = scsi_show_rq, +#endif + .init_request = scsi_mq_init_request, + .exit_request = scsi_mq_exit_request, + .initialize_rq_fn = scsi_initialize_rq, + .busy = scsi_mq_lld_busy, + .map_queues = scsi_map_queues, +}; + + +static void scsi_commit_rqs(struct blk_mq_hw_ctx *hctx) +{ + struct request_queue *q = hctx->queue; + struct scsi_device *sdev = q->queuedata; + struct Scsi_Host *shost = sdev->host; + + shost->hostt->commit_rqs(shost, hctx->queue_num); +} + static const struct blk_mq_ops scsi_mq_ops = { .get_budget = scsi_mq_get_budget, .put_budget = scsi_mq_put_budget, .queue_rq = scsi_queue_rq, + .commit_rqs = scsi_commit_rqs, .complete = scsi_softirq_done, .timeout = scsi_timeout, #ifdef CONFIG_BLK_DEBUG_FS @@ -1845,7 +1873,10 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost) cmd_size += sizeof(struct scsi_data_buffer) + sgl_size; memset(&shost->tag_set, 0, sizeof(shost->tag_set)); - shost->tag_set.ops = &scsi_mq_ops; + if (shost->hostt->commit_rqs) + shost->tag_set.ops = &scsi_mq_ops; + else + shost->tag_set.ops = &scsi_mq_ops_no_commit; shost->tag_set.nr_hw_queues = shost->nr_hw_queues ? : 1; shost->tag_set.queue_depth = shost->can_queue; shost->tag_set.cmd_size = cmd_size; diff --git a/include/scsi/scsi_cmnd.h b/include/scsi/scsi_cmnd.h index 76ed5e4acd38..91bd749a02f7 100644 --- a/include/scsi/scsi_cmnd.h +++ b/include/scsi/scsi_cmnd.h @@ -57,6 +57,7 @@ struct scsi_pointer { #define SCMD_TAGGED (1 << 0) #define SCMD_UNCHECKED_ISA_DMA (1 << 1) #define SCMD_INITIALIZED (1 << 2) +#define SCMD_LAST (1 << 3) /* flags preserved across unprep / reprep */ #define SCMD_PRESERVED_FLAGS (SCMD_UNCHECKED_ISA_DMA | SCMD_INITIALIZED) diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h index 2b539a1b3f62..28f1c9177cd2 100644 --- a/include/scsi/scsi_host.h +++ b/include/scsi/scsi_host.h @@ -80,8 +80,10 @@ struct scsi_host_template { * command block to the LLDD. When the driver finished * processing the command the done callback is invoked. * - * If queuecommand returns 0, then the HBA has accepted the - * command. The done() function must be called on the command + * If queuecommand returns 0, then the driver has accepted the + * command. It must also push it to the HBA if the scsi_cmnd + * flag SCMD_LAST is set, or if the driver does not implement + * commit_rqs. The done() function must be called on the command * when the driver has finished with it. (you may call done on the * command before queuecommand returns, but in this case you * *must* return 0 from queuecommand). @@ -109,6 +111,16 @@ struct scsi_host_template { */ int (* queuecommand)(struct Scsi_Host *, struct scsi_cmnd *); + /* + * The commit_rqs function is used to trigger a hardware + * doorbell after some requests have been queued with + * queuecommand, when an error is encountered before sending + * the request with SCMD_LAST set. + * + * STATUS: OPTIONAL + */ + void (*commit_rqs)(struct Scsi_Host *, u16); + /* * This is an error handling strategy routine. You don't need to * define one of these if you don't want to - there is a default From patchwork Thu May 30 11:28:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 10968633 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E30991398 for ; Thu, 30 May 2019 11:28:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D526028B03 for ; Thu, 30 May 2019 11:28:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C8EDC28B0F; Thu, 30 May 2019 11:28:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 619EB28B03 for ; Thu, 30 May 2019 11:28:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726924AbfE3L2X (ORCPT ); Thu, 30 May 2019 07:28:23 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:41562 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726699AbfE3L2R (ORCPT ); Thu, 30 May 2019 07:28:17 -0400 Received: by mail-wr1-f65.google.com with SMTP id c2so3963693wrm.8; Thu, 30 May 2019 04:28:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0zslCchXkU9/iE2gw3QRd9dcZ6RTFcWD2DAc5YM+PJU=; b=l/k58YGHILVSQ3nCQBqu7l0MU9SYbdSF3ri5oKMWxBw/9EO1UR+Zs57slpEHC4BfJw ZLELcw/QSBiSq7Jh0wbnj3uynynmL864AH/3COtA/u5hTMo3m0lNJWdhlXUhUlzsYoyy fkjjmovfRx2HZJ8Xk3Ouek+ofPuCSXcHfIb1ox3iCCO087iziUEHTXX2t1y/bSb2MFha dmUSImR2uxEoiI/vokVk9xaQIPx8ZMP4uYPpd11XhIDf5MoTREoTv9ZPRiHKk2cbyPBx EkyUW3hAwsXthKIxjPmtJ8VbMilnfy4XcH6/yRs3m5V2pPOgf663vx7HlIPJOvDRqPHs GeCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=0zslCchXkU9/iE2gw3QRd9dcZ6RTFcWD2DAc5YM+PJU=; b=Dw8e0whjCLHOSxVSyZ3k84OgOjBv8eb9hvPxODmos0CU1FFeobzASrQk0UzZ3fVHOA H4q1uFMc++GrA8ANElCIGzOB2bpjCI6vTdNwj5VJPGgWBvDh4J2aEaYdAecBabMA/Qew bT32jXZubQyjBzKxs6oZZHwpZuIUERI8xOJdARN4Xa3MCZpNPQKRM31COE021/fB0+42 KHXgJuqt8AO84dXNQxnInEmL4tsViX9u0AP3i3V22PpINUYy4h+PdpQdOQLgiS64VjwI jI+Z7gjV85atAw2wlxcu1zXt2ib8qodT7BsaLEG8a7OTUdIbN+kwzA9ZL6n7aREmy5kY TVLQ== X-Gm-Message-State: APjAAAWi+wKfNj6IQI0xzLtZF8kZuQpz5MArUSLRmIngElkwn0fgb024 4794iMGZNsC3mzEGYv1kLp+0wweV X-Google-Smtp-Source: APXvYqwQ1ZolElu//tR8q8fWIFyB08c1e/wWz3z13PLYJQojuLdZFVZt1KGebgNtncrsD4/yItIvJQ== X-Received: by 2002:a5d:6389:: with SMTP id p9mr2330104wru.297.1559215695004; Thu, 30 May 2019 04:28:15 -0700 (PDT) Received: from donizetti.redhat.com ([2001:b07:6468:f312:f91e:ffe0:9205:3b26]) by smtp.gmail.com with ESMTPSA id o14sm2601855wrp.77.2019.05.30.04.28.14 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 30 May 2019 04:28:14 -0700 (PDT) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jejb@linux.ibm.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, stefanha@redhat.com Subject: [PATCH 2/2] virtio_scsi: implement request batching Date: Thu, 30 May 2019 13:28:11 +0200 Message-Id: <20190530112811.3066-3-pbonzini@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190530112811.3066-1-pbonzini@redhat.com> References: <20190530112811.3066-1-pbonzini@redhat.com> MIME-Version: 1.0 Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Adding the command and kicking the virtqueue so far was done one after another. Make the kick optional, so that we can take into account SCMD_LAST. We also need a commit_rqs callback to kick the device if blk-mq aborts the submission before the last request is reached. Suggested-by: Stefan Hajnoczi Signed-off-by: Paolo Bonzini Reviewed-by: Hannes Reinecke Reviewed-by: Ming Lei --- drivers/scsi/virtio_scsi.c | 55 +++++++++++++++++++++++++++----------- 1 file changed, 40 insertions(+), 15 deletions(-) diff --git a/drivers/scsi/virtio_scsi.c b/drivers/scsi/virtio_scsi.c index 8af01777d09c..918c811cea95 100644 --- a/drivers/scsi/virtio_scsi.c +++ b/drivers/scsi/virtio_scsi.c @@ -375,14 +375,7 @@ static void virtscsi_event_done(struct virtqueue *vq) virtscsi_vq_done(vscsi, &vscsi->event_vq, virtscsi_complete_event); }; -/** - * virtscsi_add_cmd - add a virtio_scsi_cmd to a virtqueue - * @vq : the struct virtqueue we're talking about - * @cmd : command structure - * @req_size : size of the request buffer - * @resp_size : size of the response buffer - */ -static int virtscsi_add_cmd(struct virtqueue *vq, +static int __virtscsi_add_cmd(struct virtqueue *vq, struct virtio_scsi_cmd *cmd, size_t req_size, size_t resp_size) { @@ -427,17 +420,39 @@ static int virtscsi_add_cmd(struct virtqueue *vq, return virtqueue_add_sgs(vq, sgs, out_num, in_num, cmd, GFP_ATOMIC); } -static int virtscsi_kick_cmd(struct virtio_scsi_vq *vq, +static void virtscsi_kick_vq(struct virtio_scsi_vq *vq) +{ + bool needs_kick; + unsigned long flags; + + spin_lock_irqsave(&vq->vq_lock, flags); + needs_kick = virtqueue_kick_prepare(vq->vq); + spin_unlock_irqrestore(&vq->vq_lock, flags); + + if (needs_kick) + virtqueue_notify(vq->vq); +} + +/** + * virtscsi_add_cmd - add a virtio_scsi_cmd to a virtqueue, optionally kick it + * @vq : the struct virtqueue we're talking about + * @cmd : command structure + * @req_size : size of the request buffer + * @resp_size : size of the response buffer + * @kick : whether to kick the virtqueue immediately + */ +static int virtscsi_add_cmd(struct virtio_scsi_vq *vq, struct virtio_scsi_cmd *cmd, - size_t req_size, size_t resp_size) + size_t req_size, size_t resp_size, + bool kick) { unsigned long flags; int err; bool needs_kick = false; spin_lock_irqsave(&vq->vq_lock, flags); - err = virtscsi_add_cmd(vq->vq, cmd, req_size, resp_size); - if (!err) + err = __virtscsi_add_cmd(vq->vq, cmd, req_size, resp_size); + if (!err && kick) needs_kick = virtqueue_kick_prepare(vq->vq); spin_unlock_irqrestore(&vq->vq_lock, flags); @@ -502,6 +517,7 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost, struct virtio_scsi *vscsi = shost_priv(shost); struct virtio_scsi_vq *req_vq = virtscsi_pick_vq_mq(vscsi, sc); struct virtio_scsi_cmd *cmd = scsi_cmd_priv(sc); + bool kick; unsigned long flags; int req_size; int ret; @@ -531,7 +547,8 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost, req_size = sizeof(cmd->req.cmd); } - ret = virtscsi_kick_cmd(req_vq, cmd, req_size, sizeof(cmd->resp.cmd)); + kick = (sc->flags & SCMD_LAST) != 0; + ret = virtscsi_add_cmd(req_vq, cmd, req_size, sizeof(cmd->resp.cmd), kick); if (ret == -EIO) { cmd->resp.cmd.response = VIRTIO_SCSI_S_BAD_TARGET; spin_lock_irqsave(&req_vq->vq_lock, flags); @@ -549,8 +566,8 @@ static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd) int ret = FAILED; cmd->comp = ∁ - if (virtscsi_kick_cmd(&vscsi->ctrl_vq, cmd, - sizeof cmd->req.tmf, sizeof cmd->resp.tmf) < 0) + if (virtscsi_add_cmd(&vscsi->ctrl_vq, cmd, + sizeof cmd->req.tmf, sizeof cmd->resp.tmf, true) < 0) goto out; wait_for_completion(&comp); @@ -664,6 +681,13 @@ static int virtscsi_map_queues(struct Scsi_Host *shost) return blk_mq_virtio_map_queues(qmap, vscsi->vdev, 2); } +static void virtscsi_commit_rqs(struct Scsi_Host *shost, u16 hwq) +{ + struct virtio_scsi *vscsi = shost_priv(shost); + + virtscsi_kick_vq(&vscsi->req_vqs[hwq]); +} + /* * The host guarantees to respond to each command, although I/O * latencies might be higher than on bare metal. Reset the timer @@ -681,6 +705,7 @@ static struct scsi_host_template virtscsi_host_template = { .this_id = -1, .cmd_size = sizeof(struct virtio_scsi_cmd), .queuecommand = virtscsi_queuecommand, + .commit_rqs = virtscsi_commit_rqs, .change_queue_depth = virtscsi_change_queue_depth, .eh_abort_handler = virtscsi_abort, .eh_device_reset_handler = virtscsi_device_reset,