From patchwork Fri Apr 1 13:19:52 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 8724501 Return-Path: X-Original-To: patchwork-qemu-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id C26419F38C for ; Fri, 1 Apr 2016 13:20:34 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id A3D0E203AE for ; Fri, 1 Apr 2016 13:20:33 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8DD4B20251 for ; Fri, 1 Apr 2016 13:20:32 +0000 (UTC) Received: from localhost ([::1]:44322 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1alz03-0000LN-R5 for patchwork-qemu-devel@patchwork.kernel.org; Fri, 01 Apr 2016 09:20:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56293) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1alyzf-0008Pc-OE for qemu-devel@nongnu.org; Fri, 01 Apr 2016 09:20:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1alyzd-0001uA-VN for qemu-devel@nongnu.org; Fri, 01 Apr 2016 09:20:07 -0400 Received: from mail-wm0-x244.google.com ([2a00:1450:400c:c09::244]:33480) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1alyzd-0001tH-LC for qemu-devel@nongnu.org; Fri, 01 Apr 2016 09:20:05 -0400 Received: by mail-wm0-x244.google.com with SMTP id i204so4581231wmd.0 for ; Fri, 01 Apr 2016 06:20:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=fGy80Wp9s0C2yT8XMnq0nMBa6JcC7yDYjHVjuKh5Mtc=; b=iRTsV9WbK49VaYH6eo9hXIRWRurGOpLIXVL2yq82FAWm9fVRZf7eZne5hQJzk0Ww0L iksf2Hp4RabVrAmJ1y1NHTemtbAjEwwU8PZ6x6hzBMtYk5b/sAZbbUVk+Nbi1eDSV6IZ ZvSCjfC1hou7Ir9qparxQKKy71B8/RQ1F5eI6WnqSvjGouAsuDxP7znThHrJNGkd6ZHn m9XxSJhitXGps4t5GWfoAhnnxq9EAfEI56C1xr4zX/C7kuM44Q/XFa2ITixcM3z3M/A3 hFlVcv2OwzbM9Mxpj7P1nmEUlO+6VVsctB7t7URMtV/+ZZPe8dptobr1iAyKH/lV4Qd3 kFjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=fGy80Wp9s0C2yT8XMnq0nMBa6JcC7yDYjHVjuKh5Mtc=; b=WdCfqOya39ExMOJJExtFkZtjDVS5uySHI35vp5TzbTOqa62s4z+LJpxkm60jQBiX8i 6/4c11TQxRj0t9ND+MHzBoIa3PxGofHNAXmHUUjxEf7/m36VRqRwhgNA4a+01Hw5/U59 ZvmFMhZ18PPo5OlPgdGAMraOB21WMiuIHNeOyGLETywDVlS8la9XRSvHCMjhThGjmjh9 WTTuka7TP1LRpMNdfHHXMR8EGchkTfgkcoInRoIYJOJ1IuclkJNVusEN9klUdiOtGIWL dmQUsybzLLvLupW5VoCtzMuYSYk6zvoLhwCwMGewG+9zkneQvq3sKJrv4FX5KFY/kp/V rvcg== X-Gm-Message-State: AD7BkJKMJEvyS4bo+cqWKSbJUIQCIvaPWt0/p1SMfoyDP2Nyhk8uZ7BUhsuktvr4vYa6bg== X-Received: by 10.28.111.14 with SMTP id k14mr4025304wmc.82.1459516805116; Fri, 01 Apr 2016 06:20:05 -0700 (PDT) Received: from 640k.lan (94-39-141-76.adsl-ull.clienti.tiscali.it. [94.39.141.76]) by smtp.gmail.com with ESMTPSA id a1sm14032334wje.43.2016.04.01.06.20.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Apr 2016 06:20:04 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Fri, 1 Apr 2016 15:19:52 +0200 Message-Id: <1459516794-23629-8-git-send-email-pbonzini@redhat.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1459516794-23629-1-git-send-email-pbonzini@redhat.com> References: <1459516794-23629-1-git-send-email-pbonzini@redhat.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::244 Cc: famz@redhat.com, tubo@linux.vnet.ibm.com, mst@redhat.com, borntraeger@de.ibm.com, stefanha@redhat.com, cornelia.huck@de.ibm.com Subject: [Qemu-devel] [PATCH 7/9] virtio-scsi: use aio handler for data plane X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In addition to handling IO in vcpu thread and in io thread, dataplane introduces yet another mode: handling it by aio. This reuses the same handler as previous modes, which triggers races as these were not designed to be reentrant. Use a separate handler just for aio, and disable regular handlers when dataplane is active. Reviewed-by: Cornelia Huck Signed-off-by: Paolo Bonzini --- hw/scsi/virtio-scsi-dataplane.c | 43 ++++++++++++++++++++++++--- hw/scsi/virtio-scsi.c | 65 ++++++++++++++++++++++++++++------------- include/hw/virtio/virtio-scsi.h | 6 ++-- 3 files changed, 86 insertions(+), 28 deletions(-) diff --git a/hw/scsi/virtio-scsi-dataplane.c b/hw/scsi/virtio-scsi-dataplane.c index 21d5bfd..a497a2c 100644 --- a/hw/scsi/virtio-scsi-dataplane.c +++ b/hw/scsi/virtio-scsi-dataplane.c @@ -38,7 +38,35 @@ void virtio_scsi_set_iothread(VirtIOSCSI *s, IOThread *iothread) } } -static int virtio_scsi_vring_init(VirtIOSCSI *s, VirtQueue *vq, int n) +static void virtio_scsi_data_plane_handle_cmd(VirtIODevice *vdev, + VirtQueue *vq) +{ + VirtIOSCSI *s = (VirtIOSCSI *)vdev; + + assert(s->ctx && s->dataplane_started); + virtio_scsi_handle_cmd_vq(s, vq); +} + +static void virtio_scsi_data_plane_handle_ctrl(VirtIODevice *vdev, + VirtQueue *vq) +{ + VirtIOSCSI *s = VIRTIO_SCSI(vdev); + + assert(s->ctx && s->dataplane_started); + virtio_scsi_handle_ctrl_vq(s, vq); +} + +static void virtio_scsi_data_plane_handle_event(VirtIODevice *vdev, + VirtQueue *vq) +{ + VirtIOSCSI *s = VIRTIO_SCSI(vdev); + + assert(s->ctx && s->dataplane_started); + virtio_scsi_handle_event_vq(s, vq); +} + +static int virtio_scsi_vring_init(VirtIOSCSI *s, VirtQueue *vq, int n, + void (*fn)(VirtIODevice *vdev, VirtQueue *vq)) { BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(s))); VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); @@ -54,6 +82,7 @@ static int virtio_scsi_vring_init(VirtIOSCSI *s, VirtQueue *vq, int n) } virtio_queue_aio_set_host_notifier_handler(vq, s->ctx, true, true); + virtio_set_queue_aio(vq, fn); return 0; } @@ -71,9 +100,12 @@ static void virtio_scsi_clear_aio(VirtIOSCSI *s) int i; virtio_queue_aio_set_host_notifier_handler(vs->ctrl_vq, s->ctx, true, false); + virtio_set_queue_aio(vs->ctrl_vq, NULL); virtio_queue_aio_set_host_notifier_handler(vs->event_vq, s->ctx, true, false); + virtio_set_queue_aio(vs->event_vq, NULL); for (i = 0; i < vs->conf.num_queues; i++) { virtio_queue_aio_set_host_notifier_handler(vs->cmd_vqs[i], s->ctx, true, false); + virtio_set_queue_aio(vs->cmd_vqs[i], NULL); } } @@ -104,16 +136,19 @@ void virtio_scsi_dataplane_start(VirtIOSCSI *s) } aio_context_acquire(s->ctx); - rc = virtio_scsi_vring_init(s, vs->ctrl_vq, 0); + rc = virtio_scsi_vring_init(s, vs->ctrl_vq, 0, + virtio_scsi_data_plane_handle_ctrl); if (rc) { goto fail_vrings; } - rc = virtio_scsi_vring_init(s, vs->event_vq, 1); + rc = virtio_scsi_vring_init(s, vs->event_vq, 1, + virtio_scsi_data_plane_handle_event); if (rc) { goto fail_vrings; } for (i = 0; i < vs->conf.num_queues; i++) { - rc = virtio_scsi_vring_init(s, vs->cmd_vqs[i], i + 2); + rc = virtio_scsi_vring_init(s, vs->cmd_vqs[i], i + 2, + virtio_scsi_data_plane_handle_cmd); if (rc) { goto fail_vrings; } diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c index 38f1e2c..30415c6 100644 --- a/hw/scsi/virtio-scsi.c +++ b/hw/scsi/virtio-scsi.c @@ -374,7 +374,7 @@ fail: return ret; } -void virtio_scsi_handle_ctrl_req(VirtIOSCSI *s, VirtIOSCSIReq *req) +static void virtio_scsi_handle_ctrl_req(VirtIOSCSI *s, VirtIOSCSIReq *req) { VirtIODevice *vdev = (VirtIODevice *)s; uint32_t type; @@ -412,20 +412,28 @@ void virtio_scsi_handle_ctrl_req(VirtIOSCSI *s, VirtIOSCSIReq *req) } } -static void virtio_scsi_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq) +void virtio_scsi_handle_ctrl_vq(VirtIOSCSI *s, VirtQueue *vq) { - VirtIOSCSI *s = (VirtIOSCSI *)vdev; VirtIOSCSIReq *req; - if (s->ctx && !s->dataplane_started) { - virtio_scsi_dataplane_start(s); - return; - } while ((req = virtio_scsi_pop_req(s, vq))) { virtio_scsi_handle_ctrl_req(s, req); } } +static void virtio_scsi_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq) +{ + VirtIOSCSI *s = (VirtIOSCSI *)vdev; + + if (s->ctx) { + virtio_scsi_dataplane_start(s); + if (!s->dataplane_fenced) { + return; + } + } + virtio_scsi_handle_ctrl_vq(s, vq); +} + static void virtio_scsi_complete_cmd_req(VirtIOSCSIReq *req) { /* Sense data is not in req->resp and is copied separately @@ -508,7 +516,7 @@ static void virtio_scsi_fail_cmd_req(VirtIOSCSIReq *req) virtio_scsi_complete_cmd_req(req); } -bool virtio_scsi_handle_cmd_req_prepare(VirtIOSCSI *s, VirtIOSCSIReq *req) +static bool virtio_scsi_handle_cmd_req_prepare(VirtIOSCSI *s, VirtIOSCSIReq *req) { VirtIOSCSICommon *vs = &s->parent_obj; SCSIDevice *d; @@ -550,7 +558,7 @@ bool virtio_scsi_handle_cmd_req_prepare(VirtIOSCSI *s, VirtIOSCSIReq *req) return true; } -void virtio_scsi_handle_cmd_req_submit(VirtIOSCSI *s, VirtIOSCSIReq *req) +static void virtio_scsi_handle_cmd_req_submit(VirtIOSCSI *s, VirtIOSCSIReq *req) { SCSIRequest *sreq = req->sreq; if (scsi_req_enqueue(sreq)) { @@ -560,17 +568,11 @@ void virtio_scsi_handle_cmd_req_submit(VirtIOSCSI *s, VirtIOSCSIReq *req) scsi_req_unref(sreq); } -static void virtio_scsi_handle_cmd(VirtIODevice *vdev, VirtQueue *vq) +void virtio_scsi_handle_cmd_vq(VirtIOSCSI *s, VirtQueue *vq) { - /* use non-QOM casts in the data path */ - VirtIOSCSI *s = (VirtIOSCSI *)vdev; VirtIOSCSIReq *req, *next; QTAILQ_HEAD(, VirtIOSCSIReq) reqs = QTAILQ_HEAD_INITIALIZER(reqs); - if (s->ctx && !s->dataplane_started) { - virtio_scsi_dataplane_start(s); - return; - } while ((req = virtio_scsi_pop_req(s, vq))) { if (virtio_scsi_handle_cmd_req_prepare(s, req)) { QTAILQ_INSERT_TAIL(&reqs, req, next); @@ -582,6 +584,20 @@ static void virtio_scsi_handle_cmd(VirtIODevice *vdev, VirtQueue *vq) } } +static void virtio_scsi_handle_cmd(VirtIODevice *vdev, VirtQueue *vq) +{ + /* use non-QOM casts in the data path */ + VirtIOSCSI *s = (VirtIOSCSI *)vdev; + + if (s->ctx) { + virtio_scsi_dataplane_start(s); + if (!s->dataplane_fenced) { + return; + } + } + virtio_scsi_handle_cmd_vq(s, vq); +} + static void virtio_scsi_get_config(VirtIODevice *vdev, uint8_t *config) { @@ -725,17 +741,24 @@ out: } } +void virtio_scsi_handle_event_vq(VirtIOSCSI *s, VirtQueue *vq) +{ + if (s->events_dropped) { + virtio_scsi_push_event(s, NULL, VIRTIO_SCSI_T_NO_EVENT, 0); + } +} + static void virtio_scsi_handle_event(VirtIODevice *vdev, VirtQueue *vq) { VirtIOSCSI *s = VIRTIO_SCSI(vdev); - if (s->ctx && !s->dataplane_started) { + if (s->ctx) { virtio_scsi_dataplane_start(s); - return; - } - if (s->events_dropped) { - virtio_scsi_push_event(s, NULL, VIRTIO_SCSI_T_NO_EVENT, 0); + if (!s->dataplane_fenced) { + return; + } } + virtio_scsi_handle_event_vq(s, vq); } static void virtio_scsi_change(SCSIBus *bus, SCSIDevice *dev, SCSISense sense) diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scsi.h index eef4e95..ba2f5ce 100644 --- a/include/hw/virtio/virtio-scsi.h +++ b/include/hw/virtio/virtio-scsi.h @@ -139,9 +139,9 @@ void virtio_scsi_common_realize(DeviceState *dev, Error **errp, HandleOutput cmd); void virtio_scsi_common_unrealize(DeviceState *dev, Error **errp); -void virtio_scsi_handle_ctrl_req(VirtIOSCSI *s, VirtIOSCSIReq *req); -bool virtio_scsi_handle_cmd_req_prepare(VirtIOSCSI *s, VirtIOSCSIReq *req); -void virtio_scsi_handle_cmd_req_submit(VirtIOSCSI *s, VirtIOSCSIReq *req); +void virtio_scsi_handle_event_vq(VirtIOSCSI *s, VirtQueue *vq); +void virtio_scsi_handle_cmd_vq(VirtIOSCSI *s, VirtQueue *vq); +void virtio_scsi_handle_ctrl_vq(VirtIOSCSI *s, VirtQueue *vq); void virtio_scsi_init_req(VirtIOSCSI *s, VirtQueue *vq, VirtIOSCSIReq *req); void virtio_scsi_free_req(VirtIOSCSIReq *req); void virtio_scsi_push_event(VirtIOSCSI *s, SCSIDevice *dev,